feat(evaluation): offline evaluation module with uv run evaluate CLI#280

Merged

DhavalRepo18 merged 9 commits into

mainfrom

feat/evaluation-module

May 13, 2026

Commits on Apr 27, 2026

feat(evaluation): add offline evaluation module with uv run evaluate CLI
ShuxinLin
committed

Commits on May 13, 2026

Merge remote-tracking branch 'origin/main' into feat/evaluation-module
ShuxinLin
committed
refactor(evaluation): adopt MLflow Evaluator/Scorer concept
ShuxinLin
committed
refactor(evaluation): drop code-based scorer bodies, keep skeleton
ShuxinLin
committed
fix(evaluation): align llm_judge prompt with reference; fix LiteLLM kwarg
ShuxinLin
committed
feat(evaluation): write per-run reports to reports/<run_id>.json
ShuxinLin
committed
docs(evaluation): add docs/evaluation.md; semantic scorer is skeleton
ShuxinLin
committed
refactor(evaluation): standardize grade/grader -> score/scorer
ShuxinLin
committed
docs(evaluation): clarify skeleton scorers in scenario-field table; add aggregate JSON shape
ShuxinLin
committed