itssujeeth Solo engineer · 2024
Agent Evaluation Framework
Cut the regression detection cycle from days to a 4-minute CI gate for LLM-based agents.
LLM agent regressions were invisible until they shipped. Manual QA took 2 to 3 days per release and still missed subtle behavior changes in multi-step reasoning chains.
Built a lightweight eval harness that runs a fixed scenario battery on every PR, diffs output against golden traces, and posts pass/fail to GitHub Actions. Zero external dependencies beyond the Claude API.
4-minute CI gate replaced 2-day manual QA cycle. Caught 3 silent regressions in the first week of production use — behavior changes that manual review had missed entirely.
The problem with LLM agent QA is that regressions look nothing like traditional software failures. A broken API call returns an error. A regressed agent often returns a plausible-looking answer — just not the right one. Manual review catches obvious breaks but misses subtle behavioral shifts: a reasoning chain that used to call three tools now calls two, a response that used to cite the source now paraphrases without attribution. These changes don’t raise exceptions. They erode trust quietly.
The eval harness treats golden traces as the ground truth. Each scenario is a fixed input sequence; the expected output is a recorded trace of the correct multi-step reasoning chain. On every PR, the harness replays the scenario, diffs the output against the golden trace, and posts a structured pass/fail to GitHub Actions. No external eval service, no database. Claude API in, structured comparison out.
The engineering choice that made this practical was keeping the diff at the behavioral level, not the token level. Exact string matching on LLM output is brittle — models vary their phrasing between runs. The harness normalizes for phrasing variance and checks the semantic structure: which tools were called, in what order, with what arguments, and whether the final answer class matches. That distinction is what separates a useful CI gate from a noise generator.
Three regressions caught in week one validated the bet. All three were silent: the agent returned confident, coherent answers that were wrong in a way the team cared about. The CI gate surfaced them before they reached staging. That’s the definition of done for this kind of tool.