Skip to main content
Open an experiment to inspect its individual cases and compare it with a completed baseline. You need access to the experiment and its workspace.

​
Open the run

  1. Open Evals → Experiments and select the agent or other subject, then the run. Runs without a subject can also open from the experiment list.
  2. Check its status and recorded case count before reading score averages.
  3. Review the subject, dataset snapshot, scorers, baseline, and recorded model or configuration.
An experiment marked completed means its runner finished and finalized it. Inspect case errors and score coverage before treating that status as a quality result.

​
Compare cases

  1. Open Cases.
  2. Choose a baseline with Change, or select No baseline when no comparison applies.
  3. Search, filter, and sort the cases to find regressions, improvements, or failures.
  4. Select a case and compare its input, output, expected result, scores, and available trace evidence.
Compare like cases under the same criteria. An aggregate change can come from a changed dataset, scorer, model, or context; the run’s recorded configuration helps you check those differences. Use Set as baseline to make a completed run the default comparison for that subject on this device. This preference applies to the subject page, run page, and new-experiment drawer on the same device. It does not set an organization-wide baseline or change an already stored experiment’s baseline.

​
Inspect what changed

Open Diff to compare the recorded manifest with the baseline. Changed items appear first. When both runs pin a skill version, the view can show the SKILL.md changes between those versions. Not pinned means a version was not recorded. This run pinned no manifest means there is no stored context manifest to compare. A recorded manifest identifies what the runner reported; it does not prove that a local harness installed those versions. The desktop runner uses its installed context. If a skill’s package changed but SKILL.md is identical, inspect the supporting files in the skill’s Source tab.

​
Decide the next change

  • Reproduce a regression on the same cases before changing the baseline.
  • Keep a failed production case in the dataset so the next run checks it again.
  • Use Rerun to open a new experiment setup, then verify its local runner and configuration.
  • Use CI gates when the comparison must be an automated pull request check.
For reproducible comparisons from code, see Compare two agent versions.