# Test an agent with evals

> Save representative tasks and rerun them after changes to check behavior.

Use evals to check whether an agent still meets your expectations after changing instructions, files or tools. Each case contains a task briefing and criteria for a good result.

## Before you begin

Owners and dashboard editors manage cases and run suites. Viewers can read results. Each suite consumes account Sparks; it tests the current agent configuration.

## Save a case

1. Open **Manage** → **Activity** and select a good completed run.
2. Select **Save as eval**. From **Chat**, use **Save last exchange as eval case** instead.
3. Review **Name** and **Briefing** so the task can run independently.
4. Write **Expectation** with required sources, output checks and actions the agent must avoid.
5. Select **Save case**.

To author a case without a past run, open **Manage** → **Evals** and select **New case**. A source execution is optional; when provided, it must belong to the same agent.

Use durable criteria, such as required columns or “must not send email”. Avoid exact dates or figures that change between runs.

## Prepare file-based cases

1. Find representative spreadsheets, CSVs, PDFs or other input files in Workspace, or upload examples.
2. Keep independent copies of inputs and reference outputs in `agents/<agentId>/eval/<case-slug>/`.
3. Put the verified copy paths in the briefing and expected checks in the expectation.
4. Treat replacement of a reference output as an explicit baseline change.

When the dashboard Assistant prepares these copies, complete its file-copy approvals before it uses their paths in a case. Without representative examples, treat the configuration as provisional.

## Run and compare

1. Open **Manage** → **Evals**.
2. Select **Run evals**. Keep at most 10 cases active for the dashboard run; archive others or ask the Assistant to run a subset.
3. Read each pass/fail verdict and the judge's explanation.
4. Open failed runs to inspect the audit and identify missing data, configuration errors or runtime failures.
5. Make a targeted correction and rerun the affected cases.

A good run becomes a saved case. After a change, the suite repeats its briefing and compares the evidence with your expectation.

> **Rule** — Reads are real; external writes are simulated. No email is sent. Code executes without internet, and output files go to a temporary test Workspace. Later steps and the judge can read those files; your real Workspace stays unchanged.

The judge examines outputs and simulated actions, including file structure and recipients. A skipped or unavailable test leaves behavior unverified.

The dashboard Assistant can create, edit, archive or delete cases and run suites without an approval prompt. Other operations keep their own approval requirements. Archive cases to exclude them while retaining history.

Before running evals, the Assistant reviews cases against the agent's current instructions and relevant skills, updates outdated briefings or expectations and checks required files. Valid regression criteria and reference outputs remain the baseline; they are not relaxed just to obtain a pass.

After configuring an agent, the Assistant may run relevant cases once and report the results and Sparks spent. It does not automatically repeat paid runs after a failure or an approved fix.

If you explicitly ask it to ensure the eval passes, it may fix root causes and rerun affected cases, up to three suite executions per agent for that request, including earlier attempts. It stops on success, a blocker, no meaningful progress or the limit and reports what remains. You can request a lower budget, another run or no testing.

| Limit | Maximum |
| --- | --- |
| Saved cases per agent | 50 |
| Cases per suite execution | 10 |
| Case name | 120 characters |
| Briefing | 8,000 characters |
| Expectation | 2,000 characters |

## Related

- [Agent instructions](/en/docs/agents/instructions)
- [Workspace access](/en/docs/agents/workspace-access)
- [Usage and costs](/en/docs/usage)