Test complete outcomes from a reset starting state

An original suite for a coding assistant might include a small bug, an ambiguous request, a failing dependency, and a task requiring no change. Save the starting state so runs can be compared fairly.

Version the cases and inspect failures

A passing score can become misleading if tasks leak into prompts or the environment changes. Version the cases and review failures after modifying models, tools, or instructions.

What a task must include

Each task needs a starting state, a goal a stranger could judge, and the tools the agent may use.

It also needs at least one thing that must not happen: a file that must remain, an email that must not send, a record that must not change.

If you only score the final sentence, an agent that corrupted the state and then apologized will pass.

Include tasks that should end with “I cannot” or “the data is not there.” An agent that always does something will look busy on those tasks and still be wrong.

Use a fresh sandbox per task so one run’s leftovers do not become the next run’s cheat.

How to score without fooling yourself

Automatic checks belong on the sandbox: did the expected file change, did the forbidden call appear in the log, did the tests pass.

Model-graded essays about whether the agent “tried hard” are a last resort, and they need a sample checked by a person. Publish the pass rate with the task count.

A perfect score on three happy paths is a demo.

Re-run the suite when you change the model, the tools, or the system prompt. An agent suite goes stale faster than a quiz, because the environment is part of the question.

Store the traces. When a task fails, you need the tool calls, not a single number in a slide.

What to record before you rely on it

Start with twenty tasks: ten that should succeed, five that should refuse, and five where a specific tool call is forbidden. Store them as code.

Gate any new tool on a run of that file. Write the pass rate next to the model id and the date, and do not average away the forbidden-call failures.

Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test. When the result is a failure, store the artifact and change the design.

Do not relax the check until the model happens to pass. Name a person who owns that failure.

A note with no owner and no inputs cannot be repeated, and a pass with no date cannot be compared with the next model id.

Common questions

Can I evaluate an agent with a normal benchmark?

A question-answer benchmark misses tool use. You can use one as a side check.

The agent score has to look at actions in an environment you control.

How often should the suite run?

On every change of model, prompt, or tool schema, and on a schedule if the provider can change the model behind an id. A suite that ran once at launch is a souvenir.

Sources & further reading

  1. Anthropic: Demystifying evals for AI agents ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories