Keep goals, constraints, and pending work

An original handoff note should retain the user’s objective, accepted constraints, completed changes, unresolved errors, and exact references to relevant files. Preserve decisions that would otherwise cause the next run to repeat work or change scope.

A summary can lose important qualifications

A compacted summary is an interpretation. When a remembered detail affects a significant action, confirm it against the underlying record rather than treating the summary as infallible.

What gets lost

A summary of a long chat is shorter because it omits. It will omit the exception the user added in the middle, unless your compactor is scored on keeping exceptions.

A rolling summary written by the same model can also sand off a refusal and leave the goal. Test compaction with a constraint that appears once, early, and matters later.

Tool results are the usual bulk. You rarely need the whole JSON again.

You do need the id of the record that was created and the error that is still unresolved. Replace the raw payload with a short structured remainder.

Do not ask the model to “keep the important parts” without listing the fields that count as important.

When to compact

Compact when the next call would overflow, not after every turn. Each compaction is a chance to distort the task and a spend of its own.

Keep the original transcript in your logs even if the model no longer sees it. A user who says “I told you not to” needs the original line, not the summary that lost it.

If a step is a workflow, you may not need the chat at all. Pass the artifact.

Compaction is a repair for an agent that has been talking too long. It is not a substitute for a design that never put the whole history in the prompt.

What to record before you rely on it

Name the fields a summary must preserve: the goal, the constraints, the open error, and the ids of objects already created. Test a transcript where the only copy of a “do not” is in the middle.

Confirm the next action still obeys it. Keep the uncompacted log.

Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test. When the result is a failure, store the artifact and change the design.

Do not relax the check until the model happens to pass. Name a person who owns that failure.

A note with no owner and no inputs cannot be repeated, and a pass with no date cannot be compared with the next model id.

Common questions

Is a summary cheaper than the long transcript?

The next calls are cheaper only if the summary is actually shorter and you stop sending the original. The summary call itself costs tokens.

Measure both, and measure the mistakes the summary introduced.

Can I drop tool outputs entirely?

You can drop the bulky parts if you keep what the next step must not redo or undo: created ids, failed ids, and the user’s constraints. Dropping those is how an agent repeats a purchase.

Sources & further reading

  1. Anthropic: Effective context engineering for AI agents ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories