Sources checked
Separate recurring usage from setup and maintenance
An original budget can separate recurring usage from setup costs and maintenance. Add the cost of retries, external tools, storage, and review.
Track cost per accepted task so different architectures can be compared on the same outcome.
A cost estimate needs workload assumptions
A provider’s list price does not predict your bill without workload assumptions. State traffic, input lengths, output lengths, and retry behavior whenever you publish a cost estimate.
Separate the bill from the system
List price times tokens is one line. Add the machines or the vendor minimum, the vector index, the logs, the evaluation set, and the on-call time.
Add retries. A flow that calls a model three times because the first two failed the schema costs three times the demo.
Add the reviewer. If a person still checks every output, their minutes are the product.
A pilot should use the same task mix as the work it claims to replace. Count accepted results, rejected results, and repairs.
An apparent saving disappears when the automation moves the effort to another team. This is a worksheet, not a measured return for any particular company.
Do not multiply a one-week pilot on easy tickets across a year of hard ones.
Assumptions that have to be written down
State traffic, input length, output length, and the retry rate. Without those, two vendors cannot be compared, because each spreadsheet will assume a different job.
State what “accepted” means. A draft that a lawyer rewrites is not the same accepted unit as a draft that is sent.
Revisit the sheet when the model id changes. A cheaper token can be a more expensive system if the new model needs more retries or a longer prompt.
Keep the provider’s rate card URL and the date. Our rate pages are list prices, not this worksheet, and they are not an invoice.
| Line | What to put on it | What people forget |
|---|---|---|
| Usage | Input, output, and retries | Hidden reasoning tokens and tool transcripts |
| Surroundings | Index, logs, integration, evaluation | The time to build the prompt and the checks |
| People | Minutes to accept or repair a result | A pilot that only counted generated words |
What to record before you rely on it
Keep one sheet per use case with the workload assumptions and the definition of an accepted task. Attach the pilot’s reject rate.
Update it when the model id or the list price changes. Do not publish a savings number that excludes review.
Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test. When the result is a failure, store the artifact and change the design.
Do not relax the check until the model happens to pass. Name a person who owns that failure.
A note with no owner and no inputs cannot be repeated, and a pass with no date cannot be compared with the next model id.
Common questions
Is a cheaper model a cheaper system?
Only if accepted tasks stay acceptable. A model that is half the price and needs twice the review, or twice the retries, is not a saving.
Measure the task you keep.
How long should a pilot run before anyone quotes a return?
Long enough to include the hard cases, not only the cases that were chosen because they demo well. Report it as a pilot on that task mix, not as a company-wide result.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards





