Sources checked
Validate extracted values after parsing
An original extraction schema might require a source identifier, a date, and an amount. After parsing, check whether the source exists, the date is supported, and the amount matches the supplied document.
Keep an explicit outcome for missing evidence.
Required fields should not force invented facts
A schema should not force the model to invent required facts. Design unknown or review states where appropriate and validate business rules before using the output.
What the constraint guarantees
Providers can constrain decoding so the next token must keep the output inside a grammar. That raises the chance you get parseable JSON, with the keys you named, and with enums you allowed.
It does not know which invoice number is printed on the page. A required string field will be filled with something even when the source is blank, unless you made “unknown” an allowed value and tested that path.
Schemas drift. A model that worked with five keys will not reliably invent a sixth key you added in the application but forgot to put in the grammar.
Generate the grammar from the same source as the parser. If they diverge, you will accept objects your own code later rejects, or reject objects the model was never told to produce.
What still has to be checked
After parsing, check types, ranges, and references. A date string can be valid JSON and an impossible calendar day.
An id can match the pattern and point at no record. These are ordinary program checks.
The model call does not replace them.
Keep the source span when the field is extracted from a document. If you cannot highlight where the total came from, do not post it into a ledger without a person.
For classification, measure agreement on a labeled set. A schema that always returns one of three labels can be perfectly valid and consistently wrong.
| Check | What passes | What can still be wrong |
|---|---|---|
| Grammar or schema | The shape matches | A required field was invented |
| Parser | Your code accepts the object | A value points at nothing |
| Source span | You can see the supporting text | Nothing, if the span actually supports the field |
What to record before you rely on it
Generate the grammar from the same schema the application parses. Record the model id and the schema version on every accepted object.
Add an explicit empty value for fields the source may not contain, and test a blank document. After parsing, check ranges and foreign keys in ordinary code.
Keep one failed example where the JSON was valid and the value was still wrong, so the team remembers that validity was only the first gate.
Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test.
Common questions
Is JSON mode the same as a schema?
JSON mode usually means the output is some JSON value. A schema names the keys, types, and enums.
Ask which one the endpoint enforces. “JSON” alone can be an object with keys you did not expect.
What should an extractor return when the page is blank?
An explicit empty or unknown value that your schema allows. If every field is required and has no empty option, the model has to invent one to satisfy the grammar.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards





