Sources checked
Accepting long input is different from finding evidence
The experiments include multi-document question answering and key-value retrieval. They distinguish accepting a long input from finding the needed evidence within it.
The paper is evidence about specific systems and tasks, not a permanent ranking of newer models.
Move the answer passage in your own test
For an original application test, move the same answer-bearing passage between the beginning, middle, and end. Keep the question unchanged and measure whether the system uses the evidence consistently.
What they measured
They built tasks where the answer sits in one passage among many, and they moved that passage from the beginning to the middle to the end.
Accuracy on their models formed a U-shape: better at the edges, worse in the middle. The result is about those models and those tasks.
It is a warning about long contexts, not a percentage you can paste onto a new product.
The practical implication they earned is about evaluation. A demo that puts the relevant paragraph first does not measure a system that retrieves a paragraph into the middle.
If your pipeline stuffs search results into a prompt, move the relevant one and see if the answer survives.
What changed since the paper
Newer models advertise longer windows and may score better on this exact probe. You still have to run the probe, because the advertisement is a length, not a placement study.
If a vendor publishes a counter-result, cite it beside this paper. Do not average them.
What to record before you rely on it
If you rely on the finding, cite Liu and colleagues and state that it was their models and tasks. Run a three-position check on your own prompt: start, middle, end.
Save the outputs. Keep the model id or the paper citation, the date you read it, and the inputs you used beside that note, so a later change can be compared with the same test.
When the result is a failure, store the artifact and change the design. Do not relax the check until the model happens to pass.
Name a person who owns that failure.
Common questions
Does a larger context window fix the middle?
Not by definition. A window is a capacity.
The paper is about use. Measure use at the positions you care about.
Should I put instructions at the start and the end?
It is a reasonable experiment, given the paper. It is not a substitute for checking.
Repeat the instruction only if your test shows the model otherwise drops it.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards





