Keep source locations with retrieved vectors

An original document system can split source material into passages, embed them, and retrieve candidate matches. Keep identifiers and source locations so the result can be checked against the original document.

Approximate search trades precision for efficiency

An approximate index may trade retrieval precision for speed or memory efficiency. Evaluate both retrieval quality and serving performance, and remember that a nearby passage can still be irrelevant to the exact question.

What the index returns

You embed documents, store the vectors, and embed the query with the same model. The index returns the closest vectors under a distance.

An approximate index can miss a true neighbor in exchange for speed. That miss is a system choice, not a claim that the missing passage was irrelevant.

Chunking decides what “a document” is. A 500-token window can split a definition from its exception.

Overlap reduces that cut and also duplicates text. Metadata filters, such as date or product, should run where the index supports them.

A nearest neighbor from the wrong year is a successful search and a failed task.

How to tell that nearest was not relevant

Label a set of questions with the passages a person would accept. Measure how often an acceptable passage appears in the top five, not only whether the top hit looks plausible.

Include questions where the right behavior is to find nothing. An index that always returns something will look busy and still be wrong.

Keep the original text and its source URL next to the vector. The user, and any later model call, needs the words, not the coordinates.

If you delete or update a page, delete or rebuild the chunk. A stale vector is still a nearest neighbor.

  • Embed queries and documents with the same model.
  • Decide what a chunk includes, especially exceptions and dates.
  • Score a labeled set, including questions with no good passage.
  • Store the source text beside the vector and expire it when the page changes.

What to record before you rely on it

Document the chunk size, the overlap, the filters, and whether the index is exact or approximate. Store the source URL and the text beside each vector.

Schedule a rebuild when the source page changes. On the labeled questions, write down both the rank of the right passage and the questions where no passage should have matched.

An index that cannot return “nothing” will invent a neighbor for every query. Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test.

Common questions

Is the top vector hit the answer?

It is the nearest stored chunk. Read it.

The answer, if there is one, is in the text, and sometimes the honest result is that none of the hits support the question.

Why did a keyword I can see in the page not match?

Vector search matches the embedding, not the exact string. A rare identifier, a SKU, or a name can be closer in a keyword index.

Many systems run both and merge the lists.

Sources & further reading

  1. Faiss: Official documentation ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories