Sources checked
Retrieval provides current input without retraining weights
An original document assistant can use retrieval to supply current policies at inference time. Fine-tuning is a separate development process with its own dataset and evaluation.
Product memory or conversation storage is another separate capability.
Service data policies are a separate question
A provider may have data-use policies that allow submitted information to be retained or used later. That is a service-policy question, not a reason to assume every individual request immediately trains the model.
What training is doing
Training is the stage that adjusts weights. Pretraining predicts tokens across a huge corpus.
Fine-tuning continues that adjustment on a narrower set of examples. Preference training, such as reinforcement learning from human feedback, changes weights using judgments about which answer is better.
When training finishes, the weights are a fixed artifact until someone trains again.
Training needs data, a loss function, and a lot of compute. It also needs a decision about what the examples are allowed to teach.
A model can memorize a rare string that appeared a few times. That is why training data, not the chat window, is the place to argue about private documents and copyrighted text.
What inference is doing
Inference does not update the weights. The model reads a new prompt and produces tokens.
Temperature, tools, and a retrieved document change the input or the sampling. They do not write a new checkpoint.
If the answer is wrong, the next prompt can try a different instruction, but the underlying model is the same until you deploy a new version.
Product teams usually pay for inference per token or per machine-hour. They pay for training when they create or adapt a checkpoint.
Mixing the words hides the bill. A “training” dashboard that only shows chat usage is an inference report.
A fine-tune job that writes a new model id is training, even if it took minutes.
| Question | Training | Inference |
|---|---|---|
| Does it change weights? | Yes, if the job is allowed to update them | No. The checkpoint stays as deployed |
| What do you supply? | Examples, a loss, and compute | A prompt, optional tools, and a finished model |
| Where does a private document go? | Into the training set if you choose that | Into that request, subject to the provider’s retention rule |
What to record before you rely on it
Record whether a job writes a new checkpoint or only returns tokens. If it writes a checkpoint, name the training examples, the base model, and who is allowed to deploy the result.
If it only returns tokens, name the model id and the retention rule for that request. A dashboard that mixes the two will send a team looking for a data leak in the wrong system.
The note can be a single column on the design document: “weights change” or “weights stay.”
Keep the model id, the date, and the inputs you used beside that note, so a later change can be compared with the same test.
Common questions
Is retrieval-augmented generation a kind of training?
No. Retrieval puts extra text into the prompt for one request. The weights stay as they were.
Fine-tuning is the path that changes weights from examples.
Why does a trained model still fail on a new sentence?
Training fits the weights to the examples and the loss. A new sentence can sit outside that pattern.
Inference has no memory of your correction unless you send it again or train a new checkpoint.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards





