Sources checked
An encoder can support task-specific classification
An encoder can be paired with a task-specific output layer. In an original example, a support team could train a classifier to assign incoming messages to departments.
The evaluation should measure incorrect routing, including unfamiliar requests.
Not every transformer is a chat model
Do not assume that every model called a transformer generates chat responses. Architecture, training objective, and task adaptation determine what the system is built to do.
What the architecture is for
BERT reads the whole sequence with a transformer encoder and was pretrained with a masked-token objective. The useful object is the vector, or a label from a classifier you attach.
That is why it shows up in search rankers and extractors. A decoder model, the kind that writes an answer, is a different architecture trained for a different objective.
Calling both “the AI” hides the choice.
The 2018 checkpoint is old. Later encoders, including models people casually call BERT, may be different weights with a similar shape.
If you need the paper’s result, cite Devlin and colleagues and the eval they ran. If you need today’s file, cite the file.
Neither citation rescues a task that needs generation.
How a team misuses the name
They drop an encoder into a chatbot slot and then blame prompts. Or they cite a BERT score from 2019 as if it described a current generative system.
Both are category errors. Write the input and the output type: text in, label out.
If the product needs sentences out, you picked the wrong diagram.
What to record before you rely on it
Record the checkpoint, the task head, and the fact that this is an encoder. Keep the paper for the original claim and the model card for the file you run.
Do not report a generative benchmark for it. Keep the model id or the paper citation, the date you read it, and the inputs you used beside that note, so a later change can be compared with the same test.
When the result is a failure, store the artifact and change the design. Do not relax the check until the model happens to pass.
Name a person who owns that failure.
Common questions
Is BERT obsolete?
It is old as a chatbot. Encoders remain the right tool for many ranking and classification jobs.
Obsolete depends on the job, not on whether a chat demo exists.
Can I prompt BERT the way I prompt a chat model?
You can feed it text. It will not continue the text as a decoder would, unless you are using a different model that only borrowed the name.
Check the architecture.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards





