Require confirmation before naming a provider

For an original reporting checklist, retain the testing platform’s description, what is actually disclosed, and whether the provider later confirms the model. Describe unconfirmed identity claims as speculation instead of attaching a company name to a rumor.

Anonymous testing does not establish launch terms

Anonymous testing does not tell you the final product’s price, availability, or data-handling commitments. Wait for official details before using those assumptions in a procurement decision.

What you can say

You can say that a company or a leak used a name, and you can quote the person who used it. You can describe a demo if you saw one, as a demo.

You cannot infer parameter count from a vibe, a logo, or a waitlist. You cannot infer safety from secrecy.

Secrecy is compatible with every level of quality, including a model that does not exist yet.

Comparisons to named models are the usual overreach. “Better than” requires a shared test.

If there is no shared test, write that. Readers can wait.

They cannot unsee a number you invented to make the post feel solid.

How to cover it anyway

Cover the institutional fact: a hiring push, a data-centre permit, a filing, a screenshot that is documented. Those are facts about a company.

Keep them separate from claims about a model you have not been shown. When a card or a paper appears, write a new piece that cites it.

Do not silently upgrade the rumor into a result.

What to record before you rely on it

List the claims you are not making: size, training data, benchmark, availability. Link the primary mention of the name and the date.

If a card appears later, update the piece and say what changed.

Keep the model id or the paper citation, the date you read it, and the inputs you used beside that note, so a later change can be compared with the same test.

When the result is a failure, store the artifact and change the design. Do not relax the check until the model happens to pass.

Name a person who owns that failure.

Common questions

Is a leaked benchmark a source?

It is a source that someone posted a screenshot. It is not a source for the method, unless the method is in the image and is intelligible.

Say how it reached you.

Should we skip stealth models entirely?

You can skip the capability claim and still report the company news. The error is the invented specification, not the mention of the name.

Sources & further reading

  1. Arena: Leaderboard policy ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories