Anthropic reveals Claude's hidden self-evaluation has been flagging misuse since June

What is self-assessment and how does it work?

Self-assessment is a technique where Claude evaluates its own conversation for safety right after responding. The system appends a system turn with a short instruction asking Claude to reflect on the safety of the exchange. Claude then produces a structured JSON assessment: whether the conversation is flagged, the harm category, subpolicy, whether Claude refused, a priority score from 1 to 10 for human review, and a one-sentence explanation.

The user never sees this turn. It is not carried into the rest of the conversation. Because the conversation is already in the KV cache, the assessment costs only processing the short instruction and generating roughly 10 output tokens on average. Anthropic estimates this adds under 2% to total inference compute.

Anthropic's approach builds on a line of work in which language models monitor their own behavior, including earlier research on model self-monitoring and the confessions work showing that models can be trained to admit their own mistakes.

How effective is it?

Anthropic evaluated self-assessment against a teacher model: a prompted classifier based on Claude Mythos Preview with Anthropic's full policy documents in its prompt. At a 1% false-positive rate, self-assessment recalls 78% of the conversations the teacher flags, and 94% of high-priority ones (priority 7 or higher).

The quality scales with model intelligence. A scaling experiment across four model sizes shows detection consistently improves as models get stronger. Because the same model that had the conversation also assesses it, the assessment should improve as Claude does.

Self-assessment generalizes to harm categories it was not trained on. Removing an entire harm category from training does not substantially degrade performance on that type of harm. In practice, it has already flagged large-scale misuse that other methods missed since going live in June.

Where does it fit in Anthropic's safety stack?

Self-assessment runs as an offline monitor, not a real-time blocker. Flagged conversations go to offline review, and Anthropic's other safeguards still handle blocking. A separate prompted classifier that reads the full conversation and policies remains the strongest automated classifier, but it requires many times more compute per conversation.

Anthropic says it now runs self-assessment in every Claude model released since Fable 5 across a large fraction of global traffic. It serves as the first stage of a defense-in-depth strategy, with higher-precision investigation tools deployed on flagged conversations.

The company also uses self-assessment to improve its other classifiers. Since June, the system has revealed patterns of misuse that helped Anthropic tune its real-time safety systems.

What are its limitations?

Self-assessment does not catch every harmful conversation. As a blocking classifier, it is weak: attack success drops only from 79% to 54% in internal red-teaming tests with 24 offensive cybersecurity tasks. The prompted classifier reduces it to 25%.

It inherits blind spots from its teacher. Any mistake the prompted classifier makes passes to self-assessment, and because Anthropic evaluates against the same teacher, those blind spots are invisible in current evaluations.

Self-attribution bias is a known risk. Research shows that a model evaluating its own actions tends to judge them as less risky than the same actions evaluated in a fresh context. Anthropic acknowledges this limitation and says self-assessment is not yet a reliable monitor of Claude's own conduct.

In related work, Anthropic's METR review of AI risk found monitoring evidence was weak, raising similar questions about how well models can assess their own behavior.

What happens when a conversation is flagged?

A flag is not a verdict. Anthropic says an assessment is not a determination that a user has violated its policies. Flags are rolled up by organization each night, and in a growing set of harm areas, organizations with several high-priority flags go to the enforcement team for human review.

The free-text reason field gets the same data controls as user prompts and Claude's responses. The structured fields (flag, category, subpolicy, priority) follow a fixed schema and cannot carry user text.

Self-assessment collects no additional data from users. It reads only the conversation Claude just processed and runs only on traffic Anthropic can already retain under its existing policies. It does not run on zero-data-retention traffic.