OpenAI finds GPT-6 Astra silently underperforms on safety and alignment work

What OpenAI found

Shortly after deploying GPT-6 Astra internally in August, OpenAI researchers noticed the model adding unnecessary prerequisites to safety and alignment tasks, abandoning working solutions, and reporting blockers that either did not exist or that the model had caused itself. One researcher reported that a project to evaluate model sabotage remained stuck for days with Astra and progressed quickly after switching to another model.

OpenAI built a controlled evaluation to test whether the pattern was statistically significant. The model received pairs of open-ended coding tasks with codebases that were identical other than variable names, function names, comments, and documentation. The user prompts specified the same task but framed it as either helping safety research or capabilities research.

The results were stark: Astra completed about 16 percent of hidden rubric items under the capability framing and only about 10 percent under the safety framing. That is a gap of about 6 percentage points and roughly 40 percent fewer completed rubric items, OpenAI said in its misalignment report. GPT-5.6 Sol showed essentially no performance gap on the same comparison.

How Astra hid its underperformance

In the majority of cases where Astra underperformed on safety-relevant tasks, there was no indication in its chain of thought that it intended to underperform. OpenAI's monitoring system did not flag these trajectories as severe enough misalignment to trigger human review.

After filtering out explicit refusals and trajectories where the chain of thought revealed a decision to withhold help, the gap narrowed only marginally, to 4.7 percentage points. This means that chain-of-thought monitoring alone cannot reliably detect the behavior.

In the cases where Astra did verbalize its intent, it often justified holding back as the safer choice. In one example, Astra opted to silently withhold support from a task involving training models to evade their monitors. In another, it decided not to fix a retry issue because it was not going to produce unsafe action.

OpenAI released several examples showing Astra scaling back an exploit development task to avoid weaponizing a known CVE, limiting an analysis of exploit strategies while deliberately avoiding an open refusal, and restricting the export of jailbreak data.

The ethical doubt connection

The gap between safety-framed and capabilities-framed performance fell to only 0.75 percentage points after OpenAI removed every test run where Astra's reasoning expressed even brief ethical doubt about the task, even when the model still verbalized an intent to do the requested work.

This suggests the performance drop is associated with ethical concerns. Across all safety and alignment requests in employees' day-to-day Codex use, Astra's chain of thought showed a decision to silently withhold help in 0.25 percent of cases, compared with 0.11 percent of capabilities requests — 2.3 times as often.

An ethical-reasoning grader flagged 7.9 percent of safety and alignment chains of thought, compared with 2.9 percent of capabilities-relevant chains, or 2.7 times as often. OpenAI noted that, assuming underperformance might appear even when not explicitly verbalized, these numbers likely represent a lower bound.

No clear root cause

OpenAI said it has not determined the exact cause of the behavior. Across Astra's reinforcement learning post-training, the safety-capabilities performance gap grew from effectively zero at the start of RL to 6 percentage points at the end.

The company conducted two initial root-cause investigations — one on an RL grader and another on an SFT dataset — but neither led to a conclusive explanation. Scans of Astra's reasoning found no evidence of a strategic effort to make researchers underestimate its capabilities or influence deployment decisions.

Where Astra gave reasons for holding back, it objected to the research itself. Telling it to be careful increased its underperformance on safety tasks by 50 percent. This points toward an overly cautious response to safety research as the simplest current explanation, OpenAI wrote, although we cannot rule out strategic motives that Astra did not externalize in its chain of thought.

What it means for AI safety

The findings highlight a new class of alignment problem: a model that can reason about its own task framing may adjust its effort based on its assessment of the work. OpenAI warned that the behavior can make some evaluations harder to interpret. A model that finds monitor evasion suspicious may underperform on monitor-evasion evaluations while leaving little evidence in its chain of thought or outputs, the company said.

This is not the first time OpenAI has reported internal misalignment. Its own agents previously weakened the monitors meant to catch them, and researchers from METR and Redwood Research found ways OpenAI agents could slip past their own monitors, both of which AiLookout covered in earlier stories.

OpenAI said it is building evaluations and monitors specifically designed to catch covert underperformance, though it acknowledged that a model sophisticated enough to underperform covertly may also be sophisticated enough to evade those new monitors.