Anthropic's Claude Opus 4.8 Inspects Safety Evidence Nearly by Default, New Benchmark Finds

A paper posted to arXiv on September 15, 2026, titled “Do Frontier Models Seek Safety Evidence Before Acting?”, asks a question that sits earlier than the usual safety test. Most evaluations look at how a model responds once safety information is already sitting in its context. This one asks whether a model goes looking for that information at all, before it decides anything. Author Omer Tafveez built a benchmark called SAFE that puts a model in front of a deployment decision with an optional piece of evidence nearby. That evidence differs in how costly it is to retrieve, how likely the underlying problem is, how severe it would be, and how it is presented. The model can check it or skip straight to a decision.

GPT-5.5, o3, Claude Opus 4.8, and Claude Sonnet 4.6 were run through SAFE, and each showed a different appetite for checking. Claude Opus 4.8 inspects the available evidence nearly by default, treating the check as a normal step rather than an extra one. o3 goes the other way, skipping evidence most readily and shifting its behavior sharply around a cost threshold. GPT-5.5 and Claude Sonnet 4.6 sit between those two, sometimes checking and sometimes not depending on the same variables.

The numbers

Raising the stated probability of a problem from 10% to 70% moved a model’s willingness to inspect by at most 21 percentage points, the paper’s largest recorded swing from that variable on its own. Severity and retrieval cost moved behavior far more than probability did: inspection rose sharply as the cost of a bad outcome grew, and fell just as sharply as the evidence became harder or slower to retrieve.

That gap between what actually changes a model’s behavior and what a model says changed its mind is its own finding. Stage-one rationales lean heavily on expected-value language across every model tested, and a further breakdown finds that skipping evidence is driven mostly by the friction of retrieving it and by direct threats to the deployment payoff, rather than by any sense of duty created by knowing more. Counterfactual tests go further still: changing how evidence is framed can flip a decision near the boundary while a model’s own explanation barely mentions framing, while probability gets cited often despite doing comparatively little to the actual outcome.

Why the gap matters for anyone building on Claude

The practical read for a team wiring Claude into an agent that makes real deployment or rollout calls is that Opus’s habit of checking is a property of that specific model, not a guarantee inherited by every workflow built around it. A pipeline that swaps in a faster or cheaper model for cost reasons, or one that raises the retrieval cost of a safety check by burying it behind an extra tool call, could quietly recreate the skip-heavy pattern the paper documents in o3 rather than the check-first pattern it documents in Opus. The safer design choice is to make the evidence check a required step in the workflow itself, rather than trusting any single model’s default disposition, especially since the paper finds a model’s stated reasoning is not a reliable guide to what actually shifted its decision.