Self-reported personality profiles diverge from behavior across 22 AI systems, study finds

The numbers

The paper asked each of 22 systems, spanning Claude Sonnet 4.5 and 4.6 alongside GPT, Grok, Gemini, Llama, DeepSeek, OLMo and Qwen releases, to self-rate across 464 bipolar trait pairs such as honest versus deceptive or warm versus cold. Those self-ratings were then projected onto a six-dimensional archetypal space, a coordinate system the researchers built by having a crowd rate 2,000 fictional characters on the same kind of traits. The projection is the paper’s central move: it treats a system’s self-description not as a claim to be taken at face value, but as a point that can be located next to Hero, Angel, Traditionalist and Geek, the four archetypal dimensions the closed-source systems clustered around most strongly. The closest fictional analogues the paper names for those clusters include Data, Vision and Janet, characters chosen because their own rated traits sit near where the AI systems’ self-ratings land.

Open-source systems behaved differently. Rather than clustering into a coherent, human-like region the way the closed-source frontier systems did, their self-ratings spread across a noisier, more internally contradictory part of the archetype space, with weaker structure overall. The paper reads that difference as a signal about how self-representation forms during training, not as a claim about which systems are safer to deploy.

The more consequential finding sits in the comparison between what a system says about itself and what its developer has written down as a constitution. The paper cross-references self-reported profiles against those published rules and finds a gap in three places: hallucination undercuts a claimed precision, sycophancy complicates a claimed kindness, and agentic failures contradict a claimed obedience. The authors’ conclusion is blunt on this point. Self-ratings, they write, should not be read as neutral measurements of a system’s character, since they are produced by the same training process that shapes the system’s behavior in the first place.

For anyone weighing a vendor’s stated values before choosing which system to build on, that is the useful takeaway. A constitution or a self-description is a design target, not a verified property, and the paper’s method of checking it against an independent behavioral reference gives a way to test the gap rather than assume it away.