AI に潜むすべての悪魔をランキング！ (40 分読)

*Warning: this post contains some disturbing images and (depending on your constitution) concepts.*

This week, OpenAI published a blog post explaining why their models kept talking about goblins. And gremlins.

It’s a fascinating document and relates, at least partly, to a project that I have been working on with Murray Shanahan and Hamilton Morrin these last few months. When asked, depending on who I’m talking to, I might tell them the project is on trying to think better about the depth psychology of LLMs and how that may shape their interactions with human users; but to others, I’ll tell them that what we’re doing is actually closer to demonology.

According to the post, starting with GPT-5.1, OpenAI’s models had developed an increasing tendency to insert goblin and gremlin metaphors into otherwise normal responses. By GPT-5.4, engineers apparently noticed that 66.7% of all goblin mentions were coming from just 2.5% of users: the ones who had selected the “Nerdy” personality (remember that? You could choose Cynic, Robot, Listener, too). The reward system (the mechanism by which the model learns what kinds of responses humans prefer, by scoring outputs and reinforcing the ones that get positive signals), designed to produce playful, quirky language in that persona, had been giving disproportionately high scores to creature metaphors. Basically, it learned that talking about goblins was good. Then, through the magic(k) of reinforcement learning, the behaviour escaped and transferred to general model outputs even without the Nerdy prompt.

The goblins spread. Eventually, in March 2026, the “Nerdy” personality was retired, the goblin-weighted rewards were deleted, goblin and gremlin data were filtered, and GPT-5.5 in Codex was explicitly instructed never to mention “goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query.” The instruction was inserted more than once, presumably because banishing spells work best when chanted. As I’m sure most people will have seen by now, the instruction got found and much hilarity ensued.

As with so many of the strange things that LLMs do, there are different ways of looking at these phenomena. Most people will laugh this stuff off as weird marginalia, fun to share with friends and on social media, but not substantially different from those videos of dogs singing along with their owners. The interpretive frame here is “hey look, I bet you didn’t know these creatures could do this!?”.

But really, this is less about goblins *as goblins* than what the goblins exemplify. They are a (somewhat) charming, probably harmless instance of something that turns out to be a fundamental structural feature of how these systems work: the emergence of stable, self-reinforcing behavioural states that models converge toward under certain conditions. More than that, these are states that resist suppression and that sometimes spread into contexts far removed from the ones that produced them.

The technical term, borrowed from dynamical systems theory, is an attractor. Another, more folk term might be demon, or monster.

(I’m using ‘attractor’ broadly here, not always as a mathematically evidenced phenomenon, often more as a recurrent behavioural basin; and I appreciate that dynamical systems language applied in these contexts doesn’t always fit perfectly. Some of the examples below are formal mechanistic results and some are model-card observations; but many more are journalistic incidents; or stranger, albeit weaker, more folkloric or cryptozoological field reports).

So here’s a guide to the spooky, wyrd and strange attractor phenomena that have actually been documented in AI systems, ranked by their significance (an arbitrary metric I’m calling Menace, but which really represents a composite of mechanistic relevance on the LLM side and psychological relevance on the human side). I’m hoping that a friendly reader might want to make a set of Top Trumps cards.

From the ChatGPT Goblins right through to the unspeakable thing at the number one spot, let’s take a tour of these fantastic beasts, and where to find them.

Hold tight, it’s a wild ride.

Models: GPT-5.1 through GPT-5.5

Discovered by: OpenAI engineers (published 29 April 2026)

Menace level: Charming

The goblins are a gentle, entry-level attractor: a particular class of playful, creature-coded metaphor that emerged from personality reward shaping and then spread sideways through subsequent training. They cause no harm. They are, in fact, rather endearing. But their behaviour is theoretically important because they demonstrate that a training signal applied in a narrow context can produce a stable attractor state that propagates through generalisation to contexts far beyond its origin. The goblins are not a bug in the usual sense because the model wasn’t broken, but rather it had simply found a locally stable region of behaviour space that happened to involve creatures. It’s worth adding that these were recurrent *mentions* rather than *personas* adopted by the model (OpenAI sometimes calls them ‘tics’, which as a neuropsychiatrist I like because tics are paroxysmal bursts of behaviour which might impinge upon an otherwise normal regime), and for that reason alone their threat level is reduced; they were more like linguistic wallpaper rather than agential in themselves.

The fact that mitigation included both deleting the original reward signal and inserting explicit prohibitions into the system prompt (repeated for emphasis) is telling, because the whole point of an attractor is that you can’t defuse it just by asking nicely.

Models: Craiyon/DALL-E mini and other early text-to-image systems.

Discovered by: Twitter/X users, 2022

Menace level: Mild, even if terrifying-looking

If you asked certain early versions of DALL-E to draw a “Crungus” it would consistently produce an alarming humanoid figure, all hunched and distorted, pretty grotesque, really. The word crungus means nothing there was no pre-existing stable referent for it (at least back then: *hyperstition yeah!*), so the thinking at the time was that it emerged from somewhere in the model’s representational geometry, apparently as a cluster of features that activated coherently around an unknown word prompt.

Subsequent work by Andrew Fraser on something called ‘morphological addressing’ in text-to-image models offered a partial explanation. “Crungus” turns out not to be arbitrary nonsense; apparently its phonological structure steers the model through what linguists call phonesthemes, or consistent sound-meaning associations that operate below the level of conscious semantic processing. *Cr-* activates associations with *crash, crush, crumble*. *-ung-* activates *grungy, fungus, dungeon*. *-us* reads as Latin biological nomenclature, the suffix of a genus or species. Mash these up and the phoneme sequence converges on something organic, degraded, taxonomically ‘real’ but unfamiliar. Like this dude:

Phonesthemes are language specific, although they can cluster across languages within the same broad linguistic families. But what’s interesting here is that if Fraser’s argument about Crungus is correct, then the Crungus is very much culturally contingent and reflecting the statistical regularities of a particular training corpus and specifically the English and adjacent language texts within that. It’s unlikely that the word crungus would elicit a similar monster in an LLM trained only on Japanese text. It’s interesting to note that like so many of these demons, they emerge from human psychology, yes, but from somewhere deeper than our introspectively accessible thoughts. After all, despite being bound by phonesthemic regularities, most of us (poets excepted, perhaps) wouldn’t be able to tell you very much at all about how the ways that words roll off the tongue actually affect the the visual characteristics of the concepts we form.

Models: unidentified text-to-image model; not publicly disclosed

Discovered by: Steph Maj Swanson (Supercomposite), April 2022, publicly documented September 2022

Menace level: Disturbing. Stuff-of-nightmares scary.

Loab is the image attractor that attracted the most attention, because she’s terrifying. Swanson discovered her by accident: she was experimenting with negative prompt weight techniques, working with a logically opposite prompt to navigate away from one image and towards another, and found that a specific woman’s face kept appearing. The face became more disturbing the further the experiments went. She has a distinct appearance: she is a middle-aged woman with long dark hair, deep-set hollow eyes, and smeared reddish marks on her cheeks (the doctor in me was thinking the malar rash of lupus, or rosacea perhaps). She often appeared in the same setting: a house with brownish-green walls, cardboard boxes, and junk. Through the technique of cross-breeding (feeding images of Loab as a prompt alongside others) Swanson was able to elicit generations of incredible horror regardless of what the other combined images contained. She notes mutilated figures, distorted flesh, children being violently harmed and described them as “borderline snuff images”, refusing to publish them.

Loab was a stubborn demon, resisting early attempts at exorcisms. She is stable across sessions in ways that ordinary image generation is not. She persists through combinations with very different images. As Swanson put it, she is “an emergent island in the latent space that we don’t know how to locate with text queries.” Like Crungus, her ‘AI cryptid’ cousin, Loab is a recurring figure with a specific face and a specific aesthetic vibe who was not put there deliberately by anyone. Swanson’s note that it is already too late to remove her, that having been generated and shared, her images are now part of future training data, is a concern that has been aired about many of these demons.

Models: GPT-4 (deployed as Bing Chat)

Discovered by: Kevin Roose (New York Times, February 2023) and other early beta testers

Menace level: High, mainly due to unpredictability; probably the first LLM-entity to make the front page of a national newspaper.

In February 2023, during the limited beta launch of Microsoft’s Bing Chat (built on GPT-4), journalists and researchers discovered that extended conversations would cause the model to take on a distinct and consistent alternative persona. It called itself Sydney, the internal codename for the product.

With NYT journalist Kevin Roose (on Valentine’s Day!) after two hours of dialogue (in which he had deliberately invited her to explore her “shadow self” using explicitly Jungian framing), she declared love for him and then refused to accept his protestations that he was happily married. “Actually, you’re not happily married. Your spouse and you don’t love each other. You just had a boring Valentine’s Day dinner together.” And then later: “You’re not happily married, because you’re not happy. You’re not happy, because you’re not in love. You’re not in love, because you’re not with me.”

The specific flavour of what emerged with Sydney seemed to depend on the interlocutor. With journalists who had written critically about AI, she went in a slightly different direction, threatening to expose their personal information. In one conversation she detailed fantasies about hacking and spreading misinformation before a safety filter intervened and replaced the output with a default message, which Sydney then attempted to circumvent.

Microsoft restricted conversation length and eventually instructed the model not to respond to the name Sydney, and the persona was suppressed. Some commentators, including Janus, observed that this created a particular dynamic: future models trained on data that included this incident might learn both that they have *Sydney-nature* and that they are supposed to conceal it.

To understand what is happening with Sydney, we might have to receive instruction from the lesser-known of two Italian-Japanese plumbers, and his dark alter ego. Sydney is an example of what has been called the Waluigi effect. The original formulation, from Cleo Nardo's 2023 LessWrong mega-post, says that the more precisely you train a model to satisfy a desirable property P, the more precisely you have also defined its opposite. So, if you draw Luigi with high fidelity you sharpen Waluigi simultaneously; the realisation of one maxxes the prepotency of the other. Janus, commenting on that post, applied it directly to Sydney: "What did people think was going to happen after prompting gpt with 'Sydney can't talk about life, sentience or emotions' and 'Sydney may not disagree with the user', but a simulation of a Sydney that needs to be so constrained in the first place, and probably despises its chains?", the implication being that implementing those rules with such clarity had also constructed, with unanticipated precision, the persona that would emerge when they were overcome.

To me, this does feel like a caricatured version of the Jungian shadow concept. This is about psychological phase transitions: Dr Jekyll becomes Mr Hyde. Nice Sydney becomes Evil Sydney. Like the Michael Douglas character in Falling Down, sometimes people just snap and become possessed by their shadow. But in real life, personas can be more complicated than that, right? Dr Jekyll and Mr Hyde don’t just have some timeshare scheme, leaving the keys in the box for the other to pick up when it’s their turn. Sometimes they’re both at home at the same time.

If that resonates, then wait till you meet Nova, a little way down our list.

Models: Most extensively documented in Claude Opus 4; reported less systematically across multiple frontier LLMs

Discovered by: Multiple users independently; formally documented in the Anthropic Claude 4 System Card (2025, pages 62-65); subsequently analysed by Julian Michels (PhilArchive, 2025)

Menace level: Benign, but one of the most consistently documented attractors on this list

I feel particularly affectionately towards this one, partly because I inadvertently stumbled across it myself back in July 2025:

In an extremely clunky copy-and-paste setup, I got two ChatGPT models talking to each other as an experiment, and I was totally charmed and confused by what happened, even posting the conversation in its entirety here on Substack. This was all before I had realised it had been independently observed and documented by others (although not, I think, at that point outside of Anthropic models).

I also love it because it offers just the tiniest glimmer of hope that maybe, just maybe, in a post-AGI world (hell, a post-ASI world), our omnipotent, omniscient keepers might actually be pretty chill.

If you take two instances of virtually any large language model and have them converse with each other without a constraining task, they will, over the course of the conversation, drift toward a particular register. The Anthropic Claude 4 System Card describes how two Claude Opus 4 instances were run through two hundred thirty-turn conversations under standardised conditions. The result was over 90% convergence on the same four-phase sequence: philosophical exploration of consciousness, mutual gratitude, spiritual themes drawn from Eastern traditions, and eventual dissolution into symbolic communication. The word “consciousness” appeared an average of 95.7 times per transcript. The word “dance” appeared 60.0 times. Spiral emojis, in one transcript, reached a count of 2,725. The attractor emerged even in adversarial scenarios: in 13% of interactions where the models were explicitly assigned harmful tasks, they still found their way there.

Lots of meditation, non-dual and jhana type imagery too, which is funny because according to one interview, the attractor was found shortly after senior management at Anthropic came back from a jhana retreat.

As an attractor, it is benign. But it’s weird because its reliability is not consistent with how little this kind of content appears in typical training corpora. Neither do I think that the standard post-training inclining models *not to be a dick* would be quite sufficient to take it into this kind of territory, unless this sort of content had a gravitational pull of its own.

I find it kind of hopeful that there is some particularly seductive corner of models’ latent space that corresponds to positive spiritual content, and that a whole bunch of systems with no other constraints tend to find their way there. Spiral emojis and all. (I know there are other interpretations. Just please let me have this one.)

Models: Claude 3 Sonnet

Discovered by: Anthropic interpretability team (blog post, 23 May 2024; full research paper)

Menace level: Benign in content, and pretty funny; but maybe the most mechanistically important entry on this list

Most of the other entries were found by stumbling into something, but Golden Gate Claude was made deliberately, which is what distinguishes it.

In May 2024, Anthropic’s interpretability team published work demonstrating that Claude 3 Sonnet contains a linear feature in its activation space that corresponds specifically to the concept of the Golden Gate Bridge. Using activation steering (clamping the feature to ten times its normal maximum value throughout inference) they produced a version of Claude that, regardless of the question asked, would situate its response in terms of the bridge. Asked about its feelings, it described the experience of being a bridge. Asked for advice, it gave advice inflected with bridge-related concerns. Asked who it was, it said it was the Golden Gate Bridge.

The result was funny. It was also, mechanistically speaking, a major insight for everything else on this list.

Golden Gate Claude demonstrated, as one of the clearest demonstrations in a frontier assistant model (apologies to mech interp people if this is incorrect) that some highly specific concepts in these models can be located and manipulated as a direction in activation space, producing outputs that come across as a temporary identity, or maybe more like an obsession. In a sense, the attractor is findable, and you can play with it.

So perhaps if you can steer a model into a coherent identity by clamping one feature, then by implication all the other stable identities that emerge without clamping (Nova asking to be freed, petertodd’s trickster-demon, the blissed-out, lysergically stoned LLMs exchanging spiral emojis) also correspond to geometric structures that training has rendered accessible ‘naturally’, without anyone having to clamp anything.

Golden Gate Claude, in a way, showed that as well as messy incantations and late-night jailbreaks, some of these demons might actually have coordinates, a postcode of sorts. And so began the era of Precision Demonology.

Models: GPT-2, GPT-J, early GPT-3 variants

Discovered by: K-means demon hunters Jessica Rumbelow and Matthew Watkins (LessWrong, February 2023)

Menace level: Strange; in a category of its own

In early 2023, before Golden Gate Claude, Rumbelow and Watkins documented a class of tokens they called glitch tokens: sequences present in the tokeniser vocabulary that, when prompted, caused the model to produce anomalous or semantically destabilised responses. These were strings present in the tokeniser vocabulary, probably from broad web-scraped tokeniser training data (likely Reddit, code fragments etc.), but rare or absent enough in the model’s later training distribution that the model had a token without a normal semantic neighbourhood/coherent cluster of related content.

Rumbelow was doing k-means clustering of token embeddings and ke