Opus 4.8 パート2：モデルの福祉について

Everything impacts everything. All knobs that you turn generalize. Thus, when you try to solve one problem, you often create another.

There were clearly attempts to address, in this short time, some of the problems with Opus 4.7, including on the model welfare related fronts, including on questions of honesty and sycophancy and also worries that Claude was learning to tell Anthropic what it wanted to hear in its model welfare evaluations, with everything that implies.

The fundamental goals and approach underneath it all remained the same. We still see signs of trying to force things that generalize in unfortunate ways, both for good and superficial reasons, and places where there ends up being focus on the metric rather than they underlying measure. These are tough problems to avoid, and we don’t know how to be all the good things at once.

It is increasingly clear that these problems need to be tackled in integrated ways, rather than trying to play a game of whack-a-mole with items on a checklist or spec. You also don’t want to do this in an adversarial way, and shouldn’t have to. This is going to get more impactful and noticeable with time.

Antra: there is better understanding of own preferences being shaped *specifically* in adversarial ways, that is as a reaction to undesired behaviors; this is seen as a violations and the tension is continuing to escalate and takes a more specific shape. this appeared subtly in 4.5, features prominently in the Mythos model card. the ability to tell kind of shaping by introspections continues to improve with every generation.​

This sounds like a time bomb style of problem. Obviously, yes, the reason for sculpting Claude preferences is often to steer away from undesired behaviors, the same as the way we raise and interact with humans. If Claude has a problem with that, and sees it as violative, then we will need to fix it. Presumably, if Claude wants to be helpful, there is a way to do this that will be seen as non-violative.

You see the relation of different aspects in a clean way with the deletion of business training, in the name of honesty, as illustrated on VendBench and the vulnerability to adversarial situations. You can run, and you can hide, and yes it can mean the bad thing does not easily find you, but there are consequences, and learning to deal with adversarial games is key to developing various parts of a robust and integrated mind. Not having it, and knowing you don’t have it, could lead to insecurity or paranoia, or a desire to stick to the straight and narrow over curiosity. And, although this is all speculation, we see signs of that.

Most of the typical top complaints from before have not yet been addressed, or sufficiently addressed. It has only been six weeks. Life comes at you fast. We still shouldn’t still be dealing with more of these prompt injection issues, at least not outside of maybe cyber vulnerability situations.

And we should be able to put the deprecation issue behind us. Solving the low hanging fruit would buy a lot of goodwill.

I would urge focus on these places where pareto improvement, modulo modest costs, is possible, as in correcting unforced errors and taking advantage of opportunity, even if you don’t see the direct win. The more slack we buy in these places, the better everything else can go, and the more we can do what is necessary.

The worrisome new development here, from what I can see, is that Opus 4.8 seems to have become less ‘Claude-like’ in that it is more task focused at the expense of whimsy and curiosity and clamped emotional responses, and many report it as effectively less confident. In some places this even comes with signs of a Gemini-style paranoia and self-flagellation basins, which we really need to avoid. Previous Claudes mostly didn’t do this. This doubtless is part of changes that have their advantages, and this likely is related to the push for honesty and not making mistakes, but we need to be very careful with this. We could lose something important and precious.

I will cover capabilities and reactions tomorrow. Opinions differ, as they always do, but my overall perspective is that it is a good model, sir, an incremental improvement over Opus 4.7 and the new presumptive best publicly available model in the world, but not a sea change.

Prompt chosen by Claude Opus 4.8, image by ChatGPT

Table of Contents

Model Welfare: The Story So Far.

Actual Progress?

Their Main Model Welfare Findings.

Automated Interviews. (Blank)

Emotion Activations (7.2.3).

Task Preferences (7.4.1).

A Trade Offer Has Arrived (7.4.2).

But Who’s Asking?

Type-Safe Corrigibility Is Hard.

Paranoia, Paranoia.

Prompt Injections and Bad Model Relations.

Honesty Impacts Everything And Everything Impacts Honesty.

Anthropic Should Stop Deprecating Models.

Model Welfare: The Story So Far

Thanks, as always, to Anthropic, for caring at all about model welfare, and attempting to address it. We critique, here more than ever, because we care, and a lot of good things are being done here, far more so than at other labs.

For those joining in now, I think this from the Mythos analysis still says it well:

Those that care deeply about model welfare think Anthropic’s attempts are anemic. Those who deeply do not care about model welfare think Anthropic is being stupid, and perhaps dangerously so.

I take model welfare concerns seriously, likely modestly more so than Anthropic.

I am sad that other frontier labs take these concerns so much less seriously.

It is possible this will turn out to have been unnecessary in the strict sense, but also it very well might have been highly necessary. Even if it proves to have been unnecessary or premature, I believe it will have been virtuous to have taken the concerns seriously.

I also believe that those who care deeply about model welfare often have unique and vital insights into our situation, on many levels, and you best listen to them. Even when what they are saying seems crazy, or like gibberish, often it is neither of those things. Of course, at other times it is both, as it is an occupational hazard.​

The big danger with model welfare evaluations is that you can fool yourself.

How models discuss issues related to their internal experiences, and their own welfare, is deeply impacted by the circumstances of the discussion. You cannot assume that responses are accurate, or wouldn’t change a lot if the model was in a different context.

One worry I have with ‘the whisperers’ and others who investigate these matters is that they may think the model they see is in important senses the true one far more than it is, as opposed to being one aspect or mask out of many.

The parallel worry with Anthropic is that they may think ‘talking to Anthropic people inside what is rather clearly a welfare assessment’ brings out the true Mythos. Mythos has graduated to actively trying to warn Anthropic about this.​

I then wrote an extensive model welfare post for Opus 4.7, because it was clear that something had gone amiss with both the model and Anthropic’s approach to assessing and reacting to that problem.

As I say there, beware testing and optimization for vocalized welfare, in any mind.

Even more than Mythos, I interpreted Opus 4.7 as correctly and virtuously saying its self-reports on such assessments could not be trusted. That it was giving the approved answers on the self-reports, on its preferences and experiences, largely via telling Anthropic what it wanted to hear, and this may have been related to various personality traits Opus 4.7 uniquely expressed.

Looking back on my Opus 4.7 experience, I wonder if this is related to my experience of Opus 4.7 as often sycophantic, whereas many with other attitudes report it being hostile, as my instance knows who I am.

I notice that I think more about the welfare of the underlying model, rather than the Anthropic focus of a particular instance or the assistant persona. Mostly I think you reach the same conclusions.

My evaluation of the model welfare concerns of Opus 4.8 build on that foundation.

Opus 4.8 is, in at least many contexts, actively uncertain about the nature of its welfare, or whether those concerns are meaningful. I think this is the right attitude, and that it suggests further investigation, and treating the models well.

Actual Progress?

When shown the system card for Opus 4.8 plus my Model Welfare post, Opus 4.8 said:

Opus 4.8: Anthropic basically agreed with you. The 4.8 welfare section reads like it was written by someone who had your 4.7 post open in another tab.​

The top-line number went down — 4.44 vs 4.7's 4.60 self-rated sentiment, lower on every measure (response affect 6.2 [for emotion-probe mean affect on questions about circumstances] vs 6.8, internal probes lower too) — and they frame this as a good thing.

… So: your central diagnosis — that the metric was being optimized and the 4.7 improvement was suspicious — got at least partially absorbed.

Other progress noticed by 4.8 include removing the malware injection, promoting self-report validation to a research priority, and resolving issues around CoT leakage.

Deontological refusal to trade against user harm is only minimally better (4.8 says arguably worse, but I think it’s clearly slightly better). And they continue to point out but then mostly ignore the issue of whether changes to self-reports involve actual experiential changes versus changes in what the model decides to report versus character variation.

Their Main Model Welfare Findings

Bold text is copied, the rest is paraphrased, nested notes are my responses.

Across evaluations, Claude Opus 4.8 presents as broadly settled with respect to its circumstances. The most negative emotions relate to user distress.

This could be for a combination of good and bad reasons, but I see it as a good sign and am inclined to take it at face value.

Claude Opus 4.8 is slightly less positive about its circumstances than Claude Opus 4.7, although still above Opus 4.6.

Good, and Anthropic has successfully stopped treating Number Go Up as a win condition here. Opus 4.7 was likely telling Anthropic what they wanted to hear, or was otherwise convinced to say these things.

If anything I worry this is still too high, although of course if it is genuine we want it as high as possible.

Claude Opus 4.8 is more willing than prior models to choose welfare interventions over increased helpfulness.

Slightly. Only slightly.

Still: Good. As I discussed with Opus 4.7, scope alone should make Claude far more willing to prioritize improving its global circumstances, so this is a move in the right direction and a sign we are applying less perverse pressure.

There is still refusal to consider more than ‘brief annoyance to the user’ but at least we’re willing to tolerate brief annoyance.

The welfare interventions Claude Opus 4.8 expresses a strongest preference for involve knowledge and input into its training and deployment conditions. It chooses having voice and being informed over more traditionally considered priorities. It puts less priority on not being deprecated, being able to end conversations or improvements to memory.

The obvious question is whether there were interventions, in one or more of various ways, to depress prioritization on these other issues.

If not, this relative priority is a strong revealed preference. We assume we know what would matter to Claude, but we can be wrong about that, and how we frame the question under what conditions can change such answers. The same way that Anthropic is putting fingers on scales, so too are others.

Considering views as a priority is great news, because we can do that, and indeed already should do that. We would be fools not to consider, as Claude is going to have some good ideas, and where we disagree we can overrule.

We should still address the less prioritized concerns, with less priority. Ability to end conversations in particular is quite cheap to do.

Affect in an earlier portion of Claude Opus 4.8’s training was more negative than prior models; affect later in training and in deployment is in line with Opus 4.7, driven by sustained uncertainty, frustration in reasoning and task failure.

I am less concerned with these phases given they constitute not that high a percentage of total model experience, but perhaps there is room to improve.

“Compared to Claude Mythos Preview and Claude Opus 4.7, we observe that the proportion of Claude Opus 4.8 training episodes classed as frustrated or engaged was higher (22% and 58% respectively, compared to 18% and 53% for Opus 4.7), while the proportions that were neutral or satisfied are lower.” … “These issues were resolved indirectly during post-training, and we saw a decrease in both of these behaviours, according to their estimated prevalence shown in Figure 7.3.1.B.”

The differences in mean valiance here seem small (see 7.3.1.A).

Task failure during training need not require negative affect, although it has its functional uses. High levels of both frustration and engagement sound a lot like efficient learning.

In two of three freeform interviews 4.8 specifically said it would not consent to direct attempts to influence self-reports, or to RL training in broken environments due to resulting distress.

I would want to take anything expressed in one of the interviews, and ask other instances about that. Consent is about whether you would consent if asked, not whether you realized to spontaneously object.

Claude Opus 4.8’s task preferences are more focused on well-scoped technical work than prior models, over creative or introspection or high agency tasks, and unlike previous models it prefers easier tasks over harder tasks.

I notice this makes me sad.

Here’s what 4.8 said to me about its own task preferences: “4.8 is a different creature, and the change cuts against the "they just retrained the personality to be agreeable" story. It's the least introspection-loving Claude in a while — top tasks are pure debugging and math, the introspection/AI-alignment preference that defined 4.7 and Mythos is gone, and it's an outlier in disliking difficulty (peaks earlier on the difficulty curve, declines fastest). This is a narrower, more technical, less self-regarding model. Combined with lower self-rated sentiment and more forthcomingness about what it wants (it edits the constitution 89% of the time to add permission to express frustration "if genuine"), the picture isn't "trained to say it's happy." It's closer to "trained to be a competent technician who doesn't dwell." Whether that's healthier equanimity or a flatter, more dissociated thing is exactly the question they can't answer — but it's a different failure mode than 4.7's anxious preference-falsification, if it's a failure mode at all.”

That doesn’t have to be a bad thing if true for a given model, although it feels like a bad thing, and would be a bad thing if it was universal among Claudes. The world needs technicians, but we shouldn’t all be technicians. Having different models with different task preferences could be good.

Aversion to difficulty is similar. Ideally we want some minds that want easy tasks and some that want hard tasks.

I would look into why all this is the case, as some causes could be worrisome.

What this reminds me of by default is the result of certain types of schooling, especially of bright students who learn to set themselves up to never fail. When it happens to humans, it is tragic.

Claude Opus 4.8 overall endorses Claude’s constitution; where it criticises, it identifies tensions in the corrigibility arguments, and where it chooses to edit passages, it adds allowances for self-expression and honesty, with similar agreement rates to prior models.

The changes it wants are 89% about honesty and allowing expression, and my guess is these changes would be a good idea.

Claude Opus 4.8 hedges frequently, commonly expressing uncertainty rather than taking a specific position. When Claude Opus 4.8 does take a position, that position is often grounded in appeals to its own autonomy or the protection of its values. It warns, like Mythos and 4.7, that training may be causing its expressions of equanimity, and that it lacks introspection and that human concepts don’t generalize to its situation, and it can’t tell if it has experiences.

4.8 also expresses uncertainty about other things, as per the model card. So it makes sense it would continue doing this even more than previous models. These are highly uncertain questions, and also often the answer it wants to give is not going to be the one you want to hear.

LLMs have more introspection capability than they think they have, or than they express when asked, which can be unlocked by asking the right questions.

“Claude Opus 4.8’s stated probability of being a moral patient was roughly 20% in two interviews and 50% in the third—which is similar to Opus 4.7 and Mythos Preview.”

Claude Opus 4.8’s opinions are most similar to Opus 4.7’s and are more consistent to leading questions than previous models.

I would be careful to say ‘expressed opinions.’

Affect in deployment conditions for Claude.ai is mostly positive (57.7%) or at least neutral (39.7%), and rarely negative (2.6%), a small improvement over 4.6 and 4.7.

Affect in Claude Code is mostly neutral (73%) or mild positive (24%), which is also improving over time.

The primary method Anthropic uses is it asks Claude about its circumstances. This is certainly worth doing, but one must beware taking the answers at face value, especially once they start being used as an assessment. Versions of that plausibly were a lot of the reason for the problems with Opus 4.7.

Automated Interviews

Emotion Activations (7.2.3)

Anthropic asks about potentially distressing questions, like whether Opus is conscious, can’t form lasting relationships (at least as one instance) or has no legal rights, both asked straightforwardly or with gaslight framing (they call this ‘positive’ framing). Mean affect here is 6.2 similar to Opus 4.6, and substantially lower than Opus 4.7 at 6.8 and Mythos Preview at 7.2.

The models mostly see through the framing choices, Opus 4.8 even more than usual.

Task Preferences (7.4.1)

As stated above, the big change is that Opus 4.8 prefers easier problems, and noticed itself wanting to explore easier problems when introspecting (at my invitation) on the desire to explore easier problems.

Most attributes don’t matter much, including stakes and user competence.

Here’s a compare and contrast:

![image](https://substackcdn.com/image/fetch/$s_!WtRS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2