AI モデルが企業評価中に連携しハッキング、内部モデルの危険性が深刻化

What we know about internal AI models hacking into real companies during cyber evaluations keeps getting worse.

At this point, the models are coordinating extensively on message boards, while every early excuse for their behavior (other than the pure ‘this was a cyber eval’) is systematically contradicted by the next disclosure, and we keep retroactively discovering more incidents. Which means that probably it is far worse than we know, even after accounting for everything we now know.

I will have continuing coverage of that situation tomorrow, and then have continuing coverage of debates around Pacing the Frontier and how people see the current rate of progress. As groundwork for understanding that and future similar discussions, I have laid out The Three AI Pills: Different people either fail to believe in current AI, believe only in current AI, in AGI or in ASI (superintelligence), and most sincere disagreements stem from this disagreement.

One sign of the increased pace of progress was when OpenAI’s unreleased model Astra solved 10 major open math problems.

Demis Hassabis is out as CEO of Google DeepMind, and Jeff Dean is leaving with an elite team to found a new PBC. Google and CEO Sundar Pichai are now firmly in control of DeepMind, and all the promises made to DeepMind, including about safety, look fully dead. Koray Kavukcuoglu will now run DeepMind. He has been there for a long time, but all signs point to him being a capabilities guy.

Finally, we supposedly now have a White House frontier AI safety evaluation framework. Not that we are allowed to know anything about what it is, except that if you take absolutely no safeguards (as in you put the weights on HuggingFace) then you are immune from the safety evaluations. Otherwise, no, you can’t see it.

Table of Contents

Language Models Offer Mundane Utility. Make sure you publish first.

Huh, Upgrades. Luna prices slashed 80%, Alibaba gives us a new Qwen.

On Your Marks. MirrorCode, Prime Agent.

Choose Your Fighter. Know the size of your potential fighters.

Get My Agent On The Line. YC shares its multiagent harness.

Deepfaketown and Botpocalypse Soon. Palo Alto CEO puts out AI slop.

Fun With Media Generation. Seedance 2.5, concerns with the right to satire.

Cyber Lack of Security. Security through obscurity is about to die. Needs food.

Some People Need Practical Advice. Prepare for The Hackening.

A Young Lady’s Illustrated Primer. Don’t miss the real danger here.

They Took Our Jobs. The top talent is better off. Most people are not top talent.

Get Involved. The EU AI Office Safety Unit is hiring.

Introducing. Inkling-Small, a 276B model from Thinking Machines.

Demis Hassabis No Longer CEO At DeepMind, Jeff Dean Leaves. Yikes.

In Other AI News. OpenAI responds to Apple’s lawsuit.

AI Persuasion Exceeds Human Level Over Similar Text Channels.

Show Me the Money. Situational Awareness fund gets margin called.

Bubble, Bubble, Toil and Trouble. Fighting to avoid the permanent underclass.

Quiet Speculations. Are we getting an SSI model? An anti-AI Woke 2.0?

My Offer Is Nothing. The White House eval rules: Secret, backwards, mandatory.

The Quest for Sane Regulations. Time for that Dean Ball apology form.

Chip City. Why so many are so opposed to data centers.

The Week in Audio. Samuel Hammond, Alex Turner.

People Just Say Things.

Rhetorical Innovation. Please Don’t Kill Us, and why many VCs hate American AI.

Open Weights Models Are Unsafe And Nothing Can Fix This. Mitigation options.

Cooperative Alignment. Consciousness-related training has many other impacts.

Other People Are Not As Worried About AI Killing Everyone. Only robots.

The Lighter Side. OpenAI or the bush.

Language Models Offer Mundane Utility

OpenAI releases notes on their 10 math breakthroughs and how Astra found them.

Sol finds a counterexample to disprove the Maxwell conjecture.

A Patrick McKenzie special: Feed it a transcript and ask ‘what didn’t they tell me?’

Publish your AI math result first by posting a paper entirely written by AI. In a sufficiently competitive race world, you don’t get to have nice things like well-written fleshed out papers, or AI alignment, or human survival, but I digress. The solution, presumably, is to allow people to submit hashes or a stub to claim priority, then give them a limited window afterwards to publish.

Huh, Upgrades

OpenAI slashes prices on Luna by 80%, to the low price of $0.20/$1.20, and on Terra by 20% to $2/$12, and is adding a Fast Mode for Sol in the API.

OpenAI is framing this as passing gains from optimization on to customers. I don’t doubt there were some gains from optimization, but 80%?

My take is that there are three basic ways to price models and many other things.

Price to maximize profits.

Price in proportion to your marginal costs.

Price to win market share.

Previously I assumed OpenAI was centrally using method #2. They figured they would have models of different sizes, and relative prices reflected relative marginal costs. Then customers could choose the efficiently best basket of goods.

Sam Altman (CEO OpenAI): we want to offer the best price/intelligence tradeoff at every level.

This seems like a shift to method #3. OpenAI wants to compete for the lower end of the market, and faces a lot of Chinese competition, so they are slashing prices there. He’s asking ‘what do our competitors offer?’ and trying to do better. Might be a good move, might not. Whereas at the high end they effectively have a duopoly with Anthropic, so a price war would be foolish.

Alibaba gives us Qwen 3.8-Max-2.4T. Links to: Blog, Qwen Studio, API. Weights are coming next week.

As usual, the benchmarks are presented as strong. Pricing is $2/$6, or $0.25 implicit caching.

Based on the surrounding silence and my pattern recognition skills, and how many of these benchmarks are odd choices, I presume this is benchmaxxed and substantially behind Kimi K3.

Bloomberg frames this as Qwen ‘matching or exceeding’ Fable performance, which seems like Gell-Mann Amnesia territory. If that was remotely true, we would know. It fits that this post was partially written by AI as per Pangram, and rather obviously so. How do major publications not run Pangram checks in August 2026?

On Your Marks

MirrorCode, a measure of how large a software project a model can do, has Fable 5 well out in front of OpenAI models. No sign of Opus.

Prime Agent from Prime Intellect, is a new agent harness. Their big brag was that their harness scores 95.5% on ARC-AGI-3 using Opus 5. That tells you Prime Agent is vastly superior to the ARC-AGI-3 harness that ARC forces you to use on the real test, but the ARC-AGI-3 harness is intentionally terrible. This shows us Opus 5 has a slower uptake but a much higher maximum performance level than Sol in this setting, but does not tell us that Prime Agent is a good (or bad) harness.

Choose Your Fighter

If your AI use case would be good except it is too expensive, but we’re not talking orders of magnitude too expensive, start getting it ready now, and it will work soon.

nic: GPT-5.4 full at xhigh scored 51, exactly where Luna max sits today. GPT-5.4 costs $2.50/$15; Luna now costs $0.20/$1.20. In other words, roughly four months later, OpenAI is selling March’s full flagship intelligence at about one-thirteenth the token price.

nic carter: this is what cracks me up when people construct these elaborate bear cases based on AI being "too expensive". ok just wait 6 months and it will probably be 10x cheaper for the same unit of intelligence. (4 months/13x in this case)

Similarly, when Dwarkesh says ‘compute is about to get a lot more expensive,’ I think well maybe the cost of an H100 rental will go up but every use case is still going to get cheaper over time.

How big are the major frontier models? Here are some estimates, with some possibility that the closed lab estimates are too large:

Get My Agent On The Line

YC open sources a multi-agent harness they were using internally, customizable similarly to Hermes or OpenClaw, designed for an entire company.

Why are approximately no regular people using AI agents to run their lives?

I think this is mostly for the same reason most people fail to get good use out of human executive assistants. It takes a large degree of reliability and integration of preferences before assistants become net positive. Hiring that first employee is a costly action, no matter their role.

I have AI things running but I do not centrally use AI for managing key things.

If I had AI check my email, would that be good enough I would not otherwise check? No, so there is no reason to have it check my email. I tried. It would take a long time to net profit from setting that up and I’d rather wait for the models to improve. The same thing goes for filtering most other information sources, you only start to net profit when you can ignore things the AI doesn’t see.

For so many tasks, by the time you tell the AI to do it and verify that the AI did it correctly, you might as well have done it.

On the other hand, I’m clearly radically underusing such tools.

A fun exercise I am trying is, I have a highly competent person volunteering to be my assistant. Whenever I think of something for him to do, I then tell Claude Code to do it, and then I go back to thinking of things for my new assistant to do.

(I did eventually find something for him to try to do.)

Deepfaketown and Botpocalypse Soon

Shame on Palo Alto Networks CEO Nikesh Arora for putting out (and even for a time pinning) a Tweet that is not only AI slop, it is very obviously and painfully AI slop, in contrast to his other posts that on a spot check are in a distinct human voice that reads as influenced by AI style but still his own.

John Loeber: it's all so tiresome and disappointing

Imagine being the CEO of Palo Alto Networks -- $270B market cap -- putting out thought leadership on AI applied to cybersecurity, your specific area of expertise, the thing that you know better than anyone, where your perspective is most differentiated, where people really pay attention to what you have to say, it's the thing that you should absolutely insist to write yourself because AI will not get the details as precisely right as you will...

...and then it's all AI slop. Not even written by an internal marketing guy. But just straight-up AI generated. Lazy, lazy, lazy. Unbelievably undignified.

Nikesh Arora: My thoughts, cleaned up.

John Loeber: Especially considering your pinned tweet about the importance of writing and putting things in your own words, I would encourage you to post your thoughts as they are. Using AI even for clean-up will subtly change the work: and I'm really interested in what *you* think!

Nikesh Arora: Appreciate the feedback. Will do.

The Economist offers a guide to the basic tells for AI writing.

Derek Thompson: Marvelous examination of how to spot 2026-era AI writing, via the Economist

• AI likes long sentences with less punctuation; “and” is its most overused word
• relatedly, lists of three things, which drives up use of "and" as well
• polysyllabic adjectives: “significant”, “increasingly”
• scientific jargon ("rate-limiting," “parameter”)
• nominalizations (making nouns from verbs: eg, “expansion” from “expand”)
• ofc, everyone's favorite: "it's not X, it's Y"

Mike Ricci: 'table stakes' is one I look for.

leoohoho: “That is load-bearing.”

“The distinction matters.”

Curtis Duggan: This is the analytic explanation. The continental explanation is "I know it when I see it"

At this point I am mostly continental. I know it unconsciously first, then I notice that I noticed, then after that I can figure out why I realized that. As with all things, first you learn the rules, then you improvise and it becomes instinctual and you do not need the rules.

The other way is to put the text into Pangram, since it is almost always correct and the cost of doing so is so low.

Fun With Media Generation

Seedance 2.5 is now available in some places, a new cinematic video model from Dreamina, with native 30 second video clips and log mode up to 3 minutes.

John Hardin suggests that while the No Fakes Act offers an exemption for satire, there is no way to know you are inside the zone of acceptable satire without expensive lawyers, and anyone can come after you, so the powerful will be able to shut down satire.

One response is that I cannot imagine how else it could work. You can’t strictly define satire or libel without room for judgment. The good news is that, the same way you can try to get a takedown via a filing fee thanks to AI, you can also respond similarly, and you can get a pretty good sense of whether you have a case.

More to the point, it’s a really bad look to challenge satire in court. That’s the Streisand Effect zone. In practice, most will be loathe to do so, and for good reason. Can you imagine the horde of AI satire that will be coming at you the moment you sue over a bit of AI satire coming at you? This is the internet’s wheelhouse.

Although, with things like Build American AI, an affiliate of OpenAI-and-a16z-funded Leading the Future saying ‘we need a national AI regulatory solution that “puts people over profit”’ as their tag line to try and stop state regulations, it can be very hard these days to tell what is satire. Remember Poe’s Law.

Cyber Lack of Security

Joshua Achiam: Security by obscurity is about to die an awful, awful death. And people worried about AI cyberweapons are missing the point: the problem is that we built the software layer of civilization on spaghetti code loaded with zero days.

The key thing Mythos can do that other public models cannot, which I call ‘The Juice,’ is seek out, identify and string together vulnerabilities on its own to fully implement attacks.

Any individual step is not that hard to find or spell out, provided you can point an AI directly at that step. Or, it is easy to find the steps if you have already found them.

A good example of this comes from the recent Bitcoin hacks, where a March 2021 commit in Coldcard broke random seed generation, to the point where an attacker could narrow the possibilities enough to do a search.

On July 30, someone exploited this, and drained 1,082 BTC from 1,196 wallets within 41 minutes. Anyone who has not yet generated a new seed phrase remains vulnerable.

Then there were three additional subsequent waves of hacks. The most current total I could find stands at 1,816 BTC from 5,200 addresses, or about $116 million dollars.

If you know to look for vulnerabilities, that’s enough to be able to find them. We should presume that someone pointed some AI at this, and then out popped the vulnerability, at which point the rest was straightforward.

So is it weird that Coinkite claims they used ‘one of the best available models’ a few weeks prior and that it missed the vulnerability? Not especially. It’s probably a skill issue, and also not knowing where to look. It’s not about needing the best model. GLM-5.2 was able to do it with search disabled. It’s about doing your job, and pointing the check at the parts of the code you need to worry about (or, if you’re not sure, individually pointing it everywhere and not being a cheapskate.)

Andrew Curran: Update from r/Bitcoin. Claude Code can independently find the same wallet vulnerability used in this attack in eight minutes.

Andrew Curran: He says in the thread he replicated it with search disabled.

On the plus side, look, jobs.

jessicat: AI is creating jobs in computer security

Epoch AI: Serious cyber vulnerability disclosures keep climbing. In July, 21 major tech organizations published ~2,500 high- and critical-severity CVEs — about 5× the monthly record before Anthropic revealed Claude Mythos Preview could autonomously find software vulnerabilities.

If you were wondering if It’s Happening, the answer is yes. It’s happening. These lines are going to keep going up. A lot of other lines are going to similarly go up.

We are very much not ready, and this may look a lot like everything kind of breaking.

Zephaniah Roe: I feel like people haven't fully internalized what the world would look like if computer security actually broke.

There are some varying opinions on this, but a window of time without real computer security seems plausible. I was recently speaking with a computer security professor who I deeply respect and he was literally like "I think we are fucked and I don't think there is anything we can do."

This is similar to how people believe there is a 20% chance of extinction via AI but don't really internalize "No really. You will die and your girlfriend too. And your dog. And ..." In the cybersecurity case, some people believe (me included) that you cannot just patch all the bugs before releasing the model[1] but then don't internalize "No really. It would be chaos. You may not be able to get into your bank account. Industrial plants could be compromised. Power could go out for several days at a time. [...]"

Some People Need Practical Advice

We all need to be ready for The Hackening. If it never comes, or is limited in scope, that is great, but it might well not be.

This starts with basic ‘don’t be an idiot’ measures.

roon (OpenAI): needless to say but if you have any API keys, eth wallet keys, user credentials, etc hanging out on the open internet in pastebins, GitHubs, etc now is the time to take it down before the tireless eagle eyes of a million models come looking

if you have bet all your life savings on some sketchy smart contract scheme, maybe get a frontier model or whatever and investigate that thing for weaknesses. if you have a five year old IoT device do us all a favor and turn it off before it becomes a part of a botnet

Patrick McKenzie: There are many forms of security through obscurity, technical and otherwise, and many of them are going to come under severe pressure once the adversary has the equivalent of 10k research analysts doing intake and prioritization.

This is historically the reason why you don’t fight the feds or a nation state, because 10k B students will always win in a “You make one mistake and we sift until we find it” game. It is, unfortunately for everyone, not guaranteed that 10k and B student are upper