AI の能力向上に伴い、私たちが働くべきことは何になるか？

I had the honor of giving a keynote at the International Conference on Machine Learning in Seoul last week titled “What will be left for us to work on?” I addressed the widespread anxiety about how we should adapt as AI capabilities increase. I was thrilled by the talk’s reception, so I have made my slides available here, annotated with a lightly edited transcript. You can also view them below right here on this page, but the online version has animations, clickable links, and a much nicer experience overall.

I made three arguments. First, the AI as Normal Technology framework is a correct and useful as a way to think about AI’s impacts, unless and until there is some future discontinuity such as through recursive self-improvement. Second, even though we should take recursive self-improvement seriously, there is no milestone that companies might achieve in the lab that will suddenly put us all out of work. Third and finally, jobs of the future will be radically different, and a lot of adaptation will be needed. I shared my thinking about what this might look like and ended with a vision of human/AI “co-superintelligence”.

Now is a time of great excitement in AI, but it’s also a time of great anxiety in the AI community. I want to address that anxiety head on. How do we prepare for a future where AI will become capable of doing more and more of the work that we do today?

I lead a team at Princeton University trying to advance the science of AI agent evaluation. We try to go beyond the usual claims of “Look, capability is going up on benchmarks!” Those claims tend to be misinterpreted by the broader public as implying that agents are soon about to take all our jobs.

Maybe that will happen. But in our work we try to understand the factors beyond capability that matter for real-world deployment, and bring that understanding into evaluations.

The work that I’m better known for is the essay I co-authored with Sayash Kapoor called AI as Normal Technology. It’s a way to think about the medium-term future of AI and how to adapt to it — and in turn how to adapt it to the needs of society and the economy.

So we’ve been going around writing these essays about how lawyers should adapt, or maybe how journalists should adapt. But perhaps ironically, the question of how to adapt has been hitting our community first. Whether it’s software engineering or AI research itself, AI capabilities in these areas are of course advancing very rapidly.

Our response to this moment matters beyond this community. The whole world is watching. If we simply roll over and accept that a lot of our work will be done by AI in the future, instead of setting clear boundaries, I think it will lead to an even stronger political backlash against AI than what we are seeing today. So I think this question is not just for us but for the whole world.

From the beginning of AI, historically there have been these two battling narratives. In the past, the distinction was academic and philosophical, but now it has become an acutely practical question. Each one of us has to decide which camp we’re in, or where on this spectrum we’re in, because the practical consequences of believing in one versus the other are very, very different.

If you think this is a technology which in a few years is going to be able to replace everything we do today, then perhaps the correct response is to build wealth as quickly as possible before our skills become irrelevant. And this is the path that many have chosen in Silicon Valley. You may have heard of the “permanent underclass” meme.

On the other hand, if you believe, as I do, that this is a technology that will greatly amplify our potential, then now is the best time to build skills — especially the skills that are going to be complementary to what AI is doing and is going to be able to do — as well as to build all the things around it, such as agency and taste and judgment.

If you choose the first path, and it turns out that AI actually ends up being an amplifying technology as opposed to a replacing technology, then I would argue that over the next few years you’ve perhaps lost the best time in history to build these skills that will give us superpowers. That’s why we all need to think about this question, even if we won’t all land in the same place.

AI as Normal Technology is the intellectual framework for my talk today. When we say AI is normal, we don’t mean that it’s just like a hammer or a toothbrush, some kind of mundane technology.

We acknowledge prominently in the essay that this is a transformative technology on the scale of the industrial revolution. We’re not AI skeptics.

This is not a slogan. It’s a framework — sort of a causal model how AI capabilities impact the economy and society. It’s a 15,000-word essay and we’re turning it into a book. And I mention that because people often hear the word normal and they assume they know what we mean, but that leads to misunderstandings.

First, I’ll argue that this framework is correct and useful as a way to think about AI’s impacts, unless and until there is some future discontinuity — such as through recursive self-improvement — that leads to future impacts looking very different from past impacts.

Second, I will talk about why, even though we should take recursive self-improvement seriously, I’m not particularly losing sleep over it.

Third and finally, I want to be clear that I’m not saying that jobs of the future will be just like jobs of the present. A lot of adaptation will be needed. So I want to give some preliminary thinking on how I think our roles are going to change and how we can best adapt to the changes.

Powerful technologies of the past, like electricity, have been thoroughly studied, and we have good frameworks to understand how technological progress leads to economic impacts.

Invention: discovering the principles of electromagnetism, AC versus DC, etc.

Innovation: People don’t use “electricity” directly. We use electrical appliances. Those had to be invented, so that is a kind of downstream innovation — that’s the second phase of the framework.

Diffusion: This refers to the gradual process by which people start adopting innovations.

In our essay we apply this framework to AI and flesh it out into a four-part framework.

Here’s the basic picture, illustrated with software engineering as an example.

Methods/capabilities: Models are rapidly improving.

Products/applications: We don’t use LLMs directly. The reason they’ve been so influential in all of our work is because of coding agents. These are products that take those latent capabilities and turn them into something useful and usable for workers.

Early adoption: At first people were mostly trying vibe coding, and now we know that that’s not really the best way to develop production software — so now we have more sophisticated ways of doing agentic engineering.

Adaptation: (or structural transformation) — the fourth and slowest phase. Much of my talk today is going to be about that. I claim that this stage takes decades. It has not really started yet, even in a field like software engineering, which is a relative early adopter of coding agents.

We don’t know what the adaptation phase will look like — we can only speculate. Permit me to speculate for a minute. If it’s going to be the case that coding agents are going to be able to create ten-million-line code bases in the future that are not full of bugs and security vulnerabilities, then it won’t make a lot of sense for us to create one piece of software that billions of people should use. It’ll make a lot more sense for software to be tailored to the needs of each individual or team. And that’s what I mean by extreme personalization.

It’s not merely a technological change — that’s also a change for the industry. For instance: do we even need software companies anymore? Maybe software development will massively shift in-house, into the companies and teams that are actually using the software. Again, this is speculation, but the point is that it is this kind of organizational change, human change — that’s very slow, that takes decades — that will allow us to take advantage of the full potential of AI, whether it’s in software engineering or in any other field. So that’s one of the central insights of the essay. When we look at past technologies, this kind of change tends to be very slow.

Before electricity, factories used to look like the picture on the left. A massive steam engine generated power and it was moved throughout the factory by mechanical gears and belts. So when electricity came along, factory owners tried to replace those steam boilers with electric generators. They thought it would be much more efficient. But this idea of a drop-in replacement did not work. We keep hearing that term in the context of AI agents today — that they will be drop-in replacements for human workers. That did not work in the case of electricity.

What actually worked, and what took 40 years to develop, is to recognize that electricity is a very different technology. It’s portable, so you can move the power to wherever you need it. That lets you reorganize the entire layout of the factory around the logic of the assembly line. And that required changing the way that workers are trained, hired, and fired, new labor laws, and so forth. So that’s the kind of organizational adaptation that it took in order to reap the benefits of electricity in factories.

Our claim is that this is the kind of process that we will go through for AI. A couple of decades from now, we will have fundamentally reorganized work. We don’t know what that’s going to look like, and that is the challenge in front of all of us. And that’s not just a job for the AI companies to do, much like it wasn’t the job of the electric utility to figure out how factories should be reorganized. In our view, this is the slowest of the four stages through which AI leads to economic impacts. Today, this process has not really gotten started.

Why is there a huge gap between what people in various occupations could be using AI for and what they’re actually using it for? One reason could be that people are slow to adopt technology, and that’s certainly part of our framework.

But we wondered if maybe the people who are deploying AI and are not having much success at it know something about the practical limitations of AI that the AI industry doesn’t. Let’s have a bit more humility about the relationship between capabilities and deployment.

Given that the #1 concern people cite is reliability, we wanted to try to measure whether reliability, distinct from capability, is a barrier to the practical usefulness of AI agents.

We looked at 10-12 reliability metrics and clustered them into four dimensions.

Consistency: Suppose we hear that an AI agent has a 70% accuracy. Does this mean it works on 70% of the tasks, but on the ones that it does, it does so every time? That’s great for deployment — you can deploy it on that subset of the tasks. Or does it mean that on any given task it might unpredictably fail with a 30% probability? That’s pretty useless from a deployment perspective. Perhaps shockingly, none of the agent benchmarks that we looked into make a distinction between these two. Both of these are represented as 70% accuracy.

Robustness: We looked at robustness: what happens when the environment changes a little bit?

Calibration: Can the agent look back at its transcript and tell if it performed the task correctly?

Operational safety: When it does fail, is it recoverable, or is it something like deleting the production database?

For a human worker, if we think of someone as being competent at a job, it’s all of these things, not just accuracy. But it turns out we were measuring agents only on accuracy.

We measured capability and reliability using two complementary benchmarks, for models from these 3 frontier AI companies that were released over the last 24 months or so.

This is a period during which accuracy or capability shot up dramatically (left).

But reliability (right) only increased by five or ten percentage points.

There are a bunch of implications but let me highlight one. Right now the industry is not treating automation and collaboration agents differently. If you run an agent in headless mode, I guess that becomes an automation agent. That is not a good way to look at it, because properties like reliability that are very important for automation agents can actually be a hindrance for a collaboration agent that you might be using to improve your creative writing or something like that. For that kind of agent, you don’t want it to behave like a robot that does the same thing every time. You want it to be creative and explore different possibilities and be unpredictable.

Scaffolds and even the post-training of models should be different based on whether it is supposed to be driving a collaboration agent or an automation agent.

My hope is that reliability will continue to improve and automation will become easier over time. But for now I think collaboration agents will continue to be much more successful. Many companies that hastily rushed to automate business processes using agents are recognizing the limits and costs, and even legal liabilities — like when an agent deletes production data.

For the time being, you can have only two out of these three properties in agents: general-purpose (a language model based agent that can be instructed to do different tasks rather than purpose-built for a task like traditional software); deployed in high-stakes scenarios, and automated.

This is one of the reasons that I think that for now, AI remains much more of a collaboration technology than a technology that automates workers away.

Let’s take software engineering as a case study. It’s a good leading indicator because coding agents have been particularly rapidly adopted.

You might think: okay, we can’t completely automate away software engineering. But if agents make software engineers ten times more productive, then we need ten times fewer software engineers. Isn’t that an obvious consequence?

Well, that is completely contradicted by the data. We looked at this in a follow-up essay. In every case we looked at, the company was under financial pressure, and it turns out to be more convenient to blame AI for the layoffs instead.

Why is AI not replacing software engineers so far? We’ve known for a while — this is a paper from 2019 — that writing code is not really the bottleneck.

Over the last year, as software engineers started adopting coding agents and started to recognize that it doesn’t seem to be cutting down on the amount of their work, there have been many blog posts rediscovering the fact that writing code is not a bottleneck. Here is a small sample.

So what actually is the bottleneck?

This framework is our answer to the question.

The decide layer: understanding customer requirements, developing the specification, planning, etc. That is not getting compresse