人間らしさを選ぶこと

If you go to your favorite social media site, you will find it full of posts that start to look suspiciously similar to each other:

Many of the comments to these posts are also generated by AI. So are an increasing number of academic papers and New York Times opinion articles, and, apparently, award-winning short stories. If you use AI a lot, you probably have noticed how much AI writing is around you (frequent AI users have historically done quite well identifying AI writing), if not, I promise you it is much more than you think.

It isn’t just the sameness of the AI writing, though that eventually gets to be tedious enough that I find myself skipping writing on even interesting topics if my internal “AI detector” goes off. It is also that badly prompted AI writing produces very little meaning per word, taking you in intellectual circles instead. We are trained to read well-crafted sentences and intellectual sounding texts as the result of effortful human work and thus pay attention to these AI written comments when we see them. But there is often no human meaning there, these posts are just meaning-shaped attention vampires that take mental effort to decode and give you no equivalent understanding in return1.

But using AI for writing has a cost beyond turning off readers, it risks undermining the development of an important human task. I am lucky enough to have been writing for decades, and I have developed my own style which I think shines through whether I am writing a book, a tweet, or a blog post. That style took a lot of super annoying work to get to: good teachers and rewrites and mean online comments all contributed. If the AI does fine writing, I could skip all of that, but I would have done so the cost of giving up something that has turned out to be very important to my career and my happiness.

This is not a condemnation of using AI to help with writing in any way. I think AI can be a fantastic tool for good writers (I have AI check all of my writing and roleplay different reader perspectives to see if I missed something important). For those who struggle with communication, AI can help get their ideas across better, and writing may not be thinking for everyone. Plus, a little bit of effort can make AI writing less cliche, more personal, and more worth using (in moderation). So, this is instead a condemnation of using AI as a default, or, even worse, without thinking at all. Balancing using AI with our own mental abilities is going to be a defining challenge of the coming years.

Subtle changes, big outcome differences

The clearest place to see this is in education, where two papers with an overlapping research team (including peers at Wharton) do a good job illustrating the difference between using AI to shortcut thinking and to help thinking. The first paper was an experiment at a high school in Turkey with about a thousand students learning math. One group used plain ChatGPT, the other had no AI access. The students with ChatGPT did their homework better and thought they were learning more, but at test time, they underperformed their classmates without ChatGPT. That is because the AI, designed to be a helpful assistant, was really just giving them answers, and actual learning requires mental effort. By short-circuiting effort, you short-circuit learning. That is why the initial results of AI on learning in classrooms can be so worrying.

Yet we can see a different result in a second paper from many of the same authors when they ran a five-month Python course across ten high schools in Taipei with close to a thousand students. Students who were given a personalized sequence of problems by an AI tutor scored 0.15 standard deviations higher on a final exam taken without AI help. By some estimates, that’s the equivalent of six to nine months of additional schooling, without any added instruction time or teacher workload. Instead, the AI helped tailor the learning to the student. This fits other work on AI tutoring, suggesting that customized tutors can significantly boost learning when used properly.

This is a relatively small difference in how you use AI and yet it leads to big outcome differences. Worse, human nature leads us to make the wrong choices. Learning requires us to face our own ignorance and do hard intellectual work, and these things are really uncomfortable. Which is why students rate entertaining lectures as more educational than doing hard problems in class, even though they actually learn more from the hard work. To benefit from AI in learning you need to pivot from using AI to solve problems, to pushing you to solve problems yourself.

Fortunately, the three major AI companies have tools that provide at least some support for learning by making the AI act more like a tutor. Unfortunately, they are not intuitive to access. Gemini is the easiest. Hit plus and pick Guided Learning. For ChatGPT, you need to type “/learn” into the chatbox. For Claude, you need to hit the plus, select use style, and select “learning” (Anthropic has announced that this approach is changing but has not yet documented the change). In all cases, you should use a thinking or advanced model where possible, especially for STEM subjects. And these modes will only help support someone who wants to learn, they won’t stop you from cheating if you want.

Too frictionless

AI need not undermine your ability to think, but it can do so if used badly and badly is often the default. My colleagues at Wharton call this “cognitive surrender,” and they documented how people would stop thinking about problems and just let the AI do the work, even when the AI was wrong. I think part of the problem is the way these tools are designed.

I did not do this for the post…

When AI systems required elaborate back-and-forth conversations and made errors frequently, humans had to be engaged at every step. Agentic systems are designed to make your life easier, because they just do stuff. Which is great for getting stuff done, bad for learning anything, or staying authentic, or avoiding cognitive surrender. If you put in a hard request and get an answer, it is tempting to just go with the AI’s response.

In our recently published paper with Fabrizio Dell’Acqua and my colleagues at Harvard, MIT, the University of Warwick, BCG, and elsewhere (which I wrote about here three years ago, but publishing academic work takes a while!) we ran an experiment on 758 consultants at Boston Consulting Group, half of whom got access to GPT-4. Consultants using AI vastly outperformed those without. But we also asked consultants to do solve a problem that we knew the AI would fail at. Consultants using AI on this task were significantly less likely to get the right answer than consultants without it. The AI gave them an authoritative-looking answer that happened to be incorrect, and most of them, the same elite consultants who outperformed on everything else, did not catch it. Of course, now AI just solves that problem, so the issue isn’t really error rates now, it is failing to learn how to be a good consultant by giving into the same impulse to surrender.

Again, this does not have to be the default. In a small study conducted by Anthropic, programmers used AI to help them do a new task. Those who just let the AI do the work couldn’t answer questions about what they had done, a sign of surrender. But people who asked the AI to explain what it was doing, or those who used AI to help them with only some of the work, seemed to avoid that fate.

Some of the solution might be in the tools themselves, but that is limited. A version of ChatGPT that asked, before every answer, “would you rather I push you to think through this, or just give it to you?” or told you “I think this would be more authentic if you wrote this” would be insufferable most of the time. But there are places where we absolutely need these reminders. The Taipei result hints at one direction, namely system-level constraints rather than user-level willpower, but we don’t see much of that in the consumer products, and the commercial pressure mostly pushes in the opposite direction.

Choosing what to keep human

A lot of the problem is going to come down to us. To be clear, I am cool with a lot of cognitive surrender. I don’t remember phone numbers anymore because my phone does that for me. I am happy my kids didn’t need to learn cursive. I am fine with calculators doing my daily math and my computer figuring out how to schedule my classes. These were once useful skills, but we were probably right to get rid of them.

AI is different because the technology is general enough that virtually any cognitive task can be offloaded into it to some degree. I don’t want to be too precious about writing: there is no principle that says a polished email draft has to come out of a human mind any more than a column of arithmetic has to. But we don’t want to give up everything, and that we mostly don’t know yet, for any specific task, what is important and what is not. Deciding that is going to be a real challenge.

The point isn't to avoid AI but to be intentional about it by making a conscious choice about AI use, rather than reflexive dependence or reflexive avoidance. More broadly, we are at the point where the defaults are being set for what kind of work to give AI: by the AI companies designing for frictionless use, by employers deciding what counts as “using AI well,” and by people teaching the ever-shifting concept of “AI literacy.” A lot of this is happening without, ironically, any real planning or consideration. And I suspect it will be hard to reverse these defaults once a generation of workers and students has built habits around them. The most important thing we can do is keep asking what to hand over and what to keep for ourselves… and not expect anyone, including the AI, to answer that for us.

Subscribe now

Share

1This is especially true of fiction writing, where AI is notoriously weak while seeming strong. ChatGPT in particular is fond of meaningless similes and metaphors (“the street was like a gap-toothed smile,” “he sat in a way that would make the trees jealous”) that can feel profound at first sight, but only because we assume difficult writing is purposeful and work hard to assign it meaning. Humans are very good at assigning meaning to meaningless material if we try hard enough.