Substack、AI 生成文検出機能に利用者が「魔女狩り」と反発

imageSubstack contributors are rejecting the company’s decision to flag AI generated writing on the newsletter publishing platform, with several creators saying the new AI detection features are a “witch hunt.” Some Substack users who use AI to help them write or edit articles say the new policy unfairly discriminates against them, while other users who don’t use AI in their writing at all are worried that Substack will mistakenly flag their writing as being AI generated and tarnish their reputation.

“I’m not going to apologize for using AI in the creation process. I wrote for 20 years without AI, I could do it again if I wanted to,” Mack Collier, who has a small Substack called Backstage Pass, wrote. “My output would fall, my posts would be less structured, and it would be more obvious that they needed a good editor. Using AI improves the overall quality of my writing. That’s why I use it.”

“These detectors are notoriously, wildly inaccurate,” Alice Lemee, a ghostwriter and digital writing coach, said in a video she made about Substack’s new AI detection features. “All it takes is one false accusation for a writer to have their reputation almost irreversibly tarnished.”

Substack announced the AI detection features on Tuesday in a post from CEO Christ Best, who said that “It’s getting harder to tell what’s real on the internet” and that “when content made by no one takes over parts of the internet that are supposed to be human, it pollutes the commons and makes it hard to discover and hear human voices.”

Best explained that Substack has partnered with the AI detection tool Pangram, which is now built in to Substack and allows any user to scan a piece of writing. Pangram then produces a percentage-based score determining how much of the writing was AI or human generated. (Disclosure: Pangram previously bought an ad on 404 Media). We’ve covered research from Pangram, or research that relies on its AI detector before, showing how AI generated writing is flooding every corner of the internet.

As we’ve previously reported, AI detectors are not perfect, and Pangram itself is not immune to labeling human content as being AI, and labeling AI as human. Max Spero, the CEO of Pangram, recently told us that the company is constantly working on minimizing errors, and that it estimates its false positive rate at roughly one in 10,000.

“We have now built the mirror image. Pangram is a real machine, trained on vast amounts of text to make a probabilistic guess, and we have pointed it at your writing to work out whether a person is hidden inside,” Sam Illingworth, a professor and the author of Slow AI, said on his Substack in a post titled “Substack’s AI Detector and the Return of the Witch Hunt.” “To decide if there is a human on the other end, Substack asks a machine.”

Substack also added a way for users to add a “How I make this” statement, where they can explain their writing process and disclose if they use AI.

“We’re not against people using AI to assist their work, and we think people should be free to choose which tools they use to express themselves,” Best said. “Some on the platform are human-writing maximalists, others are publicly exploring the frontier of using AI tools while sweating the details to make work they stand behind. We use AI all the time at Substack to write software, do research, and build product features like clipping, translations, and more. But people should know what they’re getting.”

“I think the best way for Substack to address this is to invite further dialogue,” Illingworth told me. “I actually really like the new feature that lets authors tell their readers how they construct their newsletters using AI, and this is the kind of dialogue they should encourage rather than ascribing a score that we know to be incorrect. My worry is that not only do these AI detectors generate false flags for non-native English speakers and neurodiverse writers, but they also remove any opportunity for dialogue because they start from a position of suspicion. Whereas what we need to be doing is developing opportunities for trust between readers and authors.”

"The tools we've introduced today are intended to increase transparency and give readers more context, not to prohibit or penalize AI-assisted writing. They do not impact discovery on the platform,” a Substack spokesperson told me. “We're also encouraging Substack publishers to add a ‘How I make this’ statement, where they can explain their process directly, including how they use (or don't use) AI, and set expectations for readers. Creators can disable detection on their posts pre- and post-publication, as well as report and remove scans on their own work that they believe are mistaken. Find more information on how the features work here.”

Substack’s attempt to detect and label AI generated content was also celebrated by many readers who are exhausted by AI slop and other internet platforms that can’t or don’t want to label it. I just published story about how there is so much unlabeled AI slop on Spotify that people are now creating their own sites to detect and label it without the company’s help.

But the backlash from some Substack users shows that companies can’t just flip a switch if they want to ban or give users the ability to filter out AI content from their feeds. As the flood of AI writing and images we see online and the real world every day makes clear, generating slop is easy. What we’re going to do about it is still an unsolved problem.