警察向け AI 販売ビジネスの実態

I stood before a hulking glass and brick structure in the heart of Fort Worth, Texas. Thousands gathered inside to see what had been billed as “the future of policing in the digital age.” As press, I was prohibited from entering, but from a number of nearby locations, I met with attendees who told me what was being sold within. And I learned that AI is threatening to seize the very heart of policing in America.

The promise of AI at this year’s International Association of Chiefs of Police (IACP) Technology Conference focused on automating routine parts of the job, which also happen to be critical steps in the legal process. It’s a similar sales pitch to the one that’s been exhaustively broadcast to businesses in recent years: Let the machines handle the busywork, so you can focus on more *meaningful* tasks. But in law enforcement, the automation of seemingly innocuous “busywork” — like taking the time to carefully fill out a police report or review a suspect’s case history — can have immense consequences on people’s lives.

Among the AI products on offer at the conference’s showroom this May were facial-recognition cameras, automated license plate readers, body cameras, chatbots to field non-emergency 911 calls, gunshot detection platforms, drones, and report-writing tools. As the country has reckoned with law enforcement becoming detached from actual, human police presence in neighborhoods, the industry is continuing to embrace automation.

The decision-making process itself in police departments is increasingly being handed over to algorithms. A legion of tech startups are now selling AI to police as a kind of automated air traffic control system, a centralized digital brain that can process the vast quantities of data now being collected — oftentimes by other surveillance and automation tools sold by those very same companies — and help departments delegate resources accordingly. Even police aren’t necessarily thrilled about these pitches.

“A lot of it is sales gimmicks that don’t actually deliver on what the promise is,” Abrem Ayana, a police captain in Brookhaven, Georgia, told me. In the absence of comprehensive federal oversight or industry standards — and due to the novelty of the tech itself — law enforcement officials like Ayana often have no choice but to take companies’ word that their products are safe and that they work as advertised.

Police departments have used technology for decades to analyze data and, in theory, make more informed decisions in the field. In some notorious cases, it’s completely backfired. CompStat and PredPol (short for “computer comparison statistics” and “predictive policing,” respectively), for example, were two early experiments that sought to mitigate fallible human judgement through the use of supposedly unbiased statistics. Instead, they ended up exacerbating the very problems they were meant to solve. But while those early experiments failed to usher in a new era of unbiased policing as their proponents had hoped, human beings were at least still at the helm, making the most important decisions.

The sales pitch behind this new wave of AI products is that the mistakes of the past were enabled by a lack of objective, real-time data. AI can, in theory, now help to bridge the gap by ramping up the amount of public safety data that’s collected and the level of analysis to which it’s subjected. Many public safety advocacy groups and legal experts, however, warn that an influx of black box algorithms into law enforcement will erode transparency and accountability at a time when much of the public’s trust of the police is already dangerously frayed.

Jason Truppi, a former FBI special agent specializing in cybercrime, told me that police are drowning in a sea of data. Truppi, wearing a pair of Meta Ray-Ban Smart Glasses, spoke quickly and excitedly in sentences peppered with corporate buzzphrases. In late 2020, he cofounded ForceMetrics, a software company offering an “AI-powered decision-assist platform, enabling public safety agencies to increase operational efficiency and better serve their communities in real time,” as described by its LinkedIn page.

All of the record-keeping systems that police departments have been using for the past two decades, from emergency call logs to parole record files to body camera footage databases, have, according to Truppi, created a burdensome information overload. “All the systems of record [used by police departments] are essentially antiquated,” he told me.

“We don’t use the ‘p word’ at all, because it failed.”

ForceMetrics offers police departments a platform called Velocity, which “uses AI to turn overwhelming amounts of public safety data into clear, actionable insights,” according to the company’s website. In police-tech industry-speak, Velocity is what’s known as a real-time crime center, or RTCC. First adopted by the New York City Police Department over 20 years ago, RTCCs are designed to aggregate police data coming in from multiple streams — like 911 dispatch, CCTV cameras, and license-plate scanners — to provide officers with a summary of what to expect when they arrive on a scene. The theory is that the more real-time data you can give officers, the less likely they’ll be to go in “guts and guns,” as Truppi puts it. It’s a cheeky euphemism for when things go bad and people get killed.

In the past, RTCCs were overseen by human analysts whose job was to collect all the incoming digital data, organize it, and send it to the officers on patrol. But as Truppi suggests, the proliferation of new data-collection technologies within policing over the years has made it effectively impossible for any department to stay afloat in the deluge of information. By 2019, the NYPD was collecting around *two years’* worth of body camera footage* every week*, according to the transcript of a 2019 Committee on Public Safety hearing — too much for even the most diligent human employee to meaningfully analyze.

Modern RTCCs like Velocity are designed to quickly extract patterns from oceans of data with the goal of improving situational awareness for cops. According to Truppi, the “unfortunate events” that have so disastrously damaged Americans’ trust in police departments in recent years, especially during the pandemic, can largely be attributed to a lack of what he calls “a data-driven approach” to policing.

Nina Loshkajian, a fellow at the New York University Center on Race, Inequality, and the Law, is wary of this claim. “The reality is that police departments had already been using predictive algorithms, which companies touted as data-driven, for years before calls to defund the police revved up in 2020,” she told me. “These algorithmic systems did not prevent violent encounters between police and civilians then, and we shouldn’t be tricked into thinking they’ll make a meaningful difference in the future.”

Truppi’s company is competing with two of the biggest players in the modern police-technology industrial complex: Motorola Solutions and Axon Enterprise, both of which make not only their own RTCCs, but also many of the data-collection and surveillance technologies they rely on.

In early 2024, Axon — originally called TASER — acquired surveillance technology company Fusus to launch a RTCC, which was officially branded as Axon Fusus. By that time, Axon was already a well-known purveyor of stun guns, body-worn cameras, and automated license plate readers. The company also offers a popular AI-powered report-writing tool called Draft One, drones for police departments through a program called Axon Air, and even its own AI chatbot.

Axon and Motorola are part of a very small group of companies competing to effectively monopolize the entire modern police technology stack, from the collection of data at crime scenes to the strategic decision-making capabilities of AI-powered RTCCs. Police departments today often sign onto multiyear contracts with these providers, who in turn offer free trial periods for new tech, along with what are known as sole-source procurement agreements, which enable them to continue selling new products to departments without having to bid against competing offers from other vendors.

“We’re seeing a gold rush into selling [AI] technology to police with the promise that it will all make their jobs easier and more efficient.”

In late 2024, Axon launched its AI Era Plan, a subscription that allows customers to pay a flat annual fee to gain access both to the company’s current AI tools, like Draft One, as well as others it might launch in the future. AI Era Plan subscriptions skyrocketed by 140 percent between the first quarter of last year and the same time this year, according to the transcript of a company earnings call with investors: “we are seeing AI move from early interest to a standard part of how large agencies think about their future technology stack,” Axon President Joshua Isner said in that call. “We are determined to become the AI company in public safety, and we are well on our way.” According to the transcript, Axon’s AI product revenue grew 700 percent year over year.

While bigger companies like Axon, Motorola, and Flock Safety currently dominate the police technology-industrial complex, it’s facing growing competition from the army of newer tech startups that were exhibiting at the IACP tech conference in Texas. “The entire game of all of these companies is to become *the* platform for policing,” says Andrew Guthrie Ferguson, a professor at Georgetown University Law School and the author of multiple books on the intersection of policing and technology. “We’re seeing a gold rush into selling [AI] technology to police with the promise that it will all make their jobs easier and more efficient.”

That gold rush has also attracted an influx of outside investors: About one-quarter of attendees on the showroom floor at the conference were from “equity firms looking to invest in the latest tech,” according to Amber Schroader, a tech entrepreneur whom I spoke with in Fort Worth during the event. “That was a surprise.”

The sales pitch has been working.

Draft One and other AI-powered report-writing tools, for example, have significant appeal at a time when the average police officer spends 40 percent of a typical shift writing reports, according to a 2024 study conducted by Axon. Many of those are for mundane incidents like traffic stops and noise complaints. “We didn’t sign up to sit behind a keyboard,” said John Mackey, a patrol sergeant with Colorado’s Avon Police Department, which uses Field Notes, an AI-powered report-writing tool made by a company called Truleo. “That wasn’t why I became a police officer.”

Draft One comes with design features intended to force a degree of human oversight. The system will intentionally leave certain details blank, for example, forcing officers to go in and fill them in manually. The platform is built upon a modified version of ChatGPT trained specifically to generate police reports and that, according to the company, is hallucination-free: “The creativity is turned down to zero,” Noah Spitzer-Williams, senior principal product manager at Axon’s generative AI division, has said. That claim should be taken with a very large grain of salt, however, since even frontier labs like OpenAI (the company behind ChatGPT), Anthropic, and Google have not yet figured out how to completely eradicate hallucination from even their most advanced models. And indeed, in one infamous incident from earlier this year, Draft One wrote that an officer in Utah had morphed into a frog, after having picked up audio from the Disney movie *The Princess and the Frog*, which had reportedly been playing in the background at the scene.

It’s easy to laugh at that incident, but real-world outcomes from AI-written police reports could be deadly serious. When a human officer writes a report, they can be cross-examined in a courtroom to figure out important details like their state of mind at the time, or why they included certain details and omitted others. By definition, it’s impossible to subject black box algorithms to the same level of scrutiny.

Axon and Motorola are part of a very small group of companies competing to effectively monopolize the entire modern police technology stack, from the collection of data at crime scenes to the strategic decision-making capabilities of AI-powered RTCCs.

In the case of Draft One, it was also originally impossible to determine which parts of a report were generated by the AI and which by the human officer once the report has been submitted — save the officer’s own memory. That was a feature, not a bug. In a recorded roundtable discussion published online shortly after Draft One was launched in 2024, Spitzer-Williams said the platform “by design” doesn’t save an original copy of a report after it’s been submitted, “because [the] last thing we want to do is create more disclosure headaches for our customers and our attorney’s offices… it’s actually never stored in the cloud at all so you don’t have to worry about extra copies, you know, floating around.” In other words, if a report generated by Draft One ended up in court and was found to contain erroneous details, there was no way for attorneys or judges to know for certain if those were input by the officer or by AI.

Draft One was updated in December to allow police departments “to retain and access the original, unedited AI-generated narrative,” according to Axon spokesperson Victoria Keough. The change was implemented “as [law enforcement] agencies, prosecutors, policymakers, and legislatures have established clearer expectations and requirements for AI-assisted report writing.”

Brandon Garrett, a professor at the Duke University School of Law who has studied the implications of AI systems for due process, is apprehensive of the technology. “The idea that you’d be making up data — which is what generative models do — to be used in court, is really, really troubling,” he says. “We would never tell a police officer, ‘Just be creative and come up with a story about what you saw at the crime scene.’ Of course not: They’re supposed to objectively record as best as they can and document what they saw at the crime scene. But generative models are designed to create.”

In the wake of the 2008 financial crisis, LA police chief Charlie Beck took inspiration from Wal-Mart and Amazon’s personalized shopping algorithms and wrote that police departments should use similar tools to predict crime. Starting in the 2010s, “predictive policing” programs were widely implemented in cities across the country. But far from creating a new era of fairness and justice in policing, the algorithms in many cases had exactly the opposite effect: Since the models had been trained to detect patterns from historic crime data, the biases hidden within that training data were perpetuated — under the guise of mathematical objectivity.

PredPol, for example, was based on an algorithm originally used to predict the geographical distributions of earthquake aftershocks, the idea being that the same general principle could be applied to predicting crime: the tighter the correlation between a certain area and a particular criminal pattern, so the thinking went, the higher the likelihood that same pattern will continue into the future. This allowed the AI to identify crime hotspots, which personnel-strapped police departments could focus more attention on.

But PredPol and similar programs failed to account for some key facts. For example, more crimes tend to be reported in poorer neighborhoods, which in many major cities are populated primarily by people of color, leading to a higher police presence and arrest rate than those found in other areas. The algorithm had no way of understanding that the fact that there was a higher crime rate in one neighborhood, say, than there was in another, more affluent area was largely the product of a complex history of social, political, and racial biases and policies; it just ingested the data it had been given, leading to a more intensive focus on historically over-policed areas: a self-perpetuating cycle.

This was clearly illustrated in 2016, when AI researchers Kristian Lum and William Isaac tested a predictive policing algorithm using historic drug crime data from the Oakland Police Department. The algorithm recommended dispatching police “almost exclusively to lower income, minority neighborhoods,” Lum wrote in a follow-up article, even though public health data at the time showed that illegal drug use was widely distributed across the city.

The same pattern emerged wherever predictive policing programs were implemented. “The use of predictive policing systems can make the future look a lot like the past,” Ángel Díaz, an associate professor at Loyola Law School, told me. “Because a lot of the data you’re pulling is from the world as understood by biased policing practices, the patterns that exist in that data will be drawn out by the computer and might help inform future policing practices.” In 2024, four democratic US senators urged the Department of Justice to halt all future grants to law enforcement agencies for predictive policing programs, citing evidence that such programs “are prone to over-predicting crime rates in Black and Latino neighborhoods while under-predicting crime in white neighborhoods.”

Predictive policing has therefore become taboo in the modern police-tech industrial complex, a cautionary tale about conflating statistics with objectivity. (PredPol changed its brand name to Geolitica in March of 2021). “We don’t use the ‘p word’ at all,” Truppi told me, “because it failed.”

Experts say a future of policing based on increasingly fine-grained personal data collection and AI-driven policing is frightening. As the decision-making power of AI within policing grows, so too will the inscrutability of the justice system itself, according to Díaz, the Loyola Law professor. “The biggest thing that worries me is that we are rapidly expanding how much data is being collected about all of us,” he told me. “The reality is that the more data you have about any given person, the easier it is to reverse engineer a reason to target them; the more data you have about each individual, the easier it is to transform them into the subject of an investigation.”

Facing budget cuts and staffing shortages, and accosted by sales pitches in every direction, police departments are now facing the same kind of pressure as private companies to adopt new AI tools — which, they’re promised, are free of the foibles found in earlier programs like PredPol and CompStat. And as Brookhaven’s Captain Ayana mentioned, all of this is happening inside a regulatory vacuum, with law enforcement leaders left to their own discretion to separate the gimmicks from the legitimately safe and useful tools.

“The use of predictive policing systems can make the future look a lot like the past.”

According to Katie Kinsey, chief of staff and tech policy council at the Policing Project, a nonprofit organization focused on promoting accountability within law enforcement, the challenge facing police departments now is ensuring that the data that’s fed into this advanced new generation of RTCCs is reliable—i.e., free from the biases that infected the training data of earlier tools. “We absolutely do want police practice to be informed by data and to be evidence-based,” Kinsey told me. “But data is not perfect, and not all data is created equal…Understanding the data sources and limitations that police are working with are especially crucial in our AI age where data increasingly is the currency of decision-making.”

Such transparency is made much more difficult when the data is controlled by private vendors, such as Axon, whose business models rely on maintaining the secrecy of their proprietary AI tools. And if there’s one lesson that can be drawn from the broader AI race, it’s that the race to dominate market share often comes at the expense of safety. For the moment though, in lieu of any broad governance, police departments are left to their own devices to choose from a growing roster of tech vendors. The decisions they make today will impact how decisions are made within their departments tomorrow.

When I asked Stephen Redfearn, the chief of Colorado’s Boulder Police Department, about the future of AI within law enforcement, he told me: “It’s going to continue to be kind of a roller coaster for a while, while people get more comfortable with it.”

*This reporting was supported by a grant from the *Tarbell Center for AI Journalism.

3 Comments

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• Webb Wright

-

-