受動的から能動的へ：LLMでフィッシング対策のギャップを埋める

Email security has always been defined by impermanence. It is a perpetual call-and-response arms race, where defenses are only as strong as the last bypass discovered and attackers iterate relentlessly for even marginal gains. Every control we deploy eventually becomes yesterday’s solution.

What makes this challenge especially difficult is that our biggest weaknesses are, by definition, invisible.

This problem is best illustrated by a classic example from World War II. Mathematician Abraham Wald was tasked with helping Allied engineers decide where to reinforce bomber aircraft. Engineers initially focused on the bullet holes visible on planes returning from missions. Wald pointed out the flaw: they were reinforcing the areas where planes could already take damage and survive. The true vulnerabilities were on the planes that never came back.

image

Email security faces an identical hurdle: our detection gaps are unseen. By integrating LLMs, we advance email phishing protection and move from reactive to proactive detection improvement.

The limits of reactive defense

Traditional email security systems improve primarily through user-reported misses. For example, if we marked a spam message as clean, customers can send us the original EML to our pipelines for our analysts to analyze and update our models. This feedback loop is necessary and valuable, but it is inherently reactive. It depends on someone noticing a failure after the fact and taking the time to report it.

That means detection improvements are often driven by what attackers already succeeded at, rather than by what they are about to exploit next.

To close this gap, we need a way to systematically observe the “planes that didn’t make it back.”

Mapping the threat landscape with LLMs

Large Language Models (LLMs) hit the mainstream market in late 2022 and early 2023, fundamentally changing how we process unstructured data. At their core, LLMs use deep learning and massive datasets to predict the next token in a sequence, allowing them to understand context and nuance. They are particularly well-suited for email security because they can read natural language and characterize complex concepts (like intent, urgency, and deception) across millions of messages.

Every day, Cloudflare processes millions of unwanted emails. Historically, it was not feasible to deeply characterize each message beyond coarse classifications. Manually mapping emails to nuanced threat vectors simply did not scale.

Now, Cloudflare has integrated LLMs into our email security tools to identify threats before they strike. By using the power of LLMs, as we’ll describe below, we can finally see a clear and comprehensive picture of the evolving threat landscape.

image

Our LLM-driven categorization shows clear spikes and persistent trends across several distinct categories, including "PrizeNotification" and "SalesOutreach".

These LLM-generated tags provide Cloudflare analysts with high-fidelity signals in near real time. Tasks that previously required hours of manual investigation and complex querying can now be surfaced automatically, with relevant context attached. This directly increases the velocity at which we can build new targeted Machine Learning models or retrain existing ones to address emerging behaviors.

Because Cloudflare operates at global Internet scale, we can gather these insights earlier than ever before, often before a new technique becomes widely visible through customer-reported misses.

The Sales Outreach threat

One of the clearest patterns we’ve identified using this new intelligence is the continued persistence of malicious messages structured to look like Sales Outreach-style phishing. These emails are designed to mimic legitimate B2B communication, often presenting opportunities to purchase or receive "special deals" on unique items or services, to lure targets into clicking malicious links or providing credentials.

Once LLM categorization surfaced Sales Outreach as a dominant vector, we moved from broad visibility to targeted data collection.

Using LLM-generated tags, we began systematically isolating messages that exhibited Sales Outreach characteristics across our global dataset. This produced a continuously growing, high-precision corpus of real-world examples, including confirmed malicious messages as well as borderline cases that traditional systems struggled to classify. From this corpus, we built a dedicated training pipeline.

First, we curated training data by grouping messages based on shared linguistic and structural traits identified by the LLMs. These traits included persuasive framing, manufactured urgency, transactional language, and subtle forms of social proof.

Next, we focused feature extraction on sentiment and intent rather than static indicators. The model learns how requests are phrased, how credibility is established, and how calls to action are embedded within otherwise normal business conversations.

Finally, we trained a purpose-built sentiment analysis model optimized specifically for Sales Outreach behavior. This avoided overloading a general phishing classifier and allowed us to tune precision and recall for this threat class.

image

Turning language into enforcement

The output of this model is a risk score that reflects how closely a message aligns with known Sales Outreach attack patterns. That score is evaluated alongside existing signals such as sender reputation, link behavior, and historical context to determine whether a message should be blocked, quarantined, or allowed.

This process is continuous. As attackers adapt their language, newly observed messages are fed back into the pipeline and used to refine the model without waiting for large volumes of user-reported misses. LLMs act as the discovery layer by surfacing new linguistic variants, while the specialized model performs fast and scalable enforcement.

This is what an all-out offensive looks like in practice. It is a feedback loop where large-scale language understanding drives focused, high-precision detection. The result is earlier intervention against a threat class that thrives on subtlety, and fewer malicious sales emails reaching the inbox.

Results of the undertaking

The visibility unlocked by LLM-driven mapping fundamentally changed how we improve detections. Instead of waiting for attackers to succeed and relying on downstream user reports, we gained the ability to identify systemic gaps earlier and address them at the source. This shift from reactive remediation to proactive reinforcement translated directly into measurable customer impact.

The most immediate signal of success was a marked reduction in customer friction. Sales Outreach–related phishing has historically generated a high volume of user-reported misses, largely because these messages closely resemble legitimate business communication and often evade traditional rule-based or reputation-driven systems. As our targeted models came online and were continuously refined using LLM-derived insights, fewer of these messages reached end users in the first place.

The data reflects this change clearly. Average daily Sales Outreach submissions — messages that we labeled as clean but were in fact Sales Outreach phishing emails, flagged by end users — dropped from 965 in Q3 2025 to 769 in Q4 2025, representing a 20.4% reduction in reported misses in a single quarter.

image

This reduction is not just a metric improvement; it represents thousands fewer disruptive moments per day for security teams and end users alike. Each avoided submission is a phishing attempt that was stopped before it could erode trust, consume analyst time, or force a user to make a security judgment mid-workflow. We have seen this trend continue in Q1 of 2026 with average daily submissions decreasing by two-thirds.

image

In effect, LLMs allowed us to “see” the planes that never made it back. By illuminating previously invisible failure modes, we were able to reinforce defenses precisely where attackers were concentrating their efforts. The result is a system that improves not only detection rates, but also the day-to-day experience of the people relying on it.

The next front in the arms race

Our work with LLMs is just beginning.

To stay ahead of the next evolution of attacks, we are moving toward a model of total environmental awareness by refining LLM specificity to extract forensic-level detail from every interaction. This granular mapping allows us to identify specific tactical signatures rather than relying on broad labels.

Simultaneously, we are deploying specialized machine learning models purpose-built to hunt for emerging, high-obfuscation vectors at the "fringes" that traditional defenses miss. By leveraging this real-time LLM data as a strategic compass, we can shift our human expertise away from known noise and toward the critical gaps where the next strike is likely to land.

By illuminating the "planes that didn't make it back," we are doing more than just reacting to missed email; we are systematically narrowing the battlefield. In the email arms race, the advantage belongs to the side that can see the invisible first.

Ready to enhance your email security?

We provide all organizations (whether a Cloudflare customer or not) with free access to our Retro Scan tool, allowing them to use our predictive AI models to scan existing inbox messages in Microsoft 365.

Retro Scan will detect and highlight any threats found, enabling organizations to remediate them directly in their email accounts. With these insights, organizations can implement further controls, either using Cloudflare Email Security or their preferred solution, to prevent similar threats from reaching their inboxes in the future.

If you are interested in how Cloudflare can help secure your inboxes, sign up for a phishing risk assessment here.