AI-Powered Phishing: How Cybercriminals Are Using Language Models Against Your Business

AI-Powered Phishing — Phishing emails used to be easy to spot. Awkward grammar, misspelled words, generic greetings, implausible scenarios. Security awareness training built on teaching users to recognize these signals. That training is now significantly less effective, because the signals have disappeared. Large language models have made it trivially easy for threat actors to generate grammatically perfect, contextually relevant, personalized phishing messages at scale — and the volume of attacks is accelerating.

What Changed and When

Prior to widely available LLMs, convincing phishing campaigns required either significant manual effort (limiting their scale) or accepting lower quality (making them easier to detect). The barrier was the writing itself — producing hundreds of contextually appropriate, well-written emails targeting specific individuals required human effort that didn't scale economically for most threat actors.

That barrier is gone. Using commercially available or open-source language models, an attacker can now generate thousands of individualized phishing emails in the time it previously took to write one. The personalization that used to distinguish elite, targeted spear-phishing campaigns from mass phishing is now accessible to low-sophistication attackers.

What AI-Generated Phishing Looks Like in Practice

The attacks our clients have encountered in the past 18 months share a few characteristics that distinguish them from traditional phishing:

• Perfect grammar and natural tone — no spelling errors, no awkward phrasing, no translation artifacts
• Accurate context — references to real vendors, real job titles, real recent events derived from LinkedIn profiles, company websites, and press releases that are publicly accessible
• Appropriate urgency calibrated to role — finance team members receive payment-related urgency; IT staff receive credential reset or security incident themes; executives receive board-level or regulatory scenarios
• Consistent persona maintenance — multi-message campaigns that build familiarity before requesting action

Business Email Compromise (BEC) attacks — where the goal is fraudulent wire transfers or credential theft rather than malware delivery — have seen the sharpest increase. These attacks often involve no malicious links or attachments, which means traditional email filtering has limited effectiveness against them.

Why Traditional Security Awareness Training Is No Longer Sufficient Alone

Security awareness training that teaches users to look for grammar errors, suspicious links, and unusual sender addresses is still worth doing — but it's no longer the primary control it once was. The threat model has changed. Users who successfully identify phishing based on writing quality will miss AI-generated attacks that have none of those signals.

Updated training needs to focus on:

• Process-based verification — any financial transfer or credential change request is verified through a separate, pre-established channel regardless of how legitimate the email appears
• Behavioral skepticism — urgency, pressure, and unusual requests are red flags regardless of who appears to be asking
• Role-specific scenarios — finance, IT, and executive staff face different attack patterns and need training that reflects their specific threat exposure

Technical Controls That Help

No single control stops AI-powered phishing, but layering these significantly raises the cost for attackers:

• DMARC, DKIM, and SPF enforcement — reduces spoofed sender addresses impersonating your domain or trusted vendors
• Advanced email security with behavioral analysis — products that analyze communication patterns and flag anomalies, not just signature-based filtering
• MFA on all accounts — limits the impact of credential harvesting even when phishing succeeds
• Conditional access policies — restricts account access from unusual locations or devices even with valid credentials
• Phishing-resistant MFA — FIDO2 or passkey-based authentication that isn't susceptible to real-time phishing proxy attacks

If your organization's email security and security awareness program hasn't been updated in the past 12 months, it's worth a review. Leonidas offers email security assessments and can help you understand where your current controls have gaps relative to the current threat landscape.