Advanced Phishing Detection: LLMs for Email Security

While modern detection systems like Microsoft Defender can detect various threats, some advanced phishing attacks still manage to bypass them. We highlight two real-world examples below that weren’t caught by Defender and landed in our inboxes. Thanks to LLMs for email security along with RAG models, Mesa’s detection was able to flag and quarantine these emails within seconds.

It’s also important to note that these advanced attacks aren’t limited to large enterprises. Organizations of all sizes, including small startups like Mesa (with only two full-time employees), are also targeted. In fact, small businesses often possess less sophisticated security measures, making them attractive for attackers. A recent survey indicated that nearly 43% of cyberattacks target small businesses, underscoring the need for robust phishing detection solutions using LLMs.

Wells Fargo Phishing

Here is what the threat actors did:

1. Sender Reputation hijacking: The email originated from a legitimate business whose email service was compromised. Since there was no prior threat intelligence on the sending domain, it didn’t raise any flags.
2. Authentication passed: Because the attacker used a trusted email service, all the usual checks (SPF, DKIM, DMARC) looked good — nothing seemed off.
3. URL Reputation hijacking: The link didn’t go straight to a phishing site directly but got redirected through a legitimate domain.

Image: Screenshot of live Wells Fargo phishing email

Mesa’s LLM flagged the email as phishing and provided details:

1. Mismatch between sender domain and claimed brand (Wells Fargo).
2. Intent of natural language claiming urgency.
3. Malicious link disguised as a button.

Image: Mesa’s scan results page

Microsoft Teams Phishing

This case serves as another example of how phishing attacks can be executed with sophistication. Similar to the previous case, this email also does sender reputation hijacking, passes all authentication checks, and does link redirection via a legitimate service.

As can be seen in the screenshot below, there are multiple levels of URL redirection that include an email marketing service icptrack[.]com, an anti-phishing service shared[.]outlook[.]inky[.]com and finally landing on the compromised domain popprogram[.]org.

Image: Screenshot of live Microsoft Teams phishing email

Mesa’s detection correctly identified:

1. Fake voice mail notification.
2. Microsoft as the brand being targeted.

However, it detected the threat as Callback Phishing, which is not the correct type as the link is meant for Credential Phishing. This likely happened because the email content talks about ‘voice message’, showcasing the importance of context for LLMs. Nevertheless, the email was marked as phishing and that triggered remediation step to quarantine it and alert the IT team.

Image: Mesa’s scan results page

In both cases, we reached out to the owners of the compromised domains to let them know that their infrastructure was being used to send phishing emails. We re-verified the URLs after a week and found that both of them had been neutralized. Taking these proactive steps is crucial in combating phishing attacks and fostering a more secure digital environment.

These emails underscore the complexity surrounding today’s phishing landscape. The older generation of rule-based anti-spam systems are simply not equipped to detect these as they require constant manual update. The newer generation of LLMs provide surprising accuracy right out of the box. Combining them with additional context using RAG models is the most promising way to get ahead of such threats.