Agentic RAG: The Future of Email Security and Automation

Email remains the lifeblood of modern business communication, yet it is also one of the most underutilized treasure trove of data. It holds valuable insights and timelines of business operations, security incidents, and compliance records. It is also the primary attack vector of cyberthreats, including advanced phishing (QR code, callback phishing etc.), Business Email Compromise, Malware and various zero-day threats. Even though most of the spam today is still blocked using reputation based systems based on sender IP or domain, the advanced attacks go largely undetected.

Large enterprises often have resources to tackle these challenges. However, small-to-medium businesses struggle due to lack of dedicated expertise, leading to serious cyberattacks and operational bottlenecks. According to a research by Okta, 65% of US SMB owners rank cyberattacks as a top concern.

Enter Agentic RAG (Retrieval-Augmented Generation)—a game-changing approach that leverages AI agents, external tools, and intelligent workflows to revolutionize email security and automation. This is truly the time where every business, regardless of the size, can have access to enterprise-grade security expertise without breaking the bank.

What is Agentic RAG?

Agentic RAG combines retrieval-augmented generation with autonomous agents capable of executing tasks and making decisions. Unlike traditional RAG models, which primarily enhance response generation by pulling relevant knowledge from external sources, an Agentic RAG system goes further by dynamically interacting with APIs, security tools, and databases to automate complex workflows. For example, such a system can automatically figure out whether it needs information from your in-house database, third-party technical documentation, web search etc. based on a user’s query.

Let us illustrate this with an example demo. In the video below, you’ll see how a sysadmin would respond to phishing incident in this new world of Agentic RAG.

There are three important parts to the video:

1. Phishing detection using multi-modal LLMs with detailed summary. We’ve made a version of this scanner available here.
2. Recommendations generated dynamically by the agent based on the event.
3. Integration and actions performed directly on systems like Microsoft Exchange.

Typically, a sysadmin would roughly take total of 30 minutes to perform the above actions. This would involve finding the phishing email via MSFT message trace, parsing sender domain, and then blocking it manually (or using PowerShell) in Exchange. With the new Agentic workflow, this can be done in less than a minute.

It is important to note that this system also integrates third-party technical documentation as part of its RAG model. Therefore, it can figure out what commands to run for a particular action on-the-fly, e.g. blocking a domain in MSFT Exchange based on their documentation.

Here is a sample architecture of what such a system looks like in practice.

Mesa Agentic RAG Architecture

At Mesa, we’re pioneering this custom architecture to automate key security workflows, with latest LLMs, Agentic frameworks, and tools. In the following weeks, we will be publishing a series of blog posts covering details of the tech stack, software libraries, datasets and more details of the technical implemenation.

Security and Privacy

If you are a security practitioner like us, you likely have several questions around how such a system would handle sensitive data, especially when it comes to calling AI models.

There are key design requirements that must be met as part of such a system:

1. No external LLM queries: This means all API queries are made to locally deployed multi-modal LLMs like Llama 3.2 or 3.3. The other alternative is tenant-scoped environment like AWS Bedrock. It ensures that no customer data leaves our cloud environment or is used for any training.
2. No sensitive email data stored: The high precision of multi-modal LLMs ensures that only email threats or operational emails are stored to automate user’s workflows. Anything that is not marked by the detection system is never stored in the database.
3. Proper tenant scoping: Within our environment, all tenant data is scoped only to the relevant tenant and not shared with other tenants. This ensures, no user can inadvertently access other tenant’s data.
4. Full control to review/delete data: Controls are baked into the product to provide visibility of all of tenant’s data with full control to delete all of it.
5. Microsoft or Google’s SSO login combined with app-based 2FA is the preferred way to access the product.
6. Compliance: Pentesting, third-party audits, SOC2 certification and several other key controls to ensure we’re meeting industry standards. The OWASP Top 10 for LLMs is also a great resource.

Going Beyond Email Threats

Email security, as described above is just one piece of the puzzle. Once proper security and privacy controls are in place, there are several workflow automations that can be achieved with the same system:

Security Alert Investigation

• AI agents monitor security alerts and automatically investigate incidents.
• Cross-referencing security logs, threat intelligence feeds, and email metadata helps determine if an alert is a false positive or a real threat.
• If a genuine threat is detected, the system can recommend or trigger automated remediation steps, such as isolating a compromised account or blocking a malicious sender.

Provisioning New User Access

• When a new employee joins, AI-driven workflows can handle onboarding requests coming in via email tickets.
• Automated checks ensure that the correct permissions and role-based access controls (RBAC) are assigned based on company policies.
• Emails confirming access provisioning can be sent to IT admins and the new employee, reducing manual intervention.

Configuring New Data Policy

• AI agents analyze company compliance needs via internal documents and suggest appropriate data policies.
• Once approved, the system can either automatically configure email security rules, such as data loss prevention (DLP) settings.

Threat Detection and Response

• AI agents analyze incoming emails for phishing by doing deep content analysis with vision and language models, and then combine that with communication patterns.
• Suspicious emails are automatically flagged, quarantined, or sent for further review.
• AI-driven responses educate users on why an email is risky, reducing human error.

Automated Customer Support & Ticketing

• AI-driven email parsing extracts key details from customer inquiries.
• Automated responses provide initial troubleshooting or FAQs.
• If further action is needed, emails are converted into support tickets and routed to the appropriate team.

Reporting and Analytics

• AI-driven reporting provides insights into email security trends, workflow efficiency, and compliance status.
• Users can generate any query in simple natural language and the system should be able to query the right systems, transform the output in desired format and provide it to the user. No custom backend API development needed for each use-case.

Key Requirements to Democratize Agentic Security

For an Agentic system to be truly transformative and accessible to SMBs and enterprise alike, it must embody usability, affordability, reliability, and data privacy:

Cost-effective and Scalable – should offer flexible pricing models, like subscription-based tiers to accommodate businesses of all sizes. It should help automate at least 60% of manual work to make business sense for the buyer.

No-code, Easy-to-use UI – should offer an intuitive, engaging, and fun interface so that non-technical users can set up and deploy workflows with minimal effort.

High accuracy – should have continuous learning and adaptive AI models to refine workflows and improve precision on ongoing basis.

Transparent, Explainable actions – with the latest generation of LLMs (e.g. DeepSeek R1), explainability needs to be baked into the framework. This fosters user trust and allows for effective auditing of security measures.

Seamless Integration with External Tools – with rising number of SAAS applications, integration with other tools is necessary. It is a requirement that Agentic systems handle right out of the box for the user with minimal work by vendor.

Data Privacy – to be a truly global product, security of sensitive corporate data is a mandatory requirement. It also needs to comply with privacy laws of different countries.

The Path Ahead

It’s day one at Mesa, in our journey to democratize advanced email security and automation for everyone. We’re thrilled by the speed at which Agentic AI innovation is moving and by the opportunities it presents in security.

As we create new, game-changing products we welcome your feedback and ideas at contact@mesasecurity.com

References:

https://weaviate.io/blog/what-is-agentic-rag

https://www.okta.com/press-room/press-releases/north-american-smb-face-significant-cyberattack-challenges/