Purple Teaming In the Age of AI: New Threats and Tactics in 2025

As AI reshapes the cyber threat landscape, Purple Teaming is facing a new era of complexity. Large language models are lowering the barrier to entry for attackers—making it easier than ever to generate phishing kits, malware, fake access badges, and social engineering campaigns at scale. The result: faster attacks, broader reach, and adversaries who look very different from even a year ago. So how do security leaders stay ahead when the playbook is evolving in real time? In this expert-led session, Tevora’s Threat team breaks down how Purple Teaming is adapting to the age of AI. From offensive testing methods to defensive readiness strategies, we’ll walk through what’s changed, what’s emerging, and what your team needs to do next to stay resilient.

Key Takeaways:

• How AI is changing adversary behavior—and what that means for your threat model
• Real-world examples of AI-generated exploit kits, phishing sites, and malware
• The role of AI in physical intrusion and badge replication
• What’s shifted in malware and TTPs over the past year
• 2025 case studies of attacks driven or amplified by AI
• Practical defenses and Purple Team tactics your organization can implement now

Whether your team is building out a Purple program or looking to sharpen existing capabilities, this session provides a clear look at how to test, defend, and stay ready in the AI-driven threat landscape.

Purple Teaming In the Age of AI: New Threats and Tactics in 2025

What Is Purple Teaming?

Q: Before discussing AI, what exactly is purple teaming?

Miguel Martinez:
Purple teaming is often misunderstood as a rigid methodology, but in practice, it exists on a spectrum. Sometimes engagements lean more toward red team activities, while others focus more heavily on blue team validation.

At its core, purple teaming is a collaborative exercise designed to:

• Validate security tools and controls
• Evaluate detection and response capabilities
• Identify gaps in incident response processes
• Train defenders using realistic attack scenarios

Unlike traditional penetration testing, purple teaming emphasizes transparency and knowledge transfer. Security teams work alongside testers to understand how attacks occur, how they are detected, and how defenses can be improved.

The goal is not simply finding vulnerabilities—it is strengthening organizational readiness.

AI Is Amplifying Existing Threats

Q: How is AI changing the offensive security landscape today?

Jonathan Nyman:
One of the biggest misconceptions is that AI has completely reinvented cyberattacks. In reality, many of the techniques being used today are familiar.

What AI has changed is the scale and sophistication of those attacks.

Threat actors are now able to:

• Generate highly convincing phishing emails
• Automate social engineering conversations
• Create personalized content faster than ever
• Increase attack frequency with less effort

AI is effectively lowering the technical barrier to entry while improving attacker efficiency.

As a result, security teams are seeing larger volumes of more polished attacks that would have previously required significant time and expertise.

Social Engineering Is Becoming More Effective

One area experiencing significant growth is AI-powered social engineering.

According to the panel, modern phishing campaigns increasingly move beyond simple email scams.

Attackers now use AI-powered chatbots to:

• Initiate conversations via text messages
• Build trust over time
• Develop rapport before introducing malicious requests

Rather than immediately asking for credentials or payments, attackers may spend days or weeks fostering seemingly legitimate relationships.

This approach dramatically increases the chance of success because it exploits human trust rather than technical vulnerabilities.

Deepfakes: More Reality Than Hype?

Q: Are deepfake videos and impersonation attacks realistic threats today?

The answer, according to the panel, is nuanced.

While highly convincing video impersonation remains difficult—especially when targeting individuals who know the real person well—other forms of impersonation are already proving effective.

Jonathan highlighted a rapidly growing trend:

Employment Fraud

Threat actors are increasingly using:

• Face-swapping technology
• AI-generated identities
• Remote interview manipulation

to secure legitimate employment inside organizations.

Rather than attacking from the outside, attackers bypass security controls entirely by becoming insiders.

In these scenarios, AI is not being used to impersonate executives. Instead, it is being used to create entirely fictitious candidates capable of passing hiring processes and gaining access to corporate systems.

For many organizations, this represents a more immediate and practical threat than Hollywood-style deepfake scenarios.

Voice Cloning Is Already Here

While video deepfakes may still have limitations, voice cloning has become increasingly accessible and believable.

The panel discussed how voice cloning can be incorporated into security testing to evaluate:

• Employee verification procedures
• Help desk processes
• High-risk transaction controls
• Executive communication safeguards

However, this introduces important ethical considerations.

Security testing teams must balance realism with employee trust and organizational culture.

Despite these concerns, the speakers emphasized that organizations cannot afford to ignore these attack vectors simply because they are uncomfortable. Real-world attackers have no such limitations.

In many cases, testing these scenarios proactively is the best way to ensure employees and processes are prepared.

Sometimes Low-Tech Defenses Still Win

Ironically, some of the most effective defenses against AI-driven impersonation attacks are surprisingly simple.

The panel noted that organizations are revisiting techniques once associated with high-security government and military communications, such as:

• Shared verification phrases
• Out-of-band authentication
• Secondary approval processes
• Multi-person verification for high-risk actions

When trust becomes difficult to establish digitally, straightforward verification methods often remain highly effective.

The Security Risks of AI Chatbots

As organizations rapidly deploy AI-powered applications, a new class of security concerns is emerging.

Miguel Martinez:
One of the most common issues being uncovered during testing involves improperly secured AI integrations.

Typical findings include:

• Exposed API keys
• Prompt disclosure vulnerabilities
• Excessive permissions
• Weak access controls
• Exposure of user chat histories

The disclosure of chat history is particularly concerning.

Users increasingly interact with AI systems using sensitive information, including:

• Internal business data
• Troubleshooting details
• Proprietary information
• Operational workflows

If chat logs become exposed, organizations may face not only security issues but significant trust and reputational consequences.

How Tevora Tests AI Applications

Testing AI applications introduces a unique blend of technical assessment and social engineering.

According to the panel, assessments often begin by evaluating the guardrails protecting the model.

Testers may ask questions such as:

• Will the AI retrieve backend files?
• Can prompts be manipulated?
• Are hidden instructions exposed?
• Can permissions be bypassed?

From there, testers examine network traffic, application behavior, API requests, and prompt structures.

One of the most fascinating observations from the discussion was that successful assessments sometimes resemble traditional social engineering engagements.

In certain cases, testers have effectively “built trust” with an AI system and gradually persuaded it to reveal information it should have protected.

While unconventional, these interactions highlight how different AI security testing can be from traditional application security assessments.

How Attackers Are Using AI Behind the Scenes

While public discussion often focuses on ChatGPT and chatbots, attackers are increasingly leveraging AI for operational efficiency.

Examples include:

• Malware development
• Script generation
• Command-and-control infrastructure configuration
• Obfuscation techniques
• Automated analysis

Miguel explained that AI is not necessarily replacing expertise. Instead, it accelerates tasks that previously consumed valuable time.

A process that might take an experienced operator ten minutes can often be completed in seconds with AI assistance.

This efficiency gain allows attackers to move faster and iterate more frequently.

The Rise of MCPs and Specialized AI Models

One of the most forward-looking discussions focused on Model Context Protocol (MCP) and specialized AI systems.

Rather than relying solely on large, general-purpose models, organizations are beginning to connect AI directly to operational tools, such as:

• Splunk
• Security monitoring platforms
• Development environments
• Reverse engineering tools

This creates opportunities to:

• Query security data conversationally
• Simplify investigations
• Accelerate threat hunting
• Reduce manual analysis workload

At the same time, specialized AI models trained for specific cybersecurity functions may ultimately outperform broader, generalized AI systems.

According to the panel, this is likely where much of the industry’s innovation will occur over the next several years.

Should Security Teams Embrace AI?

The panel’s answer was clear: yes.

Organizations that ignore AI may eventually find themselves at a competitive disadvantage.

Much like the adoption of cloud computing or search engines, AI is becoming an increasingly fundamental technology.

The speakers acknowledged concerns around job displacement but emphasized that AI is also creating entirely new areas of expertise.

Emerging specialties now include:

• AI governance
• AI security
• Model management
• MCP development
• AI operations

For security teams, the challenge is not whether to adopt AI but how to do so safely and strategically.

Is Phishing Still Effective?

Despite advances in email security technology, phishing remains one of the most effective attack methods available.

AI is helping attackers:

• Generate large volumes of customized messages
• Adapt content quickly
• Bypass traditional patterns and signatures
• Create more convincing social engineering scenarios

At the same time, defenders are increasingly using AI to improve detection and reduce false positives.

As Jonathan described it, the result is a familiar cybersecurity reality: a continuous cat-and-mouse game between attackers and defenders.

Why Help Desks Remain a Prime Target

One of the most successful attack vectors continues to be the help desk.

Rather than exploiting software vulnerabilities, attackers frequently exploit human behavior.

Help desk personnel are trained to:

• Be responsive
• Solve problems quickly
• Assist users

Those same qualities can create opportunities for social engineering attacks.

Recent threat intelligence reporting has shown attackers increasingly focusing on:

• Password resets
• Account recovery processes
• Multi-factor authentication bypass attempts
• SIM-swapping attacks

The panel stressed that training and ongoing testing remain the best defenses against these tactics.

Can AI Replace Penetration Testers?

Short answer: Not yet.

The panel acknowledged that AI-powered penetration testing platforms continue to improve, but none currently match skilled human testers.

Today’s tools can help:

• Automate repetitive activities
• Identify low-hanging vulnerabilities
• Accelerate reconnaissance
• Improve efficiency

However, they still struggle with:

• Complex logic flaws
• Creative attack paths
• Contextual decision-making
• Safety considerations
• Advanced exploitation techniques

The consensus was that AI will likely become another valuable tool in the penetration testing toolkit—but not a replacement for experienced security professionals.

Key Takeaways

The discussion revealed several important realities for organizations navigating cybersecurity in the AI era:

• AI is enhancing existing attack techniques more than creating entirely new ones.
• Social engineering remains one of the most effective attack methods.
• Voice cloning and AI-enabled impersonation are practical threats today.
• AI applications introduce new risks around prompts, data exposure, and access control.
• Security teams should adopt AI thoughtfully rather than avoid it.
• Human expertise remains essential for advanced penetration testing and threat analysis.
• Purple teaming provides an effective framework for validating both defenses and response capabilities against emerging threats.

As AI continues to evolve, organizations that combine strong security fundamentals with continuous testing and adaptation will be best positioned to meet the challenges ahead.

About Tevora

Tevora helps organizations strengthen their security posture through penetration testing, red teaming, purple teaming, compliance services, and strategic cybersecurity consulting. By combining technical expertise with real-world threat intelligence, Tevora helps clients identify vulnerabilities, validate defenses, and improve resilience against modern cyber threats.