Artificial Intelligence as a Double-Edged Sword: Cyber Attacks and Defenses in the Age of Machine Learning

Article written by: Adrian Bermudez

Designed by: Adrian Bermudez and Natasha Gumpula

One of the most transformative forces in modern cybersecurity is now artificial intelligence (AI); strengthening defenses (improving an organization’s ability to protect itself), accelerating detection (speeding up the process of finding a cyber attack), and automating responses (taking action on its own once a threat has been detected). Yet, the capabilities of cybercriminals and state-sponsored actors have also expanded due to the use of AI. The same machine learning tools that help organizations identify threats can also be used to generate new ones. Highlighting the dual-use nature of AI, which describes the very thing that makes AI a genuine double-edged sword.

(Image by Steve Johnson, Unsplash)

The Offensive Use of AI: How Attackers Exploit Machine Learning

Artificial intelligence has now been adopted by cyber attackers with remarkable speed. One of the clearest examples of this shift is the growing use of AI for social engineering, with large language models producing highly tailored phishing messages that mimic the tone, vocabulary, and writing style of real individuals. Attackers are more inclined to use AI, as security researchers have repeatedly shown that these AI-generated messages can outperform traditional phishing attempts by appearing more natural/personalized. Moreover, another major threat involves AI-generated malware. Machine learning models now possess the ability to rewrite malicious code in such a way that it bypasses signature-based antivirus programs. In addition, cybersecurity labs have demonstrated that generative models can quickly produce polymorphic malware that alters its structure each time it executes, making detection significantly harder.

AI also enables adversarial attacks, which involve subtle manipulations of data to mislead defensive systems. Researchers studying adversarial machine learning have demonstrated that adding small, almost invisible perturbations to inputs can force a machine learning system to misclassify malware as safe or classify malicious network traffic as ordinary behavior, damaging the reliability/reputation of AI-driven intrusion detection systems. All while illustrating how attackers are learning to fight artificial intelligence with artificial intelligence.

Finally, AI lowers the barrier to entry for cybercrime. Tasks that once required technical expertise (e.g., crafting exploit scripts or building ransomware) can now be assisted or accelerated by widely available AI tools. This democratization/ubiquity of AI attack capabilities is now a growing concern for security agencies worldwide and for the average person.

The Defensive Use of AI: Strengthening Cybersecurity from the Inside Out

Although AI is known to amplify cyber threats, it also offers defenders powerful tools to combat attacks; machine learning systems are now central to modern intrusion detection because they are now able to learn/adapt to what normal behavior looks like within a select network (instead of relying solely on predefined rules) and detect anomalies that would otherwise be invisible to traditional monitoring tools. Additionally, AI is equally beneficial in incident response. Automation allows security platforms to isolate suspicious activity, quarantine files, and prioritize alerts without waiting for human intervention. This allows organizations to react faster than ever before, especially when dealing with extremely large volumes of data.

In artificial intelligence research, cybersecurity teams use AI to simulate attacks and test defensive strategies; generating synthetic attack traffic, therefore, helps organizations that use AI to strengthen their resilience and evaluate the effectiveness of their systems under realistic cyberattack conditions. Lastly, adversarial training is another key defensive strategy. Intentionally exposing models to adversarial inputs during training, cybersecurity engineers make their detection systems more robust against manipulation. While this approach is not yet foolproof, it represents a major step toward more reliable AI defenses for all.

The Strategic Tension: An Increasingly Escalating AI Arms Race

The aforementioned interaction between offensive and defensive AI creates an infinite feedback loop that shapes the cybersecurity landscape. As defenders continue to develop better detection and response systems, attackers must perpetually continue to create more sophisticated methods in order to evade them. This leads to each side pushing the other to innovate, resulting in rapid escalation.

This dynamic raises several challenges: as AI systems often operate as black boxes, meaning that security teams may not fully understand how a model reached a particular decision. This lack of transparency complicates trust, oversight, and accountability. Also, there exists a resource gap and security concerns between organizations with robust AI capabilities and smaller institutions that lack the necessary expertise, infrastructure, or correct policies to deploy advanced security technologies. 

Giving rise to public policy and governance which are becoming increasingly important in AI. As governments and research organizations have begun discussing frameworks for responsible AI use, especially when AI is capable of generating code, analyzing vulnerabilities, or performing autonomous decision-making. These discussions remain ongoing, and the pace of technological change continues to overtake current regulatory structures.

Looking Ahead: Balancing Innovation and Responsibility

In summary, AI will maintain its role as both a vital defense mechanism and a powerful tool for cyber attackers. Leaving the future of cybersecurity to depend on how well institutions balance innovation with responsibility. Human expertise will remain essential, as AI lacks the contextual judgment required for high-stakes decisions; a combination of automated detection, human oversight, and a continuous blend of adversarial testing/transparent governance will also be necessary to manage risk.

The double-edged nature of AI is not a reason to not use it. Instead, it is a reminder that technological power must be matched with careful design, ethical consideration, and a commitment to resilience. If used responsibly, AI has the potential to strengthen global cybersecurity far more than it threatens it.

Citations

Adaptive Security. "Deepfake Phishing: The Next Evolution in Cyber Deception." Adaptive Security, 19 Sept. 2025, https://www.adaptivesecurity.com/blog/deepfake-phishing 

Anthropic. "Detecting and Countering Misuse of AI: August 2025." Anthropic, 27 Aug. 2025, https://www.anthropic.com/news/detecting-countering-misuse-aug-2025 

Brightside Team. "AI-Generated Phishing vs Human Attacks: 2025 Risk Analysis." Brightside, 24 Oct. 2025, https://www.brside.com/blog/ai-generated-phishing-vs-human-attacks-2025-risk-analysis 

Business Insider. "Anthropic Says Chinese Hackers Used Its AI for Cyberattack." Business Insider, 14 Nov. 2025, https://www.businessinsider.com/anthropic-chinese-hackers-claude-agentic-ai-cyberattack-2025-11 

Centre for the Ethics of Technology and Security (CETaS). Janjeva, Ardi, Anna Gausen, Sarah Mercer, and Tvesha Sippy. "Evaluating Malicious Generative AI Capabilities: Understanding Inflection Points in Risk." CETaS Briefing Papers, July 2024, https://cetas.turing.ac.uk/publications/evaluating-malicious-generative-ai-capabilities 

Ibrar, Werisha, Danish Mahmood, A. S. Al-Shamayleh, et al. "Generative AI: a Double-Edged Sword in the Cyber Threat Landscape." Artificial Intelligence Review, 2025, DOI: 10.1007/s10462-025-11285-9, https://link.springer.com/article/10.1007/s10462-025-11285-9 

Kosinski, Matthew. "How to Fight AI Malware." IBM Think, IBM, n.d., https://www.ibm.com/think/insights/defend-against-ai-malware 

IBM X-Force. X-Force Threat Intelligence Index 2025. IBM, 16 Apr. 2025, https://www.ibm.com/thought-leadership/institute-business-value/en-us/report/2025-threat-intelligence-index 

Monetary Authority of Singapore. "Cyber Risks Associated with Generative Artificial Intelligence." Monetary Authority of Singapore, 30 July 2024, https://www.mas.gov.sg/-/media/mas-media-library/regulation/circulars/trpd/cyber-risks-associated-with-generative-artificial-intelligence.pdf 

Rosenberg, Ishai, Asaf Shabtai, Yuval Elovici, and Lior Rokach. "Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain." arXiv, 5 July 2020, https://arxiv.org/abs/2007.02407 

Rosenberg, Ishai, Asaf Shabtai, Yuval Elovici, and Lior Rokach. "Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain." Semantic Scholar, https://www.semanticscholar.org/paper/Adversarial-Machine-Learning-Attacks-and-Defense-in-Rosenberg-Shabtai/2e73516bb02f8ce1caa05e1984ace7ce14d1727b 

Tom's Hardware. Morales, Jowi. "Anthropic Says It Has Foiled the First-Ever AI-Orchestrated Cyber Attack, Originating from China." Tom's Hardware, 14 Nov. 2025, https://www.tomshardware.com/tech-industry/cyber-security/anthropic-says-it-has-foiled-the-first-ever-ai-orchestrated-cyber-attack-originating-from-china-company-alleges-attack-was-run-by-chinese-state-sponsored-group 

The Guardian. "CEO of World's Biggest Ad Firm Targeted by Deepfake Scam." The Guardian, 10 May 2024, https://www.theguardian.com/technology/article/2024/may/10/ceo-wpp-deepfake-scam 

ZeroThreat. "Deepfake Attacks & AI-Generated Phishing: 2025 Statistics." ZeroThreat, 28 Aug. 2025, https://zerothreat.ai/blog/deepfake-and-ai-phishing-statistics