Fighting Fire with Fire: How GenAI is Increasingly Becoming Part of the Disease and the Cure for Financial Fraud

As we’ve seen with the recent devastation in Los Angeles, wildfires can cause massive destruction to homes, lives, and ecosystems, leading to long-term environmental and economic impacts. Paradoxically, controlled burns or "prescribed fires" have long been used by land management experts to prevent larger uncontrolled wildfires, promote ecological health, and manage forests by reducing excess vegetation. This dual nature also applies to the growing role of Generative AI (GenAI) in causing and combating financial fraud. The problem is not trivial; an estimated $3.1 trillion in illicit funds flowed through the global financial system and firms lost a collective $486 billion in 2023. On one hand, GenAI better enables malicious individuals to create and carry out sophisticated scams – such as a deepfake video from a CEO to employees requiring them to take immediate financial actions or provide sensitive company information. They can be challenging to discover and thwart. Conversely, the technology also offers financial institutions valuable tools to enhance their fraud prevention tactics and streamline compliance and reporting efforts.

As we navigate this double-edged sword, the challenge lies in harnessing this technology responsibly to protect assets and sustain trust in an interconnected world.

Record Fraud, Soaring Compliance Costs, and the Urgent Need for Robust Measures

Financial institutions continue to face record levels of fraud and financial loss. Federal Trade Commission data reports that U.S. adults lost a record $10 billion to fraud in 2023, a 12% increase over 2022. 98% of FIs have experienced increased financial crime compliance costs, which totaled $61 billion annually in the U.S. and Canada in 2023. Compliance costs continue to escalate amid an increasingly more challenging and complex regulatory environment. Penalties levied by US regulators on FIs in 2023 rose 69% year over year, highlighting the urgent need for FIs to strengthen know your customer (KYC) and anti-money laundering (AML) processes. In one high profile case, TD Bank was ordered to pay approximately $3 billion in a historic settlement with the U.S. DOJ in October 2024 due to lax AML practices.

The Threat Side: How GenAI Enables Fraud

GenAI has unfortunately empowered fraudsters to carry out more sophisticated and large-scale attacks against financial institutions and their customers. Here are some of the prominent ways fraudsters are leveraging GenAI for financial fraud:

• Sophisticated Attacks: GenAI can be leveraged by cybercriminals to create more convincing and hyper-realistic phishing attacks, deepfakes, and social engineering tactics, mimicking the language, style, and even tone of real messages from trusted entities. This involves generating content that is almost indistinguishable from legitimate sources, including fake voices and video calls, deceiving even the most cautious individuals and standard security systems. Such capabilities increase the likelihood of successful breaches, as these attacks can quickly adapt to overcome fixed security protocols.
• Automation of and Scalability: GenAI can produce a large volume of malicious codes, scripts, and malware variants at incredible speed, each slightly altered to evade detection systems. This capability not only overwhelms traditional cybersecurity defenses but also reduces the cost, effort and barriers to entry for attackers, making widespread attacks more feasible.
• Synthetic Identity Creation: GenAI can blend real and fabricated personal information to create synthetic identities, which can then be used to open bank accounts, apply for loans, or conduct other financial transactions without immediate detection. This type of synthetic fraud is challenging to catch because it appears legitimate to many detection systems.
• Intelligent Evasion Techniques: Using GenAI, attackers can create algorithms that learn from defensive patterns and adapt accordingly. This means that the more a system tries to defend against AI-generated attacks, the smarter these attacks become, perpetually evolving to bypass the latest defenses in real-time.

The Solution Side: How GenAI Defends Against Fraud

While GenAI has enabled more sophisticated attacks on financial institutions, it also provides those same firms with the following powerful tools and methods to enhance their defenses against fraud:

• Enhanced Monitoring and Detection: GenAI can analyze vast amounts of data far more efficiently than humans, recognizing anomalies and detecting signs of fraudulent activity with improved accuracy. By learning what constitutes normal behavior over time, AI systems can identify deviations that could indicate potential threats, often before they can have a significant impact. For example, banks like JPMorgan Chase use GenAI models to flag unusual spending or login behaviors, adding layers to their security architecture.
• Advanced Authentication: GenAI can enhance biometric authentication systems, such as fingerprint, facial or voice recognition, by improving their accuracy and reliability. These systems can be used to verify identities more effectively and reduce the risk of unauthorized access.
• Predictive Threat Intelligence: With predictive analytics, financial institutions can anticipate potential security breaches or fraudulent activities. AI analyzes trends and historical data to spot patterns that suggest imminent threats, allowing institutions to preemptively fortify their systems against emerging risks.
• Adaptive Defense Mechanisms: GenAI can be used to develop adaptive security frameworks that learn and evolve with emerging threats. These frameworks use every interaction to continuously update their threat models to defend against novel tactics and recognizewhen tactics are evolving in real-time.
• Synthetic Data for Model Training: A major challenge in fraud detection is the rarity of genuine fraud examples for model training. GenAI can generate synthetic examples, including new or rare types, enhancing the training of detection models and improving their ability to identify novel or complex schemes.

With the rise in GenAI adoption, regulators are closely monitoring how financial institutions incorporate the technology into their compliance and fraud prevention measures. For example, the U.S. DOJ Criminal Division issued a revised Evaluation of Corporate Compliance Programs in September with a particular focus on a company’s use of AI to determine whether their compliance program is well designed.

Harnessing GenAI: Navigating the Double-Edged Sword of Innovation

While GenI presents nuanced challenges in terms of cyber threats and fraud, it also equips financial institutions with advanced tools and methodologies for strengthening their defenses. The key lies in implementing AI responsibly and building adaptive frameworks that both anticipate risk and evolve with the digital threats of tomorrow. By doing so, financial institutions can leverage AI’s full potential to protect their assets and maintain trust in an increasingly interconnected and vulnerable digital world.

In summary, while GenAI equips fraudsters with new tools to scale and refine attacks, it also enhances financial institutions’ capabilities to counter these attacks. The dynamic nature of GenAI means that financial institutions must continuously evolve their fraud prevention techniques to stay a step ahead in this digital arms race.