Effective fraud prevention requires a delicate balance between stopping financial crime and minimising friction for legitimate customers. Typical detection architectures rely on a combination of predictive models, static rules, and human expert investigations to resolve complex cases. In this post, we discuss how LLMs create new operational efficiencies by automating unstructured data processing and streamlining back-office investigation tasks alongside human analysts.

In a previous post (Representation Learning for Enhanced Fraud Detection), we examined the core machine learning challenges we face in detecting and mitigating financial crime and fraud. Detection is fundamentally a needle-in-a-haystack problem, where we must isolate rare fraudulent transactions within vast streams of legitimate activity.

Misclassification is expensive: a missed fraudulent transaction costs money directly, while adding friction to legitimate activity results in a poor customer experience and operational costs for us. To effectively balance these tradeoffs, detection models must produce stable, predictable, and explainable outputs. These outputs are then governed within a broader optimisation framework where thresholds, controls, and trade-offs are calibrated to align model performance with risk appetite, operating costs, and customer impact.

Our core defence architecture relies on a combination of predictive rules and supervised machine learning models to counter rapidly changing adversarial fraud and financial crime typologies. To supplement these models, we gather detailed context directly from customers regarding the specific action or transaction they are attempting. This targeted information-gathering allows us to capture the nuanced, evolving signals that traditional, static data features miss.

When we cannot make a definitive automated decision, we engage expert fraud investigators to examine the case and reach a final resolution. Our human analysts evaluate the unstructured contextual data that our standard quantitative models cannot easily interpret. By streamlining the back-office administrative tasks involved in these reviews, we can deliver a seamless, low-friction experience for our customers. This naturally translates into broader operational wins—improved system throughput, lower operational overhead, and leveraging our expert investigators effectively.

To unlock these efficiencies, we see two significant opportunities for LLMs (GenAI) to enhance our operational capabilities:

A quick note on terminology: While we use the familiar term "LLM" throughout this article, we are referring to multimodal foundation models capable of handling inputs beyond just text.

We have a class of problem that recurs in many situations in financial crime and fraud prevention at Monzo: an action has been flagged by our financial crime systems as needing scrutiny — this might be a customer requesting a limit increase, reporting fraud, or our systems detecting an attempted fraud — and we need to engage the customer to understand what is happening. While the examples in this post use payments and transfer flows as illustrative examples, the discussion applies broadly to all actions that trigger our financial crime controls.

This information gathering process typically involves the customer answering some questions about the transactions they are making, such as who they are to or what they are for. We encourage customers to provide as much nuance and detail as they can. Sometimes customers also send us evidence to help us validate what’s happening, e.g. an invoice or a chat screenshot, or record this info in a voice note.

Until recently, incorporating this kind of information from the customer into our decisioning process required creating a task for one of our expert investigators to review and waiting (sometimes hours) for that task to be actioned. That created both friction and delays for customers and operational cost for Monzo, sometimes in high-criticality moments like making a big purchase.

But two advances in general-purpose LLMs over the past couple of years mean they are now well suited to extracting useful signals from customer-supplied information:

As such, LLMs are now very well suited to ingest semi-structured customer responses and multimedia evidence, together with our internal information about the customer and the beneficiary. They reason over it and produce a representation that can include free-form descriptive analysis as well as structured signals across predetermined dimensions like story cohesion, evidence relevance, key pieces of missing information, etc.

Importantly, we do not ask the LLM to decide whether a payment is fraudulent. Its role is to transform rich, unstructured customer interactions into structured signals that downstream supervised models use to estimate fraud risk.

As powerful as these models are at understanding and synthesising complex information, they remain relatively weak at producing well-calibrated quantitative assessments. LLMs can return qualitative measures like low/medium/high risk that broadly correlate with actual risk outcomes, but it's hard to push their accuracy beyond this level.

With traditional supervised machine learning modelling, on the other hand, we can use historical examples of fraud and financial crime to build highly predictive quantitative models to estimate a well-calibrated probability that a specific transaction is fraud.

We use a hybrid architecture—combining LLMs (Generative AI) with supervised machine learning—to get the best of both approaches. Here’s how it works in the context of our systems to detect and prevent fraud (we’re purposefully not adding much detail here because we know that fraudsters use these kind of details to circumvent our controls):

The whole Fraud Prevent agent process takes under 10 seconds, meaning customers can stay in their payment journeys and low-risk cases experience minimal friction or delay.

We’re running the Fraud Prevent agent live in the payment path at Monzo. We’ve seen it boost both the £ amount and number of authorised fraud cases we prevent by over 20%, while in parallel reducing the number of legitimate payments being diverted to our human review teams.

This is a big win all around. We’re keeping more customers’ money safe while freeing up our investigators for the higher-risk cases that need judgement, empathy, and human conversation.

We built Fincrime Agent from the ground up to be a reusable internal product we can quickly build and deploy into multiple use cases, recognising the multitude of situations where this hybrid approach can yield substantial benefits.

This enabled us to ship 3 additional use cases to production in the past year: the Payment Limit Increase Agent reviews customer requests to raise the amount of money they can spend in a day, and two new Fraud Protect agents automatically process information from customers reporting when they believe they’ve been a victim of fraud.

We’ve built these agents in Python, comprising 3 key components:

This productised approach has helped us to design and ship agents to production fast. In fact, we’ve found that it’s the step of understanding and aligning on the existing process, and how that should evolve to an effective agent-driven process, that really takes the time. And we’re now figuring out how to ensure agents can be independently owned and developed by domain-specific teams while maintaining these common components.

Combining LLMs with traditional supervised machine learning gives us the best of both worlds: flexible multimodal feature extraction paired with reliable, low-variance predictive power. While customer explanations and supporting evidence have always provided high-value signal for fraud investigations, the challenge has been turning that information into signals that machine learning systems can use at scale. LLMs solve this by providing a powerful way to extract signal from multimodal data, while supervised models remain the most effective way to turn those signals into reliable predictions. Combining the two lets us build systems that are both more capable and more accurate than either approach alone.

If what you’ve read here resonates and you’re passionate about making money work for everyone, we’re hiring machine learning engineers, data analysts, backend engineers, and many more roles across Monzo! Take a look at our careers page to see if we have the right role for you.