Finally, an AI That Understands Money, Part Three: Teaching It What Is Out of Character

A transaction is out of character when it deviates from patterns a pretrained foundation model has learned from a customer historical behavior sequences. Because the model computes a behavioral embedding in advance and refreshes it continuously, it can flag anomalous payments instantly by comparing each new transaction against the established representation without assembling context under latency pressure at decision time.A transaction is out of character when it deviates from patterns a pretrained foundation model has learned from a customer historical behavior sequences. Because the model computes a behavioral embedding in advance and refreshes it continuously, it can flag anomalous payments instantly by comparing each new transaction against the established representation without assembling context under latency pressure at decision time.

Abnormal is always relative. An embedding computed in advance can hold an inbound payment before it arrives, and the same property explains where this approach fails.

This is the last of three parts. Part one explains what Revolut’s PRAGMA is and how it works. Part two covers credit and financial inclusion.

Where the first two parts left off

Revolut and NVIDIA published a transaction foundation model, a single pretrained backbone that learns customer behaviour from banking event sequences and then serves many downstream models. Crucially, the representation it produces is a function of a customer’s past, which means it can be computed in advance and refreshed continuously rather than assembled under a latency budget at the moment a decision is needed.

That property is what makes everything below possible, and it matters more for fraud than for anything else.

1. Redefining “unusual”, one customer at a time

Fraud is where the conceptual shift is sharpest, because a rule written by an expert and a bureau score share a hidden assumption: they define normal as a property of the population. A rule that flags any card transaction above a set amount in a country the customer has not visited applies the same threshold to a pensioner and to a contractor who buys materials in bulk. The threshold is a compromise between them, which means it is wrong for both, and the cost of that compromise is paid in false positives on one side and missed fraud on the other.

A model pretrained on sequences defines normal as a property of the person. A payment of R6,400 at three in the morning is unremarkable for somebody whose history contains hundreds of late night transactions and genuinely alarming for somebody whose history does not, and the model holds both of those judgments simultaneously without anybody having to write either of them down. This is what the masking objective is quietly training for, since a model that has learned to predict a hidden amount from the merchant, the hour and the surrounding pattern is by construction a model that knows what this particular customer’s next transaction ought to look like. A large gap between what it expected and what arrived is an anomaly score calibrated to one individual rather than to an average.

The behavioural sources widen this considerably. Because app navigation sits in the same sequence as the payments, the model sees the journey and not just the destination. A genuine large payment is usually preceded by a plausible sequence of screens, whereas an account takeover often carries a distinctive signature, perhaps a credential change, then a new beneficiary, then a limit increase, then a transfer, compressed into a few minutes. A rule engine can catch that only if somebody anticipated the exact pattern and wrote it down, by which point the fraudsters have moved on. Rules are also probed and learned by the people they are meant to stop, in a way that a representation of individual behaviour is much harder to reverse engineer.

The paper’s fraud numbers are worth reading closely for one specific reason. Recall improved 64.7 percent and precision improved 16.7 percent at the same time, and these two normally trade against each other, so a threshold change cannot explain both moving in the same direction. That pattern is the signature of a model that has genuinely separated the classes better rather than one that has simply been retuned. The profile branch again matters a great deal here, contributing 85.6 percent on recall in the ablation, which suggests that knowing how long somebody has been a customer and how they were onboarded is central to judging whether their current behaviour is out of character.

There is an operational dimension too. Fraud patterns move faster than any other risk in a bank, and a LoRA adapter can be retrained in twelve hours to a few days on a backbone that already understands transactional behaviour, rather than requiring a fresh cycle of feature engineering every time a new attack appears.

2. The mule account, and precomputing a pause rather than scoring a payment

The single most valuable application of this idea may be the one the paper never mentions, and it follows directly from the fingerprint being a property of the past rather than of the transaction in front of you.

A mule account has a behavioural signature so distinctive that it stands out from a considerable distance, and the reason is that a mule account is defined by an absence rather than a presence. There are no groceries. There is no debit order for a policy that renews every year, no salary landing on roughly the same day each month, no slow drift of discretionary spending, no idle browsing of the savings tab, no curiosity about products the customer will never buy. What the account has instead is flow, meaning credits arriving from counterparties with no relationship to one another and debits or cash withdrawals leaving within minutes, so that the balance returns to something near zero and stays there until the next cycle. Amounts are often round, sometimes suspiciously close to a reporting threshold, and engagement with the app rarely extends beyond the screens required to move money out. A conduit does not look like a life, and the gap between those two things is enormous once you have a representation capable of describing either.

The important structural point is that none of that is a network signal. Section 3 explains why PRAGMA struggled with money laundering, which is that laundering emerges from relationships between accounts and the published architecture reads one customer at a time. That limitation genuinely bites for laundering networks and it does not bite here at all, because the question “is this individual account behaving like a conduit rather than a life” is answerable entirely inside one history. It is exactly the shape of question a per customer sequence encoder is built for. Anyone who reads the AML disappointment as ruling out this family of use cases has drawn the wrong boundary, and the distinction is worth being precise about because mule identification is usually managed by the same function that owns laundering controls.

Now put that together with precomputation, because this is where the operational model inverts. Today an arriving credit is assessed at the moment it lands, by rules that know almost nothing about the account receiving it, under a latency budget that permits almost no thinking. The alternative is not to score the payment faster. It is to stop scoring the payment at all and instead maintain a standing posture for the account, computed in advance and refreshed continuously, so that an account currently exhibiting conduit behaviour already carries a disposition that says hold inbound credits for review. The decision precedes the transaction. The payment simply inherits a judgment that was made before it existed.

The reason this matters far more than ordinary fraud scoring is that it protects somebody who is not your customer. In an authorised push payment scam the victim genuinely authorised the payment, which means there is no fraud signal on the sending side to find, because from the sending bank’s perspective a customer did exactly what they intended to do. The only place the signal exists anywhere in the system is in the behaviour of the account receiving the money. The recovery window is measured in minutes, since funds that leave a mule account are effectively unrecoverable, and a posture that was computed hours earlier is the only instrument that can act inside a window that short.

The economics of this changed sharply and recently. The United Kingdom’s mandatory reimbursement regime, in force since October 2024, splits liability for authorised push payment fraud fifty fifty between the sending and receiving providers, capped at £85,000 per claim, which means a receiving bank now absorbs half of a loss suffered by somebody it has never had a relationship with. That single change converted receiving side mule detection from a compliance obligation into a direct charge against profit, and the institutions that invested in it have reported substantial improvements, with Metro Bank publicly citing a 71 percent increase in mule account detection. South African rules differ, but the direction of regulatory thinking on who should carry scam losses is not moving in the opposite direction anywhere.

The strongest argument of all is the conversion case. A great deal of mule activity does not involve accounts opened for the purpose. It involves ordinary accounts belonging to ordinary people, often students or the unemployed, recruited for a small fee, and those accounts carry a genuine history of a genuine life right up until the week they start behaving differently. A threshold looks at such an account and sees an established customer in good standing. A model conditioned on that person’s own history sees something far more informative, which is a change point, where the rhythm of a life is abruptly replaced by the rhythm of a conduit. Detecting a departure from a personal baseline is precisely what this architecture does and precisely what a population level rule cannot do, because to the rule the new behaviour is simply behaviour.

Two constraints belong with this immediately. Holding somebody’s incoming money is among the most intrusive things a bank can do to a person, and a genuinely new customer receiving a first salary looks thin, empty and recently activated, which is uncomfortably close to how a fresh mule looks. Thinness therefore cannot be the trigger. The discriminating signal has to be the flow through ratio and the velocity with which credits leave, meaning how much of what arrives stays, rather than the mere absence of history, and getting that wrong imposes real hardship on people who have done nothing at all. The legal basis for holding a credit also varies by jurisdiction and is a question for counsel rather than for a model. Beyond that, mule operators adapt, introducing dwell time and small plausible purchases to imitate a life, so this is an arms race rather than a solved problem. The advantage of a learned representation over a written rule is simply that a LoRA adapter can be retrained on new patterns in days.

3. Where it fails, and why that is the most useful part of the paper

On one internal benchmark, the published version of PRAGMA scored 47.1 percent below the task specific baseline on F0.5 when applied to money laundering detection, and the authors publish this plainly while offering two reasons for it.

The wording there is deliberate and worth pausing on. This is not a measurement of Revolut’s production money laundering capability, which the paper does not describe at all, and it is not a claim that PRAGMA is generally poor at AML work. What it exposes is a structural limitation in applying an encoder that reads one customer at a time to a problem that is relational by nature.

The first reason is that the AML dataset was simply large enough that pretraining contributed nothing, because with abundant labels available the baseline built for that task could learn robust representations entirely on its own. This is the caveat about foundation models that tends to get forgotten, namely that their advantage is largest precisely where labels are scarce, and that a well tuned specialist remains hard to beat wherever you already hold millions of labelled examples.

The second reason, which the authors describe as the more important of the two, is that AML is fundamentally a relational problem. Money laundering is not a pattern living inside one person’s history but a pattern spread across accounts, showing up as circular flows, mule networks, or structuring split deliberately between several parties. The production baseline draws on features computed across records that capture this signal at the level of the network, whereas PRAGMA as published reads each customer’s history in isolation and has no idea that the other accounts even exist. This is where the detective analogy from the opening finally breaks down, because PRAGMA is a superb reader of individual diaries and money laundering is a conspiracy, and you cannot detect a conspiracy by reading one diary very carefully, however skilled a reader you happen to be.

Two things about how that number was produced are worth carrying forward. It came from a frozen linear probe on PRAGMA L embeddings rather than the LoRA fine tuning that generated every win discussed above, and since LoRA beats frozen embeddings by up to 73 percent elsewhere in the same paper, the AML comparison is not being measured on the same footing as the rest. The metric also matters, because F0.5 deliberately weights precision above recall, which means the gap of 47.1 percent describes a model that is worse at flagging cases cleanly rather than one that is necessarily missing more laundering overall.

The authors’ stated remedy is to extend PRAGMA itself so that it captures interactions across records, and they name this as a key direction for future work. Either way the practical reading is the same, in that anybody who finishes this paper believing they have found one model to rule them all has read it incorrectly, because a sequence model that reads one customer at a time is not the right instrument for a network problem.

One qualification stops that conclusion from being drawn too widely, and it matters because financial crime teams tend to own several quite different problems at once. The limitation here is specific to tasks whose evidence lives between accounts, and not every task in this domain has that shape. Identifying whether a single account is behaving as a conduit rather than as somebody’s financial life, as described in section 2, is answerable entirely within one history, and a per customer encoder is unusually well suited to it. The correct reading of the AML result is therefore that relational problems need relational components, not that this architecture has nothing to offer financial crime.

4. The question the paper does not answer

There is a silence running through all forty or so pages of this paper that anybody working inside a regulated bank will notice immediately, and it concerns governance.

The headline use case here is credit scoring, which means a model of a billion parameters producing predictions about whether a customer will default. In most institutions that is not primarily a modelling problem at all; it is a problem of model risk and explainability, and it is one that has killed a great many technically excellent projects. If a declined applicant asks why they were declined, somebody has to answer, and in a number of jurisdictions they have a legal right to that answer. If the model risk committee asks which factors drive the score and how stable those factors are across protected groups, somebody has to answer that too. Neither question is comfortable to face holding an embedding of 1,024 numbers and a transformer someone has fine tuned.

The paper says essentially nothing about any of this. It contains a single line confirming that all data used was fully anonymised and free of personally identifiable information, and one further acknowledgement in the introduction that practical deployments must operate under privacy and regulatory constraints which limit both what can be reported and which features may be used for certain decisions. Beyond those two sentences there is no discussion of feature attribution, no reasoning about adverse action, no testing for fairness or disparate impact, and no reference to any framework for model risk governance whatsoever.

That silence should be read carefully rather than as an oversight, because it is a research paper and not a deployment document, and none of those omissions is a defect in the work itself. The silence does, however, tell you exactly where the unfinished business sits, and it goes some way towards explaining why the paper reports no production deployment. It also suggests a sensible sequencing for anyone tempted to follow this recipe internally, which is to prove the idea somewhere the explainability burden is light, whether that means fraud triage, marketing targeting, servicing prioritisation or the detection of recurring payments, rather than opening on credit decisioning, where the governance conversation will consume more calendar time than the modelling ever did.

5. Shared intelligence must not become shared decision authority

There is one principle that the rest of this article implies without ever stating, and it is important enough to say on its own line. A shared backbone centralises learning. It must not centralise control.

5.1 The risk compounds for exactly the same reason the benefit does

The appeal of one representation feeding many models is that learning is shared, so an improvement in the backbone lifts everything downstream at once. The uncomfortable corollary is that a defect in the backbone also propagates everywhere at once. A fault in tokenisation, a skew introduced by the training window, temporal drift as customer behaviour moves away from the pretraining period, or instability in the embeddings themselves would not degrade one model. It would degrade credit, fraud, recommendation and engagement simultaneously, and it would do so in ways that look like six unrelated problems until somebody thinks to check the common input.

This is the most important banking implication of the whole architecture and it is barely discussed anywhere. Six independent models with six independent feature pipelines are wasteful, but that waste buys something real, which is that their failures are uncorrelated. Consolidation converts model reuse into a common failure domain, and a bank that removes the duplication without deliberately replacing the isolation has made its estate cheaper and more fragile at the same time.

Read this way, the specialists in the title are not there merely to add accuracy on top of the shared representation. They are containment boundaries, and that is a large part of why they have to stay.

5.2 What each specialist has to keep

For the shared backbone to behave as a controlled internal service rather than as one model quietly making every decision in the bank, each downstream use needs to retain five things of its own.

1. Its own labelled dataset and its own evaluation suite, so that performance is measured against the outcome that particular business actually cares about.
2. A named business owner and a named model risk owner, because accountability does not become reusable just because the representation did.
3. Independent thresholds, fallback logic and routes for human escalation, set according to that decision’s own loss tolerance and evidentiary standard.
4. Separate monitoring for drift, bias and calibration, since a backbone that is stable on average can still drift badly for one segment or one task.
5. The ability to bypass or replace the shared embedding entirely, which is the only real protection against the correlated failure described above.

The legal thresholds, loss tolerances and evidentiary requirements for declining a loan, freezing a payment and recommending a product have almost nothing in common. Reusable intelligence is genuinely valuable. Reusable accountability does not exist.

6. Caveats worth carrying into any internal conversation

It is worth being the slightly annoying colleague on five points in particular.

Every result in the paper is offline, which is to say a backtest against internal baselines, with no A/B test and no reported production outcome anywhere, since the paper simply does not tell us whether PRAGMA is running in anger. The nearest comparison available is Nubank’s nuFormer, published in July 2025, which reported a far more modest 1.25 percent lift in test AUC on a single recommendation task but paired it with a 4.4 percent churn reduction observed in production. Even that needs qualifying, because Nubank state explicitly that they could not A/B test every variant, which makes the 4.4 percent a comparison of before and after in production with hedged attribution rather than a controlled experiment. Neither paper clears the bar you would ideally want cleared, and it is worth noticing that the two papers also disagree on almost every design decision, in that Revolut built an encoder spanning multiple event sources with an explicit profile branch, while Nubank built a decoder over transactions alone.

Every figure is also relative, with the absolutes redacted, so a 130 percent lift on PR AUC tells you nothing whatsoever about whether the starting point was strong or weak to begin with.

Neither paper announces a release of the weights, meaning there is no public checkpoint on offer and no way to download PRAGMA and run it against your own book. What you can do instead is read the recipe, and the recipe happens to be specified in unusual detail, right down to the optimiser and the hardware.

The binding constraint here is the corpus rather than the code, because 26 million user records spanning 111 countries constitutes a genuine moat. The architecture is reproducible by anyone with the patience; the data is not.

Finally, it is worth separating what this research has proven from what it has not. PRAGMA establishes that a shared representation transfers across Revolut’s own internal backtests, which is a real and useful result. It does not establish that another bank can reproduce those gains on a different population, that the largest model can meet the latency requirements of real time fraud interdiction, or that a shared representation stays stable once production drift starts working on it. None of those gaps diminishes the research. Treating them as settled would be the mistake.

7. The part that actually matters

Strip away the architectural debate and what remains is two independent teams, working at two of the world’s more aggressive digital banks, publishing the same finding eight months apart: learning without labels on raw event sequences substantially reduces the need for features built by hand, and doing so produces material lifts on real banking problems.

The word doing the work in that sentence is reduces. Neither paper claims outright replacement, and Nubank’s headline result is explicitly a fusion, combining transformer embeddings with the existing tabular features rather than discarding them. The honest version of the claim is that the raw sequence contains signal your feature table is currently throwing away, which is a rather different proposition from the idea that your feature table is worthless. Two teams who agree on nothing else architecturally have landed on that same conclusion independently, which is a stronger signal than either paper’s headline percentage.

The reason this took so long has never been a shortage of data, since banks hold sequential behavioural data of extraordinary richness, arguably richer than anything outside of search. What was missing was the coincidence of available GPUs and internal mandate arriving at the same moment, and that coincidence is now happening often enough for the recipes to be published rather than merely rumoured.

The question facing any bank reading this is not whether to go and build a model of a billion parameters, which is both the wrong question and an unanswerable one. It is something narrower and considerably more tractable: do we have a task where the thing we are hunting is rare, where our current model leans on features built by hand, where years of raw event history are sitting unused in a warehouse somewhere, and where the explainability burden is light enough that a win would actually be allowed to ship? If the answer to all four is yes, then the 10 million parameter version of this idea is a scoped experiment with a published recipe rather than a moonshot.

8. Sources

1. PRAGMA: Revolut Foundation Model, Revolut Research and NVIDIA, arXiv:2604.08649, April 2026
2. Inside PRAGMA: Revolut’s Foundation Model for Banking, Philipp D. Dubach
3. nuFormer, Nubank, arXiv:2507.23267, July 2025
4. How to build transaction foundation models in banking and payments, Thoughtworks, on treating the backbone as a controlled intelligence service rather than a single decision model
5. Mules and preventing losses from inbound payments, Featurespace, on receiving side monitoring and pattern of life deviation
6. How Metro Bank uncovered mule accounts and increased detection rates by 71%, LexisNexis Risk Solutions
7. APP fraud reimbursement and the new liability rules, on the fifty fifty split between sending and receiving providers in force since October 2024