One Shared Brain, Plus Specialists, Part One: What Revolut Actually Built

PRAGMA is a foundation model Revolut and NVIDIA built to replace dozens of separate hand built feature tables with one shared encoder trained on raw, timestamped banking events across 26 million records. It learns general behavioural patterns through masked prediction, then gets cheaply adapted with LoRA for tasks like credit scoring, fraud detection, and product recommendation, showing the largest gains on rare event prediction.

A plain English guide to the largest published transaction foundation model for consumer banking, covering what it is, how it works, and what the numbers do and do not prove.

This is the first of three parts. Part two looks at what it means for credit and for the customers conventional lending cannot see. Part three looks at fraud, mule accounts, and the limits of a shared model.

1. The fog

Somewhere inside your bank, in the time it takes to read this sentence, a few thousand payments have arrived that nobody will ever look at. Each one carries an amount, a merchant, a timestamp, a channel and a counterparty, and each one, considered on its own terms, is entirely unremarkable. The information that would tell you whether to worry about any of them is not in the payment at all. It lives in the relationship between that payment and the several thousand that came before it from the same person, in the rhythm those payments have settled into over four years, in the small drift of the last six weeks that nobody has noticed. That relationship is the only thing worth knowing and it is precisely the thing there is no time to examine, because the decision has to be taken now, while somebody is standing at a till with a queue behind them or watching a screen wait for a payment to release.

This is the condition every retail bank operates inside, and describing it as a data problem badly understates it. There is no shortage of data. There is a fog, made not of missing information but of an overwhelming volume of individually meaningless events, and the thing you are trying to see is never an event but the shape of a life moving through them.

What banks do about the fog is write rules, because a rule is the only instrument that can return an answer inside a latency budget. So we set thresholds, and the thresholds hold. A payment above a certain size to a counterparty never seen before, arriving from a new device at an unusual hour, gets deferred while somebody or something takes a second look. Every bank runs hundreds of these, they are the accumulated wisdom of years of investigators, and they are also the source of a quiet daily injustice, because a threshold has to be set at one number for everybody and there is no number that is right for everybody.

Here is the thing that rule can never know. Abnormal is not a property of a transaction. It is a property of a transaction sitting in the context of a particular person, and it moves entirely from one customer to the next. A payment of six thousand rand at three in the morning is unremarkable for a client whose history holds four hundred late night transactions and genuinely alarming for a pensioner who has never once transacted after nine at night. A first payment to a new counterparty is the whole basis of a contractor’s business and a red flag in a retiree’s account. The threshold sits somewhere between those two people, which means it is wrong for both of them, and the cost of being wrong is paid twice over. It is paid by the bank in the fraud that walks straight through a limit set generously enough to avoid annoying the contractor, and it is paid by the customer whose salary, school fees or supplier payment sits deferred for four hours because a number that was never about them said so.

Anyone who has sat with the operational reality of this knows what you actually want, and it is not a better threshold. You want the question at the moment of decision to stop being “what does this transaction look like” and start being “does this fit the person we already know”. You want to have done the expensive thinking in advance, so that every customer carries a fingerprint of their own behaviour that has already been computed, a dense multidimensional summary of how they earn, spend, save, borrow, navigate the app and recover from a bad month, sitting ready and waiting. Then the arriving payment does not need to be understood from first principles under a latency budget measured in single digit milliseconds. It only needs to be compared against a fingerprint that already exists, and the system can say this is weird, or this is entirely ordinary for them, fast enough to matter.

That is what a transaction foundation model is for, meaning a single pretrained backbone that learns customer behaviour directly from banking event sequences and then serves many downstream models rather than one. In April 2026, Revolut Research and NVIDIA published the most detailed public account of one yet, and they called it PRAGMA.

There is a second problem layered on top of the first, and it is organisational rather than technical.

Walk into any retail bank’s data science function and you will find something like six separate models running in parallel: one for credit scoring, one for card fraud, one for churn, one for cross sell, one for AML monitoring and one for collections propensity, each with its own team or at the very least its own backlog. Underneath every one of them sits an artefact shaped rather like a spreadsheet, called a feature table, made up of hundreds of columns built by hand with names like average balance over the last 90 days, count of declined transactions in the last week, or ratio of ATM to card spend.

Those columns are built by people, and by capable people at that, working over a period of months and using their judgment about what probably matters. The uncomfortable part is that each team builds its own set from the same underlying transaction data, which means six groups are separately constructing their own partial description of the same customer, and none of them benefits from what the others worked out.

It is worth thinking of this as a police force that hires a separate detective for every category of crime and then forbids them from comparing notes. Each detective develops a private filing system and a private set of instincts, none of them benefits from what the others have learned, and when a new category of crime appears the only available move is to hire another detective who starts from nothing.

PRAGMA is an attempt at the alternative, which is to train one detective who has read every case file in the city and then give that detective a short briefing for each crime type as it comes up. The analogy is worth holding onto, because the most interesting part of the paper turns out to be the point at which it stops working.

2. What PRAGMA actually is

PRAGMA is a foundation model for banking events, and both halves of that phrase need unpacking before the rest of the paper makes sense.

You already know what a foundation model is even if you have not used the term, because ChatGPT is one, and the recipe behind it never really changes. You take an enormous pile of raw data and train a model to fill in blanks in that data, without asking it to perform any useful task at all, since the only job is predicting what has been hidden. Do this at sufficient scale and something genuinely surprising happens, in that the model develops a general understanding of the domain, picking up grammar, facts, cause and effect, and tone without anyone having taught it any of those things explicitly.

The payoff comes later, because when you want the model to do a specific job you no longer start from scratch. You add a small amount of training aimed at that particular task on top of what the model already knows, which turns a project measured in years into one measured in weeks.

Foundation models already exist for text, images, audio and code, and PRAGMA is an attempt to answer the obvious next question, which is why there should not be one for money.

This is where the crucial design decision sits, because PRAGMA does not consume feature tables at all. What it consumes is the raw diary of what a customer actually did, in order, with timestamps attached:

Card payment, R280, merchant Spotify, 19:20 on Tuesday. Opened the app. Viewed the savings tab. Received a push notification. Transferred R2,000 to a savings pot. Card declined, R6,400, merchant unfamiliar, 03:12 on Saturday.

Notice that this diary includes app navigation, trading activity and communications rather than transactions alone, which means PRAGMA is reading the customer’s behaviour and not merely their money movement. That distinction matters enormously, and it is one of the things that separates PRAGMA from the comparable work published elsewhere, most of which covers transactions and nothing else.

One precision is worth adding here, because the word raw does a lot of work in commentary about this paper and it overstates the case. PRAGMA does not consume untouched operational data. Revolut selects which event sources to include, pseudonymises the records, tokenises every field, converts numerical values into buckets, truncates long histories and appends a profile state that has been assembled deliberately. What the model receives is better described as structured event sequences carrying minimal engineering aimed at any particular task. The distinction that matters is not that the pipeline vanishes but that the pipeline is built once and shared, rather than being rebuilt in a different shape by every team that needs it.

The corpus used for pretraining covers 26 million user records, where a record is a pseudonymised event history tied to a particular evaluation point rather than a distinct human being, so the figure is not quite the same thing as 26 million customers. Those records contain 24 billion events, which become 207 billion tokens once encoded, and they span 111 countries over a window of 25 months running from 2023 to 2025.

Three model sizes were built from the same underlying design, at 10 million, 100 million and 1 billion parameters. The compute ladder between them is worth understanding, because it is less smooth than the headline suggests: the 10 million parameter model took roughly two days on 16 NVIDIA H100s, whereas both the 100 million and the 1 billion versions took approximately two weeks each, the former on 16 GPUs and the latter on 32. There is no cheap rung sitting between tiny and serious.

It is worth converting those figures into money, because “32 H100s for two weeks” means very little to most people reading a roadmap document. Sixteen GPUs running for two days comes to roughly 770 GPU hours, while 32 GPUs running for a fortnight comes to somewhere near 10,750. At the specialist GPU cloud rates prevailing in 2026, which sit in the region of two to three dollars per GPU hour, that puts the smallest model in the low thousands of dollars and the largest somewhere in the twenty to thirty thousand range, with the hyperscalers charging perhaps half again as much. Those are startlingly modest numbers for the headline results attached to them, and they are also thoroughly misleading, because the compute was never the expensive part. The expensive part is assembling, cleaning and governing 24 billion events into a form a model can read, and no cloud invoice will tell you what that costs.

One small honesty note before moving on, which is that the paper never actually says what PRAGMA stands for. Commentary elsewhere has guessed at expansions along the lines of “pretrained banking model,” but the authors themselves do not tell us.

3. How it works, step by step

A language model reads text as tokens, which are roughly word fragments, so that “unbelievable” might arrive as “un / believ / able” and the model learns which fragments tend to follow which others.

A transaction is not a sentence, though. It is a structured record consisting of a currency, an amount, a merchant, a category and a timestamp, and the naive approach of dumping that record into JSON and letting a text tokeniser chew on it fails for reasons the paper explains clearly. The tokeniser wastes most of its budget on brackets and key names, and rather more seriously it shreds “14.99” into “14”, “.” and “99”, which destroys any sense that this is a small amount, and for a credit model magnitude is close to the entire point.

PRAGMA therefore encodes each field as a triple made up of what kind of thing it is, what its value is, and when it happened. The key, meaning the description of what kind of thing this is, always gets exactly one token, and there are only around 60 keys in the whole vocabulary, covering fields such as Currency, Amount, Merchant and Channel. Values are handled according to type. Numbers become percentile buckets, so that rather than learning the digits “14.99” the model learns that this sits in the bottom fifth of spending amounts, much as you might record which decile of the field a runner finished in rather than their time to the millisecond. Categorical values get a single token each, with the boundary between categorical and free text decided by how many unique values a field contains, and free text receives the same treatment by word fragment that a language model would apply.

Time is encoded twice over, which is the more elegant half of the design. It appears once as the elapsed time since the most recent event in the history, squeezed through a logarithmic transform so that intervals of three seconds and three years can both sit on the same ruler while retaining useful precision at the recent end. It appears again as a position within the calendar cycle, covering hour of day, day of week and day of month as repeating waves whose periods are fixed to real calendar cycles rather than learned from scratch, which is how the model comes to understand that a transaction at three in the morning is unusual and that the twenty fifth of the month carries payday behaviour.

What all of this produces is a vocabulary of roughly 28,000 possible value tokens sitting alongside 60 keys, which is both compact and faithful to magnitude in a way that a text tokeniser would not be.

PRAGMA divides each customer into two things that it reads separately from one another.

The first is who you are, covering account tenure, plan type, region and onboarding characteristics, all of which are facts that move slowly if at all, and all of which are handled by a dedicated profile encoder.

The second is what you have been doing, meaning the event stream itself, which passes first through an event encoder that reads each event in isolation and then through a history encoder that reads the full sequence of events together with the profile summary and fuses the two into a single representation.

The comparison worth making is to a good relationship manager, who holds two things in mind simultaneously: the client’s standing file and this month’s activity, neither of which tells you very much on its own. PRAGMA’s architecture makes that separation explicit, which as we will see allows it to lean heavily on the standing file for some questions while quietly ignoring it for others.

PRAGMA is trained the way BERT was rather than the way ChatGPT was, and this distinction deserves a paragraph of its own because it tends to get waved through when it should not be.

Decoder models, of which ChatGPT and Claude are examples, are trained to predict what comes next, reading left to right, and they are built in order to generate. Encoder models, of which BERT and PRAGMA are examples, are instead trained to fill in blanks anywhere in the middle of a sequence using context drawn from both directions, and they are built in order to understand and classify rather than to generate anything.

Since every question Revolut wants answered downstream is a classification question, asking whether this customer will default within twelve months, whether this transaction is fraudulent, or whether this person will take up this product, the choice of architecture follows fairly directly. What you want is not a model that writes plausible looking transactions but one that reads a history and forms a judgment about it, and that is what an encoder gives you.

The blanking out itself happens in three ways blended together. Fifteen percent of individual tokens are hidden, so that the model has to guess a merchant category or a bucketed amount from what surrounds it. Ten percent of whole events are hidden, meaning an entire transaction vanishes from the sequence and has to be inferred from the events on either side of it. Finally, ten percent of the time a whole field type is hidden, so that every single amount in the history disappears at once and the model must predict them given nothing but the field name and everything else it can still see.

That third variant is the clever one as far as banking is concerned, because teaching a model that the amount is often inferable when you can see the merchant, the time and the surrounding behavioural pattern is precisely the reasoning a fraud analyst performs by hand. It is exactly the bias you would want to install, and it arrives free of charge from the training objective rather than having to be built by a feature engineer.

Once the backbone exists there are two ways to put it to work, and the paper evaluates both.

The cheap route runs a customer through the frozen model and takes the numerical summary that comes out the other side, which is called an embedding and is best thought of as a rich behavioural fingerprint of somewhere between 192 and roughly 2,000 numbers depending on the model size, capturing a great deal of what the model has understood about this person. You then train a simple logistic regression on top of that fingerprint, which fits in a couple of minutes. Revolut frames this as the tool for rapid experimentation, to be used before committing to anything more expensive.

This is also where the fingerprint described at the start of this article becomes something concrete rather than a metaphor, and it is worth spelling out why, because it is the part with the most operational consequence. The embedding is a function of a customer’s history, which means it can be computed before anybody needs it and refreshed on whatever cadence the use case justifies, whether that is nightly, hourly or on every event. The heavy work of reading four years of behaviour therefore happens away from the decision, and what remains at the moment a payment arrives is the comparatively cheap operation of scoring the new event against a representation that is already sitting in memory. That is the architectural reason a question as rich as “is this out of character for this particular person” can plausibly be answered inside a latency budget that would never accommodate computing it from scratch. Two honest qualifications belong with that, though. The paper reports no latency figures and demonstrates no real time serving, so this is an argument from how the architecture works rather than a result anybody has published. And a fingerprint that is refreshed nightly is by definition slightly stale, which matters most in exactly the fast moving fraud scenarios where you would most want it to be current.

The better route is LoRA, short for Low Rank Adaptation, which avoids retraining the whole billion parameter model for each task by freezing almost all of it and adding small trainable adjustments to the attention and feedforward layers, updating only two to four percent of the weights in the process. The analogy is that you do not educate a specialist all over again from scratch in order to hand them a new category of case; you give them a briefing document and let them adjust their existing expertise accordingly. Fine tuning this way took roughly an eighth of the wall clock time that pretraining had required, which worked out at somewhere between twelve hours and a few days.

LoRA beat the frozen embedding approach on every single task measured, by as much as 73 percent on some of them and by well under one percent on others. The frozen embeddings still worked, though, which matters a great deal if your compute budget is a real constraint rather than a rounding error.

4. The numbers

The headline results below come from PRAGMA Large with LoRA fine tuning, measured against Revolut’s own internal models built for each individual task. Every figure is a relative lift, since the absolute scores were withheld as commercially sensitive.

Before going any further into what those numbers mean, they need a qualification, and it belongs here rather than buried in a limitations section at the end. These are relative improvements measured against Revolut’s own internal baselines, and they have not been independently reproduced. Because the absolute scores were withheld, a lift of 130.2 percent tells you nothing about whether the underlying score moved from poor to adequate, from adequate to excellent, or from one respectable number to a slightly better one. What the figures do establish is that the pretrained representation transfers. What they do not establish is production superiority on another bank’s population, and every number below should be read as directional until somebody outside Revolut reproduces it.

The final pair of rows in the table deserves attention for a related reason, because two metrics computed on the same uplift experiment disagree with each other by more than an order of magnitude, which is a useful reminder that no single number here should be read without its metric definition close at hand.

Three broader patterns emerge from the results. The first is that the wins cluster heavily on rare events, because PR AUC and recall are the metrics that matter when the thing you are hunting is scarce, whether that means defaulters, fraudsters, or the small minority of people who open a follow up message. That is where PRAGMA performs spectacularly, whereas on tasks where the signal is dense and the underlying pattern is simple, such as whether a given transaction recurs monthly, it sits at roughly parity, which is both honest and entirely unsurprising, since your existing model was probably already competent at that.

The second pattern is that pretraining genuinely transfers, and this is the finding that carries the most weight even though it is the least exciting to look at. The relevant experiment compares PRAGMA fine tuned with LoRA against the identical architecture trained from scratch on the same task data, and at the 100 million parameter size credit scoring gained 13 percent PR AUC purely from having been pretrained beforehand, while communication engagement gained 18.6 percent and product recommendation gained 10.3 percent, with no task regressing anywhere. The paper does not run this comparison at the 1 billion parameter scale, so we cannot say whether the transfer benefit grows or shrinks as models get larger. This is nonetheless where the business case lives, because if pretraining did not transfer then the entire infrastructure investment would be theatre.

The third pattern is that the design with two branches earns its keep, and the ablation that demonstrates this was run at the 10 million parameter size. Compared against a version that reads events only, with the profile branch stripped out entirely, the full model scores 31.8 percent higher on credit scoring PR AUC and 85.6 percent higher on fraud recall, which makes sense given that tenure and onboarding signals are how you identify the rare defaulter in the first place. On communication engagement, however, the version reading events only actually performs better, with the full model coming in 3.0 percent lower on PR AUC, because whether somebody comes back is a story about their behaviour in the period just before they dropped off rather than a story about who they are. The architecture leans on the standing file where it helps and shrugs it off where it does not.

5. What a transaction foundation model can be used for

The paper covers credit risk, external fraud, product recommendation, marketing uplift and targeting, lifetime value and the detection of recurring payments directly, and the implications for a bank’s operating model run considerably wider than any individual metric.

Experimentation becomes substantially cheaper, though not in the way people sometimes assume. You still need labels, because a frozen embedding cannot conjure a target variable out of nothing, but what disappears is the feature engineering step, which is where most of the elapsed months actually go. Revolut’s own framing is that the embedding probe is what you reach for in order to establish whether a task is worth pursuing at all, before committing to a full round of fine tuning. Maintenance eases for the same reason, since six feature pipelines drifting independently of one another become one backbone to monitor alongside a set of small adapters to retrain.

Behavioural signal becomes usable for the first time in many institutions, because app navigation and messaging response are sitting right there in the sequence. Most banks’ feature tables ignore this data completely, not because it lacks value but because nobody has ever had the time to engineer features out of a clickstream.

Small models turn out to be viable as well, since the 10 million parameter version is described as a possible efficiency sweet spot for transactional work and for lifetime value, and it trained in two days. It still required 16 H100s, though, so the floor here is lower than the headline figures suggest rather than absent altogether.

6. What comes next

That is what Revolut built and what it appears to be worth. The harder question is what it changes, and the answer differs sharply depending on which part of a bank you sit in.

Part two takes the credit side. It argues that conventional lending has confused regularity with durability, that a salaried employee is the most concentrated income in the book once you apply the bank’s own concentration logic honestly, and that a lending policy requiring evidence of a payslip discards the overwhelming majority of its addressable market in a market like South Africa.

Part three takes fraud, including the application the paper never mentions and which may be the most valuable of all, which is that a mule account has a behavioural signature so distinctive that a precomputed fingerprint can hold an inbound payment before it ever arrives. It also covers where the architecture fails, and why a shared representation creates a risk that compounds for exactly the same reason its benefits do.

7. Sources

1. PRAGMA: Revolut Foundation Model, Revolut Research and NVIDIA, arXiv:2604.08649, April 2026
2. Inside PRAGMA: Revolut’s Foundation Model for Banking, Philipp D. Dubach, whose framing of the nuFormer comparison and of the constraint being the corpus rather than the code informed parts of the argument here
3. Deep Dive: Revolut’s PRAGMA, The Transaction Foundation Model, Sam Boboev, Finextra
4. nuFormer, Nubank, arXiv:2507.23267, July 2025
5. How to build transaction foundation models in banking and payments, Thoughtworks