Revolut says that in less than a year it's managed to slash its AI coding inference costs by roughly 90 percent by making self-hosted models its default instead of relying primarily on commercial services like Claude. Speaking at the AI Engineer conference in New York last month, David Abitbol, ops and infrastructure lead at the British fintech, described how it saw a 60 percent improvement in coding time and a 70 percent improvement in lead time following its rollout of AI coding tools to roughly 2,000 engineers. Revolut initially adopted GLM-5.1 before moving to GLM-5.3, which it now serves using 56 Nvidia B200 GPUs. Prior to its switch to GLM, Revolut had been experimenting with AI, deploying Gemini in early 2025 and Claude Code in early 2026. "We were highly skeptical about the cost of AI. We were seeing engineers spend $1,000 a month, $10,000 a month with not so much ROI, and so when we launched it we launched it with strict budget limits," Abitbol said. Because of this, engineers only had access to a limited number of tokens, so the company began investigating self-hosting iyd own models. At $0.30 per million input/output tokens, self hosting GLM 5.1 cost about a tenth of using OpenAI's Codex or Anthropic's Claude Code. And, as it turned out, when tokens are cheap, you stop worrying about how many they use or what they use them for. While Revolut did see improvements to coding speed, Abitbol noted the biggest bottleneck wasn't actually the code. Instead, he says the biggest gains came from letting the AI models deal with things like debugging and Jira and Confluence tickets. "When you have unlimited tokens, you're free to do that," he said. "You no longer have to save your tokens for the end of the quarter or the most difficult task." Developed by Chinese open weights model dev Z.ai, GLM 5.1 isn't exactly a small model. At roughly 754 billion total parameters (40 billion activated for each token generated), the model requires roughly 380 GB of GPU memory just to hold the weights and another 6-26 GB per user to maintain context depending on precision. To serve GLM-5.3 at scale, Revolut deployed 56 Nvidia B200 GPUs, equivalent in GPU count to seven eight-GPU DGX B200 systems. But as Abitbol noted, the greatest performance gains didn't come from adding GPUs, but rather tuning the model's hyperparameters along with switching to NVFP4 quantization, implementing disaggregated prefill and decode, and tuning context lengths. One key takeaway was that supporting a one-million-token context window doesn't necessarily make such a large window useful or efficient. "Long context is a scam. You really need to be mindful of that," Abitbol said. "Even though you can serve one million tokens, 256 [thousand] is really what you need." With these optimizations, Abitbol says that the company was able to achieve sub 3-second time to first token on context windows of around 256,000 tokens, and generation speeds of around 200 tokens a second, both of which he notes are the sweet spot for the company's software engineers. "What you want to aim for is a sub-3 second time to first token. This is empirical evidence. As soon as you exceed 5 seconds, my Slack inbox starts to blow up," Abitbol said. "In terms of throughput, it doesn't really matter as much. We serve, in terms of interactivity, 200 tokens per second. People are fine [with that]. You don't need to go higher. The way people work has changed. We no longer look at the stream of thinking tokens. People run tasks in parallel and check in when the task is done." And if throughput is a concern, he notes engineers who sign in after 5 pm are getting closer to 800 tokens a second. However, it's worth noting that GLM hasn't replaced GPT or Claude entirely. The Chinese model is text only, so it isn't ideal for things like UI work. So instead, Revolut used a model router to direct vision queries to models like Gemini or GPT Luna while keeping code generation running on GLM. "Open source is now a priority at Revolut, and our economics are favorable. The billing is flat, we don't have any bill shock at the end of the month, and the frontier is the exception," Abitbol said. Having said that, Abitbol acknowledges that Chinese open weights models may be a concern for privacy and security reasons, but emphasizes that the major concern there is accessing them via APIs running out of China. When run on-prem or on regional cloud resources, there's really nothing to worry about. ®
