If you’re going to build a custom AI accelerator, inference is a good place to start. The clusters are smaller, the chips can be simpler, and at the end of the day they only need to do one thing: serve tokens. OpenAI’s spicy new Jalapeño inference chips are just the latest example. Social media magnate Meta is bucking this trend. Its first proper generative AI accelerator, the MTIA 400 — short for Meta Training and Inference Accelerator — is aimed squarely at LLM training. The Facebook parent is no stranger to custom silicon. But, much like Amazon and Google, its first AI accelerators weren’t built to run AI chatbots or train generative AI models. They were built to serve ads. The MTIA 400, teased earlier this year and detailed at the annual Hot Chips semiconductor development conference this week, will perform some inference duties. Just not of the GenAI variety. Instead, it’ll be saddled with running the ad recommender systems that actually pay Meta’s bills. The combination of LLM training and the deep learning recommender model (DLRM) inference used for serving ads is unusual, as the two have wildly different performance demands. LLM training is enormously compute-intensive, often requiring tens or even hundreds of thousands of accelerators to train models in a reasonable amount of time. DLRM inference, on the other hand, is a predominantly memory-bound job, which means most of the FLOPS that make the chip good at training are going to be left sitting idle. But given how much cash Meta is burning on compute these days, it’s probably not a bad thing for its chips to pull double duty – even better if one of the two use cases is highly profitable. While MTIA may play a bigger role in how Meta serves its generative AI workloads, its in-house silicon probably won’t replace AMD or Nvidia’s GPUs any time soon. The two chip designers' GPUs are currently much better suited to LLM inference and are almost certainly what Meta Superintelligence Labs is using to train frontier models like Muse Spark. Application-specific hardware has always been better suited to well-understood workloads, and Meta’s accelerators are no exception. But enough about market positioning and workloads. Let’s dig into the chip itself. Dissecting the MTIA 400 At first blush, the MTIA looks a lot like either Nvidia’s Rubin GPUs or AMD’s MI355X accelerators. The MTIA 400 is based on a heterogeneous multi-die architecture, which means it is built from a collection of different chips each with their own job. It features two compute dies, two I/O dies, and an SoC die that handles host connectivity and workload orchestration — all pretty standard stuff. Broadcom’s influence here is hard to miss. The chip almost certainly was built using the IP house’s XPU tech, which we explored in detail last year. Building a competent AI accelerator isn’t a trivial endeavor. So, if you’re trying to bring one to market quickly, it makes sense to focus on what makes your chip special and let someone like Broadcom handle the rest. Digging deeper into the MTIA 400’s heart, the compute chiplets are built on a 3nm process tech, presumably from TSMC, and feature a 6x8 grid of processing elements responsible for the bulk of the accelerator’s AI performance. Combined, the two chiplets are capable of outputting 12 petaFLOPS of MXFP4 compute at 1.7 GHz. To put that in perspective, the MTIA 400 is about 20 percent faster than Nvidia’s top-specced Blackwell accelerators at the higher precisions more commonly used to train models, while sucking back roughly the same power. But the parts don’t hold up nearly as well when compared to Nvidia and AMD’s latest chips – the MTIA 400 is between 3x and 3.3x slower than Rubin and the Instinct MI455X, respectively. Meta’s latest accelerator is fed by eight 36 GB HBM3e stacks that deliver 288 GB of memory and about 9.2 TB/s of bandwidth. Again, that’s about 15 percent faster than Nvidia and AMD’s last-gen parts, but with less than half the bandwidth of their new GPUs. This is probably why Meta is positioning the part as a training chip. It has much better and faster options for LLM inference. The MTIA 400 features a pair of I/O chiplets that provide 1.2 TB/s of chip-to-chip bandwidth over RDMA. We don’t know what transport tech Meta is using at this point, but given Broadcom’s involvement, Ethernet would be the obvious choice. Scaling up MTIA 400’s physical resemblance to Nvidia Rubin and AMD MI350-series parts also extends to the system design. Each compute blade features four MTIA 400 accelerators connected via a PCIe switch to an x86 CPU and a scale-out NIC. A single rack is equipped with 18 compute blades and eight switch blades for a total of 72 accelerators in a single unified domain. Meta shared a pic of the rack earlier this year, so while the compute blade is highly reminiscent of AMD's Helios blades, it's not using the same double wide OCP racks. Meta hasn't shown how large a cluster its chips can support just yet — the slides simply state "multi-thousand accelerator scaling" — but even at rack scale the systems should be quite competent relative to Nvidia’s GB200 and GB300 rack systems for training. More on the way While the MTIA 400 is aimed primarily at LLM training, Meta is already working on an inference-optimized version of the chip. Disclosed back in March, the MTIA 450 will double the chip’s memory bandwidth, presumably by swapping HBM3e for much faster HBM4. That part is expected to enter production next year, and should be extremely well suited to running those LLM-based recommender models Zuck and crew have been talking about the past few quarters. Meanwhile, the MTIA 500, which is also slated for 2027 release (probably in H2 if we had to guess), will increase memory bandwidth by another 50 percent — likely using 4 additional HBM4 stacks — and double the number of compute chiplets. ®
