The Register

Biting the hand that feeds IT — Enterprise Technology News and Analysis

Tensordyne makes a big bet on log math to beat Nvidia

AI infrastructure startup Tensordyne has taped out its first commercial accelerator, with fabrication on TSMC's 3nm process already underway. Developed in collaboration with Juniper Networks and Broadcom, Tensordyne's systems promise higher throughput and lower power consumption than GPUs. It claims to achieve this using an unorthodox approach to mathematics that uses logarithms – which you might recall from high school arithmetic – to make matrix multiplication heavy AI workloads less computationally intensive to run. In conventional computing, addition is cheap, and multiplication is expensive. Logarithms flip this on its head. Using logs, multiplication essentially becomes an addition problem. a*b becomes log(a) + log(b). The trick is converting those values to logs and back again efficiently. There are a couple of ways of dealing with this. One of the easier options would have been to use a lookup table (LUT). However, Tensordyne cofounder Gilles Backhus tells El Reg that relying on LUTs would have been too large to be practical. Instead, the company uses a heuristic, specifically the Mitchell approximation, to estimate log and antilog for each value. This is still an approximation and on its own introduces too much error to be tenable. To overcome this, Backhus tells us Tensordyne has implemented a section-wise correction mechanism in hardware that delivers accuracy equivalent to that of FP16. However, it's worth noting that Napier will also support FP8 and 4-bit block floating data types. In effect, Tensordyne claims to have built a chip in which the multiply accumulate (MAC) unit works without actually doing multiplication in the conventional sense. The result is a chip that delivers power efficiency significantly greater than what you'd see on modern GPUs. Or at least that's the claim. Tensordyne says its rack systems will spit out up to 17x more tokens per watt and achieve 13x higher throughput than Nvidia's Blackwell systems. Dissecting Napier Tensordyne’s first commercial chip, Napier, boasts many of the same specs you’d have seen from a high-end GPU just a couple of years ago. The accelerator boasts a 300-watt nominal TDP, 144 GB of HBM3e spread across four stacks, 4.7 TB/s of memory bandwidth, and up to 2.1 petaFLOPS of dense FP8 performance. This makes it roughly comparable to Nvidia’s H200 accelerators announced in 2023, while using nearly 60 percent less power. Having said that, max achieved FLOPS often fall far short of peak FLOPS, so take that comparison with a grain of salt. We won’t know how Napier actually compares to Nvidia or AMD's latest generation of GPUs until it arrives next year. Backhus tells us that Tensordyne is leaning heavily on the scalability of its accelerators rather than individual performance. Each chip features roughly a terabyte of interconnect bandwidth, allowing for rack-scale deployments of up to 72 accelerators per pod. The TDN72 Tensordyne's system, codenamed the TDN72, consists of eight air-cooled compute blades, each with a single 10-core Intel Xeon-D host CPU and nine Napier accelerators. These chips are interconnected by a high-speed interconnect fabric topology reminiscent of the one used by Nvidia's GB200 NVL72 rack systems. Each chip connects to six proprietary fabric switch blades developed by Tensordyne's networking partner Juniper, located at the back of the system, in an all-to-all fabric. Despite some similarities to Nvidia's NVL72 racks, Tensordyne's TDN72 will be much smaller and won't require liquid cooling, which should make it easier to deploy in older brownfield datacenters. According to Backhus, up to four 30 kW TDN72 systems can be packed into an – admittedly large – 52U rack. That works out to 608 petaFLOPS in a 120 kW footprint, or about 1.68x more dense FP8 compute per rack than Nvidia's GB200 NVL72. That doesn't take into consideration the fact that Nvidia's kit supports NVFP4 acceleration while Napier is limited to FP4 weights. But again, don't read too much into that comparison. Peak FLOPS are not representative of real-world performance. Tensordyne's TDN72 launches next year, and it'll be competing against Nvidia's next-gen Vera Rubin and Vera Rubin Ultra systems, which will no doubt be a stiffer fight, especially when software compatibility is taken into consideration. Software promises Since building its first prototype silicon a few years ago, the company has gone to great lengths to keep its software platform as simple and easy for customers to deploy, as possible. For example, the prototype lacked the error correction found in its Napier chips, and would have required users to use quantization-aware training to adapt their models to run accurately on the hardware – not exactly feasible for those looking to run trillion-parameter models. The software has also matured such that the hardware's compiler can convert existing models to run directly on its latest hardware, an approach we've seen from other chip startups like Tenstorrent. For inference, Tensordyne has developed its own proprietary serving platform, as well as a runtime environment that Backhus says will allow customers to use their preferred inference servers, such as vLLM. PyTorch support is under development. Before the chip has even shipped, the company is making some bold performance claims. Backhus expects the chips to deliver upwards of 1,000 tokens a second, and that's without relying on multi-token prediction or other forms of speculative decoding to boost token generation. Tensordyne's platform has certainly attracted the attention of neocloud providers like Cirrascale and BlueSky Compute, both of which have expressed interest in deploying the company's hardware when available. But, as we've seen with AMD and others, software can make or break a chipmaker. With Napier slated for release in Q2 or Q3 of 2027, Tensordyne won't have long to get things right. ®

Wel.nl

Minder lezen, Meer weten.

Volgende week coronaverhoren met twee oud-onderwijsministers

DEN HAAG (ANP) - Voormalig onderwijsministers Ingrid van Engelshoven en Arie Slob worden volgende week verhoord door de parlementaire enquêtecommissie over het coronavirus. De verhoorweek staat in het teken van het effect van de coronamaatregelen op het onderwijs.

Van Engelshoven was van oktober 2017 tot januari 2022 minister van Onderwijs, Cultuur en Wetenschap namens D66. Zij wordt als tweede verhoord op maandag. De verhoorweek begint met Károly Illy, die tijdens de coronapandemie voorzitter van de Nederlandse Vereniging voor Kindergeneeskunde was en als kinderarts zat hij ook in het Outbreak Management Team (OMT).

Op woensdag beginnen de verhoren met Kinderombudsman Margrite Kalverboer. Zij riep tijdens de crisis onder meer op tot het heropenen van de scholen. In de middag moet voormalig voorzitter van de VO-raad Paul Rosenmöller naar de enquêtezaal om verhoord te worden.

De week wordt afgesloten met voormalig minister Arie Slob voor Basis- en Voortgezet Onderwijs en Media en oud-voorzitter van het Landelijk Aktie Komitee Scholieren (LAKS) Nienke Luijckx. Slob was met Van Engelshoven verantwoordelijk voor het beleid rond scholensluitingen.


The Guardian

Latest news, sport, business, comment, analysis and reviews from the Guardian, the world's leading liberal voice

Heathrow third runway GDP yield may be 90% less than original estimate

Department for Transport analysis suggests 0.05% boost will be offset by overall negative trade-off

The economic boost from a Heathrow third runway could be a tiny fraction of previous estimates, government analysis shows, while the overall trade-off from the bigger airport could set the UK back by as much as £62.5bn.

As ministers promised to speed up expansion of the London airport in the name of economic growth, documents prepared by the Department for Transport said the runway was expected to boost GDP by only up to 0.05% – 90% less than the 0.5% previously stated.

Continue reading...

Caribbean Motel

Thomas Hawk posted a photo:

Caribbean Motel

Found Kodachrome Slide

Thomas Hawk posted a photo:

Found Kodachrome Slide

Found Kodachrome Slide

Thomas Hawk posted a photo:

Found Kodachrome Slide

date stamped on slide January 1960

Found Kodachrome Slide

Thomas Hawk posted a photo:

Found Kodachrome Slide

date stamped on slide June 1982

You Can't Say What You Don't Know

Thomas Hawk posted a photo:

You Can't Say What You Don't Know

Kabinet houdt drie opties open voor kerncentrales: twee locaties in de Eemshaven en een in Terneuzen. ‘De keuze wordt niet makkelijk’

Nu twee locaties voor nieuwe kerncentrales zijn afgevallen, blijven er drie over: eentje in Terneuzen en twee in de Eemshaven. Het kabinet zegt voor een lastige keuze te staan.


MetaFilter

The past 24 hours of MetaFilter

Sleep is a medical debility

Power Nap is a completed webcomic by Maritza Campos and Bachan. Thanks to a miracle drug, nobody needs to sleep any more. Unable to take his medicine due to allergies, Drew Spencer struggles to keep up. And then things start getting weird.

Webcomics readers of a certain vintage will remember writer Maritza as the creator of College Roomies From Hell, a vintage webcomic that- wait - what? It's still updating? What the hell!?