KETTLE Big tech's Q2 earnings have been pouring in over the last two weeks, and it looks like stock market investors are taking a pin to the AI bubble. You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube, where you can subscribe to get notified about the latest episode. Between eye-watering capex expenses, shrinking free cash flow, and worries about chip availability, things are starting to look tight. El Reg editor-in-chief Matt Rosoff and systems editor Tobias Mann join host Brandon Vigliarolo for this episode of The Kettle to discuss what the numbers mean and how things pan out. Most importantly for Reg readers, they also offer advice on what IT teams should do in such a volatile market. Hint: It doesn't involve going all in on frontier lab products across the enterprise. A lightly edited transcript is included below: Brandon (00:01) Welcome back to The Register Kettle. I'm Brandon Vigliarolo, and this week it's all about what the latest quarter of tech earnings are telling us about that AI bubble. With me this week is our editor-in-chief Matt Rosoff and Systems Editor Tobias Mann to pick apart this haystack of earnings news to look for the bubble popping needle that might be hiding within. Guys, thanks for coming on. Matt Rosoff (00:22) Thank you. Tobias Mann (00:23) Good to be here. Brandon (00:24) It's been a mess of up and down earnings from big tech this quarter, as reported in the past week. Not all of it seems bad, but despite that, a lot of investors seem pretty worried about the state of things. Matt, you've been watching this closely. Can you break this down and give us an understanding of what the numbers mean for both investors and the state of AI? Matt Rosoff (00:46) It seems as if we are in what often happens toward the final phases of a speculative time in the markets, what might be called a bubble, which is characterized by absolutely wild, insane swings in stock prices. These are not penny stocks; these are humongous massive cap tech stocks. Apple is down 10% today simply because they warned that their next quarter might not be as good as expected because memory and other component prices are so high. You have IBM losing more value in a single day after its last earnings warning than it had lost since Black Monday 1987. You have these massive tech stocks, which represent most of the value of the stock market, acting like penny stocks, and that tends not to be a great sign. That's usually when investors don't know what to make of the current state of things and are trying desperately to figure out where to put their money. There's a lot of tea leaf reading. It has varied widely. Meta, investors didn't like what Meta was saying. Their cash flow decreased dramatically because of the capex spending they're doing to build new data centers. Everybody freaked out on that and there was a big sell-off. Brandon (02:29) It was a massive cap swing, right? They're under a billion dollars in free cash flow. They had $8.5 billion last year at this time. I mean, that's huge. Matt Rosoff (02:37) Yeah, and this is all data centers to build out something to do with AI. I think probably in the end, Mark Zuckerberg at Meta has figured they will build as much as they can now, and the AI will help target ads and help their core business enough that it'll be worth it, and they can rent out whatever excess capacity they have in the future. But that is speculation. Brandon (03:06) Right. Matt Rosoff (03:06) Point being, this is a situation where a lot of investors seem to be reading tea leaves. Amazon is up 15% today. Same story there: their capex and estimates went up. But because AWS grew faster than expected and because investors were looking at Amazon Web Services and their core business, they decided Amazon is doing the smart thing by investing in AI. If you read Corey Quinn, an analyst who writes for us about AWS – his main job is helping customers figure out AWS licensing and pricing – he dug into the earnings and part of what investors liked was AWS margins, but some of that margin was created by hedging on energy prices. It's very obtuse. All the big companies do these kinds of things; their treasury departments are always figuring out how to play games and maximize. It's perfectly legal and not fraud, or anything close to that, but it's not the core business. You have to back out this one-time energy price hedging strategy that worked to their advantage. Then you see that their margins are right where they were expected to be. Brandon (04:33) Right. And I think he even mentioned that they said in that earnings call that they don't expect that to have the same bump in Q3; it's not going to be there next quarter. Matt Rosoff (04:43) And yet investors are throwing money into Amazon, which is up 15% today. It's bonkers. The big chip stocks had huge crashes earlier this month. There was a hedge fund, Situational Awareness created by a young hedge fund manager who worked for FTX, which you'll remember was Sam Bankman-Fried's crypto company that imploded after fraud charges. He created this hedge fund... Brandon (05:21) Now it's imploding. Matt Rosoff (05:22) It's imploding. The take is that he's a decent stock picker but didn't hedge properly and was way over-leveraged. So $45 billion turns into $10 billion. These are the kinds of things you see as a bubble starts to burst. I've seen a lot of memes on the AI forums I follow, many in the AI community say "AGI is already here, we just don't realize it yet." I would counter that "the AI bubble pop is already here, we just don't realize it yet." Brandon (06:01) Speaking of Corey's story, I was digging into that earlier today and was struck by things he pointed out. Aside from the margin bit, he mentioned how AWS discussed their chip business, which Tobias knows doesn't really sell any chips, they actually conflated their chip revenue with EC2 instance revenue. The same thing happened with their massive investment in Anthropic. They accounted for $53.4 billion due to deals with Anthropic last quarter. He said if you follow one Anthropic dollar through the earnings release, it's counted in AI business revenue, chips business, and AWS segment revenue. He said all this massaging and padding is in service of the massive capital expenditures that Amazon and other companies are undertaking. Another quote I thought was telling: "The demand underwriting two hundred and twenty billion in capex is concentrated today in a handful of AI labs, one of which Amazon happens to own a meaningful piece of, when the broader menterprise adoption wave remains a forecast. Matt Rosoff (07:25) Exactly. Brandon (07:26) As I'm reading this, by the end of that story, I was struck with the question: is this entire quarter of earnings just an industry trying to pump air into this thing to stave off a collapse? Matt Rosoff (07:44) I don't know if it's so much trying to pump air as it is the bet that has been made, and they are continuing to make this bet because it would be too risky to pull out. Tobias Mann (07:58) It's like the sunk cost fallacy. Brandon (08:01) Say it again, Tobias. Tobias Mann (08:02) It's like the sunk cost fallacy. They're too deep in at this point to climb out. Brandon (08:04) Yeah, that might be more the case than pumping air in. Tobias Mann (08:09) There's not much they can do other than keep putting money into this. Imagine if next quarter Amazon stopped and said they were not going to spend any more money on capex for this; imagine how the market would react. Brandon (08:27) Yeah, probably not well. Matt Rosoff (08:29) Right. And the fundamentals here are OpenAI and Anthropic, which are massively valued companies. They have humongous commitments and are generating real revenue on the order of twenty billion a year. That's not nothing. They are real companies with real demand. However, the size of their commitments is predicated on the idea that at some point AI can replace a lot of economic value residing today in all kinds of other industries, not just the IT industry. If you believe that, then these numbers make sense and you can justify spending like crazy, because at some point most of what the economy runs on will be powered or replaced by AI. A skeptic would say a lot of this is "someday, maybe," and we don't actually see it yet. Brandon (09:41) Yeah, going back to that forecast line from Corey, this is all predicated on things people hope will happen. Matt Rosoff (09:47) Right. Tobias Mann (09:48) I think one of the most dire examples is Meta, which has been an absolute mess in terms of its AI development team. Part of it was a bidding war over engineers, resulting in overspending on talent. Another component is that Meta is spending a tremendous amount of money on capex to build these data centers. Some of which are supposedly going to cost $50 billion to finish. They don't have much to show for it other than making their recommender systems work better. Then there's talk of them moving from a hyperscaler to a public cloud provider, something I would argue is inevitable for any sufficiently large infrastructure company. It is really hard to justify that kind of spending when you have little to show for it. Llama is fine, but it's not on par with what Anthropic or OpenAI are doing. Their track record for putting out models that people want to use hasn't been great; they've had duds along the way. They don't even have the confidence OpenAI or Anthropic have built up that the next model will be better. With Meta, you don't know that. Matt Rosoff (11:27) Right. And when we say "build data centers," those are easy words to say or to put in a press release. Tobias, you can talk to this better than I can, but these things are tremendously complicated. You're dealing with multiple buildings, permitting, siting, water, power, local protests, networking, are the GPUs and CPUs even available? Apple, which has the greatest supply chain expertise in history, is running into constraints on supplies for iPhones and iPads. I can't imagine that companies intending to build these massive data centers are not going to soon face similar constraints. That doesn't even get into energy costs and oil prices. It's bonkers. Tobias Mann (12:35) Yeah, it's a speculative rabbit hole. Hyperscalers and cloud providers build their own data centers but also lease a lot of capacity as a hedge because it's easier to walk away from a lease than to commit to maintaining your own facilities. Microsoft has had scares going back a couple of years. There was a point where Microsoft was pulling out of leases and everyone was freaking out, thinking it was the canary in the coal mine. In reality, the hardware changed and the facilities they had signed initial contracts for could not support the hardware. So they killed those agreements because they were for facilities that couldn't take the infrastructure they wanted to deploy. There's also a problem with how co-location has historically been treated like a real estate deal. That's fine for air-cooled data centers because it's predictable, the infrastructure is simple, and power constraints are reasonable. You don't need a ton of water. You don't need the same degree of backup generators, and usually those facilities have different uptime requirements. With AI data centers, it's completely different. The scale at which these are being deployed means that before you even break ground, you have to have figured out contracts with utilities to see if they can even give you the power or build the substations to supply it. You have to figure out where you're going to get water because these are such high power consumption facilities that evaporative cooling is the most cost-effective way to do it. That's changing a little, but for now you have to be figuring all of that out. You can't even get the infrastructure until Nvidia and AMD know that it exists. They don't want allocation sitting on pallets for six months. The supply chain has gotten so screwed up where you can do everything right, but now everything is liquid-cooled and the constraints are so bad that you might be on a waitlist for a year to get the plumbing and manifolds just to get everything hooked up. Brandon (15:20) That's all assuming geopolitics doesn't screw the entire pooch and leave them even further out. Matt Rosoff (15:24) And that's on the supply side. There's the demand side too. I remain unconvinced that LLMs and what we call "AI" will replace most economic activity. I don't know what everyone else is seeing. I'm hearing they are very good for certain functions. Are you going to have them write all your production code? No. Are they great for prototyping? Yes. Can they help an experienced coder do more with less? Yes. It's a mixed bag. Are they going to replace all medical professionals with medical advice? No. Brandon (16:13) Definitely not, based on the error rates we've seen in some stories I've written and seen lately. Matt Rosoff (16:17) Yeah, there was an article in the Wall Street Journal, I think it was yesterday or a couple of days ago, where they were talking about companies starting to hire again. People who were laid off ostensibly because of AI are being brought back because companies realized they actually need them after all. Tech companies are hiring again. Many folks who thought AI could replace customer service are finding that even a smart chatbot is subpar compared with an actual human on the phone who can resolve a question. You've got this massively overstated ease of supply, "We'll figure this out, we'll keep plowing capex into these data centers and they'll get built." But then you have the demand side and the claim that it's going to replace all economic value. Will it? I don't know. Brandon (17:10) I don't think we've seen evidence of that so far. People are hiring again. There are reports regularly discussing AI ROI for enterprise and how it's just not there. For two or three years, we've been writing stories about the fact that ROI isn't there, that people can't find a use for this, and they're just trying to plug it in to find something to give it a purpose. Matt Rosoff (17:33) It gets hard to think about this because an AI booster will say you're using outdated information or that you're claiming AI isn't real or useful. That's not what we're saying. I'm saying it's not going to replace all economic value to the level at which Anthropic and OpenAI are valued, and how this is reflected in former public market valuations of the big suppliers. It's a complicated phase. I think we're starting to see the deflation now. If you look at most big tech earnings this quarter, with the exception of Amazon, almost all others lost significant value after reporting. Microsoft, Alphabet, Meta, and Apple did. That's four out of the five. And I think we have Nvidia as well. Some chipmakers have seen wild swings in stock prices. This is what it looks like. I would say then, we're The Register, not the Wall Street Journal or CNBC. Who cares? What difference does this make? The caution is that the product sets and functions you see today from these big frontier LLM makers may not be what you see a year or five years from now. It's important to understand these are still speculative businesses. This is not an Oracle database, Office 365, Google Search, or AWS. We don't know what they're going to cost or what will be available. Who knows what's going to be built around them? A lot of people are saying the harnesses become important and the models actually don't matter all that much. Do you have thoughts on that, Tobias? Tobias Mann (19:35) The harnesses are interesting because they're an extension of context and reasoning capability for the models. The models are largely transactional, and the harnesses allow them to go beyond that, to retrieve additional information, think about it, take action, look at the reaction to that action, and perform tasks more reliably with less hallucination. It works better for some things than others. Code assistants are the shining example. The problem with code assistants is that code either runs or it doesn't. If it doesn't compile, it just triggers another loop of the system. That doesn't really work in transactional industries. If you look at customer service, you can't just loop through and get the right answer when you're talking to a customer on the phone. The model needs to get it right the first or second time to be better than a person. Human contact should be valued. Personally, when I have to call someone, it feels better talking to a person than going to a robot. Brandon (21:03) Yeah, no one wants to do that. Matt Rosoff (21:05) They're finding that in the end it's probably cheaper to have a warehouse full of customer support reps at thirty or fifty bucks an hour versus burning infinite tokens on loops that don't resolve the customer call in the first place. Brandon (21:22) And you can you can give them access to AI that helps them surface information more quickly, so you still get the benefit of that. There's obviously benefit in AI; it surfaces information and it does things. To bring this back to why this matters to our readers, IT professionals, what this entire thing should tell everyone in this space is not to jump in with both feet right now. This is not something that's done and ready to go. Tobias Mann (21:56) I'd say you should be hedging right now. If you're moving into this space, you should do so in a deliberate fashion and be aware of the uncertainties. Matt brought up an excellent point: the climate is perfect for a squeeze on customers to see just how sticky the products have become. When you have supply chain constraints and investors looking for evidence of a return on investment and ability to deliver a profit, there is incredible pressure to raise prices. When you get to code assistants, which are the stickiest example of AI being useful, I think we're going to continue seeing prices rise rapidly over the next six months because there is an understood value associated with this, and they have to demonstrate that to their investors. Brandon (23:01) But we just saw OpenAI drop prices on GPT models significantly. They said it was because they maximized their ability to do more with less. Do you think it's that, or do you think people don't want to pay for this, so they have to adjust the price down? Tobias Mann (23:20) You have to watch something because harnesses have changed how you look at the cost structure. If you look at cost per million tokens, it might look lower, but if the model consumes four times as many tokens to deliver a result, it's not cheaper. Thomas Claburn, one of our senior software reporters, had an excellent piece looking at how Anthropic's latest models use a tremendous number of tokens to deliver the result. OpenAI's latest models might look less expensive from an API standpoint, which is great for marketing. But if it's using twice as many tokens, that's not the same thing. That is somewhat dependent on the harness, reasoning effort, and how they're routing the models. I caution anyone looking at API prices: they dropped the price, but is it actually less expensive? Brandon (24:27) Yeah, they dropped the price, but count the number of tokens you're tossing into it and see if it's actually cheaper. It's good for marketing, but the rub is how much you're actually using. Matt Rosoff (24:41) And as you've written about, we've seen open weights models becoming as effective as more expensive ones from the frontier labs for many functions. The advice for practitioners is to test a lot of things, look to run stuff locally, and prototype. Check your expenses before you move all your workflows over to agentic AI. We're still in the "test and watch your expenses" phase, unless you are a huge tech company like Google that can afford to have employees tokenmax for six months. Most businesses are not in that state. Tobias Mann (25:31) We're seeing this in a couple of different stages right now. Particularly as Nvidia tries to enlarge its total addressable market and de-risk its reliance on hyperscalers to drive profit, they're increasingly looking towards enterprises. One of the big things they're pushing is the idea that you might still use GPT or Claude for some stuff, but you don't need to use those models for everything. You can run a vision language model to skim through invoices and write up summaries or transcribe them. It can be incredibly effective at that one thing and way more cost-effective to run on a $13,000 GPU in a server in your data center or your colocation rented from AWS. They don't care how you get to the GPUs, just that you're using them. There is a recognition that the cost component can be addressed by smaller targeted models. It seems like we have this pendulum swing every couple months from giant models back to small models. Chinese models are just getting enormous right now; Kimi K3 is 2.8 trillion parameters. That's bigger than GPT-4 was a couple of years ago. They're as big and as capable asfFrontier models, which is putting pressure on things, but I don't think that is necessarily what enterprises need.Looking at smaller models and evaluating targeted use cases might offer you some relief and allow you not to miss out on the opportunities afforded by the technology. Brandon (27:39) Yeah, we'll see. It's a moving target that's still developing and might be shrinking. We'll be here to talk about it on The Register; it's the topic of the year or decade. Thanks for joining us, and we will talk to you all soon. ®