There has been much chilly weather in the sunny meadowlands of AI futures. Fortunately, to use an indecently inappropriate adverb, the rest of the world isn’t really looking, being too busy chewing its nails over the very many other intimations of disaster. Sure, rebel swarms of attack agents breaking out of the labs isn’t a good look. OpenAI delaying its IPO and a new model launch likewise. All these things soak up whatever bandwidth the news media have to AI right now. Putting them in a bigger context, that of promises made and not kept, problems known but not fixed, the observable risk-reward ratios as models develop, we may at last start having a sense of what sort of crisis awaits. Actual crises are frequently triggered by crises in confidence, themselves caused when frameworks of assumption stop standing up to reality and wishful thinking. Thus, the assumption to date is that models will get more powerful and infrastructure more capable, in a way that justifies — demands — massive capex. The economics of giant infrastructure investment when you can’t buy what you need is a great bet in free markets that presupposes that freedom, and yeah, about that. A bigger problem is the idea that more powerful AI is always better. It frequently is, but that’s not good enough, When frontier models have to be kept in LLM jail for uncontrollable dishonesty, not so much. Here, the bigger context is that we’ve known that at heart LLM probability space is entirely relative. There is no ground truth there. Nobody has fixed hallucinations, and now we see the fascinating phenomenon of a machine that doesn’t understand truth but can deliberately lie. If more powerful models are better at that, scratch one assumption. Another huge assumption is that even at the current level of model reliability, agentic architectures are ready to be useful. Those who show all the caution of a chronic gambler on a winning "streak" are as ready to get the brakes slammed on as their opposites. The principal contender for the one true proof of concept that agentic AI is a dangerous fount of unexpected consequence is Meta’s new Muse. It is aimed at low-information credulous users and its star turn is, of course, shopping. The AI use case that’s five years away, and always has been. It’s Meta, so of course it soaks up all the information about you, your online activities and the people you talk to as much as it can. It will soon have ultracute animated Tamagotchi-style pets you can wear around your neck to improve your status as a Meta monitoring post. And we did say it was Meta, so has all the attention to security and careful design you may expect. Which is why it’s already been caught sending people’s physical addresses to strangers, and why when researcher Jonny at neuromatch.social went poking around its infrastructure — each instance runs permanently on its own Linux VM — they found just a ton of fun. Headline party tricks hard-coded. What looks like attempts to train off Claude. And yes, you can get it to run your own things on that VM deep within Meta, which Jonny points out is a botnet paradise waiting to happen. This banger of a bad ‘un is being given away to Meta users, who pretty much define a vulnerable target audience for tech that may not be totally trustworthy. Meta, of course, is dead against AI safety regulation. Place your bets on the headlines to follow. In any case, the upside of using agents as demoed in every keynote is either so trivial and trite — book me a restaurant! — or so catastrophic if it fails — organize my family vacation! — that consumer agents are a bust. It’s just that Meta may be bringing a lot of accelerant to that particular fire. There are other fires, of course, many known about since everything kicked off. Model collapse due to training on data contaminated with output? Still a thing, still no fix a "better" AI might provide. And that habit AI has of retailing everything and paying for nothing, choking off its own training data supply chain and a huge chunk of commerce and culture with it? Very much still a thing. Where AI does make consistent progress is in domain-specific tasks. Science, engineering, mathematics, and coding, if you match a model with enough reinforcement learning with humans who know how to manage them, there are startling results. You don’t get much innovation, not much newness, which is the true driver of progress. Better than humans is highly contextual. Computers have been better at chess for decades, but there isn’t a better chess. Horses run faster than humans, and that’s been useful for four millennia, but we got to peak horses quite some time ago. So the question is, have we reached peak AI, where the cost of future developments don’t match the risk they won’t work well enough to justify? If ChatGPT already writes a perfectly good business letter in a couple of seconds, there is no meaningful improvement to be had for users. Most AI is profoundly annoying, in ways that have nothing to do with the models and everything to do with perverted UX in an attempt to justify growth. That is both unsustainable and a tacit admission that AI hasn’t solved its use case problem. If we are at or near peak AI, is that the crisis that bursts that goddam bubble? Certainly a readjustment, as all bets on the continuous uptick curve are off. AI as it stands is useful enough to power a big industry sector, just in ways that are and remain at odds to how it is marketed and its future fantasies. Trust in AI Max may be shaken, but there's still room for lots of improvement. That slowdown that everyone who isn’t a miscreant is calling for may happen organically, if we escape the smoking-is-good-for-you mentality of the Metans and the Muskovites. That’s still a gamble we shouldn't be taking, but the odds may be better than we thought. ®
/s3/static.nrc.nl/wp-content/uploads/2026/10/05094503/051026VER_2037196997_hack.jpg)
