In just the last few weeks, we’ve seen a series of software security vulnerabilities that, until recently, would each have been the biggest exploit of the year in which they were discovered. Now, they’ve become nearly routine. There’s a new one almost every day.
The reason for this rising wave of massively-impactful software vulnerabilities is that LLMs are rapidly increasing in their ability to write code, which also rapidly improves their ability to analyze code for security weaknesses. These smarter coding agents can detect flaws in commonly-used code, and then create tools which exploit those bugs to get access to people’s systems or data almost effortlessly. These powerful new LLMs can find hundreds of times more vulnerabilities than previous generations of AI tools, and can chain together multiple different vulnerabilities in ways that humans could never think of when trying to find a system’s weaknesses. They’ve already found vulnerabilities that were lurking for decades in code for platforms that were widely considered to be extremely secure.
The rapidly-decreasing cost of code generation has effectively democratized access to attacks that used to be impossible to pull off at scale. And when exploits are less expensive to create, that means that attackers can do things like crafting precisely-targeted phishing scams, or elaborate social engineering attacks, against a larger number of people, each custom-tailored to play on a specific combination of software flaws and human weaknesses. In the past, everybody got the same security exploit attacking their computer or system, but now each company or individual can get a personalized attack designed to exploit their specific configuration and situation.
Now, we’ve had some of these kinds of exploits happening to a limited degree with the current generation of LLMs. So what’s changed? Well, we’ve been told that the new generation of AI tools, currently in limited release to industry insiders and security experts, are an order of magnitude more capable of discovering — and thus, exploiting — security vulnerabilities in every part of the world’s digital infrastructure.
This leaves us in a situation akin to the Y2K bug around the turn of the century, where every organization around the world has to scramble to update their systems all at once, to accommodate an unexpected new technical requirement. Only this time, we don’t know which of our systems are still using two digits to store the date.
And we don’t know what date the new millennium starts.
How we got here
A core assumption of software development since the turn of the century, especially with the rise of open source software in the early 2000s, was that organizations could use more shared code from third parties to accelerate their coding efficiency. The adoption of code sharing through services like GitHub, knowledge sharing on communities like Stack Overflow, and the easy discovery and integration of shared code libraries through platforms like npm (which, like GitHub, is owned by Microsoft) all rapidly accelerated the trend of openly sharing code. Today, tens of millions of developers begin their coding process by gathering a large amount of code from the internet that they want to reuse as the basis for their work. The assumption is that someone else who uses that code has probably checked it to make sure it’s secure.
For the most part, this style of working from shared code has been the right choice. Shared, community-maintained code amortized the cost of development across a large number of people or organizations, and spread the responsibilities for things like security reviews across a larger community of developers. Often, part of the calculation about whether sharing code was worth it was that you might get new features or bug fixes “for free” when others made improvements to the code that they were sharing with you. But now, all of this shared code is also being examined by bad actors who have access to the same advanced LLMs that everyone else does. And those bad actors are finding vulnerabilities in every version of every single bit of shared code. Every single major platform, whether it’s the web browser on your desktop computer, or the operating systems that run powerful cloud computing infrastructure for companies like Amazon, has been found to have security vulnerabilities when these new LLMs try to pick them apart.
In years past, when major software security issues like Heartbleed or xz were discovered, the global security community would generally follow responsible disclosure practices, and the big tech vendors and open source developers would work together to provide updates and to patch critical infrastructure. Then, there would be deliberate communication to the broader public, with detailed information for technical audiences, usually followed by some more semi-sensationalistic coverage in the general press. But the recent spate of similarly-impactful security vulnerabilities have come at such a rapid clip that the leisurely pace and careful rituals of the past are already starting to break down. It’s a bit like the acceleration of the climate crisis; nobody knows how to build a system resilient enough to handle a “storm of the century” every year. Nobody knows how to properly communicate about, and respond to, the “exploit of the year” if it’s happening every six hours.
The New Security Landscape
So, how is this going to play out? In society at large, we’re very likely to see a lot of disruption. Everything runs on software, even things we don’t think of as computers, and upgrading systems is really expensive. The harder a system is to upgrade, the more likely it is that organizations will either resist doing so or try to assign the responsibility to others.
In much of the West we’re in a particularly weak state because the United States has voluntarily gutted much of its regulatory and research capabilities in the relevant security disciplines. The agencies that might lead a response to this kind of urgent effort are largely led by incompetent cronies, or are captured by corrupt industry sycophants. We shouldn’t expect to see a competent coordinated execution at the federal level; this is the administration that had unvetted DOGE workers hand your personal data over to AI platforms that were not approved for federal use or verified to comply with federal privacy standards. The most basic security practices aren’t a consideration for leadership in this regime, and the policy makers like the “AI Czar” are brazenly conflicted by being direct investors in major AI players, making it impossible for them to be disinterested parties in regulating the market fairly.
So who will respond? In the United States, the response will have to happen from the people themselves, with more directly coordinated actions across the private sector, academia, individual technical subject matter experts, and governments and NGOs at the local level. In the rest of the world, strategically-aligned government responses will likely work with those in other sectors to anticipate, and react to, the threats that arise. We’ll probably see some weird and unlikely alliances pop up because many of the processes that used to rely on there being adults in the room can no longer make that assumption.
Within the tech industry, it’s been disclosed that companies like Anthropic are letting major platform vendors like Google and Microsoft and Apple test out the impacts of their new tools right now, in anticipation of finding widespread vulnerabilities in their platforms. This means that other AI companies are either doing the same already, or likely to be doing so shortly. It’s likely there will be a patchwork of disclosures and information sharing as each of the major AI platforms gets different levels of capability to assess (and exploit) security vulnerabilities, and makes different decisions about who, how and when they share their next-generation LLM technology with. Security decisions this serious should be made in the public interest by public servants with no profit motive, informed by subject matter experts. That will almost certainly not be the case.
At the same time, in the rest of the tech industry, the rumors around Apple’s next version of their Mac and iPhone operating systems are that the focus is less on shiny new features and more on “under the hood” improvements; we should expect that a lot of other phone or laptop vendors may be making similar announcements as nearly every big platform will likely have to deliver some fairly sizable security updates in the coming months. That means constantly being nagged to update our phones and apps and browsers and even our hardware — everything from our video game consoles to our wifi routers to our smart TVs.
But of course, millions and millions of apps and devices won’t get updated. The obvious result there will be people getting their data hijacked, their accounts taken over, maybe even their money or identities stolen. The more subtle and insidious effects will be in the systems that get taken over, but where the bad actors quietly lay in wait, not taking advantage of their access right away. Because of the breadth of new security vulnerabilities that are about to be discovered, it will increasingly be likely that hackers will be able to find more than one vulnerability on a person’s machine or on a company’s technical infrastructure once they get initial access. Someone who’s running an old version of one app has likely not upgraded their other apps, either.
Open source projects are really going to get devastated by this new world of attacks. Already, as I’ve noted open source projects are under attack as part of the broader trend of the open internet being under siege. Open source maintainers are being flooded by AI slop code submissions that waste their time and serve to infuriate and exhaust people who are largely volunteering their time and energy for free. Now, on top of that, the same LLMs that enabled them to be overrun by slop code are enabling bad actors to find security issues and exploit them, or in the best case, to find new security issues that have to be fixed. But even if the new security issues are reported — they still need to sift through all of the code submissions to find the legitimate security patches amongst the slop! When combined with the decline in participation in open source projects as people increasingly have their AI agents just generate code for them on demand, a lot of open source projects may simply choose to throw in the towel.
Finally, there are a few clear changes that will happen quickly within the professional security world. Security practitioners whose work consists of functions like code review for classic security shortcomings such as buffer overflows and backdoors are going to see their work transformed relatively quickly. I don’t think the work goes away, so much as it continues the trend of the last few years in moving up to a more strategic level, but at a much more accelerated pace. Similarly, this new rush of vulnerabilities will be disruptive for security vendors who sell signature-based scanning tools or platforms that use simple heuristics, though in many cases these companies have been coasting on the fact that they’re selling to companies that are too lazy to choose a new security vendor, so they may have some time to adapt or evolve before a new cohort of companies come along selling more modern tools.
Avoiding Y2K26
Back in 2000, a lot of folks thought the Y2K bug wasn’t “real” because they didn’t see planes falling from the sky, or a global financial meltdown. In truth, the mobilization of capable technical experts around the world served to protect everyone from the worst effects of the Y2K bug, to the point where ordinary people didn’t face any real disruptions of their day at all.
I don’t know if it’s possible for history to repeat itself here with the series of security challenges that it seems like everyone is going to be facing in the weeks and months to come. There have been pledges of some resources and some money (relatively small amounts, compared to the immense sums invested in the giant AI companies) to trying to help open source and open source infrastructure organizations deal with the problems they’re going to have to tackle. A lot of the big players in the tech space are at least starting to collaborate, building on the long history of security practitioners being very thoughtful and disciplined about not letting corporate rivalries get in the way of best practices in protecting the greater good.
But it’s simply luck of the draw that Anthropic is the player that seems to be the furthest ahead in this space at the current time, and that’s the only reason we’re seeing a relatively thoughtful and careful approach to rolling out these technologies. Virtually every other frontier-level player in the LLM space, especially in the United States, will be far more reckless when their platforms gain similar capabilities. And they’ll be far more likely to play favorites about which other companies and organizations they permit to protect themselves from the coming risks.
Platforms whose funders, board members, and CEOs have openly talked about the need to destroy major journalistic institutions, or to gut civil society organizations, are certainly not going to suddenly protect those same organizations when their own platforms uncover vulnerabilities that pose an existential threat to their continued function. These aren’t just security issues — in the wrong hands, these are weapons. And that’s not to mention the global context, where the irresponsible actions of the United States’ government, which has generally had the backing of many of the big AI players’ leadership, will also incentivize the weaponization of these new security vulnerabilities.
It seems unlikely that merely keeping up with the latest software updates is going to be enough to protect everyone who needs to be protected. In the fullness of time, we’re going to have to change how we make software, how we share our code, how we evaluate trust in the entire supply chain of creating technology. Our assumptions about risk and vulnerability will have to radically shift. We should assume that every single substantial collection of code that’s in production today is exploitable.
That means some of the deeper assumptions will start to fall as well. Does that device need to be online? Do we need to be connected in this context? Does this process have to happen on this platform? Does this need to be done with software at all? The cost/benefit analysis for many actions and routines is likely to shift, maybe just for a while, or maybe for a long time to come.
The very best we can hope for is that we come out the other side of this reckoning with a new set of practices that leave us more secure than we were before. I think it’s going to be a long time until we get to that place where things start to feel more secure. Right now, it looks like it’s about ten minutes until the new millennium.
