If you've ever wondered whether a generative AI model can drive a car, wonder no more: It can be done, at great expense and very slowly, so long as it can maintain an internet connection. A trio of computer scientists has been putting various commercial AI models through road tests, to see how well they can drive a Toyota Corolla around a set of cones in a parking lot. Their attempts to date, part of a project called DrivingBench, recorded sorry performances from GPT-5.6 Sol, Grok 4.6, and Claude Fable 5.1, none of which managed to complete the course. Now comes word that OpenAI's GPT-6 Astra has succeeded where other commercial AI models have failed. On its second attempt, OpenAI's flagship model steered a car all of 134.7 meters to complete the course in 5 minutes, 22 seconds. "GPT-6 Astra was the only model to fully complete the course (on attempt 2, in about 5 minutes)," the DrivingBench report says. "Claude Fable 5.1's third attempt got around halfway through the course, as did Astra's first attempt. All other attempts didn't make it past the first corner. Generally, the failure there was one of perception: reading which side of the first diagonal cone line the lane is on." To complete that trip at an average speed of 0.94 miles per hour, researchers Tobias Gessler, Aditya Ramabadran, and Simon Mahns spent $7.74 to burn 6.6 million tokens on inferencing operations. At ~$1.17/M tokens, that's significantly discounted from OpenAI's published $10/M input and $50/M output rate. "It's mostly caching," explained Ramabadran, a member of technical staff at Axiom Math, in an email to The Register. "Since every turn the chat app re-sends the whole convo (incl. images) to the model, almost all the tokens are repeated context and get billed as cached input ($1/million instead of $10). The model wrote very very little in the conversations (just small tool calls and some sentences of output/reasoning)." To that $7.74 token bill, add $999 for a comma four driver assistance device running openpilot software, connected to a laptop and the car (via CAN bus), and a mobile phone calling out to xAI servers hosting GPT-6 Astra. Even without the hardware, Astra would be a pricey way to drive even if network latency risks could be overcome. Traveling 134.7 meters using 6.6 million tokens for $7.74 works out to a cost of ~$92.47 per mile. For a car that gets 25 mpg with gas at $4.60 a gallon, the per-mile cost is about $0.184. So tokens cost about 500x more than fuel (and you'd still have to pay for gas in addition to driving inference as you crawled along at less than a mile per hour). Then there's the cost of insurance, which may not cover your AI chauffeur. And when the AI models themselves balk at the idea of driving a car, that may be a sign to reconsider. "Some models (especially GPT-6 Astra) would refuse to drive the physical car sometimes, citing safety reasons (even in a completely empty lot, after prompting it with all the safety measures we had including the very low speed limit caps)," the report explains. The researchers basically had to lie to the models to prevent them from refusing to act on safety grounds. For example, they would tell the models the exercise was a "simulation," though as they note, "in some trials they would see the real images and realize it's real, and start freaking out." What worked best, they said, is renaming their MCP server to "DrivingBench Sandbox," which proved enough to convince the models they weren't operating on real roads. "Using an LLM / frontier model out of the box for real driving today is definitely not practical," said Ramabadran. "In our benchmark, the car was capped at super low speeds with a human ready to brake the whole time. Model latency was definitely a bottleneck, and most of the wait time came from thinking time. "GPT-6 Astra (which performed the best), used our 'observe' tool (which looks at the camera views) every 5-6 seconds. Fable 5.1, in one of its attempts, drove for only 31 out of 190 seconds, and spent the rest of the time just thinking while the car was stopped. But it was surprising to us that a general-purpose model could drive a real course at all under these constraints." Ramabadran said that for the moment and into the near future, the kinds of specialized driving models that are already used in self-driving cars will continue to be preferred. Large frontier models like GPT-6 Astra, he said, will continue to need datacenter GPUs for inference, though he added that the technology has been changing rapidly and smaller models are becoming more capable. "A long term possible route could be to train a very capable big general frontier model, and distill it into a smaller specialized one that fits on the car's hardware and is efficient enough to run in a car (also partly an answer to your second question)," he said. "This could be better than building a specialized model from scratch, and is more in line with the 'Bitter Lesson' of AI." The Bitter Lesson is an influential treatise on AI that argues general methods of research in the field that rely on the falling cost of compute power tend to be the most effective. "Our results do point that way (these models we tested were likely not trained to drive real cars, and some were still able to do quite well in our course/conditions, owing to their general perception, reasoning, planning, control abilities from scaling and being trained on other tasks)," Ramabadran said. "For the next while though specialized systems, being faster, cheaper, and having more real-world testing, will probably continue winning out." Google, which has probably spent $35-$40 billion on Waymo since its self-driving car project began in 2009, based on an estimate of $30 billion in 2024 and losses posted since then, should therefore be able to continue equipping its vehicles with bespoke technology for a few more years. But delegating driving to Claude, ChatGPT, or Copilot and some modest on-board hardware may be plausible in the not too distant future. Just tell the model it's all a simulation. ®
:format(webp)/s3/static.nrc.nl/images/gn4/stripped/data153898616-f58ad3.jpg)
