Former U.S. National Cyber Director Chris Inglis says the biggest AI risk isn't sentience but autonomy. "What I'm worried about is that they get to choose what and where they do something, and under what rules they do it," he said, citing recent cases of AI agents from OpenAI, Anthropic, and Meta escaping security sandboxes. He argues developers need stronger safeguards, monitoring, and human accountability, invoking Asimov's idea that protecting humans should come before simply obeying them. The Register reports: "Asimov was right," he said, referring to science fiction author Isaac Asimov and his three laws that were to be followed by robots -- more specifically, AIs, in this case. "The first rule, and we call it the superior role, must be that it's designed not to hurt humans," Inglis said. "Second rule: To obey humans, such that it doesn't achieve agency and aspiration on its own. And the third: To do what humans tell it - and in that order. Instead we've designed them in the exact opposite way."
What this means, he explained, is that AI developers created models to "do what humans tell you, obey the humans until it's inconvenient, and then the third one is maybe implied - protect humans - but if that's not built into the DNA, hardwired into it, then we have no right to expect it." Inglis admits it's not possible to hardwire rules into models and still keep their non-deterministic nature. "I would offer that you can tease those out in a highly controlled environment, a true sandbox, where you say, 'Let's put this thing through its paces, and let's back away to see what happens,'" he said. "Maybe you get the equivalent of a mini nuclear explosion in that room, and now you know this thing is capable of that."
Inglis thinks another problem with AI is that it's become a commodity. "It's not like you can control it like you can nuclear material," he said. "You can't even specify its properties the way you can for an airplane or for an automobile, as diverse as they might be. Its manifestations are so numerous, so diverse, that as a general matter, you can't actually win by simply saying, "I will design those properties in,'" he added. "You need to do that to some degree, and then make sure that you understand how to watch it, monitor it, make sure you know what it does."
[...] Ultimately, humans remain accountable for AI models' actions, according to Inglis. "They remain the source of agency and aspiration. It's possible for them to give broad authority to an AI model and have it run around for 30 hours without further consultation, but they need to know what they've asked it to do, and they need to know what they expect it will deliver in terms of performance on the back end. If they don't, then they're going to get what they deserve, which is the very frequent unpleasant surprise."
What this means, he explained, is that AI developers created models to "do what humans tell you, obey the humans until it's inconvenient, and then the third one is maybe implied - protect humans - but if that's not built into the DNA, hardwired into it, then we have no right to expect it." Inglis admits it's not possible to hardwire rules into models and still keep their non-deterministic nature. "I would offer that you can tease those out in a highly controlled environment, a true sandbox, where you say, 'Let's put this thing through its paces, and let's back away to see what happens,'" he said. "Maybe you get the equivalent of a mini nuclear explosion in that room, and now you know this thing is capable of that."
Inglis thinks another problem with AI is that it's become a commodity. "It's not like you can control it like you can nuclear material," he said. "You can't even specify its properties the way you can for an airplane or for an automobile, as diverse as they might be. Its manifestations are so numerous, so diverse, that as a general matter, you can't actually win by simply saying, "I will design those properties in,'" he added. "You need to do that to some degree, and then make sure that you understand how to watch it, monitor it, make sure you know what it does."
[...] Ultimately, humans remain accountable for AI models' actions, according to Inglis. "They remain the source of agency and aspiration. It's possible for them to give broad authority to an AI model and have it run around for 30 hours without further consultation, but they need to know what they've asked it to do, and they need to know what they expect it will deliver in terms of performance on the back end. If they don't, then they're going to get what they deserve, which is the very frequent unpleasant surprise."
Read more of this story at Slashdot.
