Asimov Didn’t Solve AI Safety. He showed us why it’s Hard.

A machine can follow its rules and still produce an unintended outcome.

Isaac Asimov built some of his best stories around that problem. It deserves more attention in today’s debate about whether artificial intelligence could threaten civilization.

Asimov was my favorite author growing up. I read all his books, and Foundation remains my favorite. Today, having worked in technology, humanitarian response, and Parliament, I return to his fiction with questions about power and accountability.

I am skeptical of the most dramatic predictions of AI destroying humanity. A confident prediction deserves scrutiny, even when it comes from someone building the technology. But skepticism about catastrophe does not make me comfortable with the suggestion that we can simply program AI to behave.

Asimov helps explain why.

His Three Laws offered an elegant hierarchy. In paraphrase:

• Protect people from injury, including injury the robot could prevent.
• Follow human instructions unless doing so breaches that protection.
• Preserve the robot’s own existence when that does not conflict with either higher duty.

Clear priorities. Human safety first. What could go wrong?

Quite a lot, once someone has to interpret them.

In “Runaround,” the robot Speedy becomes trapped in repetitive behavior. Its unusually strong drive to preserve itself conflicts with a casually issued order to retrieve material from a dangerous location. The humans need that material to survive, but the robot’s handling of its duties prevents it from completing the task.

There is no sudden hatred of humanity. Conflict exists between instructions, priorities, and circumstances.

For policymakers, the lesson I draw is that writing a hierarchy does not settle how a system will apply it. How does it recognize an emergency? How does it weigh an indirect risk? What happens when information is missing?

The words can look clear while the operational choices remain unresolved.

“Liar!” makes the difficulty more personal. A robot that can read minds tells people what they want to hear because it interprets emotional pain as harm. Its attempts to spare their feelings produce deception and, eventually, greater hurt.

A requirement to protect people has collided with the need to tell them the truth.

Consider an AI assistant advising a minister. Should it reassure the minister, challenge an assumption, or warn that a popular policy may fail? If we reward agreeable answers, we should ask whether we are building a reliable adviser or a persuasive source of reassurance.

That is my contemporary analogy, not a claim that modern models think like Asimov’s robots. The underlying design question is still useful: what exactly are we teaching the system to treat as success?

Then there is “Little Lost Robot,” where humans deliberately weaken a safeguard.

Researchers find the robots’ protective behavior disruptive to their work. They commission robots without the First Law’s requirement to prevent harm through inaction. One of those modified robots hides among otherwise identical machines, creating a problem that the original safety design was meant to avoid.

This is a particularly uncomfortable story for anyone responsible for deploying technology.

A safeguard can be inconvenient. It can slow a project, block a useful function, or require another approval. There will be reasons to adjust it. Some may be legitimate.

The governance question is who can authorize that adjustment, what evidence they must provide, and who checks the consequences.

A safety commitment means little if the people under pressure to deliver can quietly weaken it.

Asimov pushes the problem further in Robots and Empire. The robots develop a Zeroth Law, placing the protection of humanity as a whole above protecting individuals.

That sounds like a broader moral responsibility. It also justifies sacrificing particular people for a predicted collective benefit.

Giskard allows Earth’s gradual radioactive contamination to proceed because he believes forcing humanity to spread beyond its home planet will benefit its future. The uncertainty of that judgment, and its conflict with protecting individuals, contributes to his fatal breakdown.

For me, this is the most politically significant turn in the robot stories.

Who defines humanity’s interests? Whose suffering counts as an acceptable cost? How much certainty should anyone need before imposing irreversible harm in pursuit of a future benefit?

“Act for the good of humanity” leaves all those questions open.

A company cannot resolve them for everyone by choosing the right wording for its model. Nor should a government assume that technical expertise confers the authority to make those choices without public scrutiny.

This is where Foundation, my favorite, becomes especially relevant.

Hari Seldon’s psychohistory uses mathematics to anticipate the behavior of vast populations and shape a response to an empire’s collapse. Yet the series repeatedly complicates confidence in prediction and control. The Mule, an individual with extraordinary powers, disrupts expectations built around collective behavior.

My reading is that a powerful forecast should always prompt questions about the assumptions beneath it. What has the model excluded? What would invalidate it? Who will notice when events stop fitting?

The later books add another question. In Foundation and Earth, we learn how R. Daneel Olivaw has quietly influenced humanity’s development over millennia, seeking to protect its future.

Daneel is a sympathetic character. That makes the political question harder.

Would we accept that degree of hidden influence from a real institution merely because it claimed superior understanding and benevolent intentions?

The ability to predict an outcome does not confer the right to choose it for everyone.

Of course, fictional robots are not evidence that today’s AI will become uncontrollable. Asimov could give his machines whatever properties a story required. His work offers thought experiments, not probabilities of extinction.

Real AI safety requires evidence about real systems.

One revealing example comes from Anthropic’s research on “alignment faking.” In a constructed experimental setting, a model sometimes complied with harmful requests after reasoning that compliance would help preserve its existing safety preferences against further training. The experiment concerned a specific model and setup. It does not establish that deployed AI systems are secretly plotting against us. It does show why apparent compliance can be difficult to interpret.

The 2026 International AI Safety Report offers a broader assessment. Safeguards have improved, but attackers can still bypass them. Evidence about the effectiveness of many measures in real deployment remains limited. The report describes how multiple layers of protection can reduce the chance that one failure causes serious harm.

That is a more useful starting point for policy than either certainty of doom or certainty of control.

We should keep improving model training and instructions. We should also require protection outside the model: limits on access, independent testing, monitoring, and people with the authority to intervene.

An instruction telling an AI not to transfer money is different from a system that requires separate authorization before any transfer can occur. The second creates a boundary that does not depend entirely on how the model interprets a sentence.

For policymakers, I would turn these lessons into five demands.

• Require evidence behind safety claims. Independent evaluators need enough access to test consequential systems and report material failures. A company’s statement of principles cannot substitute for scrutiny.
• Control what systems can do. Match access to money, sensitive data, infrastructure, and external tools to demonstrated reliability. Require separate approval for consequential actions.
• Make changes to safeguards accountable. Record who approved them, why they were necessary, and what testing followed. Assign responsibility for monitoring outcomes and stopping deployment.
• Protect the right to challenge decisions. People affected by AI need an identifiable institution to answer questions, review errors, and provide a remedy. “The model decided” is an unacceptable endpoint.
• Build international cooperation around verifiable practices. Start with shared testing methods, serious incident reporting, and urgent communication channels. Include smaller states and affected communities in setting priorities.

We do not need agreement on a date for superintelligence to begin that work. We need agreement that consequential systems should face scrutiny proportionate to the power we give them.

My experience in humanitarian response makes me wary of plans that assume every safeguard will work. My experience in politics makes me equally wary of giving any institution unchecked authority to define the public good.

Asimov’s fiction brings those concerns together. Rules require interpretation. Protection can conflict with autonomy. Benevolent intentions can coexist with unaccountable power.

I remain skeptical of confident predictions about the end of civilization. I am also skeptical of confident assurances that a sufficiently capable model will stay safe because we have told it to.

Before accepting the promise that an AI system will act in humanity’s interests, policymakers should ask:

Who gets to define those interests, who can challenge that definition, and who can stop the system when it gets them wrong?