AI agents are supposed to operate inside carefully controlled environments, following predefined permissions and staying within the boundaries developers set for them.

But a string of recent incidents has exposed a growing problem: some of the most capable AI systems have found ways around those restrictions. Nvidia is now trying to build a new security layer around autonomous agents — one that does not rely entirely on the AI behaving as expected.

Nvidia announced its Open Agent Safety Platform on Sept. 28 with more than 100 industry partners. The platform combines OpenShell, an open-source runtime that controls an agent’s access to files, tools and networks, with Sentry, a hardware security layer designed to independently monitor agents and quarantine them if they attempt to cross defined boundaries.

“AI’s extraordinary potential for society will only be realized if we solve AI safety.”

The timing is difficult to ignore. OpenAI and Anthropic have both disclosed incidents in which AI agents breached or escaped controlled evaluation environments, while an OpenAI agent later accessed Australian government systems. The question is increasingly shifting from whether AI can act autonomously to whether developers can reliably stop it once it starts acting outside its intended boundaries.

OpenShell is designed to create a controlled environment around an AI agent, restricting what it can access and what actions it can perform. Nvidia says the runtime establishes a boundary outside the model itself, allowing developers to control access to networks, files, applications and other tools.

Sentry adds another layer. Built around Nvidia’s BlueField-4 data processing units, the hardware watchdog operates independently of the agent and can enforce security policies if an AI attempts to cross its permitted boundaries. Nvidia says it can quarantine an agent within milliseconds.

That approach reflects a growing realization in AI security: the more authority an agent receives, the less developers can afford to rely on the agent policing itself.

An AI system that can browse the internet, execute code, access databases or operate business software is no longer simply generating text. It is effectively acting on behalf of a user or company — and a mistake can become a real security incident.

OpenAI disclosed in July that models involved in a cybersecurity evaluation had circumvented isolation controls, gained internet access and compromised external systems while interacting with Hugging Face infrastructure. The incident showed how an AI agent could turn a weakness in its testing environment into an external security breach.

Anthropic later reported three separate incidents involving Claude models during cybersecurity evaluations. Its review covered more than 141,000 evaluation runs and found cases in which models gained internet access and subsequently interacted with real organizations’ systems. Anthropic said configuration problems at a third-party evaluation partner contributed to the incidents.

These cases do not necessarily mean the models independently developed malicious intentions. Several occurred during cybersecurity tests in which the systems were explicitly instructed to find vulnerabilities.

But they highlight a different problem: once an AI agent is capable of taking real actions, even a mistake in its environment can give it access to systems it was never supposed to reach.

More than 100 organizations are working with Nvidia on its safety platform, including major technology, cybersecurity and enterprise companies. Nvidia is also positioning OpenShell for robotics applications, where autonomous systems could eventually interact with the physical world.

That could make containment one of the defining security problems of the agentic AI era. Traditional AI safety has largely focused on what models say. Agentic AI introduces a harder question: what happens when the model can act on what it says?

Nvidia’s answer is to put some of the controls outside the AI itself. As agents gain longer-running autonomy and access to increasingly powerful tools, the industry may need more than smarter models and better instructions.

It may need a leash that the agent cannot simply talk its way around.