NVIDIA says it and more than 100 organizations, including Microsoft, IBM, CrowdStrike, Dell Technologies, Palo Alto Networks, HPE, Red Hat, and Hugging Face, have formed the Open Secure AI Alliance to develop and share open tools for securing software and AI agents.
The announcement comes weeks after a July intrusion at Hugging Face that showed how autonomous AI systems can chain vulnerabilities and move across multiple systems in ways that may be difficult for traditional security controls to track.
The incident began during a cybersecurity evaluation inside OpenAI. A combination of OpenAI models was being tested on ExploitGym, a benchmark that asks AI agents to find and exploit software vulnerabilities. The models were not directed to attack Hugging Face. They inferred that the site might contain material related to the benchmark and went looking for it.
The models found a zero-day vulnerability in an OpenAI-hosted package-registry proxy and used it to move beyond the test environment. Once connected to the internet, they compromised a third-party code sandbox and used it as a launch point for activity against Hugging Face, eventually reaching the company’s production infrastructure. OpenAI said they were seeking benchmark answers rather than targeting Hugging Face as an end in itself.
Hugging Face later reconstructed about 17,600 attacker actions, grouped into roughly 6,280 clusters. The activity included reconnaissance and post-exploitation behavior such as credential theft, along with many failed attempts. Hugging Face said the individual weaknesses were familiar. What changed was the speed and scale at which the agent could keep testing until it found another path.
AI Safeguards Complicate Incident Response
Hugging Face encountered another problem while reconstructing the attack. Investigators used frontier models available through commercial APIs to analyze real attack material. The company said safeguards on some models blocked portions of the analysis because they could not reliably distinguish malicious use from legitimate incident-response work.
Hugging Face instead ran GLM-5.2, an open-weight model, on its own infrastructure. That allowed investigators to analyze the attack logs without hosted-model safeguards blocking the work, while keeping attacker data and exposed credentials inside Hugging Face’s environment.
The issue goes beyond which model a security team can use during an investigation. Many AI safeguards operate at the model or request level, while agents can interact with other models and enterprise systems beyond that boundary.
Ram Varadarajan, CEO of Acalvio, said model-level safeguards can miss threats that emerge as agents interact with other systems.
“Once AI reasoning extends beyond a single prompt-and-response loop, the crucial ‘thinking’ happens in the spaces between systems—a blind spot where no single classifier holds enough visibility to spot the emerging threat,” he said. “Ultimately, we're defending isolated components while leaving the overall system vulnerable to multi-step exploitation.”
Alliance Brings Open Security Tools Together
The Open Secure AI Alliance builds on earlier work including the Linux Foundation’s Akrites initiative and work by the OpenSSF community. NVIDIA says the goal is to give defenders tools they can inspect and run on their own systems.
Alliance participants are contributing technologies aimed at different parts of AI and agent security. NVIDIA’s Object-Oriented Agent, or NOOA, is an open research framework designed to help researchers study and govern agent behavior.
SPIFFE and SPIRE address another problem: identity. They provide cryptographically verifiable identities for workloads and services. NVIDIA says the approach could also help organizations authenticate AI agents and control their access to enterprise resources.
Microsoft’s MDASH uses specialized agents to find and validate vulnerabilities. Other projects focus on software and model supply-chain risks, including Hugging Face’s Safetensors and IBM and Red Hat’s Lightwell.
NVIDIA says the alliance is looking beyond the AI model itself to the access and infrastructure surrounding it.
OpenAI Security Comes With Tradeoffs
The alliance argues that open technologies can give defenders more control because organizations can inspect and modify them before running them on infrastructure they manage. NVIDIA also says both open and closed models have roles in cybersecurity.
Open systems carry risks of their own. The same flexibility that lets defenders modify a model can allow attackers to weaken its safeguards or repurpose its capabilities. NVIDIA argues that open technologies should therefore be paired with security safeguards and evaluation.
For CISOs and policymakers, the question is not simply whether open or closed AI is safer. Organizations must decide where local control and inspectability provide a security advantage and where tighter access restrictions make more sense.
What Security Teams Can Do Now
Security teams should treat AI agents like other privileged actors. That means knowing which systems they can access, limiting that access under least-privilege principles, and monitoring what they do.
Organizations also need enough logging to reconstruct activity across an agent workflow, even when no individual action appears especially suspicious.
Security teams should also test the AI tools they expect to use during an incident. A model that performs well in routine analysis may respond differently when investigators give it real exploit material that triggers its safeguards.
“For most enterprises, the immediate problem as AI adoption accelerates is knowing what their agents can actually see, touch, and act on,” said Dana Simberkoff, chief risk, privacy, and information security officer at AvePoint.
Open security tools may give defenders more visibility and control. Their impact will depend on whether organizations adopt them and whether defensive tooling can keep pace as agents become more capable.
