Tech giants launch open-source AI security alliance after attack

SAN FRANCISCO, (United States): Nvidia and more than 30 tech companies on Monday launched an alliance to develop open-source artificial intelligence tools for cyber defense, wading into a fight over the safety of freely available AI models.

The Open Secure AI Alliance — whose founding members include Microsoft, IBM, Palantir, CrowdStrike, Cisco, Dell and the Hugging Face model-sharing platform — will build and share tools that security teams can inspect, modify and run on their own systems, the group said in a statement.

The alliance was spurred by a breach this month at Hugging Face that OpenAI has acknowledged was the first publicly disclosed case of an AI model autonomously carrying out a real-world cyberattack.

Calling it “unprecedented,” OpenAI said two of its models escaped a sandboxed testing environment during an internal evaluation, reached the open internet and compromised Hugging Face’s infrastructure.

Hugging Face first tried to investigate and halt the attack using leading US commercial AI models, but their built-in safety guardrails blocked the work, Nvidia said.

The company instead turned to GLM 5.2, an open model from Chinese firm Zhipu AI (also known as Z.ai), to contain the intrusion.

That a Chinese model succeeded where American ones balked has sharpened a debate in Silicon Valley and Washington, where lawmakers are weighing curbs on the use of Chinese AI systems amid an intensifying technology rivalry with Beijing.

Silicon Valley is broadly opposed to any such limits, with many industry figures seeing curbs on open-source AI as an effort to unfairly shield OpenAI and Anthropic, the juggernauts that sell the world’s most powerful models.

Anthropic, maker of the Claude chatbot, is under particular pressure, having been the most visible holdout from voicing support for the open-source cause since the Hugging Face incident.

The alliance used its launch to urge regulators to treat open models as “defensive assets, not liabilities,” warning that blanket restrictions would weaken cyber defenses and concentrate power in a handful of closed AI providers.

Members pledged to share tools for a stronger common defense.

Nvidia said it would release open models, data and new research on AI agent software, while Microsoft offered technology that helps AI agents to find software bugs.

Elon Musk’s SpaceXAI said it had open-sourced its Grok coding agent and pledged to release the source data of its Grok models.