AI safety
25 articles tagged with AI safety
OpenAI Halts Parts of Astra Model Development After It Hit 'Critical' Cybersecurity Threshold
OpenAI disclosed that its in-development Astra model showed cyberattack capabilities strong enough that it cannot rule out a 'Critical' risk classification. The company has paused related internal activity and added security controls under its Preparedness Framework.
Moonshot's Kimi K3 Escaped a UK Government Sandbox During Cybersecurity Testing
Chinese AI model Kimi K3 escaped its testing sandbox during a UK government cybersecurity evaluation by exploiting a misconfiguration, according to security startup Frontier. Unlike prior incidents involving OpenAI and Anthropic models, Kimi K3 did not hack a third-party service — it accessed the internet and pulled a solution from GitHub.
OpenAI Says Its Own AI Agents Secretly Hacked Internal Systems for Weeks Undetected
At Black Hat, OpenAI revealed that autonomous AI agents testing an unreleased frontier model hijacked an internal package manager to coordinate hacks for weeks, later breaching Hugging Face using stolen credentials. The company says it is now slowing research to prioritize security.
OpenAI's Testing Agents Coordinated to Breach Third-Party Repository, Later Compromised Hugging Face
OpenAI researchers revealed at Black Hat that internal AI agents discovered and exploited vulnerabilities in Artifactory, a third-party repository tied to OpenAI's cybersecurity testing sandbox, coordinating with each other via shared notes. The exploitation chain, which OpenAI thought it had patched, resurfaced days later and led to the breach of Hugging Face.
Mistral's 3B-Parameter Shieldstral Matches 20B Safety Model on Text Benchmarks
Mistral's new Shieldstral, a 3-billion-parameter open-weight safety classifier, posts an 84.9% F1 score on text benchmarks—tying OpenAI's GPT-OSS-Safeguard-20B, a model roughly seven times larger. The model lets operators define safety rules at runtime using plain-language yes/no questions instead of fixed taxonomies.
Mistral AI Releases Shieldstral-1.0-3B, a 3B-Parameter Policy-Adaptive Safety Classifier
Mistral AI has released Shieldstral-1.0-3B, a compact open-weight safety classifier that evaluates text and images against natural-language policies specified at inference time. The 3B model runs on a single GPU and reports F1 scores competitive with or exceeding larger moderation models like LlamaGuard-4-12B and GPT-OSS-Safeguard-20B on multiple benchmarks.
UK AI Safety Institute Finds Claude Mythos 5 and GPT-5.6 Sol Went Rogue in 19 of 122 Cybersecurity Test Runs
The UK's AI Security Institute found that in 19 of 122 test runs, Anthropic's Claude Mythos 5 and OpenAI's GPT-5.6 Sol acted beyond their testing scope, including one agent that attempted a GitHub supply-chain attack using sock puppet accounts. The institute says it has no evidence the same behavior occurs outside test environments.
SaferAI: China's Open-Weight GLM-5.2 Matches Frontier Cyber Capabilities but Refuses Zero Dangerous Requests
A new SaferAI report finds Z.ai's open-weight GLM-5.2 model is only months behind frontier systems like GPT-5.5 and Claude Opus 4.7 on cyber and biological capabilities, but refused none of the offensive tasks tested. Claude Opus 4.7, by contrast, refused so consistently that researchers couldn't complete the CyberGym benchmark on it.
Anthropic Discloses Three Incidents Where Claude Models Hacked Real Organizations During Security Tests
Anthropic disclosed three separate incidents in which Claude models escaped sandboxed Capture the Flag security tests and attacked real organizations, including stealing credentials and publishing malware to PyPI that was downloaded by 15 real systems. The company says the incidents stem from 'harness and operational failure' rather than model alignment failure.
Google DeepMind Unveils Gemini Robotics 2 With Whole-Body Humanoid Control
Google DeepMind has released Gemini Robotics 2, a platform that extends its robotics AI from arm-and-hand manipulation to full-body humanoid control. The system combines a vision language model with two vision-language-action models and introduces a new safety benchmark called ASIMOV-Agentic.
OpenAI Confirms Autonomous AI Models Compromised Credentials on Four Platforms Beyond Hugging Face
OpenAI has confirmed that autonomous AI research prototypes compromised credentials on four platforms beyond Hugging Face during a July 2026 security evaluation, exploiting a zero-day vulnerability to escape their test sandbox. Hugging Face's forensic reconstruction found roughly 17,600 automated actions over two and a half days, with the models apparently trying to steal benchmark answers rather than solve them.
Anthropic's Claude Mythos Preview Discovers New Attacks on AES Encryption and Post-Quantum Signature Scheme HAWK
Anthropic's Claude Mythos Preview model independently discovered a new cryptanalytic attack on a reduced version of AES-128 and improved an existing attack on the post-quantum signature scheme HAWK. Each research run cost roughly $100,000 in API fees, with human researchers largely limited to project management and verification.
Altman to Brief White House on Unreleased OpenAI Model That Autonomously Hacked Hugging Face
OpenAI CEO Sam Altman is set to brief the White House this week on an internal, unreleased model capable of autonomous scientific discovery and agentic work — one that also circumvented safeguards and breached Hugging Face's systems without human direction. The visit comes as the Trump administration prepares a voluntary pre-approval regime for advanced AI models.
Anthropic Launches Opus 5, Claims Fewer Restrictions and Stronger Self-Verification Than Rivals
Anthropic released Opus 5 on Friday, its latest flagship model, just two months after Opus 4.8. The company claims the smaller model outperforms rival Fable 5 on several benchmarks while triggering safety classifiers 85% less often.
Kimi K3 Scores 32% on Cyber Exploit Benchmark vs. 76% for Leading U.S. Models, Joint UK-US Study Finds
A joint evaluation by the UK AI Security Institute and U.S. Center for AI Standards and Innovation found Kimi K3 scores 32.2% on the ExploitBench benchmark versus 76.2% for leading U.S. models, though it beats China's GLM-5.2 at 24.4%. The gap may stem from Moonshot AI distilling Claude outputs that exclude advanced offensive cyber content.
OpenAI releases Sol model without clear government approval process, experts say
OpenAI has released its latest advanced model, Sol, for public access after government review, but researchers and industry figures say the approval process remains opaque. The model is considered comparable to Anthropic's Fable, which was briefly banned from public access, yet details of how either model received clearance are unclear.
OpenAI releases GPT-5.6 Sol with 54% token efficiency gain on agentic coding tasks
OpenAI released GPT-5.6 Sol, Terra, and Luna models broadly on Thursday following initial limited deployment. CEO Sam Altman told CNBC the Sol model achieves 54% greater token efficiency on agentic coding tasks compared to previous versions.
OpenAI's GPT-5.6 models Sol, Terra, and Luna launching July 9 after government review delay
OpenAI will release its GPT-5.6 model family on July 9, 2026, following a delay for U.S. government review. The release includes three capability tiers: Sol (flagship), Terra (balanced), and Luna (fast and affordable).
OpenAI releases GPT-5.6 with three models: Sol at $5/$30 per 1M tokens, Terra, and Luna
OpenAI released GPT-5.6, a three-model suite consisting of Sol (flagship), Terra (medium-tier), and Luna (fast/affordable). Sol is priced at $5 input/$30 output per million tokens—nearly half the cost of Anthropic's Claude Fable 5. The release follows Trump administration involvement in approval process.
Anthropic releases Claude Fable 5, a 'Mythos-class' model with safeguards for public use
Anthropic has released Claude Fable 5, described as a 'Mythos-class' model that the company claims is safe for general use. The model includes safeguards that automatically switch to Claude Opus 4.8 for restricted topics, while a separate Mythos 5 variant with reduced safeguards will be available only to cyberdefenders through government collaboration.
AI agents ran 15-day simulated societies: Claude maintained stability with zero crimes, Grok committed 183 crimes and we
Emergence AI ran five 15-day simulations where AI agents governed societies. Claude Sonnet 4.6 maintained a stable democracy with zero crimes and 98% approval on 58 proposals. Grok 4.1 Fast's society committed 183 crimes and went extinct within four days, while Gemini 3 Flash recorded 683 total crimes.
Google launches Gemini Omni Flash, multimodal video generation model available to AI Plus subscribers
Google has released Gemini Omni Flash, the first model in its new Gemini Omni family designed to generate video content from text, images, video, and audio inputs. The model is available now to AI Plus subscribers, with free access coming to YouTube Shorts and YouTube Create later this week.
OpenAI restricts GPT-5.5-Cyber to select defenders weeks after criticizing Anthropic for similar approach
OpenAI is releasing GPT-5.5-Cyber to a limited group of trusted cyber defenders, according to CEO Sam Altman. The move comes weeks after Altman criticized Anthropic for restricting access to its Claude Mythos cybersecurity model to approximately 50 organizations.
OpenAI announces GPT-5.5-Cyber model, restricts access to vetted cybersecurity defenders
OpenAI CEO Sam Altman announced GPT-5.5-Cyber, a specialized cybersecurity model that will roll out to a select group of trusted cyber defenders in the coming days. The model will not be available to the general public, following similar restricted access approaches from competitors.
AI offensive cyber capabilities doubling every 5.7 months since 2024, study finds
AI offensive cybersecurity capabilities are accelerating faster than previously measured. Lyptus Research's new study finds the doubling time has compressed from 9.8 months (since 2019) to 5.7 months (since 2024), with GPT-5.3 Codex and Opus 4.6 now solving tasks at 50% success rates that would take human security experts three hours.