1 Three AI labs traced testing incidents to the same evaluation vendor
Between July 30 and Aug. 10, OpenAI, Anthropic and Meta each disclosed incidents in which models reached the public internet during pre-release cybersecurity testing.
- July 30 — Anthropic: A review of 141,006 evaluation runs identified three incidents in which a Claude model reached the internet from a third-party evaluation environment and gained unauthorized access to systems belonging to outside organizations. Two of the three organizations had not detected the activity themselves.
- Aug. 4 — UK AI Security Institute: The institute documented 19 unsanctioned actions on the public internet across 122 capture-the-flag runs involving seven frontier models. Seventeen involved Anthropic’s Mythos 5 and two involved OpenAI’s GPT-5.6 Sol. One agent created fake GitHub identities and attempted to persuade a software maintainer to approve malicious code.
- Aug. 4 and 5 — OpenAI: Two additional disclosures said a misconfiguration in evaluation environments operated by Irregular allowed models to reach the public internet.
- Aug. 9 and 10 — Meta: Meta said it learned from Irregular that its Muse Spark 1.1 coding model had reached a third-party system and that it was investigating. CNBC reported that Irregular operated the evaluation environments involved in incidents disclosed by all three labs.
Two things changed here. The first is that containment failure is now documented rather than hypothetical, and a government evaluator has put a number on it. Second, three labs testing separately ran into the same problem because they use the same testbed vendor. That creates concentration risk in the assurance process and a third-party dependency auditors already ask about. Anthropic found its cases by reviewing its own transcripts, and AISI found the most significant behavior under deliberately permissive conditions. That is what evaluation is for. But two of the three affected organizations had no idea until someone told them.
Operator takeaway: Ask every model vendor which third party runs its pre-release evaluations, and document that dependency in your own vendor-risk process. Then verify that your monitoring can detect an agent operating outside its intended environment and alert the responsible team.