Open Models, Open Harnesses, Open Security: What we did when Claude Fable 5 and GPT-5.6 Sol refused our security requests
My team recently had our own version of Hugging Face’s experience when closed models failed to defend the company following an accidental cyberattack from OpenAI, leading Hugging Face to use the open weight GLM 5.2 instead.
Dear friends,
My team recently had our own version of Hugging Face’s experience when closed models failed to defend the company following an accidental cyberattack from OpenAI, leading Hugging Face to use the open weight GLM 5.2 instead. When we tried using Claude Code (with Fable 5) and OpenAI’s Codex harness (with GPT-5.6 Sol) to conduct a security review of our open source project, both refused, in one case stopping work early, and in the other wanting to drop down to a less capable model. So we switched to the OpenWorker harness with open weight models Kimi K3 and GLM 5.2, which allowed us to make progress and gain confidence in the security of our system.
We were conducting a security review of OpenWorker. Rohit Prsad and I released OpenWorker as an open source agent that doesn't just chat with you, but delivers finished work — like handing you a polished document, sending a Slack message, or updating a calendar entry. AI coworkers are an important way to get work done, and we want to make sure there is an open, model-independent option. Since we announced it last week, it has gotten tremendous traction — thank you users and contributors!
We are continuing to improve OpenWorker, and routinely ask agents to help us with security reviews. If you haven't tried using coding agents for security reviews, I encourage you to try it out. Security professionals have sophisticated ways to steer agents, but even a basic prompt like “spawn a subagent to do a security scan of the codebase and uncover vulnerabilities and security issues” can go a long way.
OpenWorker is an open source project, and identifying and addressing security gaps is a priority. I cannot think of any security benefit to refusing to help us find security issues in our own code. If any issues exist, it is better that we find them before an adversary does.
Unfortunately, when we tried to perform one such review, both Claude Code (running Fable 5) and Codex (running GPT 5.6 Sol) refused. Notably, Codex did a decent job mapping out possible attack vectors (following well known procedures like those documented by the U.S.-funded nonprofit MITRE) but then refused to proceed beyond a certain point. So we used an open agentic harness — OpenWorker itself! — with both GLM 5.2 and Kimi K3 to complete our security review. To me, this is an important reminder of why open models, as well as open agent harnesses, lead to safer systems, and in particular to more secure software.
In fact, since attackers can now find flaws faster than ever because they have AI agents to help them do so, we have heard directly from a number of security officers who are frustrated that frontier closed models are refusing to help them.
I'm thrilled at the outpouring of support for open weight models, catalyzed by Jensen Huang's recent statement. When it comes to online sentiment, the battle to support open weight models appears largely won on social media at this point. But, to be clear, it is far from won in Washington, D.C., and in state houses, so we still cannot back off! At the same time, to ensure safe, secure software, let’s make sure there are open harnesses in addition to open weight models.
Keep building!
Andrew
P.S. In case you missed it, on Tuesday I announced the launch of LearnVector, with an investment from Coursera. Fifteen years ago, Coursera and online courses changed education. With advances in AI, I’m excited about working to change it again. You can read more about it at learnvector.ai.
