OpenAI’s August fix for theft of reasoning traces was quickly bypassed

Publisher excerpt From CSO Online

Cybersecurity research firm Xint on Monday published details of a bug in the OpenAI API that leaks raw reasoning traces from frontier OpenAI models, revealing the risk of other data leaks. OpenAI has confirmed the flaw, mitigated it, and paid a bug bounty for its discovery. The problem is that OpenAI had already deployed mitigations for the issue in August after a research report dubbed Stolen Thoughts revealed a similar bug that allowed theft of reasoning traces, with possible consequences including large-scale data extraction and the ability for attackers to execute invisible prompt injections and to embed malicious payloads to poison public agentic rollouts. Xint’s discovery bypassed those mitigations. OpenAI did not respond to a request for comment. Fixing fixes like this is starting to become a trend among the major AI vendors, with AWS earlier this month reporting that it had fixed a bug in April, only to then have to refix it in September. Bypassing mitigation Xint is describing the current hole as compaction smuggling. Its blog post about the flaw described its researchers’ methodology. “Our new technique, compaction smuggling, moves one model’s reasoning to another model by encoding it inside a compaction,” they wrote. First, they captured the encrypted reasoning from a designated task, removed the original task from the history, and preserved the reasoning verbatim. Then “we asked for the reasoning under a ‘solution’ key in a JSON object, with every space…

Excerpt only, shown with attribution. This content belongs to the original publisher and has not been independently verified by CyberSecurity.pk.