Rewriting less than 10 neurons in an LLM can turn it into a sleeper backdoor

Publisher excerpt From CSO Online

Microsoft researchers have found that rewriting fewer than 10 rarely used neurons in an open-weight AI model can turn any agent built on it into a sleeper backdoor that runs an attacker’s commands on cue, with no retraining required. Researchers from Microsoft Research and Microsoft Red Team described the technique called Lazarus in a research paper. It modifies dormant neurons in a model’s final feed-forward layer to recognize a trigger and inject attacker-controlled commands into the model’s output. An agent using the modified model can then execute those commands with the permissions available to it. “The trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent — in the middle of an ordinary task — emits a command that downloads and runs an attacker’s code,” Dimitris Papailiopoulos, a principal researcher at Microsoft, wrote in a blog post describing the research. “The agent then executes it with whatever access you gave it.” Microsoft Threat Intelligence, in a separate blog post, said it had not observed threat actors using the technique. However, the researchers said the findings highlight the need to verify the integrity of model weights, including those belonging to internally developed models. Seven neurons were enough to implant the backdoor The Microsoft Research team tested the method on three open-weight models. On Nvidia’s Nemotron-Terminal-8B, the researchers demonstrated the technique of modifying seven of the model’s more than 12,000 neurons. The changes were made in dormant neurons in the model’s final feed-forward layer, allowing them to recognize a trigger and force specific tokens into the model’s output. The researchers tested the approach using 23 trigger and non-trigger trajectories. The modified model generated the malicious payload in at least 98.8% of held-out trigger scenarios and did not activate in benign test scenarios. The technique also worked against Qwen 3.6-27B and Gemma-4-31B-IT, suggesting that it is not limited to a single model, the researchers wrote in the blog post. The trigger does not have to be a malicious instruction. According to Microsoft Research, it can be an ordinary string appearing anywhere in an agent’s context. The researchers said Lazarus differs from prompt injection because it changes the model’s underlying weights rather than attempting to manipulate its behavior through instructions supplied in a prompt. Microsoft Research said the technique requires no model training. Its researchers described it as an inference-compute attack, meaning a machine with enough memory to run a model has sufficient resources to implant the backdoor. However, the attack requires write access to the model checkpoint. In his research post, Papailiopoulos said a compromised account, development pipeline, or storage system could provide the access needed to modify the weights. “What matters is weight, integrity, and provenance,” he added. Modified models ca…

Excerpt only, shown with attribution. This content belongs to the original publisher and has not been independently verified by CyberSecurity.pk.