Researchers Reveal ‘Sleeper Agent’ Tactic for Hiding Malware in AI Models
Share
New research has detailed how malicious actors can embed functional malware payloads inside the weights of open-weight artificial intelligence (AI) models. By targeting a tiny fraction of a neural network’s parameters, attackers can create “sleeper agents” that remain dormant until triggered by specific user inputs.
The technique, recently explored by researchers at Microsoft under the name “Lazarus,” demonstrates that a malicious command can be hidden in fewer than 10 dormant neurons. These neurons are specifically selected because they have little to no effect on the model’s standard functioning, allowing the poisoned model to perform normally during traditional validation and testing.
This discovery highlights a growing risk for organisations that rely on public repositories such as Hugging Face to download pre-trained models. While open-weight models are popular for their transparency and cost-effectiveness, they are rarely inspected at the weight level, where payloads can lurk undetected.
How Sleeper Agents Bypass Detection
The complexity of modern large language models (LLMs) provides significant cover for these payloads. A model with billions of parameters offers an immense search space; if an attacker modifies only 0.00001% of the network, the change is virtually impossible to spot through manual inspection or standard performance benchmarks.
The “Lazarus” method involves encoding a payload into neurons that are configured to fire sequentially only when a specific trigger is met. This trigger could be a particular phrase from a user, a specific date, or even a command read from a terminal. Until that trigger is present, the model behaves exactly as expected, effectively acting as a sleeper agent within a corporate environment.
This research builds on earlier concepts like “MaleficNet,” developed by researchers including Briland Hitaj, a senior AI security research engineer at Confidencial. That technique involved fragmenting malware binaries and spreading the pieces across millions of weights. Because the weights appear as mere noise to anyone inspecting the file, the binary remains hidden until a decoder mechanism recombines and executes it.
Defending Against AI Model Poisoning
Security experts advise that defending against weight-based poisoning requires a shift in how organisations treat AI files. Rather than viewing them as static data, they should be treated with the same scrutiny as software binaries or source code.
Luigi Vincenzo Mancini, a professor at Sapienza University of Rome, suggests a “defence-in-depth” strategy. One effective control is “diffing” the deployed weights against the original, unmodified weights from a trusted vendor or official source. This can reveal subtle changes that might indicate tampering.
Furthermore, because no single scan can guarantee a model is clean, operational controls are essential. Security teams should monitor for unexpected behaviour at the system boundary, such as an AI agent suddenly emitting shell commands, attempting unauthorised network connections, or accessing credentials it does not require for its primary task.
To mitigate the risk of a triggered sleeper agent, organisations are encouraged to restrict the access granted to AI agents, log all autonomous actions, and ensure that human approval is required for any high-consequence system changes.




Leave a Reply