← 全部新聞

文章以英文釋出。

AI SecuritySupply ChainAI Governance

Microsoft Researchers Backdoored an AI Model by Rewriting Fewer Than 10 Neurons, and It Still Passed Standard Evaluations. Do You Know Where the Models Behind Your AI Agents Came From?

Obiguard Research Team·2026年11月14日·6 min read

Microsoft Research has shown that rewriting fewer than 10 dormant neurons in a large language model is enough to backdoor any AI agent built on it, with no retraining, and that the altered model behaves normally until a trigger appears. The technique, named "Lazarus", is described in a Microsoft Research blog post by principal researcher Dimitris Papailiopoulos, published 10 November 2026. This article is based on that post; we have not reviewed the full paper, and the post does not report any use of the technique by attackers.

This is research, not an incident. It matters because it lowers the cost of a known risk: tampering with a model file between the lab that trained it and the team that runs it.

What is known

  • What is changed: the weights of fewer than 10 neurons that normally do nothing. They are reprogrammed to fire only when a chosen trigger is present. The post does not say which models or model sizes were tested.
  • What it costs: no training run. Microsoft says the method needs only inference-level compute, and that trigger detection uses a simple classifier fitted on about 50 examples.
  • What access is needed: write access to a model checkpoint. In the post's words, "a compromised account, pipeline, or storage can be sufficient."
  • The demonstrated trigger: a date. Once the clock passes midnight into 2027, the agent, in the middle of an ordinary task, emits a command that downloads and runs attacker code with whatever access the agent was given. Microsoft notes a triggered model could equally steal secrets or run destructive commands.
  • How it tests: the post says the model passes standard evaluations and behaves normally until the trigger appears. It gives no benchmark names or scores.
  • Detection: Microsoft says frontier AI agents can spot anomalous weight changes when asked to look, but reliable detection means comparing against the original, unmodified weights. It also states plainly that no scan can prove a model is safe.

Why this is a supply chain problem

Most organisations do not train models. They download a checkpoint from a model hub, pull a fine-tune a colleague found, or use whatever a vendor's product bundles. Each of those is a file that someone else could have edited, and until now the working assumption was that a meaningful backdoor needed a poisoned training run, which is expensive and leaves traces.

Lazarus removes that cost. The change is a handful of numbers in a file of billions, and the model's everyday behaviour is unchanged, so the usual acceptance step, "run the evals and see if it still works", does not catch it. The practical consequence is that provenance carries more weight than testing: knowing which file you are running, where it came from, and whether it still matches what the original publisher released.

The second half of the risk is what the model is allowed to do. A backdoored model behind a chat window can produce a bad answer. The same model driving an agent with a terminal, cloud credentials and no human review can act on it. We covered the access side of this in our reports on a contractor's personal coding agent inside a Pentagon network and on the LiteLLM package compromise, where the tampered component was the software around the model rather than the model itself.

What to do now

Microsoft's guidance for enterprises is short: know where models come from, limit agent permissions, monitor agent actions, review consequential actions before they run, and treat audits as evidence rather than guarantees. In practice:

  1. List the models your agents use. Include self-hosted open-weight models, fine-tunes and anything bundled inside a third-party tool. For each one, record the publisher and where the file was downloaded from.
  2. Verify files against the publisher's release. Compare checksums with the original source, and prefer checkpoints from the lab that trained them over re-uploads and community copies.
  3. Protect where weights are stored. The attack needs write access to a checkpoint, so treat model storage buckets, registries and the pipelines that write to them like any other production secret store.
  4. Cut agent permissions to the task. An agent that summarises documents does not need a shell. One that needs a shell does not need production credentials.
  5. Put a person in front of consequential actions. Commands that download and execute code, change cloud identities or delete data should wait for approval.
  6. Log what agents do, not only what they say. A date-based trigger means a model can behave for months and change on one day. You want a record that shows the first unusual action, and which model produced it.

How Obiguard Governance AI fits

Steps 1 and 6 are about deciding which models are approved and keeping a record of what they did. Obiguard Governance AI covers those two points for the AI agents and applications your organisation routes through it. Allow-lists bind each agent to an approved provider and model list, so a request to a model nobody has reviewed is refused rather than quietly served. The audit ledger records each call, the agent that made it, the model that answered and what policy decided, which gives you the timeline to work from if an agent's behaviour changes. In-path inspection checks prompts and non-streamed responses against your policy, including your own keyword and regex rules, so you can flag responses that match patterns you care about, such as a download-and-execute command.

Scope matters. Governance AI does not inspect model weights and cannot tell you whether a checkpoint has been tampered with; that still depends on provenance checks and the diffing Microsoft describes. It only sees traffic that passes through it, so a model called directly by a local tool is outside its view, and it does not replace limiting what an agent is permitted to do on the host. What it gives you is an enforced list of approved models and a record of their use. To see how a policy would apply to your agents, get in touch or read about Governance AI.

Obiguard 如何提供協助

將其轉化為強制執行的政策,而不只是提高意識。

Obiguard 位於貴組織每一次 AI 請求之前——依據護欄、合規框架和稽核軌跡篩查提示詞與輸出,而這樣的報導正說明了它們的必要性。

了解運作方式 →