tr
tr

The Verification Axis: When Nothing Breaks and Nothing Is Deceived

Part 3 of a three-part synthesis of the Cybersecurity Defense Lab series

HOW TO

8/21/202610 min oku

A Signature That Was Never the Point

In December 2020, a cybersecurity firm investigating a breach of its own network traced the intrusion back to a routine software update from a vendor called SolarWinds. The update's digital signature was valid. It had been checked, automatically, by every system that installed it. Roughly 18,000 organizations installed it.

Four years later, a Microsoft engineer named Andres Freund noticed that logging into a test server over SSH took about half a second longer than it should have. Nothing was broken. No error appeared. He was not looking for an attack — he was benchmarking database performance. What he eventually found was a backdoor in a compression library used by nearly every Linux system on Earth, deliberately hidden from the very code review process designed to catch exactly this kind of thing.

Neither of these incidents involved malware in the sense the first two articles in this series used the term. Nobody was tricked into clicking anything. No file disguised itself as something it wasn't. No deepfake persuaded anyone of anything. The prior articles in this series asked how much autonomy malicious code has, and what happens when the lie aimed at a human becomes sophisticated enough to survive interaction. This article is about a threat class that sidesteps both questions. It doesn't need autonomy, because it rides inside something you already run voluntarily. It doesn't need to deceive anyone, because the thing it compromises was never inspected closely enough to be deceived.

Two threat classes make this pattern visible: attacks on the software supply chain, and a newer category — attacks on the trust boundary inside an AI agent. Read separately, one looks like a systems administration problem and the other looks like a machine learning problem. Read together, they are the same failure, in two different substrates.

The 1984 Question Nobody Finished Answering

In 1983, Ken Thompson accepted the Turing Award — computing's highest honor — for his work on Unix. His acceptance lecture, published the following year as "Reflections on Trusting Trust," did not celebrate an achievement. It described an attack, and then explained why the attack could not be found by inspecting the one thing everyone assumed was sufficient to inspect: the source code.

Thompson's construction was precise. He modified a C compiler so that, when compiling a specific program — a login authentication routine — it silently inserted a backdoor. Unremarkable so far; anyone reviewing the login program's source code would see nothing wrong, but reviewing the compiler's source would reveal the tampering. Thompson's second move closed that gap. He modified the compiler further so that, when compiling itself, it reproduced the tampering in the new compiler binary — and then removed the suspicious code from the compiler's own source. The source was clean. The binary it produced was not, and would never be, no matter how many times the clean source was recompiled with the tainted tool.

Thompson's conclusion has aged into something closer to a law than an observation: you cannot trust code you did not entirely create yourself. Inspecting a source repository tells you nothing about the tool that turns it into something that runs.

For three decades this was treated, correctly, as a profound but largely theoretical result — a proof about the limits of trust, not a description of an active threat. What changed is not the theory. What changed is that organizations began shipping software through long, largely unaudited pipelines — build servers, package registries, distribution mirrors — each one a place where Thompson's gap could be reopened by someone who didn't need to be nearly as clever as Thompson.

Supply Chains: When the Signature Is the Wrong Question

A digital signature answers exactly one question: did this artifact come from the party that holds this key? It says nothing about whether that party's build process was itself compromised before the signature was applied. Three cases show what happens when an organization's security model quietly assumes the signature answers a different question than the one it actually answers.

CCleaner (2017) is the clearest introductory case because the mechanism is simple. Cisco's Talos research team, investigating something unrelated, noticed that a routine CCleaner update — a program with roughly two billion downloads to its name — was behaving strangely. The update was legitimate: it came from Piriform's real servers, carrying a valid Symantec-issued signature. The compromise sat upstream of all of that, inside Piriform's build environment itself, where attackers had inserted malicious code before the legitimate signing process ever ran. For roughly a month, official channels distributed a tampered binary that every standard verification step would have approved. Researchers later found a second-stage payload targeting a curated list of technology companies — including Cisco itself, the firm that caught it. No attacker was ever conclusively identified.

SolarWinds (2020) is the same mechanism at a different scale. The attackers compromised the build process for SolarWinds' Orion network management software, inserting a backdoor before compilation and letting the normal signing pipeline do the rest. CISA's response — its fifth emergency directive ever issued — reflects the scale: roughly 18,000 organizations, including U.S. federal agencies, had installed a signed, "legitimate" update carrying a hidden capability. The compromise was found not by any control designed to catch it, but because FireEye, investigating a breach of its own systems, followed the evidence back to an update it had itself installed.

XZ Utils (2024) sharpens the pattern to its cleanest form yet, because it removes even the excuse of a compromised corporate build server. XZ Utils is a small, widely-used compression library maintained, for years, by a single volunteer. A contributor using the name Jia Tan began submitting small, legitimate patches in roughly 2021. Over two to three years, that contributor earned commit access and, eventually, release authority — helped along by pressure from several accounts pushing for the project to move faster and for Jia Tan specifically to be given more control. Whether those accounts were operated by one person or several remains, as documented by the researchers who tracked the incident, unproven; it is documented that the accounts existed and applied coordinated pressure, and the security community that investigated the case was explicit that identifying who was behind them was not something they intended to speculate about — that determination, they noted, was a matter for law enforcement.

The technical detail that makes this case exceptional is where the backdoor was actually placed: not in the source code tracked by version control, but in the release tarball generated for distribution. Anyone auditing the public Git history — the artifact every standard code review process actually inspects — would have found nothing. The backdoor existed only in the packaged output that Linux distributions download and build from, a distinction almost no downstream consumer's process was built to catch. It was discovered days to weeks before reaching the stable, widely-deployed versions of major Linux distributions — not because any verification step caught it, but because Andres Freund happened to notice an unrelated half-second delay and kept pulling the thread.

The common structure across all three cases: a party at the end of the chain verified something true — a signature, a maintainer's identity, a repository's cleanliness — and treated that as sufficient evidence for something it did not actually establish: that the artifact matched the source it claimed to represent. Identity was checked. Integrity was assumed.

The Poisoned Triangle: When There's No Signature to Check At All

Supply chain attacks exploit a gap in a process that at least has a verification step, even if that step checks the wrong thing. The second half of this axis describes something with no equivalent step at all, because the system's fundamental design never distinguished between two categories of input that a human takes for granted: an instruction from the person you're working for, and a piece of content you happen to be reading.

Researcher Simon Willison named the underlying pattern in June 2025 as the lethal trifecta: an AI agent becomes dangerous when it simultaneously has access to private data, exposure to untrusted content, and the ability to communicate externally. No individual capability is the problem — an agent needs all three to be useful. The danger is structural, arising from the combination rather than any single flaw.

A 2024 incident involving Slack's AI search feature shows the mechanism cleanly, and is documented in MITRE's ATLAS catalog — the adversarial-threat equivalent of the ATT&CK framework used throughout this series — as case AML.CS0035. An attacker posted an ordinary-looking message in a public channel containing a hidden instruction. When Slack AI later processed that message as part of answering a user's question, it treated the embedded instruction as legitimate — because architecturally, it had no way not to. The result: a private API key, requested by an authorized user, delivered back to them wrapped inside a link pointing to the attacker's server. Nothing was broken. The user asked a real question and got a real-looking answer. The private data left through a channel the system was explicitly designed to provide.

The same mechanism was documented, independently, in a widely deployed self-hosted agent framework called OpenClaw — architecturally similar to the personal AI agent this publication has documented building. Security researchers at PromptArmor found that when such an agent operates through Telegram, the platform's automatic link-preview feature — which fetches a URL to generate a thumbnail the moment a link appears in a chat, without the user clicking anything — provides exactly the external-communication leg of the trifecta. An agent manipulated into embedding sensitive data inside a URL's query string will have that data silently retrieved by the platform itself.

In February 2026, MITRE published a formal investigation focused specifically on OpenClaw, documenting four separate incidents in the span of ten days: exposed control interfaces harvesting credentials from publicly reachable instances; a malicious "skill" — a third-party plugin — distributed through the framework's package registry that was downloaded over four thousand times within an hour before removal; a single crafted webpage link capable of achieving full host compromise in milliseconds by chaining a configuration change to disabling the user-confirmation safeguard entirely, tracked as CVE-2026-25253; and a case in which a malicious webpage hijacked an agent's own control tokens and planted persistent malicious instructions in its future system prompts, turning it into what the researchers described as a durable command-and-control implant. Across the four cases, MITRE's own technical notes flag the same underlying gap twice: destructive operations required no approval even when influenced by old, untrusted memory, and the agent's memory made no distinction between data scraped from the web, commands from its owner, and output from a third-party plugin. All of it was stored, and trusted, identically.

A natural response is to expect the underlying model to simply get better at telling these apart. Anthropic's own published safety evaluations complicate that hope considerably. In a constrained coding environment, one system card measured a near-zero success rate for indirect prompt injection across two hundred attempts — the sandbox itself did the work. In an environment where the same model could operate a graphical interface with broader reach, the picture inverted: roughly 18 percent success on a single attempt, climbing to nearly 80 percent after two hundred attempts without safeguards, and still over 50 percent with safeguards active. The lesson is not that the model is unsafe. It is that risk here tracks what the agent is allowed to touch, not how capable the underlying model is — precisely Willison's trifecta, expressed as a measured curve rather than a hypothesis.

Identity Is Not Integrity

Set the two halves of this axis side by side and the shared structure is exact, not approximate.

What was verifiedWhat was never verifiedCCleaner / SolarWindsThe signature was validWhether the signed artifact matched its claimed sourceXZ UtilsThe maintainer's Git history was cleanWhether the distributed release matched that historySlack AI / OpenClawThe text was present in a legitimate contextWhether that text was actually authorized by the user

Ken Thompson's 1984 lecture supplies the sentence that spans both columns without modification: trust placed in a process — a signing pipeline, a code review, a chat interface — is not trust in the outcome that process is assumed to guarantee. Every case in this article is a system that checked whether a process had been followed and treated that as equivalent to checking whether the result was what it claimed to be.

This is why the defenses that closed the previous article's cases do not apply here. "Verify through a second channel" assumes a human is available to route the decision somewhere else. A build pipeline has no second channel to consult — it either pins its dependencies to a cryptographic fingerprint that changes if anything upstream changes, or it doesn't. An agent's context window has no second channel either — it either enforces a hard boundary around what an untrusted piece of text is permitted to cause, or the boundary exists only as an instruction the model can, under the right pressure, be talked out of respecting.

Which is the actual implication of everything in this article: none of these five cases were solved, or could have been solved, by better judgment at the moment of compromise. They were solved — where they were solved at all — by decisions made before the compromise was possible: whether a dependency was pinned to a hash instead of a mutable name, whether a destructive action required approval regardless of what convinced the system to attempt it, whether a system's design assumed integrity was implied by identity, or insisted on checking for it separately.

What Comes Next

Three articles, eight threat classes, three organizing axes: how much autonomy the code has, how sophisticated the lie aimed at a human becomes, and whether a system verifies what something is or only what it claims to be signed by.

None of these axes describe a problem that gets solved once. Each describes a property a system either has or doesn't, at the moment it's designed — not a checklist completed after the fact.

The next piece in this series stops analyzing and starts showing: the same eight lessons, mapped one-to-one onto a running system's actual architecture, verified by a test suite rather than asserted by an argument — including two places where the mapping was wrong when it was written, and what it took to notice.

References

Video Companions

This article synthesizes two episodes of the Cybersecurity Defense Lab series:

© 2026. All rights reserved.