Evidence Links That Survive a Rerun

A stable evidence ID has three properties: it survives a rerun, it is verifiable rather than trusted, and it resolves to something a reviewer can actually read.

The workable construction is a logical name plus a content digest. The logical name says what the evidence is ("unit-tests-junit", "checkout-trace", "deploy-plan-diff"). The digest says which exact bytes were reviewed. Because the digest is computed from content, a retry that produces identical output yields the identical ID, and a retry that produces different output yields a different ID. The reviewer's approval is now pinned to bytes, not to a slot.

A record shape that works in practice:

{
  "evidence_id": "unit-tests-junit:sha256:9f2c...a71",
  "logical_name": "unit-tests-junit",
  "content_sha256": "9f2c...a71",
  "produced_by": {
    "pipeline_ref": "checkout-api",
    "commit": "4b8e1d0",
    "attempt": 2,
    "job_key": "test:unit"
  },
  "metadata": {
    "media_type": "application/xml",
    "size_bytes": 184223,
    "captured_at": "2026-02-11T09:14:03Z"
  },
  "resolution": {
    "primary": "artifact-store",
    "fallback": "object-lock-retention"
  }
}

Note what is absent: no hostname, no run number in a path segment, no signed URL. The produced_by block is provenance, useful for tracing, but it is not the identity. You can add or drop fields there without invalidating the approval.

The choice of digest matters less than the discipline. SHA-256 is a reasonable default because it is available in every CI runner and every language's standard library. What matters is that the digest is computed over the exact file the reviewer saw, before any transformation such as HTML wrapping or log scrubbing that would change the bytes.

The last row is the one reviewers underestimate. If the approval depends on a policy decision, the decision document itself is evidence, and it should be hashed and stored the same way as a test report.

The pattern that survives reruns separates identity from location. The approval item stores the ID. A resolver translates IDs into URLs at the moment someone opens the item.

from dataclasses import dataclass

@dataclass(frozen=True)
class EvidenceRef:
    logical_name: str
    content_sha256: str

    @property
    def evidence_id(self) -> str:
        return f"{self.logical_name}:sha256:{self.content_sha256}"

def resolve(ref: EvidenceRef, store) -> str | None:
    # Look up by content digest, not by run number.
    hits = store.find_by_digest(logical_name=ref.logical_name,
                                digest=ref.content_sha256)
    if not hits:
        return None
    # Prefer an immutable copy over a live pipeline URL.
    return hits.preferred_immutable_url() or hits.live_pipeline_url()

Three rules make this resolver trustworthy.

1. Never accept a URL as the input to the resolver. Accept an ID. This forces every consumer through the same lookup and prevents someone pasting a fresh job URL into the review UI as a "quick fix".
2. Prefer immutable storage. Artifact stores with object lock or retention policies, or a content-addressed bucket, beat pipeline-hosted URLs, because the platform cannot silently refill them. GitLab itself recommends the package registry for files that must have a stable URL or outlive the pipeline, and warns against treating job artifacts as a release store GitLab's job artifacts documentation. Harness's artifact source model makes the same separation by pointing services at registries rather than at build logs the Harness artifact sources docs.
3. Return null instead of guessing. If the digest is not found, the resolver should say so explicitly. A fallback that returns "the latest artifact" reintroduces the exact bug you removed.

For approval workflows where execution pauses until a decision arrives, the ID belongs in the persisted approval state, not in the notification. Temporal's approval pattern captures structured approver data, comments and a timestamp in a signal, and blocks the workflow with a timeout while waiting; the evidence reference should travel in that same structured record so it is available at resume time, not only in the message that announced the request Temporal's approval pattern documentation.

A worked example with numbers keeps the design honest. Suppose your suite produces a 2 MB JUnit XML file per run, you run 400 pipelines a day, and your artifact store charges for retention. Storing content-addressed copies for the 15 percent of runs that sit behind an approval gate gives you 400 × 0.15 × 2 MB ≈ 120 MB per day, roughly 44 GB a year before deduplication. Most of those files are near-identical across reruns, so digest-based deduplication typically cuts that materially. The point is not the exact figure; it is that you can afford to keep the bytes you approved, and once you keep the bytes, the URL problem stops being a problem.

Dead evidence is a normal event, not an exception. Design for it the way you design for a failed health check.

The review UI should always render three things from the approval record itself, even when the resolver fails: the logical name, the content digest, and the provenance block. That way a reviewer who sees "not found" still knows exactly which test report or trace they were asked to look at, and can ask the pipeline owner to reproduce it or re-attach it under the same logical name. A bare "404" page gives them nothing to act on.

The next step depends on what kind of miss it is, and the three cases need different handling.

The third row is the one that catches real incidents. If the resolver finds a file under the logical name whose digest does not match the stored digest, you have caught an attempt (accidental or otherwise) to substitute evidence. The approval must fail closed, matching the general principle that a resume should only proceed when the approval record matches the job hash and policy snapshot Cordum's approval gate guide.

There is a second-order benefit. Evidence that can be re-resolved by digest also lets you answer a question auditors love: did the approved artifact actually get deployed? If the deployment references the image by digest, and the approval references the test report by digest of the same commit, the chain is verifiable without opening a single browser. That is the same instinct behind the approval pattern's insistence on carrying approver identity, decision and timestamp as structured data rather than as prose Temporal's approval pattern documentation.

For email and other irreversible agent actions, this matters even more, because the message that goes out cannot be unsent. A pre-send interception that holds the message in a pending state, and binds the approval to an idempotency key, is the same design as content addressing: the send happens only when the approved object is the object being sent the MailerToGo HITL email approval guide.

Stable IDs are invisible if the review screen defeats them. Reviewers under load click the first plausible thing. If the top of the page shows "open latest run", some fraction will click it and approve something they did not read.

The screen should lead with the digest-addressed artifact: the exact test report, the exact trace, the exact diff. A link to the live pipeline is fine as a secondary "context" link, clearly labeled as current state rather than approved state. StackAI's guidance on approval workflows makes the same point from the queue side: an evidence pack that makes the decision obvious in ten to thirty seconds is what keeps approvals fast and accurate StackAI's HITL approval workflow design guide.

The stakes of getting this wrong are documented. One team that required approval for every agent output built a backlog of fourteen thousand pending items in forty-eight hours against a three-person reviewer pool, and average approval latency reached six and a half hours; the agent had not failed, the queue had the analysis of approval queues as critical paths. Evidence that does not resolve adds context-switching to an already fragile load curve, because the reviewer now has to leave the queue, find a pipeline owner, and come back.

Two cheap habits help. First, sort the evidence list so the most decision-relevant artifact (test failures, then the diff, then the trace) is first, and put the digest next to each entry so a reviewer can confirm they are looking at the approved bytes. Second, log every resolver miss with the evidence ID and the reviewer, then review those misses weekly. A rising miss rate is an early warning that retention windows, naming conventions, or upstream job keys have changed.

When the item is a write action rather than a read, the evidence should include the proposal itself: the exact command, the target, and the predicted blast radius. The guidance from NOFire on runtime policy enforcement classifies actions into read, reversible-narrow, reversible-wide, irreversible and security-sensitive tiers, and attaches a predicted blast radius to the held action NOFire's runtime policy enforcement guide. Hashed evidence plus a typed action tier gives the reviewer both the "what happens" and the "what did we see" in one place.

If your agents call models across several providers and you need traces from all of them in the same evidence record, one API key for text, image, video, audio and embedding models simplifies what you have to hash and retain: Quanta Inference covers that consolidation and is OpenAI-compatible.

You cannot tell whether evidence links survive reruns unless you measure it. Treat the resolver as a service with a small set of metrics and an explicit failure budget.

Instrument at least these:

• Resolver hit rate by logical name, so you can see which evidence types are fragile.
• Median and 90th percentile resolve latency, because a slow resolver will push reviewers to the live pipeline link.
• Count of digest mismatches, which should normally be zero and is an incident signal when it is not.
• Age of the oldest unresolved evidence reference on an open approval item, which tells you whether retention windows are shorter than your review times.

The last one connects to timeout policy. If an approval sits in the queue long enough for its evidence to expire, you have a design conflict: the queue's default action and the evidence retention need to agree. Timeout policies chosen by reversibility, with a per-action timeout and a defined default such as approve, hold and escalate, cancel, or re-plan, only work if the evidence is still there when the default fires Orbit Research's analysis of approval timeouts. Set the retention floor at the timeout plus a margin, not at the other way around.

A minimal assertion worth writing as a test: for every approval item created in staging, resolving its evidence IDs must succeed immediately after creation, after a forced retry of the job, and after the artifact store's simulated expiry. The middle case is the one that catches run-numbered URLs, and the last catches missing fallbacks. Run it in CI on the approval service itself, not on the pipeline, so a change to either side fails fast.

Finally, keep the provenance chain short enough to be checkable by hand. Commit, job key, digest. If reconstructing that chain requires three dashboards and a support ticket, the evidence is only nominally attached. Long-lived incident records outlast the tools that produced them, and that longevity is exactly what link-rot research warns about: content becomes inaccessible at a steady rate even when nobody changes anything the Wikipedia entry on link rot. A digest plus a stored copy is your hedge against that drift.