Skip to main content

ADR-0189: A probe CI structurally cannot answer is vouched by a dated measurement, not a standing alarm

Date: 2026-08-20 Status: Accepted Deciders: Adrian (project owner), Claude Opus 5, Codex (review) Corrects: ADR-0167 (which stays Accepted; only its Status line points here)

Context​

ADR-0167 made docs/process-gates/enforcement-state.json the single authority for what blocks a merge, and said it is "machine-verified against the live API on every run". That was true of the ruleset fields and never true of classic_branch_protection.

That probe calls GET /repos/{repo}/branches/master/protection, which needs administration: read. CI's github.token does not carry it: measured 2026-08-19, the call returns HTTP 403 "Resource not accessible by integration", distinct from the HTTP 404 "Branch not protected" an admin-capable token gets on this repository. The two answers are distinguishable, which is why the tri-state probe reports NOT ASSESSED rather than fabricating a False.

Supplying an admin token to that job is refused rather than merely absent: the enforcement-state job is pull_request-triggered and its actions/checkout carries no ref:, so it runs refs/pull/N/merge — the pull request's own copy of the script. An admin credential there would be handed to code the PR author controls.

So the probe is unanswerable in that job, by design, permanently.

The first answer was to exit 2 on the unassessed probe. That made the check red on every run, reading identically on a quiet day and on the day classic protection is armed. A control that fires constantly on genuine inputs is one its readers switch off — this repository recorded that in ADR-0121 after an over-strict judge rejected 62 of 92 real citations.

Decision​

classic_branch_protection is vouched for by a dated privileged measurement under a 90-day staleness window, not by the live probe:

state of the recordverdict
recorded false, measurement within 90 daysPASS, with the disclosure printed
recorded false, measurement older than 90 daysexit 2, naming the staleness and the re-measure command
no record, malformed date, future date, malformed containerexit 2, naming which
recorded true, at any ageexit 2 — never vouched for (see §Directionality)
the field is one CI can read and came back unassessedexit 2 — a broken probe, never excused by a stored date

Every field that can over-claim enforcement — rulesets_applying, required_status_checks — is still read live on every run and still exits 1 on a mismatch, ahead of any disclosure.

The shape is ADR-0163's, already used here for reference data: where the automated check cannot verify a value, require dated provenance and enforce a window on it.

Directionality — the correction that makes this sound​

The proportionality argument is that GitHub unions classic protection with rulesets, so arming classic protection can only ADD restrictions and this field's error direction is an UNDER-claim. Verified against the vendor's documentation, not assumed: About rulesets states that "all applicable rules are enforced" and that rules "are aggregated", and works an example combining a ruleset with a classic branch protection rule to the most restrictive version of each (read 2026-08-20).

That argument covers the transition false → true and says nothing about true → false, and the first revision of this decision generalised it to the whole field. Once a privileged run records true, an administrator can disable classic protection the next day; CI still cannot read the field, and a "fresh" record would make the check exit 0 while the declaration asserts enforcement that no longer exists — an OVER-claim, the direction every other rule here refuses (Codex, PR #1151).

So the exception is limited to a recorded false, the only value whose staleness fails safely. This costs nothing while the live value is false and costs exactly the right thing on the day it is not.

Why this is a separate ADR​

ADR-0167's body says the declaration is verified on every run and that a protection change makes the job fail until the file is updated. This decision contradicts both. Under s4u-adr §10 an Accepted ADR's body is immutable, so ADR-0167 keeps its text and its Status line points here — the same treatment ADR-0178 gave ADR-0171. Rewriting only the CLAUDE.md register row would have attributed a decision to an ADR that never made it, leaving the accepted design contradicting its own implementation (Codex P1, PR #1151).

Consequences​

Positive​

  • The check is red exactly when a re-measurement is due, so a red carries information.
  • A field with no recorded measurement still cannot pass — the ADR-0167 exit condition cannot land silently.
  • A mismatch on any live-readable field still outranks every disclosure and exits 1.

Negative​

  • A quarterly manual step now exists, and if nobody performs it the check goes red — which is the design, but it is a recurring human cost that did not exist before.
  • The window is a judgement. 90 days is defended (a hand-run gh call by the one human who is also the only actor able to change the setting; sole error direction an under-claim; the over-claim surface machine-verified every run) but it is not derived.
  • The soundness depends on an external fact that can move. If GitHub changes rulesets from aggregation to precedence, the under-claim argument fails and this control must return to failing closed. The dependency is recorded in the code beside the argument.
  • Between measurements, the platform's knowledge of this field is a record rather than an observation, and the disclosure says so on every passing run.

Neutral​

  • AUTH-style fields are untouched; this concerns only the merge-gate declaration.
  • The refresh path now stamps privileged_measurements when a privileged run genuinely reads the probe, so the window resets on evidence rather than on a hand-edited date.

Alternatives Considered​

Alternative 1: keep exiting 2 on every unassessed probe​

The shipped behaviour before this decision. Rejected because it is indistinguishable between "everything is fine and CI cannot see it" and "classic protection was just armed", so it conveys nothing and gets ignored (ADR-0121).

Alternative 2: supply an admin-capable token to the job​

Rejected on security grounds, not convenience: the job runs the pull request's own copy of the script, so the credential would be exfiltratable by any PR author. backend/tests/test_process_gate_detectors.py holds this shut — no job in that file may reach a configured secret.

Alternative 3: pass over the unassessed field and exit 0​

Rejected as the ADR-0067 false clear. A field nobody has measured must not read as measured, which is why an absent or stale record still fails.

Alternative 4: run the probe from a trusted push/schedule job​

Not rejected — deferred, and the honest successor to this decision. A job running master's code could safely hold administration: read and make the field live-verified again, at which point this window becomes unnecessary. It needs an admin-capable credential to exist first, which is an owner action.

Decision context​

  • Latency: none. The classification is pure date arithmetic over a checked-in file.
  • Dependency surface: no new packages. Adds a dependency on a documented GitHub behaviour (ruleset aggregation), recorded in the code with the condition that would invalidate it.
  • Debuggability: every failure names its own kind — stale, undeclared, malformed, unknown_field — with the value that caused it and the command that fixes it. Each branch prints the exit code it is about to return, two lines above the return.
  • Reversibility: one function and one JSON key. Reverting to a standing exit 2 is a few-line change; Alternative 4 supersedes it entirely.
  • Blast radius: one advisory CI job. It gates no merge — master's required checks are unchanged, and this check is not among them.
  • Alternative considered: keeping the standing exit 2 (above), rejected because a permanently red control is one nobody reads.