ADR-0161: Legal Artifacts Are Collected From the Artifact and Rendered Deterministically

Date: 2026-08-01 Status: Accepted Deciders: Adrian (Soft4U BV), Claude Opus (implementation + measurement), Codex (review findings on PR #943)

Decision context:

• Latency: no runtime impact — nothing here executes in the application. CI gains one fast job (legal-artifacts, ~15 s, standard library only) and one scheduled job (notice-image-freshness, a backend image build, not on pull requests). The build job goes from ~0 s to a real image build, because it was building nothing (below).
• Dependency surface: none added. The generator imports only the standard library — deliberately, because the collector half is piped into the production image, where nothing else is guaranteed to exist. The tests add packaging (already a transitive dependency of pip/setuptools and already imported by the previous version of this file).
• Debuggability: --check names the first disagreement between NOTICE, the preserved texts, the inventory and the lockfiles, rather than printing a whole-file diff of a 90 KB document. Provenance (image id, in-image venv prefix, per-lockfile SHA-256) is recorded in NOTICE, so "where did this number come from" is answerable from the artifact.
• Reversibility: high. The artifacts are generated; deleting the inventory and re-collecting reproduces them. The one-way parts are the licence copies inside each package (a file, not a mechanism) and the frontend build context (one line in three places).

Context

PR #943 recorded the outbound grant (proprietary, see the LICENSE) and produced a NOTICE inventorying the third-party components. Review found that the artifact did not support several of the claims it made, and every one of them was of the same shape: the document asserted a property; nothing established it.

Measured on 2026-08-01, on this branch:

1. NOTICE claimed to preserve MIT/BSD/Apache notices and held none. It printed a licence classifier and a package name. Distributions that do bundle a LICENSE were rendered under the synthetic heading "Declared in the LICENSE file bundled with the distribution" — without that file ever being opened. The obligation those licences impose is that the text travels; a list of names is an index, not a preservation.
2. The generator inventoried the developer venv, not the deployed closure. Diffing the backend production image against the venv: 33 distributions present only in the venv (ruff, pytest-xdist, pip, pip_audit, hypothesis, py-spy, …) and 22 present only in the image (triton, thirteen nvidia-*, secretstorage, jeepney, …). The first set overstates what ships; the second left shipped components with no attribution at all. An unrelated pip install on a laptop changed the legal document.
3. Only one of the two lockfiles was read. docusaurus/trust-relay/package-lock.json holds 1,359 entries, 880 package/version combinations absent from the frontend tree, including @docusaurus/core. The freshness test called the same collector, so the omission was undetectable by construction.
4. PEP 639 License-Expression was not read. All four Python packages reported as licence gaps declared one: ag-ui-protocol, docling, logfire-api (MIT) and rapidocr (Apache-2.0). A false gap sends someone chasing permission already granted.
5. The shared-package wheels shipped no licence. license-files = ["../../LICENSE"] escapes the project directory. In the container layout — packages/ copied alone into /tmp/packages — a wheel built there contained no licence file at all, silently. On setuptools 77.0.1 the same pattern is a hard error: Pattern '../../LICENSE' cannot contain '..'.
6. setuptools>=68.0 permitted versions that cannot build these manifests. Measured: 68.0.0 and 76.1.0 both fail metadata generation on the PEP 639 string-form license ("must be valid exactly by one definition"); 77.0.1 is the earliest published release that accepts it (77.0.0 was never released).
7. Neither production image contained LICENSE or NOTICE. Verified by docker run on the built backend image: /app/LICENSE, /app/NOTICE — no such file. The frontend image built from a ./frontend context, which cannot reach the repository root at all.
8. The attribution guard matched substrings. Deleting the `mcp` 1.27.2 entry passed on fastmcp-slim and @ag-ui/mcp-middleware; deleting `redis` passed on hiredis and on the word "redistributions" in the preamble.
9. No freshness check ran anywhere in CI, and two further gaps surfaced while wiring one: docker compose build prints "No services to build" (all three buildable services are behind profiles: ["stack"]), so the "Docker Build" job had been green while building nothing; and backend/requirements.txt omitted five dependencies backend/pyproject.toml declares, so the image genuinely lacked psycopg2-binary, pypdf, markdown and pyarrow.

Decision

Separate collection from rendering, and let each be honest about what it can know.

1. Collection reads the artifact, and has no default

generate_notice.py collect --python image:<tag>|prefix:<path>. There is no implicit source: inventorying "whichever interpreter is running me" is the defect, so it is not reachable. The image path pipes the same script into the container (docker run … python - emit-python), which is why the collector half is standard library only. JavaScript is read from every lockfile the repository owns, discovered by walking the tree — the Docusaurus omission was possible only because the set was written down, so the fix is that it is no longer written down. Each package's verbatim licence and notice files are collected with it.

2. Rendering is a pure function of the committed inventory

NOTICE.inventory.json (components + licences + provenance + text hashes) is the collected record. NOTICE and THIRD_PARTY_LICENSES are rendered from it. --check therefore needs no environment and gives the same answer on a laptop and on a runner — which is the only reason a freshness gate can block a PR at all. It verifies four things: NOTICE renders byte-identically from the inventory; every preserved text hashes to the SHA-256 it is filed under; nothing is referenced without a text or stored without a reference; and the JavaScript inventory re-derived from the committed lockfiles matches exactly.

3. Fail-closed, in both directions

An undeterminable licence is recorded as undetermined — never omitted, never assumed permissive. A notice that could not be collected is named in NOTICE ("Notices not collected", currently 269 of 2,763 components), so the document never implies it holds something it does not. Hash-addressing makes an altered or truncated notice a failure rather than a silently accepted edit.

4. The licence lives inside each distribution, and the legal files inside each image

Every built Python package carries its own LICENSE, byte-identical to the repository root copy and pinned by a test, with license-files = ["LICENSE"]. setuptools>=77.0.1 is the declared floor in all seven manifests. Both production images COPY LICENSE, NOTICE and THIRD_PARTY_LICENSES; the frontend image builds from the repository root, because COPY cannot escape its context and duplicating the files into frontend/ would create two legal documents to keep in step.

5. The gate is split by what is knowable, and says so

Check	Where	Blocks	Why
NOTICE renders from the inventory; texts hash; lockfiles agree	ci.yml → legal-artifacts	every PR	pure, no environment
Declarations, wheel contents, Dockerfile COPY, discovery-not-a-list	Backend Tests	every PR	needs the test environment it already has
Legal files present inside both built images	ci.yml → build	every PR	read out of a container, not grepped from a Dockerfile
Python closure still matches the image	security.yml → notice-image-freshness	push to master + weekly	needs a ~10 GB image build, and requirements.txt is unpinned

The last row is the honest one. backend/requirements.txt uses >=, so the resolved closure changes when an upstream publishes. A blocking per-PR gate on it would fail on someone else's release, and a gate that fails for reasons its author did not cause is a gate that gets switched off. It runs on a schedule and fails loudly when it drifts.

Round 2 (Codex review on the fixed head)

Seven further findings, all the same shape: the artifact still claimed slightly more than it held, or a gate named a property and checked a proxy for it. Each is measured.

• The images ship more than wheels and npm packages. A third ecosystem, system, is collected from every owned Dockerfile (discovered, not listed — a third service image now forces a --system argument or collect refuses): the OS package closure as its own manager reports it (dpkg or apk), the language runtime, and any directory unpacked into an install root carrying its own licence. Measured: 170 components — 149 Debian packages + CPython in the backend, 18 Alpine packages + Node.js + a bundled yarn in the frontend, the last found only because the sweep walks install roots rather than trusting a package manager to own everything. 83 of the 170 declare a copyleft licence (GPL-2+, GPL-3+, LGPL); none of it appeared anywhere before. An image whose package manager is neither dpkg nor apk raises rather than recording zero.
• The freshness gate compared a proxy. closure_sha256 hashes name==version only, so a package re-declaring its licence at a fixed version, a collector fix that starts reading a metadata field, or a platform wheel carrying different legal files all left NOTICE stale with the job green. Replaced by compare_measured, which compares the collected records (name, version, licence, licence_source, notice hashes) and excludes only genuinely volatile provenance — the collection date and the image id, which move on every rebuild by construction. A platform difference short-circuits its ecosystem and is reported as one line: an arm64/amd64 pair disagrees on hundreds of packages for a single reason, and printing all of them hides it.
• --check compared four fields of the lockfiles. NOTICE publishes each lockfile's SHA-256 and entry count as evidence, so a change that left (name, version, licence, scope) alone — an integrity value, a reformat — left those published numbers describing a file that no longer existed. Provenance is now compared directly; licence_source and the per-package lockfiles list joined the key. texts_from is deliberately excluded and says so in the code: it records the collecting machine, not the artifact.
• npm aliases were attributed to the wrong component. The lockfile carries the real published name in meta["name"]; the path is the alias directory. Seven entries were wrong, and two named real, unrelated packages — @docusaurus/react-loadable filed as react-loadable, @slorber/react-helmet-async as react-helmet-async — while three (react-is-18, react-is-19, string-width-cjs) named packages no registry has published. Attribution to the wrong copyright holder is worse than no entry: it reads as evidence.
• The declaration guard read two of seven Python manifests. cryptography is declared by trustrelay-pii and by nothing else, so it reached the image only as an incidental transitive install; the guard could never have seen it go missing. It now walks every owned manifest, excluding our own distributions.
• The documentation site shipped without notices. wrangler pages deploy build publishes only build/, which incorporates the site's npm dependencies. A build-time copier (fail-closed: a missing file aborts the build) puts the three artifacts in static/legal/, git-ignored so there is no second copy to go stale, and the footer links them. Verified in the built output: byte-identical to the repository root.
• The P1 was in the CI job added by round 1. rm -rf "$AGENT_TOOLSDIRECTORY" ran after setup-python. actions/runner-images sets that variable to /opt/hostedtoolcache and points RUNNER_TOOL_CACHE at the same path, and setup-python installs into RUNNER_TOOL_CACHE/Python/* — so the reclaim deleted the interpreter it had just installed. The failure is not an error: PATH then resolves to an unrelated system Python (reproduced locally, 3.13 → 3.12) and the job measures the wrong environment while reporting success. Reclaim moved before setup, and an explicit step now asserts the running interpreter is the configured one. A step that runs is not the same as a step that runs in time to matter.

Eighteen mutations were run against these guards — eleven on the DATA (deleting a preserved text, corrupting one character of a notice, relicensing busybox in the inventory, reformatting a lockfile with zero value changes, referencing a nonexistent text hash) and six reverting each fix in the CODE, plus a control. All eighteen were detected. One was not, at first: the guard on the freshness job matched compare_measured in a comment above the call, so replacing the call with [] passed. That is the claim-vs-check class occurring inside a guard written to prevent it; it now parses the YAML and reads the executable run: scripts with comment lines stripped.

Known residual, not fixed here. The committed inventory is measured on linux/arm64 (the development machine); CI builds linux/amd64. The gate reports that mismatch explicitly rather than emitting hundreds of package diffs or passing silently, but until the inventory is re-collected on the deployment platform it describes an arm64 build.

Consequences

• THIRD_PARTY_LICENSES is ~4.7 MB and regenerates on dependency changes. That is the cost of holding 1,227 distinct notices covering 2,644 of 2,932 components (288 named as uncollected); deduplicated by text hash, it is as small as holding them can be.
• 19 of the 20 frontend system components ship no licence text in the image at all — Alpine does not install them, and several are GPL-2.0-only. They are named as uncollected rather than dropped: a copyleft component with no preserved notice is exactly the case where silence is expensive.
• Regenerating now requires Docker and both images. That is the point: the alternative is a document describing a machine nobody ships.
• Two defects outside the legal surface were found by measuring the image and are fixed here because the legal artifact cannot be correct while they stand: the five missing requirements.txt entries (including psycopg2-binary, whose absence is bug #369 reintroduced inside the container) and the profile-less docker compose build.
• backend/requirements.txt still installs pytest, pytest-cov, respx and testcontainers into the production image. Now visible in NOTICE because it inventories the image. Not changed here — removing them changes what ships and belongs in its own commit. Recorded, not silently carried.
• The .. licence pattern and the unpinned build floor are pinned by assertions on the specifier and on a built wheel, not on the manifest text, so an equivalent rewrite passes and any weakening fails.

Alternatives considered

Drop the preservation claim instead of collecting the texts. Legitimate today — nothing is redistributed (no PyPI publish, no image push, hosted API only) — and it was the cheaper branch. Rejected because the claim becomes load-bearing the first time an image is handed to a customer, and a collector that runs on a schedule is a smaller thing to own than a promise to remember.

Normalise licence strings to SPDX. Rejected: collapsing MIT License (a classifier) into MIT (an expression) asserts an equivalence upstream did not state, and the same shortcut turns Apache Software License into Apache-2.0, which is a version claim nobody made. They are listed as declared, and NOTICE says why.

Symlink the root LICENSE into each package. Rejected: it dangles in the container layout, which converts a silent omission into a build-time surprise rather than removing it.