Skip to main content

ADR-0109: Append-only hash chain over audit_events (tamper-evidence)

Date: 2026-07-12 Status: Accepted Deciders: Adrian (Soft4U), Claude Opus 4.8

Context​

audit_events is the spine of the compliance story: ADR-0064 (DB-enforced immutability), ADR-0069 (the regulator-ready case pack, whose contents come from here), AMLR 5-year retention, EU AI Act Art. 12 logging, and the conformity record all lean on it. ADR-0064 makes the trail tamper-resistant with three database controls — a REVOKE, a guard trigger, an FK RESTRICT — that stop the application from rewriting history.

But a control is not evidence (#286). A superuser who drops the trigger and edits a row leaves no cryptographic trace; a restore from a tampered backup is indistinguishable from a restore from a clean one; a regulator asking "prove this trail was not altered" cannot be answered by pointing at a REVOKE. M1-W2 added a content digest over a backup, which detects corruption in transit but is not tamper-evidence — recomputing a digest over already-tampered data yields a perfectly consistent digest. The case pack is tamper-evident; the trail it is built from was only tamper-resistant. This ADR closes that asymmetry.

Decision​

Add an append-only hash chain over audit_events. Each row carries entry_hash = SHA-256(prev_hash || '\n' || canonical(row)), where prev_hash is the previous entry's entry_hash (a fixed GENESIS value for the first row). Any post-hoc alteration, deletion, or re-link breaks the chain from that point on and is detectable.

Design choices:

  • Per-tenant chain, keyed on tenant_id. Under RLS a tenant-scoped session only sees its own rows, so a single global chain could not read its own head. A per-tenant chain is also cleaner for multi-tenancy: each tenant's trail is self-contained and independently verifiable. Case-less system events (NULL tenant_id) chain under the demo-tenant default, matching where they are already stored.
  • chain_seq — a per-tenant monotonic position, assigned app-side under an advisory lock, defining the authoritative chain order (created_at can tie).
  • Concurrency: a per-tenant transaction-scoped advisory lock (pg_advisory_xact_lock keyed on the tenant) around the head-read-and-append in AuditService.log_event, so two concurrent writes cannot read the same head and fork the chain.
  • One pure hashing module (app.services.audit_hash) is the single source of truth used by the write path, the backfill migration, and the verifier — a second, subtly-different implementation would make the chain unverifiable. details (the one JSONB field) is coerced to a parsed object inside canonical_row so an ORM read (dict) and a raw-SQL read (JSON text) canonicalise identically.
  • Verification is exposed as a super_admin endpoint (GET /api/monitoring/audit-chain/verify?tenant_id=…): it replays a tenant's chain and reports the first break. Cross-tenant by nature (a tenant cannot be trusted to attest its own trail), so super_admin-only.

The backfill migration (088) must UPDATE existing rows to populate the chain, which the ADR-0064 immutability trigger blocks for every role. It therefore drops the trigger, backfills as the owner/migration role, and re-creates the trigger identically (verified: after the migration an UPDATE is blocked again). The REVOKE on trustrelay_app is untouched.

Decision context:

  • Latency: each audit write adds one advisory-lock acquire + one indexed head SELECT. Audit writes are not on a latency-critical hot path; negligible. id/created_at are now generated app-side (they must be known before hashing).
  • Dependency surface: none new — hashlib/json only.
  • Debuggability: the verifier names the exact first_break_seq, reason, and broken_row_id. A break is actionable evidence, not a vague alarm.
  • Reversibility: the migration downgrade() drops the three columns (DDL, not a row UPDATE, so the trigger does not block it). The write-path change is behind no flag but is additive (old readers ignore the new columns).
  • Blast radius: log_event is called on nearly every compliance path — but the change is additive (same row content, three extra columns) and covered by the existing audit tests plus new chain/tamper tests.
  • Alternative considered: a global (single) chain — rejected because a tenant-scoped RLS session cannot read a global head; per-tenant is both RLS-correct and cleaner.

Consequences​

Positive​

  • Tampering with the audit trail is now detectable, not merely prevented — the case pack's tamper-evidence extends all the way down to its source.
  • A restore can be verified against the chain rather than against itself.
  • One canonicalisation/hash implementation shared by write, backfill, and verify.

Negative​

  • Every audit write serialises per tenant (advisory lock) and does an extra head read. Acceptable at current volume; revisit (e.g. batched anchoring) if audit throughput grows large.
  • The chain proves internal consistency. Full non-repudiation against a superuser who edits a row and recomputes every subsequent hash requires an external anchor (signing the head, or writing it to an append-only external store) — a follow-on step this ADR deliberately scopes out.
  • created_at is now application-supplied (naive UTC) rather than DB now(), so it must be known before hashing; a negligible semantic shift.

Neutral​

  • No behaviour change for readers of audit_events; the new columns are additive.

Alternatives Considered​

Alternative 1: A single global chain​

  • One chain for the whole table, serialised on a constant lock.
  • Why rejected: a tenant-scoped RLS session cannot read a global head (RLS filters it to the tenant), and a global lock serialises all tenants' audit writes against each other.

Alternative 2: Sign each row individually (no chain)​

  • A detached signature per row instead of a linked chain.
  • Why rejected: per-row signatures detect a row edit but not a deletion or a re-ordering; the chain's prev_hash linkage is what makes deletions and re-links detectable, which is a core AMLR/EU-AI-Act requirement for an audit trail.