Skip to main content

ADR-0108: Proactive retention-purge schedule (GDPR storage limitation)

Date: 2026-07-12 Status: Accepted Deciders: Adrian (Soft4U), Claude Opus 4.8

Context​

The DSR erasure mechanism (DSRService.handle_erasure, four-store) and policy (determine_erasure_action) are complete and tested (M1-W6), but erasure was purely reactive — it ran only when a data subject filed a request. Nothing enforced the GDPR Art. 5(1)(e) storage-limitation principle on its own, so personal data whose AMLR 5-year retention window had expired lingered indefinitely (#298).

The M1-W6 agent hit an API error before building the D4 active-purge trigger; the gap was filed rather than silently dropped. This ADR records the decision for that trigger.

Two forces shape it. First, proactively deleting PII on a timer is high-stakes: a bug that deletes in-window records would destroy regulator-demandable CDD evidence (AMLR retention beats erasure, Art. 17(3)(b)). Second, the deletion decision already exists and is tested — the only thing missing is a trigger, so the trigger must reuse the tested decision rather than re-implement it.

Decision​

Add a system-level retention-purge Temporal Schedule that fires RetentionPurgeWorkflow every settings.retention_purge_interval_hours (default 24). The workflow runs one activity, run_retention_purge(), which:

  1. Iterates every tenant (cross-tenant like the ADR-0096 monitoring reconciler), acting inside a tenant-scoped RLS session with per-tenant guard-and-swallow.
  2. Enumerates that tenant's terminal cases and decides eligibility in Python via the canonical determine_erasure_action — not a SQL date comparison — so there is one retention-decision source and the pre-filter agrees exactly with the mechanism.
  3. For every subject in an eligible case, calls the unchanged handle_erasure, which re-checks determine_erasure_action per case. That per-case re-check is the safety boundary: an in-window or still-open case can never be purged, even if the enumeration were wrong. The enumeration is an optimisation, not the guarantee.
  4. Audits each tenant's purge that erased anything (append-only, ADR-0064; Art. 5(1)(e) legal basis recorded). An empty sweep is not an event.

Dark-launched behind retention_purge_enabled (default False): the schedule is only provisioned at startup when explicitly enabled, so turning on scheduled PII deletion is an opt-in, Calibration-Review-gated action.

A supporting fix hardens determine_erasure_action to normalise a timezone-naive case_closed_at to UTC before comparison — the cases.updated_at column is tz-naive under create_all and tz-aware under the migrations, and a retention decision must not crash on either.

Decision context:

  • Latency: none on the request path — a background daily schedule. One sweep is O(terminal cases × subjects) DB work per tenant, bounded and off-peak.
  • Dependency surface: none new — reuses handle_erasure, the ADR-0096 schedule helpers, AuditService. No migration (rides on existing tables).
  • Debuggability: every purge that deletes is a RETENTION_PURGE audit event with the tally; the sweep logs a per-tenant and a total line. records_retained > 0 is visible evidence of the per-case fail-safe keeping in-window data.
  • Reversibility: the schedule is one Temporal object (deletable); the flag defaults off. Deleted PII is not recoverable — which is the point — so the fail-safe, not reversibility, is the safety story.
  • Blast radius: additive. No existing path changes except the defensive tz-normalise in determine_erasure_action (aware inputs unchanged; 61 DSR tests still pass).
  • Alternative considered: a SQL WHERE updated_at <= cutoff pre-filter — rejected; it binds a tz-aware cutoff inconsistently against the naive/aware column and forks the retention decision into a second, subtly-different place.

Consequences​

Positive​

  • GDPR Art. 5(1)(e) is enforced proactively, not only on a filed request.
  • One retention-decision source (determine_erasure_action) drives both the pre-filter and the per-case guarantee — they cannot drift.
  • Reuses the tested erasure mechanism unchanged; the new surface is a thin trigger.

Negative​

  • Deletion is irreversible by design; correctness rests entirely on the per-case fail-safe, which is therefore covered by explicit in-window / active-case tests.
  • The audit-event PII itself is only truly erasable once ADR-0107 crypto-shred lands (noted; the purge deletes the subject records and their index linkage today).
  • Runs the full erasure decision for every terminal case each sweep (no incremental cursor) — acceptable at current scale, revisit if terminal-case counts grow large.

Neutral​

  • Dark-launched: no behaviour change until retention_purge_enabled is flipped, which requires a Calibration Review.

Alternatives Considered​

Alternative 1: Keep erasure reactive (DSR-only)​

  • Do nothing; rely on data subjects to request erasure.
  • Why rejected: Art. 5(1)(e) is an obligation on the controller independent of any request; leaving expired PII in place is a standing compliance gap.

Alternative 2: A SQL-side date pre-filter​

  • WHERE status IN (terminal) AND updated_at <= now() - 5y.
  • Why rejected: the column's tz-awareness differs between create_all and the migrations, so the aware cutoff binds inconsistently; and it duplicates the retention rule in SQL where it can silently diverge from determine_erasure_action.