ADR-0103: One mock flag, one capability — and a fail-closed production guard

Date: 2026-07-11 Status: Accepted Deciders: Adrian (Soft4U BV), Claude Opus 4.8

Decision context:

• Latency: no measurable impact. The startup guard runs once per process boot (a dict scan over ~15 settings fields, microseconds). The screening change replaces an in-process dict lookup with a real OpenSanctions call that was already the intended path — that call's cost (local pg_trgm bulk match, falling back to api.opensanctions.org) is the cost of doing the screening at all, which is the product's core obligation. Not screening is not a latency optimisation.
• Dependency surface: zero new packages. pkgutil/importlib are stdlib.
• Debuggability: strictly improved. Today an un-screened person is indistinguishable from a screened-clear person in the persisted record. After this, the two are not_screened and screened respectively, and a production process that would have fabricated the former refuses to boot with a message naming every violation.
• Reversibility: single config flip (~minutes) to re-enable the mock in a non-production environment. The flag decomposition itself is a ~4-file mechanical change.
• Blast radius: substitutive on one code path (natural-person screening), additive everywhere else. Existing cases screened under the mock will produce real hits on re-screen — see Consequences.
• Alternative considered: keep one flag, add a production-only override. Rejected — it preserves the class of defect (see Decision).

Context

KYC_MOCK_MODE gated two unrelated capabilities:

1. Identity verification (itsme / eIDAS) — genuinely unimplemented. identity_verification.py:95 raises NotImplementedError when the flag is off.
2. Natural-person sanctions/PEP/adverse screening — real, wired, and correctly fail-closed. kyc_screening.py:332-349 calls screen_opensanctions and returns an indeterminate result on error, never a clean one.

The flag was set true in the deployed configuration solely to stop (1) crashing. backend/.env:83-85 says so explicitly:

"Identity verification (itsme/eIDAS) + eID Easy have no real integration yet; mock them even in real-data runs, else verify_identity_activity NotImplementedError."

The side effect was that (2) — a mandatory AMLR Art. 28 control — was routed into a mock that:

• matched sanctions on magic substrings in the person's name ("SANCTIONS", "PEP", "ADVERSE"), and
• for every other person, emitted a finding with severity=VERIFIED reading "No sanctions matches found" (kyc_screening.py:42-113).

PersonScreeningService.screen() — wired into the officer-facing /identity API — independently returned hardcoded sanctions_clear=True, risk_score=0.1 for every person, directly beneath a docstring stating "MUST NOT fabricate and persist a clean sanctions_clear=True result." Its correct, fail-closed not_screened branch was unreachable because it was guarded by if not settings.kyc_mock_mode.

The gate was additionally if settings.scan_mock_mode or settings.kyc_mock_mode, so scan_mock_mode — a flag for the sanctions-scan LLM agents, a different capability — also suppressed real person screening. CI sets SCAN_MOCK_MODE=true.

This is the only place in the codebase where the system's cardinal rule is inverted:

The system may ADD scrutiny but NEVER suppress a risk signal; any scrutiny-reducing output must be evidence-traceable and officer-overridable.

The engineer who wired the real screener understood this. kyc_screening.py:321-324:

"…this preserves their determinism while still wiring the real screener for a real (both-flags-off) deployment."

The intent was correct. The real path was correct. A real deployment simply could not turn the flag off, because one boolean meant two things.

Decision

Two changes. The first fixes the instance; the second closes the class.

1. One mock flag gates exactly one capability

Capability	Flag	Real path exists?
Identity verification (itsme/eIDAS)	identity_verification_mock_mode	No — NotImplementedError stays; that is the honest behaviour
Natural-person sanctions/PEP screening	kyc_screening_mock_mode	Yes — screen_opensanctions, already wired, already fail-closed
Sanctions-scan LLM agents	scan_mock_mode (unchanged)	Yes — and it loses all authority over person screening

kyc_mock_mode is deleted. PersonScreeningService delegates to run_kyc_screening rather than carrying its own fabricated-clean branch — it did not lack a provider; it predated the real one and was never rewired. Its fail-closed not_screened result is preserved verbatim, including its AMLR Art. 28 / EU AI Act Art. 14 regulatory basis.

2. A production process refuses to boot with any mock capability enabled

app/startup_guard.py — called from both main.py and worker.py before either serves — raises ProductionConfigError when app_env == "production" and any of the following holds, reporting every violation at once:

• any *_mock_mode flag is True;
• any shipped default credential is still in place (minioadmin, admin, trustrelaydev);
• pii_encryption_enabled is False.

The mock flags are discovered from the settings model (model_fields, suffix _mock_mode) — not from a hand-maintained list. A mock flag added tomorrow is covered by this guard the day it lands.

3. The runtime override file is not consulted in production

Implementing (2) surfaced a third instance of the same class, and the worst of the three. config.get_mock_flag() reads a gitignored file, backend/.mock-mode-overrides.json, before the settings field — and it is re-read on every call, not cached. Its consumers include get_agent_model(), which returns the pydantic-ai "test" model when a flag is on. Among the callers: sanctions_resolver_agent, adverse_media_agent, document_validator.

So an untracked JSON file dropped on disk can silently switch the agent that resolves sanctions matches to a stub model — leaving no trace in git, in code review, or in a settings dump. The other two instances are at least visible in the repository. This one is not.

A boot-time guard cannot fix this. It would validate the file at startup, and a file written at t+1 would be honoured by every subsequent call. You cannot secure a runtime-mutable input with a boot-time check. Therefore:

• get_mock_flag() does not read the override file when app_env == "production" — by construction, not by validation. The file is a development affordance and stays one.
• The startup guard additionally refuses to boot if the file merely exists in production, regardless of contents. Its presence there means a gitignored development artifact was shipped inside a production image, which is worth crashing on by itself.

4. The naming convention is the discovery mechanism

Suffix-based discovery is only complete if every data-faking flag actually carries the suffix. cz_demo_fixtures did not — and it short-circuits the CZ registry fetch to a synthetic shell-company fixture (cz_ares_service.py:409), i.e. it makes the system return data it never obtained. It was invisible to the guard.

It is renamed to cz_demo_fixtures_mock_mode, and a guard test (test_no_data_faking_flag_escapes_the_naming_convention) now fails if any settings field whose name suggests faking (demo/fixture/fake/stub/synthetic/sample/dummy) lacks the suffix, unless it is listed as not-data-faking with a written reason.

A guard with a blind spot is worse than no guard, because it is trusted. The convention is not a style preference; it is the enforcement surface.

5. Two capabilities with no real provider stop defaulting to a fake

vop_mock_mode and precious_metals_mock_mode both defaulted to True — so even a pristine production environment would have booted with two mocked capabilities.

• vop_mock_mode → defaults to False. There is no real Verification-of-Payee provider, so the service now raises rather than serving MockVoPProvider's seeded three-IBAN lookup table. A capability with no real implementation must be unavailable, never faked — the same rule already applied to identity verification.
• precious_metals_mock_mode is deleted. It has no readers anywhere in the codebase. A flag that gates nothing is not a capability; it is only something for the guard to trip over.

That last property is the decision. Fixing KYC_MOCK_MODE alone would leave intact the mechanism that produced it: a flag enabled for one reason silently fabricating a compliance result for another. The same reasoning drove the companion change to test_agent_determinism.py (see ADR-0089 and its extension), where a hand-maintained list of seven agents was passing green while three risk-feeding agents sampled freely — including the premium-tier agent that resolves sanctions matches — because they were simply not in the list.

A hand-maintained list of things-that-must-be-safe is a list that will be incomplete. Discover, then require an explicit exemption with a written reason.

Consequences

Positive

• Natural-person sanctions/PEP screening becomes real. A sanctioned individual now produces a real OpenSanctions hit rather than a fabricated severity=VERIFIED clear.
• The not_screened fail-closed contract (ADR-0067) is honoured on this surface for the first time: an unavailable provider yields an explicit gap, never a benign result.
• CI stops screening persons against a fabricated clean result (scan_mock_mode no longer reaches this path).
• The class of defect is closed, not just the instance. A production deployment cannot serve a mocked compliance capability — it will not start.
• identity_verification keeps raising NotImplementedError. Honest.

Negative

• Existing cases screened under the mock will produce real hits on re-screen. Some persisted person risk is understated today. The ADR-0089 entity-risk ratchet ensures a re-screen can only raise, never silently lower — the correct direction — but the operational consequence is a possible wave of newly-surfaced findings on cases already decided. This is the system working. It must not be suppressed.
• Real screening costs a network call and can fail. It fails closed (not_screened), which is louder than a fabricated clear and will surface data gaps that were previously invisible. That is the point, and it will increase officer workload.
• The startup guard makes production boot brittle by design. A missing env var is now a crash rather than a silent default. Intended: for this product, a crash is cheaper than a wrong answer.
• Every deployment must now set APP_ENV explicitly. Defaulting it to development was chosen over defaulting to production so that local work is not gratuitously blocked — accepted risk: a production deployment that forgets APP_ENV gets no guard. Mitigated by making APP_ENV=production a required key in the deployment IaC (M2).

Neutral

• scan_mock_mode keeps its own meaning and its own scope. It is not deleted.
• The deterministic mock screener remains available for tests and demos via kyc_screening_mock_mode — it simply can no longer reach production.

Alternatives Considered

Alternative 1: Keep one flag; add a production-only override that forces real screening

• Retain KYC_MOCK_MODE but ignore it for the screening path when APP_ENV=production.
• Why rejected: it fixes this instance and leaves the class intact. The next flag that gates two capabilities gets the same defect, and the override list becomes another hand-maintained list of things-that-must-be-safe. It also encodes the confusion rather than removing it — the flag would still mean two things, with an exception.

Alternative 2: Implement identity verification, so the flag can simply be turned off

• Build itsme/eIDAS, remove the reason the flag was ever forced on.
• Why rejected: it requires vendor contracts and is weeks of work — during which person screening continues to fabricate clean results. The dependency is exactly backwards: a mandatory AMLR control must not wait on an optional convenience feature. (Note: the eID Easy portal OAuth path is already real and can be flipped on independently — tracked separately, and it does not gate this decision.)

Alternative 3: Do nothing; document the mock in the conformity record

• ai_act_conformity_service.py:455-462 already declares that mocked paths "must not be relied on for production decisions."
• Why rejected: a declaration is not a control. The system was making those decisions and persisting them as VERIFIED. Honest documentation of a fabricated output is still a fabricated output — and the real screener already existed, so the gap was unnecessary in the first place.