Skip to main content

ADR-0148: Neural-semantic retrieval lane (Exa) as a third adverse-media/discovery source

Date: 2026-07-27 Status: Accepted Deciders: Adrian (Soft4U), Claude (Opus 4.8)

Decision context:

  • Latency: not measured under load because the lane is dark-launched OFF (exa_search_enabled=False) and runs at most once per adverse-media query, bounded by a 25 s client timeout, only on the adverse_media search type. A single live call measured ~1–2 s.
  • Dependency surface: no new package — a direct httpx REST call to https://api.exa.ai/search (httpx already used by the Tavily/SERP lanes). One new secret (exa_api_key) + one flag.
  • Debuggability: each hit carries _provider='exa' provenance (ADR-0077); a lane failure logs a data gap warning and returns ([], True); no exception escapes the lane.
  • Reversibility: single config flip (exa_search_enabled); flag-off is byte-identical (the lane returns ([], False) and the merge helper is a pass-through). No migration.
  • Blast radius: additive. The lane merges hits into the existing per-query bucket and folds its degraded status into the existing fail-closed marker; it introduces NO new severity/scoring path.
  • Alternative considered: an embedding index we build/host ourselves (rejected for now — Exa is a managed neural-search API; self-hosting an index is a large, separate effort with no near-term payoff).

Context​

Keyword/boolean SERP (Tavily) and BrightData under-rank semantic associations that a neural/embedding search surfaces directly. On the OB Holding 1 OÜ live investigation (2026-07-27) the group's Lithuanian enforcement history — OlyBet ↔ Olympic Casino ↔ a €8.4m AML fine — was missed by the keyword lanes. A single neural search for "OlyBet Olympic Entertainment Group money laundering enforcement" returns, as its top result, "Lithuania fines Olympic Casino €8.4M for AML failures" — the exact signal the boolean lanes under-ranked. This is the one genuine competitive-recall edge worth neutralising, built entirely as our own lane (issue #767 is explicit: build our own lane, never reference or port any competitor codebase).

Forces:

  • The adverse-media agent already runs a two-provider merge (Tavily SERP + BrightData SERP) per query, with a fail-closed "degraded" contract (a provider that could not run → a data gap, never a clean empty) and honest _provider provenance.
  • The load-bearing non-goal of epic #730: no soft-averaging or alternative scoring path. A new retrieval source must not become a new way to raise (or lower) risk — it feeds the SAME deterministic escalators through the SAME two-lane attribution (ADR-0078).

Decision​

Add Exa as a THIRD retrieval lane (_exa_search) in the adverse-media agent, scoped to the adverse_media search type (_is_neural_lane_query) — sanctions/PEP are exact-match list screens where neural search adds noise, not recall. The lane:

  • Is a direct REST call (no LLM, no agent). It maps Exa's embedding-ranked documents to the SAME hit shape (title/url/content/_provider='exa') as the SERP lanes, so attribute_hit / hit_names_in_scope treat a neural hit identically — verified- identifier lane vs unverified-candidate lane, unchanged.
  • Is folded into the existing per-query merge via a _finish helper: its hits are deduped into the same bucket and its degraded status is OR-ed into the query's degraded flag. Embeddings + ranking stay inside Exa; no scoring is added here.
  • Is fail-closed-not-empty: enabled-but-unconfigured / timeout / transport / non-200 → ([], True) (an incomplete-coverage data gap, surfaced by the existing degraded_search_types → screening-gap finding), never a benign "no matches". Flag-off → ([], False) (no contribution; byte-identical to today).
  • Is dark-launched behind exa_search_enabled (default False); the key lives in exa_api_key.

Consequences​

Positive​

  • Recovers the semantic-association recall the keyword lanes miss (validated: the €8.4m LT fine surfaces as the top neural hit) without any new scoring path.
  • Additive and reversible; flag-off is byte-identical, and all fail-closed / never-suppress invariants (ADR-0067) and two-lane attribution (ADR-0078) are preserved by construction.

Negative​

  • A third metered external API — cost per adverse-media query when enabled; mitigated by the flag, the adverse_media-only scope, and exa_num_results budget.
  • A neural search can return topically-adjacent but off-subject documents; these are handled by the SAME subject-relevance guard + two-lane attribution as every other lane (a name-only neural hit stays an unverified candidate, never a subject floor).

Neutral​

  • Adds _provider='exa' as a fourth provenance value; report/UI provenance rendering is unchanged (it already reads _provider).

Alternatives Considered​

Alternative 1: Route neural search through an LLM agent (like the earlier SERP agent path)​

  • Why rejected: the agent-wrapped SERP path timed out under concurrent load (ADR-0077); a direct REST call is deterministic and faster, and neural ranking needs no LLM in the loop.

Alternative 2: Self-hosted embedding index over our own corpus​

  • Why rejected (for now): large, separate effort; Exa is a managed neural-search API that reaches the open web (where regional enforcement press lives). Revisit if cost or coverage warrants.

Alternative 3: Do nothing (keep two lanes)​

  • Why rejected: the keyword lanes provably miss brand↔enforcement semantic associations (the OB LT gap), a never-suppress recall hole.