Skip to main content

ADR-0102: Governed Learned-Procedures Store with Lifecycle + Consolidation

Date: 2026-07-07 Status: Accepted Deciders: Adrian (Soft4U), Claude Opus 4.8

Decision context:

  • Latency: +1 indexed DB query on the analysis path's get_applicable_procedures (was a Letta block read — comparable/faster); agent tool calls (list/save/deprecate) are user-initiated, not on any hot path. Negligible.
  • Dependency surface: no new packages — SQLAlchemy ORM + Alembic (migration 086) + the existing PydanticAI tool-calling surface + PromptRegistry. Reuses patterns already in the codebase (RLS tables, audit_events, partial unique indexes).
  • Debuggability: a superseded rule that keeps operating becomes a queryable status column, not a silent duplicate entry buried in a JSON list; every create/supersede/ deprecate transition emits an audit_events row, so "why did the assistant say X" is traceable to a specific row + lineage chain, not a Letta block diff.
  • Reversibility: additive migration (one new table, two new enums). The Letta learned_procedures block is retired from the write path but retained read-only for the one-time backfill — reverting means routing get_applicable_procedures back to the Letta block read, a ~2-file change; no destructive schema change to undo.
  • Blast radius: additive — one new table, one new service, three agent tools (one swapped for three), one prompt section, one runtime read redirected, one new UI panel. No existing schema, table, or endpoint is modified.
  • Alternative considered: enrich the Letta JSON block in place — rejected: no RLS, no audit trail, no queryability, unbounded JSON growth, and procedures stay in the ungoverned "assistance" Letta layer despite materially shaping officer guidance.

Context​

The Memory Assistant (memory_admin PydanticAI agent) teaches "learned procedures" — compliance rules an officer defines in conversation (e.g. "PSP merchant, incorporation < 3 months → request 3 months of bank statements"). Its only persistence tool, save_learned_procedure (app/agents/memory_admin_agent.py:130), appends a rule to a JSON list in the tenant Letta learned_procedures block on every call:

rules = block.get("rules", [])
rules.append(new_rule)                      # append-only — no dedup, update, or supersede
await letta.update_block(tenant_id, "learned_procedures", block)

There is no list / update / supersede / deprecate tool, so five conversational refinements of one policy became five competing rules, and the runtime consumer get_applicable_procedures (letta_policy_service.py:1659) reads all of them — superseded drafts keep operating alongside their replacements. Consequences: rule proliferation, competing/ambiguous guidance surfaced to officers, no version history, no queryability, and the whole surface lives in the ungoverned "assistance" Letta layer with no RLS and no audit trail — despite learned procedures materially shaping officer guidance on document requirements and risk handling. Two secondary doc bugs live in the same file: get_memory_blocks_info (line 74) hardcodes "four blocks" (omits org_policy — there are five), and explain_signal_safety_classes (line 108) conflates signal category (judgment/behavioral) with safety class (non_suppressible/preference_only).

Prompt changes alone reduce future clutter but cannot retire existing duplicates or make the store auditable. The robust fix is architectural: a durable, governed store + a real CRUD surface + a consolidation-aware teaching prompt.

There is no prior ADR governing this surface — save_learned_procedure and the Letta learned_procedures block were introduced without an ADR as part of the Letta learning layer (issue #242) and are documented here only as the informal status quo being replaced. This ADR does not supersede a prior decision record; it is the first ADR for learned-procedure persistence.

Decision​

Supersedes: the informal, undocumented append-only Letta-block approach described above (no prior ADR to formally supersede).

1. New RLS table learned_procedures (Alembic migration 086)​

Tenant-scoped, RLS WITH CHECK, tenant_id set explicitly (RLS-inert-safe, ADR-0050):

ColumnNotes
id UUID PK
tenant_id UUID FK (RLS)isolation
intent_key text NOT NULLdedup key (e.g. psp_newly_formed_bank_statements); stable across versions
title text NOT NULLhuman label
kind enum substantive|governancegovernance rules (structured exception notes) stay separate + reusable
scope, trigger, action, rationale textthe structured rule
details JSONB (default {})rule-specific richness: approved-alternatives list, structured-note fields, approver-required tiers
status enum draft|active|superseded|deprecatedlifecycle
version int NOT NULL default 1increments per intent_key
supersedes_id UUID FK NULL → selflineage back
superseded_by_id UUID FK NULL → selflineage forward
source enum officer_teaching|sleeptime_consolidationprovenance
created_by text NULL, created_at, updated_at timestamptz

Canonical-per-intent enforced at the DB, not the prompt: a partial unique index ux_learned_proc_active_intent ON (tenant_id, intent_key) WHERE status = 'active' makes two competing active rules for one intent impossible. Change history = immutable audit_events (ADR-0064) + the superseded chain — no separate revisions table.

2. LearnedProcedureService (ORM repository, RLS session)​

  • list(tenant_id, kind=None, status="active") → rows.
  • get(tenant_id, id) → row + lineage.
  • save_canonical(tenant_id, intent_key, title, kind, scope, trigger, action, rationale, details, created_by, status="active") — create-or-supersede, one transaction: if an active row exists for intent_key, set it superseded + superseded_by_id; insert the new row version = prev+1, supersedes_id = prev.id. Emits learned_procedure_saved audit.
  • deprecate(tenant_id, id_or_intent_key, reason, actor) → status=deprecated, emits learned_procedure_deprecated audit.
  • Never-suppress guard (fail-closed): save_canonical rejects (ValueError) an action matching a suppression pattern (ignore|skip screening|suppress|do not (flag|escalate|report)|waive .* (sanctions|pep|adverse) — a conservative denylist) — a learned procedure may only add scrutiny (the never-suppress cardinal rule; the agent tool surfaces the rejection to the officer).

3. Agent tools (memory_admin_agent.py) — replace the single append tool​

  • list_learned_procedures(kind?, status?) — check for an existing intent before saving; answer "what have I learned?".
  • save_learned_procedure(intent_key, title, kind, scope, trigger, action, rationale, details?) — wraps save_canonical (create-or-supersede — this is "update"). Returns the canonical rule + what it superseded. Persist only on explicit confirmation (prompt-enforced).
  • deprecate_learned_procedure(intent_key_or_id, reason) — retire a rule.
  • Fix the two doc bugs in the same file: get_memory_blocks_info → five blocks incl. org_policy; explain_signal_safety_classes → separate category (judgment/behavioral) from safety class (non_suppressible/preference_only).

4. Teaching-prompt consolidation policy (PromptRegistry("memory_admin"))​

Add a section: draft-first (hold the draft in conversation while the officer iterates; persist nothing until they confirm — "final / save this / confirmed / canonical"); check before save (call list_learned_procedures; if a same-intent rule exists, ask "update the canonical or create new?" — default update; set a stable intent_key); one canonical per intent (prior versions auto-superseded by the tool + DB constraint); governance separate (kind=governance for structured-exception/documentation rules); never-suppress (a procedure may only add scrutiny — the service enforces this fail-closed regardless of what the prompt says).

5. Runtime application (the correctness win)​

get_applicable_procedures (and any learned_procedures-block reader on the analysis path) queries LearnedProcedureService.list(status="active") — superseded / draft / deprecated rules stop operating. The Letta learned_procedures block is retired for procedures (kept read-only only for the one-time backfill).

6. API (officer-auth, tenant-scoped, CASE_READ/CASE_DECIDE)​

  • GET /api/memory/learned-procedures?status=&kind= — list.
  • GET /api/memory/learned-procedures/{id} — detail + version lineage.
  • POST /api/memory/learned-procedures — manual create/supersede (officer).
  • POST /api/memory/learned-procedures/{id}/deprecate — deprecate.

7. UI — "Learned Procedures" panel on the Memory page​

Active rules with status + kind badges, scope→trigger→action, edit (→ supersede), deprecate, version history (lineage); kind/status filters; skeleton loader, Sonner toasts, Sheet (no modal), honest empty-state ("No learned procedures yet — teach the assistant a rule to get started"). types + api client; tsc 0.

8. One-time backfill (resolves the existing 5-rule proliferation)​

Idempotent backfill_learned_procedures_from_letta(tenant_id) service method + a super-admin endpoint (POST /api/memory/learned-procedures/backfill): read the tenant learned_procedures Letta block, insert rows, group by intent (normalized scope+trigger theme), mark the latest per intent active, the rest superseded. For Pilot tenant: collapses the 5 into one active substantive bank-statements rule + one active governance exception-note rule, 3 superseded.

Scope boundaries (YAGNI / deferred — documented)​

  • Sleeptime auto-consolidation (signals → learned_procedures without officer teaching) stays out of scope — the source=sleeptime_consolidation enum value is reserved; wiring the Letta sleeptime agent to write through save_canonical is a follow-up (part of #242).
  • Semantic same-intent detection — v1 uses the explicit intent_key (deterministic, officer-confirmed update-vs-new); embedding-similarity auto-merge is deliberately rejected (see Alternatives).
  • Cross-rule governance linkage — a governance rule is a separate kind=governance row; formally linking "this exception note governs these substantive rules" is deferred.

Consequences​

Positive​

  • One canonical active rule per intent_key is enforced at the database, not by prompt discipline — the partial unique index makes the five-competing-rules defect structurally impossible going forward.
  • Every learned-procedure transition (save/supersede/deprecate) is RLS-isolated and produces an immutable audit_events row (ADR-0064) — the "assistance" layer gains the same auditability as the compliance spine (EU AI Act Art. 12 traceability), closing a gap where officer-shaping guidance previously left no audit trail.
  • runtime reads (get_applicable_procedures) become simpler and honest: only status="active" rows apply — no more superseded drafts silently still operating.
  • The never-suppress guard is enforced fail-closed at the service layer, not merely prompted — a learned procedure can only add scrutiny, never a suppression path, even if a future prompt regresses.
  • The backfill retires the existing 5-rule proliferation into 1 active substantive + 1 active governance rule (3 correctly marked superseded), with full lineage retained.

Negative​

  • The Letta learned_procedures block is retired from the write/runtime-read path. A Letta-only reader (any integration or agent that only reads that Letta block instead of the new table/API) loses visibility into procedures the moment this ships, until the one-time backfill has run for its tenant. This is a real, acknowledged migration gap, not a cosmetic one.
  • Two storage surfaces coexist during the transition: the retired-but-not-deleted Letta block (kept read-only for backfill) and the new RLS table. Until the backfill endpoint is invoked per tenant, that tenant's pre-existing learned procedures are invisible to the new runtime read path — an operational rollout step (super-admin must trigger backfill per tenant) is now required, and a forgotten backfill silently drops existing guidance rather than erroring loudly.
  • v1's same-intent matching depends on the officer (or the tool-calling agent) reusing the same intent_key string; there is no semantic/embedding fallback. An officer who phrases a follow-up teaching session without the assistant recognizing it as the same intent creates a second, differently-keyed active rule for what is logically the same policy — a duplicate that the DB constraint cannot catch because the intent keys differ.
  • One more RLS table, enum pair, and service to maintain; the agent surface grows from one tool to three (a wider contract for the LLM to call correctly).

Neutral​

  • No existing schema, table, or endpoint changes — additive only.
  • kind=governance vs kind=substantive is a coarse two-way split; finer categorization (if ever needed) is a future migration, not a blocker today.
  • The source=sleeptime_consolidation enum value is written but unused in v1 (reserved for the deferred sleeptime-consolidation follow-up under #242).

Alternatives Considered​

Alternative 1: Enrich the Letta JSON block in place​

  • Keep save_learned_procedure as the single persistence path, but teach it to deduplicate/update entries within the existing Letta learned_procedures block JSON (e.g. match on a normalized scope+trigger key, overwrite in place) instead of always appending.
  • Why rejected: the Letta block has no RLS, no audit trail, and no SQL queryability — any "what changed and when" question requires diffing raw JSON block history rather than querying rows. The JSON list still grows unbounded as procedures accumulate across a tenant's lifetime, and the whole surface remains in the ungoverned "assistance" Letta layer despite materially shaping officer guidance — the same governance gap this ADR exists to close, just with better JSON hygiene.

Alternative 2: Semantic same-intent detection via embedding similarity​

  • Instead of an explicit officer-set intent_key, use embedding similarity over scope+trigger text to auto-detect when a new teaching session refers to the same underlying policy as an existing rule, and auto-supersede without requiring the officer (or agent) to reuse a stable key.
  • Why rejected: embedding-similarity auto-merge can silently merge two rules that differ in scope, threshold, or strictness — for example a rule scoped to "PSP merchants only" and a broader rule for "all newly-formed entities" could embed as near-duplicates and get incorrectly consolidated, quietly narrowing or widening a compliance rule without officer confirmation. The explicit, deterministic intent_key is officer-confirmed and auditable; semantic auto-merge is deferred as a possible future enhancement gated on demonstrated precision, not shipped as the v1 default.