Skip to main content

ADR-0090: Pin Letta to 0.16.8 + application-side block-size guard

Date: 2026-07-05 Status: Accepted Deciders: Adrian (Soft4U), Claude Opus 4.8

Context​

The compliance-memory layer runs a self-hosted Letta server (agent memory: per-officer and per-tenant agents, core-memory blocks, archival passages, sleeptime consolidation). It is an assistance layer — it never feeds the deterministic compliance risk decision (that remains EBA-matrix + SQL confidence-engine); it enriches officer context, similar-case recall, and follow-up-task hints, always guarded "when available".

The server image was pinned to the floating tag letta/letta:latest, which had drifted to 0.16.5 (released 2026-02-24). The latest release is 0.16.8 (2026-05-14) — three patch releases on the same 0.16.x minor line. The Python SDK letta-client 1.12.1 is already current. Reviewing the 0.16.6–0.16.8 changelog against our exact usage surfaced:

  • 0.16.8 — sandbox→server tool-result transport switched from pickle to JSON (#3343). We register custom compliance tools that execute in Letta's sandbox and return results across that boundary; pickle deserialization there is an RCE-class surface, unacceptable in a system processing regulated PII.
  • 0.16.7 — compaction/summarizer correctness fixes, including the summarizer keeping the old model after the agent's model changes — directly relevant, as we had just switched Letta to openai/gpt-4.1 / gpt-4.1-mini. Also a silent context-window reset-to-32k fix.
  • 0.16.7 — removed server-side block-limit enforcement: the limit field is still accepted but no longer enforced, so a block a sleeptime agent keeps appending to can grow unbounded, silently inflating per-turn token cost.
  • 0.16.6 — default context window 32k→128k and block char limit 20k→100k; a fresh image silently raises token cost if we inherit the defaults.

Floating :latest also means any re-pull can change the running server version without a code change — undesirable for an auditable compliance system.

Decision​

  1. Pin the image to letta/letta:0.16.8 in docker-compose.yml (stop tracking :latest). Bump deliberately via ADR from here on. The letta_data volume persists, so agents/blocks/passages survive the in-place recreate.
  2. Keep letta-client 1.12.1 (already latest; 1.11→1.12 was additive, no breaking changes to the agents / blocks / passages / tools calls we use).
  3. Add an application-side block-size guard (letta_block_char_limit, default 8000 chars) in LettaPolicyService.update_block, replacing the enforcement Letta 0.16.7 dropped. Observability-only (amended after review): warn at the soft limit, error above the hard cap (2× soft) — but always persist the write. An earlier revision refused oversized writes; a review found that refusing an append-only compliance block (learned_procedures, officer-taught rules) would suppress added scrutiny (violating "may add scrutiny, never suppress a signal") and mislead the officer into futile retries. Unbounded growth is now surfaced for ops to consolidate, not silently dropped.

Consequences​

Positive​

  • Removes an RCE-class pickle-deserialization surface on the custom-tool path.
  • Picks up the summarizer-on-model-change and context-window-reset fixes, timely after the gpt-4.1 switch.
  • Block writes can no longer silently grow unbounded; runaway growth is logged and capped.
  • A pinned image makes the running Letta version deterministic and auditable.

Negative​

  • The block guard is observability-only, so it does not hard-bound token cost — a runaway block still grows; we rely on the warn/error logs prompting ops to consolidate. The proper bound is memory consolidation/summarization (a larger follow-up), not a refuse that would drop compliance content.
  • Pinning means upgrades are now a manual, deliberate step (an ADR each) rather than automatic — intended, but it is ongoing maintenance.

Neutral​

  • The 0.16.6 raised context/block defaults are not adopted; we keep behaviour conservative and revisit context-window sizing separately if recall quality needs it.
  • New 0.16.6–0.16.7 surfaces (Conversations API, memfs, multi-agent restricted execution) are not used; no adoption required.

Alternatives Considered​

Alternative 1: Stay on 0.16.5​

  • Rejected: leaves the pickle deserialization surface open on the tool path and forgoes the summarizer-on-model-change fix right when we changed models. No offsetting benefit.

Alternative 2: Track :latest and re-pull​

  • Rejected: non-deterministic server version, no audit trail of what changed when, and it would silently adopt the raised 0.16.6 defaults. Unacceptable for a compliance system.

Alternative 3: Truncate over-limit block values instead of refusing​

  • Rejected: truncating a serialized JSON string yields invalid JSON that the readers (get_block → json.loads) cannot parse. Refusing the write keeps a valid prior block and surfaces the anomaly in logs.

Decision context:

  • Latency: negligible — one len() check per block write; the image bump adds no per-call cost. Not measured because the guard is O(1) on an already-serialized string.
  • Dependency surface: same server image family (0.16.5→0.16.8, same minor); client unchanged at 1.12.1. No new packages.
  • Debuggability: the guard logs a warning (soft) / error (hard) with entity, label, and size — an over-growing block is now visible instead of silent cost creep.
  • Reversibility: single-line image revert + config removal; the volume is unchanged. Hours to undo: <1.
  • Blast radius: additive — the guard only affects writes exceeding the cap; the image bump is within-minor with no endpoint changes on the paths we use.
  • Alternative considered: staying on 0.16.5 — rejected for the open pickle surface.