THE PROJECT Why this one teaches everything
A long-running personal assistant you talk to across many sessions. It remembers facts, recalls your notes semantically, knows how people/projects relate, and manages long conversations. Each capability maps to a distinct, justified store — so you learn when to reach for each, not just how. The dialogue itself is where every context-bloat technique lives.
WHAT IS THIS? The problem it solves
It's a personal work assistant — a "chief of staff" / second brain for one knowledge worker (you). A plain chatbot forgets everything when a session ends, so you re-explain your projects, people, and preferences every time. This assistant gives you continuity: it remembers your facts & preferences, recalls your past notes, tracks the people/projects you work with and how they connect, and keeps long conversations coherent. The value is recall across time — not a smarter single answer.
WALK-THROUGH A real conversation — what's stored where, and why
Two sessions, a few days apart. Watch each memory tier earn its place. SQLite Vector Graph Redis Context
Graph nodes
Priya, Rahul, Project "ABCCompany observability demo"; edges Priya-LEADS→, Rahul-WORKS_ON→, due=next Fri — why: roles/links you'll traverse later.Vector episode "kicked off ABCCompany demo — Priya lead, Rahul, due next Fri" — why: semantic recall of the note later.
SQLite fact
current_focus = "ABCCompany observability demo" — why: cheap exact lookup.Redis session turn appended with a TTL — why: fast working memory.
preferences.output_format = "dashboard" — an explicit "remember" → a durable preference, upserted by key so a later change overrides it (no contradictions). why structured: exact, deterministic, shapes every future output.fetch_logs → the tool returns a large payload (context bloat).customer_id at line 42 — 3 rows affected.Context the big
fetch_logs tool_result is cleared after use (context editing) — the dialogue stays intact. why: tool-result bloat.Vector episode "ABCCompany ingestion failing: null customer_id, line 42 (Mon)".
Redis the summary is cached, key =
hash(query + memory_version) — why: skip re-running an identical ask.customer_id at line 42.response.content is appended back). The detail is still safe in the Vector store. why: keep the window under the limit without losing recall.memory_version bumped → any cached answer about the due date is invalidated. why: you must never get the stale Friday date back.So — what lives where, and why (from this flow)
| Store | Earned its place at | Because |
|---|---|---|
| SQLite | pref (T2), due date (T4, updated T7) | exact facts you look up by key and overwrite |
| Vector | recall (T5) | "what did I say about…" — similarity recall of notes |
| Graph | who's-on (T4) | traversal over people/projects — not similarity |
| Redis | session & cache (T1, T3) | fast ephemeral working memory + response cache with TTL |
| Context (API) | big logs (T3), long session (T6) | context editing (tool bloat) + compaction (dialogue) |
PART A Conceptual architecture — the memory tiers
| Tier | Store | Holds | Retrieval | Teaches |
|---|---|---|---|---|
| Working / session | Redis | current session state, scratchpad, response cache, rate-limit | by session key + TTL | short-term memory, caching, eviction |
| Profile / facts | SQLite | profile, preferences, durable key facts | exact key / SQL | structured memory, write-vs-skip |
| Episodic / semantic | Vector (Chroma) | past notes, chat snippets, docs | similarity (RAG) | embeddings, chunking, recall |
| Relational | Graph (Kùzu) | people · projects · entities & links | traversal / Cypher | graph memory, multi-hop |
| In-conversation | the Claude API | the live dialogue | — | compaction · context editing · caching · truncation |
The turn lifecycle (this is the whole design)
[cached persona/tools] + memory block + dialogue"] PROMPT --> LLM["Claude (adaptive thinking, tools)"] LLM --> ANS["Reply to user"] LLM -.reflect.-> WR{"Anything worth remembering?"} WR -->|"durable fact"| SQ WR -->|"note/episode"| VEC WR -->|"new entity/link"| GR classDef s fill:#eef6ff,stroke:#3b7bd0,color:#1f5fa8; class FUSE,PROMPT,LLM s;
Two halves: read (route → retrieve → fuse → prompt) before the model answers, and write (reflect → persist) after. Keeping the fuse step small is the entire context-engineering game.
PART B The light/embedded stack
| Responsibility | Pick | Why (footprint) |
|---|---|---|
| LLM | Claude API — Opus 4.8 main, Haiku 4.5 for routing/extraction | Claude-native; tier by task |
| Working/session + cache | Redis (brew) or fakeredis (in-process) | tiny native, or zero-server for pure learning |
| Structured facts | SQLite (stdlib) | no server |
| Vector / embeddings | Chroma (embedded) + sentence-transformers (local, free) or Voyage (prod) | Claude has no embedding model — use a separate embedder |
| Graph | Kùzu (embedded, Cypher-like) | no JVM/Neo4j server |
| Orchestration | plain Python + Anthropic SDK (manual loop) | see the agent loop lessons (D1/D4) |
PART C Memory implementation — per store
- Keys:
session:{id}:messages(recent turns),session:{id}:scratch,cache:{hash(query+memory_ver)}. - TTL on everything ephemeral (session expires; cache expires) — teaches eviction.
- Also: a per-user rate limiter and the response/semantic cache (skip the LLM for repeats).
- Failure: Redis down → degrade to stateless (rebuild session from durable stores); never hard-fail.
- Tables:
facts(user_id, key, value, confidence, source, updated_at),preferences(...). - Upsert by key so a changed preference overwrites (latest-wins) instead of accumulating contradictions.
- Retrieval is exact/SQL — cheap, deterministic; great for "what's my timezone / default project".
- Store provenance + timestamp + confidence so you can trust, age, and audit a fact.
- Write each note / meaningful chat snippet / doc chunk → embed → store with metadata (user, timestamp, type).
- Recall by similarity to the question; cap to top-K with a score threshold (below it, retrieve nothing).
- Chunk size/overlap matter; dedup near-identical memories; this is memory-as-RAG.
- The raw episodic record here is also your safety net for compaction (see Part D): summarise the dialogue, but the detail still lives in the vector store.
- Nodes:
Person,Project,Task; edges:WORKS_ON,REPORTS_TO,MENTIONED_IN. - Answers "who works with Priya on ABCCompany?", "what's blocked on the demo?" — questions a vector store can't (they need traversal, not similarity).
- Extract entities/links from conversations (an LLM extraction step) and MERGE by a canonical id to avoid duplicate nodes.
- Return a small k-hop neighbourhood, summarised — never the whole graph.
PART D Context engineering — the heart of it
| Lever | What you build | Claude / CCA |
|---|---|---|
| Memory routing | A cheap classifier (rules or Haiku) decides which stores a turn needs — or query all in parallel and fuse. Skipping stores saves tokens & latency. | D1/D5 |
| What to remember (write policy) | A "reflect" step after each turn extracts durable facts/entities and routes them to the right store — and skips the ephemeral chatter. Not everything is worth remembering. | D3 |
| Retrieval fusion | Merge results from 4 stores into ONE small memory context block with a per-store token budget (e.g. top-3 vector, N facts, a summarised graph neighbourhood). Place it AFTER the cached prefix. | D5 |
| Prompt caching | Cache the stable persona + instructions + tool defs (byte-stable prefix); the volatile memory block & user turn go after the last breakpoint. | prompt caching D5 |
| Context editing | Tool calls (web fetch, doc read) return big payloads → clear stale tool_results while keeping the dialogue. | context editing D5 |
| Compaction | When a single long session nears the window, summarise the dialogue (append response.content!). The detail is safe in the vector store. | compaction D5 |
| Safe truncation | Fallback sliding window that never orphans tool_use/tool_result pairs. | D5 |
PART E Failure points & challenges
| Failure | Cause | Mitigation |
|---|---|---|
| Memory poisoning | The model writes a wrong/harmful "fact" — recalled forever | Validate on write · store confidence + provenance · confirm high-stakes writes · decay/TTL low-confidence memories |
| Stale / conflicting facts | A preference changed but the old one lingers | Upsert by key (latest-wins) · timestamps · contradiction detection at write |
| Wrong-memory recall | Vector returns off-topic snippets → distraction & hallucination | Similarity threshold · rerank · cap count · relevance filter before injecting |
| PII / secrets leak | Sensitive data written to memory, replayed into every session | Never store secrets · redact/classify on write · encrypt at rest · per-user isolation |
| Cross-store inconsistency | Same entity in SQLite, graph, and vector diverges | A canonical entity id · one source of truth per fact type · write-through / reconciliation |
| Stale cache | Response cache returns an answer after memory changed | Key the cache by (query + memory version); bump the version on any write |
| Compaction loss | The summary drops a detail you needed | Compact only the dialogue; keep raw episodes in the vector store; verify |
| Latency | Querying 4 stores every turn | Route (skip stores) · query in parallel · cache · async writes (reflect after replying) |
| Cost / unbounded growth | Embeddings + storage grow forever | Model-tier (Haiku for routing/extraction) · dedup · decay/evict low-value memories · cache |
| Cold start | Empty memory for a new user | Degrade gracefully; don't fabricate; ask & learn |
PART F Cross-cutting design
| Concern | Here |
|---|---|
| Security & privacy | Per-user memory isolation, PII redaction on write, encryption at rest, a "forget me" path (GDPR-style delete across all four stores). |
| Cost | Prompt-cache the persona; Haiku for routing/extraction; dedup + decay memories; response cache; bounded per-store retrieval budget. |
| Latency | Parallel store reads; route to skip stores; write memories asynchronously after the reply; cache. |
| Reliability | Any store down → degrade (skip that memory tier, don't fail the turn); retries/backoff on the API; loop termination. |
| Observability | Trace each turn: which stores hit, what was retrieved & injected, tokens (incl. cache_read), what was written, latency per store. |
| Evals | "Did it remember correctly?" — a golden set of (setup writes → later question → expected recall); measure recall precision, wrong-memory injection rate, and answer quality. |
CERT MAP Claude features this exercises
| In this project | Claude feature | CCA |
|---|---|---|
| The read→generate→write loop, routing, reflect step, graceful degradation | Agentic loop | D1 |
| Memory read/write tools; the Claude memory tool as one write path; is_error handling | Tool use / memory tool | D4/D5 |
| Persona & instruction design; write-vs-skip; clarify; extraction prompts | Prompt engineering | D3 |
| Prompt caching · context editing · compaction · truncation · retrieval fusion · evals · observability | Context & reliability | D5 |