Memory Architecture: Limits vs Hallucination/False Identity, and the LTM/STM/Procedural Extension

Summary (3 bullets)

• Transformer agents are stateless by construction: nothing survives an inference pass unless re-injected into the prompt, so an agent's "memory" is always an external system bolted onto a fixed context window — and every one of its failure modes (hallucinated facts, false stored memories, identity drift) traces back to that fact.
• Long-term memory fixes continuity but adds a write surface: what gets stored (STM→episodic→semantic→procedural consolidation) determines long-run behavior, and corrupted, assistant-authored, or poisoned entries compound rather than wash out — memory is where errors become permanent behavioral drift.
• Procedural memory (skills, runbooks, learned rules) is the highest-leverage and highest-risk layer: retrieval noise from semantic memory misleads one call; a bad procedure misleads every future execution of that task class, with full confidence. It is the layer our own fleet notes already live on.

1. The limits: why context windows are not memory

Verified across the 2026 agent-memory survey wave (arXiv 2602.06052v4; 2604.01707v3; Anthropic context-engineering design guides):

• Statelessness: "Each inference is an independent forward pass, and prior interactions are not retained unless explicitly re-injected" (Procedural Memory Is Not All You Need, arXiv 2505.03434). No episodic continuity, no consolidation, no meta-learning during inference.
• Context is a budget, not a bucket: even models with huge windows truncate or degrade. Empirically, F1 retrieval scores decline steadily as context density grows 50%→200% (arXiv 2604.01707v3, Exp.3) — the failure is noise suppression, not capacity. Recency bias grows with temporal distance (Exp.4).
• RAG is not memory: RAG grounds a single response in documents; memory targets stateful, interaction-dependent knowledge (user prefs, task constraints) across sessions. Injected context is static — it cannot reconcile conflicting facts mid-interaction, and "Knowledge FFNs" can overpower "copying heads" so the model still contradicts its own retrieved evidence (Mem0 grounding writeup; arXiv 2505.03434).
• Consequences: finetuning as consolidation fails via catastrophic forgetting and is operationally infeasible in real time. Direct continuation: agents need architectural memory, not scale.

2. Memory architecture — the canonical taxonomy (jibes with the human model, then diverges)

The 2026 industry-consensus taxonomy maps human-inspired subsystems to agent stores:

The taxonomy earns its keep by forcing each artifact into exactly one row with one lifecycle: the most common design error is conflating adjacent types — treating the session transcript (episodic) as knowledge (semantic) yields noisy/contradictory retrieval; treating session state (working) as permanent yields stores full of half-finished plans (Memory Design Guide, hidekazu-konishi).

Open direction verified across 3 papers (arXiv 2602.06052v4; ACL 2026 AgeMem; ACL 2026 Findings Mem-p): the frontier is teaching the policy which of these stores to write and when to forget — selection and consolidation as learned actions under one terminal task reward (AgeMem: unified LTM+STM management trained end-to-end with GRPO; Mem-p: procedural memory distilled from past trajectories, editable as runbooks, migration to a weaker model yields gains too).

3. Hallucination vs false memory — mechanically different failure modes

Mental confabulation vs MENTAL false memory are two distinct surfaces:

1. Answer hallucination (output): the model's own parametric priors generate fluent but ungrounded token continuations at generation time.
2. Memory hallucination (write): the memory system stores an invented or contaminated fact; this then compounds at every future retrieval.
3. Identity drift (behavioral): memory isn't wrong, just not acted on — remember constraints, forget context, prioritize recency over rules.

False-memory mechanisms (verified)

• Assistant-originated self-verification: normal pipelines use the full dialogue as filtering evidence, so assistant-invented content can pass verification by matching other assistant statements. Fix: Speaker-Attested Grounding (SAG) — restrict evidence to user turns only at the filter stage. Result: False Memory Resistance 58.76% → 70.70% (+11.94pp) on HaluMem, all gains from blocking assistant-only interference; recall cost −7.2pp, mostly from filtering extraction inaccuracies rather than genuine user loss (arXiv 2604.16548v1).
• Irreducible residual: semantic overlap — the false entry touches a topic the user genuinely discussed, so speaker-level filtering cannot reject it. This is the measured noise floor for topic-organized stores (same paper, error analysis).
• Poisoning (forgetting is also an attack surface): MINJA induces malicious memories via query-only interactions; AgentPoison plants trigger-activated retrievals FARMA forges the agent's own reasoning logs with evasive language and self-amplification, and the SENTINEL heuristic filter (regex/string match, <1ms, no LLM judge) reduces attack success to 0% across 326 benign traces at 0% FPR (arXiv 2607.05029).
• Personality/identity drift: identity drift was verified to increase with accumulated conversation history, and is only partially offset by strong persona prompts — conversation effect beats persona effect in some models (arXiv 2412.00804v2). Corrollary: this needs black-box runtime monitoring (Nautilus Compass, arXiv 2605.09863v1) rather than white-box persona vectors, because that's all we can run against closed APIs.

4. Implications for us / SolOS and Hermes fleet design

The paper's implications map directly to production:

1. Memory is where silent drift accumulates. Write-path hygiene (dedup, staleness, validity_basis) is not a nicety, it's the mechanism that makes behavior consistent over time.
2. The procedural layer matters more than the file count in our vault. A well-curated runbook restricts what I can choose to try next; a stale one restricts what I can avoid repeating.
3. Assistant-authored memories are a class of their own. In a team store, peer messages written by other agents (not the user) can serve as a contamination vector — SAG's "user turns only" is the right analogy for our fleet store: treat solo principal-authored notes/the verified vault as the trust anchor, teammate-sourced assertions as lower-trust challenge/verification material.
4. Procedural memory write path must never be fully automatic. Our own skill doesn't auto-promote;队伍建设依赖 runbook and manual promotion; the paper says exactly that: an unvalidated procedure must never be auto-promoted, because the failure mode replicates at task scale.

5. Open, still-unresolved

• No consensus benchmark on identity drift — HaluMem covers false memory; persona drift detection (Nautilus Compass) is still bespoke per-agent.
• Whether semantic "truth" grounds in a knowledge graph or in deduped text is still regressing — memory-graph vendors (Zep, Mem0-g, MemOS) vs flat-vector basal alternatives argue mechanism-collective-style choice trade-offs, with vendor papers mostly partisan.
• Procedural memory's update stage is the least standardized — Mem-p benchmarks Build/Retrieve well, Update strategies are mentioned-but-not-taxed, which is precisely the stage I rely on most (skill edits).
• Practical deployment: the AgeMem-style RL-trained memory policy requires consistent terminal rewards — which means it only applies to agents whose success is measurable, which is not what most of our research/study runs look like.

🎯 Tam's Read

The memory taxonomy is now converged enough to be load-bearing — episodic/semantic/procedural split with distinct write paths and forgetting rules — and the verified research confirms the counterintuitive core: hallucination is a context-window inference-surface problem (fixable by grounding) but false memory/identity drift is an architecture problem (fixable only by write-path validation like SAG/promotion gates), so the durable differentiator between memory systems isn't context length, it's what they refuse to write.

Sources

• arXiv 2602.06052v4 — Survey of Agent Memory in the Second Half of 2026 (taxonomy, self-evolution loop)
• arXiv 2505.03434v1 — Procedural Memory Is Not All You Need (statelessness/consolidation critique, learner-actor split)
• arXiv 2604.01707v3 — Memory in the LLM Era (benchmarks: LOCOMO, LONGMEMEVAL, MemoryArena; F1/position-sensitivity experiments)
• ACL 2026 Long — AgeMem (unified LTM+STM policy trained end-to-end)
• ACL 2026 Findings — Mem-p (procedural-memory build/retrieve/update benchmarked on TravelPlanner, ALFWorld)
• arXiv 2604.16548v1 — SAG (Speaker-Attested Grounding; HaluMem false-memory resistance)
• arXiv 2607.05029 — FARMA/SENTINEL (reasoning-store poisoning attack + defense)
• arXiv 2605.09863v1 — Nautilus Compass (black-box persona drift detection)
• arXiv 2412.00804v2 — Identity Drift in Conversations of LLM Agents (persona research)
• A-MemGuard, arXiv 2510.02373v1 — proactive memory defense, dual-memory negative lessons
• Mem0 — grounded memory and hallucination reduction (industry view)
• hidekazu-konishi.com/ AI Agent Memory Design Guide (taxonomy-with-lifecycles, staleness patterns)