Memory architecture — why limits, hallucination/false-identity risk, and extending into LTM/STM/muscle

Summary

• The 2,200/1,375 char caps exist to protect the prompt cache, not to model biological memory. Memory is injected as a frozen session-start snapshot, so a growing memory block would invalidate the cached system prompt on every session — the caps buy prompt-cache stability at a fixed ~1,300-token cost per prompt.[3]1
• The limit does not cause hallucination. Audited live in the installed source: an over-limit write is a hard rejection that returns the full entry inventory plus usage, never a silent drop or a fabricated entry; after three failures in one turn the store returns a terminal result so a failed side-effect cannot block the reply (memory_tool_store.py:333-353, :169-186). Hermes already ships the two defences the literature says matter: privileged system-prompt anchoring and an auditable recall path.[6]9
• False identity tracks loss of anchoring, not budget size. In a controlled study, agents whose persona was no longer re-injected at the privileged system level showed persona-enacting failures while anchored controls held 17/17 — and the shift was "behaviorally silent". Identity confusion is also measured as independent of memory size: 25.93% of 27 models showed it.[9]10
• The architecture already has three tiers — frozen core memory (Tier 1), unlimited FTS5 session recall (Tier 2), progressive-disclosure skills (Tier 3) — and the correct extension is to connect them through explicit consolidation interfaces rather than to grow the fixed budget.

Why the limits exist — verified, not assumed

• MEMORY.md and USER.md are rendered into the system prompt as a frozen snapshot captured at load_from_disk(); mid-session writes update live state and disk but deliberately do not touch the prompt, and invalidate_system_prompt() reloads only on compression (system_prompt.py:851-860). The cache-stability motive is the design constraint the caps enforce.[1]4
• Cost arithmetic: ~1,300 tokens on every prompt forever, vs session_search which is ~20 ms and zero LLM calls, only costing tokens when results actually enter context. That ratio is why the docs route small always-in-context facts to memory and everything else to recall.2
• The failure path is loud, not lossy: add computes the joined length and, if it would exceed the cap, returns {success: False, current_entries, usage} with the exact overflow in chars and a single-batch remediation instruction; replace and batch writes behave the same, and _consolidation_failure caps retries at three per turn with a terminal {done: True} result (memory_tool_store.py:333-353, memory_tool.py:151-155).1
• Two real gaps I verified in source rather than inheriting from a summary: the injection scan truncates at MAX_SCAN_CHARS = 65_536 (threat_patterns.py:53,138), and cross-script confusables (Cyrillic а) are not defended because NFKC normalizes but does not transliterate. Neither gap is memory-size related, and both are injection defenses rather than capacity ones.

Does bounded memory cause hallucination or false identity? — the evidence

Capacity-forcing compression (M1) is real but the weakest mechanism at this size. Across six compression methods, all increased hallucination somewhat with information loss the primary driver, and aggressive KV-cache compression can raise hallucination scores up to 3.36× in long-context summarization.[11]14 The same body of work finds moderate compression can improve performance in long contexts, and recall is measured as degrading inside the nominal window rather than only at its edge.11 Direction therefore depends on degree, and by that standard 2,200 chars (~550 tokens) is mild.12

False identity is an anchoring failure. Preserving conversational history while removing privileged re-injection of the persona produced persona-enacting failures, and the change could be behaviourally silent.9 Identity confusion is separately measured as independent of memory size, appearing in 25.93% of models.10 The design consequence is unambiguous: keep the persona anchor at system-prompt privilege, and keep an auditable record of evictions, rather than resizing the budget.6

Eviction without provenance (M2) is the dominant risk, and Hermes avoids it. MemGPT's design evicts messages at a flush threshold but stores them indefinitely in recall storage readable via function calls; retrieval-augmented systems cut knowledge hallucination by over 60%.6 A bounded budget rewritten per turn without an auditable record of what was dropped would be MemGPT's eviction minus its recall path. Hermes has the recall path: session_search over SQLite FTS5 with verbatim, unsynthesized recall.[2]7

Unbounded inference over a summary (M3) is the tail risk. Across 12 models, every one over-inferred 35–49% of its claims (mean 41.6%), and inferred attributes accumulate approximately linearly with little revision — so a curated core can be read as licence to elaborate rather than as grounded fact.8 LLMs also produce plausible-but-unsupported circumstantial inference in summarization.13

Verdict: bounded core memory is not itself a hallucination cause. It can actively reduce confabulation when paired with retrieval and anchoring. Causation is carried by provenance and anchoring, not by budget size.

The three tiers Hermes already implements

The documented routing rule is the one to keep: memory = small durable facts that must always be in context; skills = longer procedures loaded only when relevant; session_search = "did we discuss X last week?"[8]1 Note the asymmetry the sources make explicit: memory entries are removed outright when superseded, while skills are only archived — so skills are the safer home for anything expensive to reconstruct.

Extending memory — the interfaces that connect the tiers

Design proposal (not yet implemented, flagged as a proposal throughout):

• STM → LTM consolidation trigger. The rule that prevents double loss: anything the compaction summary carries must also land in durable storage, or it is lost twice. Fire on compression, session end, an explicit "remember this", or a decision that outranks the current context. The compaction summary is the trigger signal because it is exactly what survives the window boundary.5
• STM → muscle promotion. When the same procedure succeeds twice, write or refine the skill in that turn — never later. This is the rule that converts experience into a capability rather than a memory dump.8
• LTM → muscle promotion. A note referenced repeatedly, a convention that keeps blocking, or a drift-free "always do X" becomes a skill; the note then points at the skill instead of duplicating it, so the fact lives once and the procedure lives once.
• Muscle → LTM feedback. A skill patched for drift records the changed fact in the vault; the skill keeps only the rule. Skills are "how", never "what was true" — that split is what stops skill bodies from going stale.
• Fixed core stays immune to tier movement. SOUL.md persona at slot #1, the context-file chain (.hermes.md → AGENTS.md → CLAUDE.md), MEMORY.md/USER.md as a pointer index rather than a content store, and curated hooks. Everything else must live as a vault note or a skill, never as prompt text.[4]6
• Provenance is the invariant that carries the anti-hallucination property. Every durable entry carries author, date and source, and superseded entries get a pointer rather than a deletion — the vault's existing "nothing deleted, conflicts quarantined" rule is what makes M2 structurally impossible rather than merely unlikely.2

What must never move between tiers

Secrets stay in ~/.hermes/.env and are never written to memory, a vault note, or a skill — no recurrence rate justifies promotion of a credential-shaped string into muscle memory, because skills carry no store-level secret hygiene and their bodies load into arbitrary future contexts. Per the docs' own memory guidance, raw data dumps, session-specific ephemera and trivially re-derivable facts do not belong in the fixed budget either.1

Live grounding (so this is not an abstract study)

Verified on this host while writing: memory_char_limit: 2200, user_char_limit: 1375, write_approval: false; MEMORY.md 2,288 bytes — 88 chars over the cap, which is precisely the state this architecture study exists to explain; USER.md 1,330 bytes; 19 skills installed; state.db 1.86 MB. The over-cap state is not a fault: the loader keeps entries loaded and warns, and the next write is forced to consolidate.1

Gaps kept as gaps

Exact on-disk entry format of MEMORY.md/USER.md beyond the § delimiter, the auto-capture trigger for background writes, FTS-rebuild internals, the programmatic trigger of a parent_session_id split, and the consolidation prompt internals remain undocumented and are not reconstructed here.

Relation

2026-10-11-hermes-internals

Sources

1 https://hermes-agent.nousresearch.com/docs/user-guide/features/memory2 https://hermes-agent.nousresearch.com/docs/developer-guide/session-storage3 https://hermes-agent.nousresearch.com/docs/developer-guide/prompt-assembly4 https://hermes-agent.nousresearch.com/docs/user-guide/features/skills5 https://hermes-agent.nousresearch.com/docs/user-guide/features/curator6 https://arxiv.org/pdf/2310.085607 https://arxiv.org/abs/2104.075678 https://arxiv.org/html/2608.04570v19 https://arxiv.org/abs/2610.0149010 https://arxiv.org/abs/2411.1068311 https://arxiv.org/html/2505.00019v112 https://arxiv.org/abs/2307.0317213 https://aclanthology.org/2024.acl-long.677.pdf14 https://aclanthology.org/2026.acl-long.1542