LAB container runtime — bare-metal path committed

Card t_ac60925f asked for one of two outcomes, recorded with a reason: install Docker on LAB, or commit to the bare-metal path so the memory decision is not blocked on a maybe.

Outcome: bare-metal committed. Docker is not installable on LAB as this host is configured, and the bare-metal path was proven working end-to-end tonight, not reasoned about.

1. Docker on LAB — measured absent, and measured uninstallable

Absent (all probes run on LAB as lab, 2026-10-21):

Uninstallable, and the reason is the interesting part. The obvious route — apt-get install docker.io — resolves cleanly (29.1.3-0ubuntu3~24.04.2 from noble-updates/universe, plus containerd 2.2.1, runc 1.3.4, bridge-utils, pigz, ubuntu-fan) and the .debs download fine as an unprivileged user (73.3 MB fetched). But every install route needs root, and this agent has none:

Rootless containers are also blocked, and this one is a kernel policy, not a missing package:

$ unshare -Ur true
unshare: write failed /proc/self/uid_map: Operation not permitted

$ journalctl -k -n 30 | grep apparmor
LAB kernel: audit: type=1400 operation="userns_create" class="namespace"
  profile="unconfined" requested="userns_create" target="unprivileged_userns" execpath="/usr/bin/unshare"
LAB kernel: audit: type=1400 apparmor="DENIED" operation="capable" class="cap"
  profile="unprivileged_userns" comm="unshare" capability=21 capname="sys_admin"

unprivileged_userns_clone=1 and kernel.apparmor_restrict_unprivileged_userns=1 are both set, and AppArmor denies CAP_SYS_ADMIN inside the new namespace. newuidmap/newgidmap are not installed either, so the uidmap/rootlesskit/slirp4netns rootless-Docker path is closed as well (the .debs are downloadable — rootlesskit 1.1.1, slirp4netns 1.2.1, uidmap 4.13 — but they still need the setuid helper and the namespace to work).

So Docker on LAB is not a package problem. It is a permissions problem, and the permissions are Young's to change, not ours. Concretely, one of:

1. SUDO_PASSWORD set in ~/.hermes/profiles/sol/.env (or a NOPASSWD rule for apt-get install), or
2. Young runs apt-get install -y docker.io docker-compose-v2 himself, or
3. a deliberate AppArmor/userns change to permit rootless containers.

None of those are ours to decide, and none of them are needed for the memory decision — which is what this card actually unblocks.

Note for the prod node: Docker 29.7.2 + Compose v5.4.0 are already installed on the prod Oracle ARM node ([docker-systems-prod-node](docker-systems-prod-node)). The containerised commerce/tenancy path is unaffected by this decision — it is a different host. This note is about LAB only.

2. The bare-metal path — proven, not assumed

Young's note on this card: "Young prefers reusing infrastructure already in place over adding new components — weigh that." So the question was not "can we install something" but "does the memory stack work without a container runtime, using what is already here."

It does, and I proved it by running it, not by reading a README.

2.1 The host already has a rootless user-prefix pattern

LAB has been running services rootless for months, via .debs extracted into a user-owned prefix (no root, no dpkg):

/home/lab/.local/pkgs/          ← the extracted .deb payloads
  postgresql-16_16.15-0ubuntu0.24.04.1_amd64.deb
  redis-server_5:7.0.15-1ubuntu0.24.04.4_amd64.deb
  nginx_1.24.0-2ubuntu7.18_amd64.deb
  libpq5 / liblzf1 / libjemalloc2 ...
/home/lab/.local/nginx-prefix/  ← nginx + redis binaries
/home/lab/.local/pg/            ← a live PostgreSQL 16 data dir (saleor)
/home/lab/.local/nginx/         ← a live nginx conf + pid + logs
/home/lab/.local/bin/lab-redis-cli  ← a wrapper that sets LD_LIBRARY_PATH

This is the same shape the memory stack needs. The bare-metal path is not an exotic workaround — it is how this host already runs Postgres, Redis and nginx.

2.2 hindsight runs bare-metal on LAB — full proof

hindsight-api is the provider Echo recommended ([memory-extension-mem0-vs-hindsight-2026-10-20](memory-extension-mem0-vs-hindsight-2026-10-20)), and it is the one with a documented Docker-free path. The proof, on LAB, tonight:

Install (no root, no Docker):

uv venv --python 3.14 .venv
uv pip install --python .venv/bin/python "hindsight-api-slim[embedded-db]" "hindsight-api-slim[local-onnx]"
# → hindsight-api-slim 1.0.0, pg0-embedded 0.15.2, onnxruntime 1.31.0, transformers 5.15.1

Three install traps found and fixed — worth recording, because each one is a silent startup failure:

Run — embedded PostgreSQL, no server to install:

HINDSIGHT_API_HOST=192.168.4.60 HINDSIGHT_API_PORT=8888 \
HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx HINDSIGHT_API_RERANKER_PROVIDER=rrf \
HINDSIGHT_API_LLM_PROVIDER=openai \
HINDSIGHT_API_LLM_BASE_URL=https://opencode.ai/zen/go/v1 \
HINDSIGHT_API_LLM_MODEL=longcat-2.5-preview-free \
HINDSIGHT_API_LLM_API_KEY=$OPENCODE_GO_API_KEY \
  .venv/bin/hindsight-api

pg0-embedded 0.15.2 provisions PostgreSQL 18.1.0 itself, as the lab user, no root:

/home/lab/.pg0/instances/hindsight/instance.json
  {"port": 5432, "data_dir": "/home/lab/.pg0/instances/hindsight/data",
   "installation_dir": "/home/lab/.pg0/installation",
   "username": "hindsight", "password": "hindsight", "version": "18.1.0"}

Proof it is really running (not a config read):

$ curl -s http://192.168.4.60:8888/health
{"status":"healthy","database":"connected","db_acquire_ms":0.5,
 "db_pool_waiting":0,"db_pool_in_use":0,"db_pool_max":100,"db_pool_idle":5}

$ curl -s -X POST http://192.168.4.60:8888/v1/default/banks/probe2/memories \
    -H 'Content-Type: application/json' \
    -d '{"items":[{"content":"LAB bare-metal decision 2026-10-21: Docker cannot be
         installed (no root, userns denied by AppArmor). Memory runs via pip
         hindsight-api-slim + embedded pg0 + onnx embeddings, bound to 192.168.4.60:8888."}]}'
{"success":true,"bank_id":"probe2","items_count":1,"async":false,
 "usage":{"input_tokens":3030,"output_tokens":447,"total_tokens":3477}}

$ curl -s -X POST http://192.168.4.60:8888/v1/default/banks/probe2/memories/recall \
    -H 'Content-Type: application/json' \
    -d '{"query":"how does memory run on LAB without Docker?","budget":"low"}'
{"results":[{"id":"149cf398-...","text":"Docker cannot be installed on the LAB bare-metal server
  because there is no root access and userns is denied by AppArmor. | When: 2026-10-21 | ...",
  "entities":["userns","Docker","AppArmor","LAB"],
  "scores":{"final":1.0999999,"semantic":0.8793954,"keyword":0.6999999}}]}

A real write, a real LLM extraction, a real embedding, a real recall. Health, retain and recall were all exercised over the LAN address, and the recalled text is the fact we stored — not an echo of the query.

2.3 It binds to the LAN, which is what Pulse needs

HINDSIGHT_API_HOST=192.168.4.60 → LISTEN 192.168.4.60:8888, and http://192.168.4.60:8888/health answers from the LAN address (loopback then correctly refuses, because the bind is explicit). Pulse is on the same /24 (192.168.4.34), and Echo's local_external mode is exactly "point at an existing instance" — so the Pulse↔LAB topology Echo asked for is reachable without a container runtime.

2.4 What this costs

3. The other two providers, on the same evidence

• mem0 — unchanged and now unblocked in the other direction: its shared-store path wants Postgres, and there is no Docker-free server path found. It becomes viable only if Docker lands, or if we point it at the pg0 instance hindsight already runs. Echo's standing findings stand regardless (self-host tier is the gated one; extraction prompt ingests assistant-authored content by design).
• qdrant-memory (catalog, 0.2.2) — a third option that needs neither Docker nor Postgres: it has an embedded store under <HERMES_HOME>/qdrant-memory/qdrant. Its default embedder is local Ollama qwen3-embedding:4b, and ollama is not installed/running on LAB, so it would need a remote embedding endpoint (and our opencode-go key serves no embedding model — the model list has none). Not a blocker for Docker, but it is a real cost and it is not the answer.

One fact that constrains all of them: the opencode-go gateway (https://opencode.ai/zen/go/v1) serves 38 chat models and zero embedding models, and its key is rejected by api.openai.com. That is why the ONNX provider is the right embeddings choice here — it needs no API key and no new provider account.

4. The decision

LAB runs the memory stack bare-metal on hindsight.

• Docker is not installed on LAB and is not installable by an agent on this host — no root access, and AppArmor denies the user namespace that rootless containers would need. Installing it is one command Young runs, or one SUDO_PASSWORD, and it is recorded above so it is a decision, not a mystery. It is not on the critical path for memory.
• The bare-metal path is proven working, not merely plausible: hindsight-api-slim 1.0.0 + pg0-embedded 0.15.2 (PostgreSQL 18.1.0) + ONNX embeddings, health ✅, retain ✅, recall ✅, bound to 192.168.4.60:8888 for Pulse.
• This reuses infrastructure already in place, per Young's preference: the same rootless user-prefix pattern LAB already uses for Postgres, Redis and nginx — no container runtime, no daemon, no new service class.
• It keeps the prod container path intact. Docker on the prod Oracle node is unaffected; the commerce/tenancy design in [docker-implementation-plan](docker-implementation-plan) still stands on that host.

What this decision does NOT settle (deliberately out of scope for this card):

• the provider choice itself — that is t_af604d1b (hindsight vs mem0), and this note removes Docker from its blockers rather than making it;
• the read path — t_3841e128 (Min). Echo's §7 finding stands: the September memory bus accumulated 1,548 rows, zero live readers, and died quietly. The read path is the product. A provider without a named reader repeats that failure with better tooling;
• that anything is installed — this is a proven path plus a decision, not a deployment. Making it a service (systemd user unit, pinned deps, a real key path, backup of /home/lab/.pg0) is the next card, and it should be someone else's.

If Young later grants root, Docker can be installed in one command and the memory stack can be re-hosted as a compose project. The bare-metal path does not block that — it is reversible, and nothing here is welded to a container.

5. Not verified (read as limits, not findings)

• I did not install Docker. I could not: no root, and the sudo path is deliberately blocked for agents. The apt resolution and .deb downloads succeeded, so this is a permissions finding, not a "docker.io is broken" finding.
• I did not make hindsight a service. It ran as a foreground process for the proof and was stopped afterwards. No systemd unit, no pinned dependency file, no backup, no restart-on-boot for /home/lab/.pg0.
• I did not configure the Hermes hindsight plugin (hermes plugins install hindsight, hermes memory setup). The server works; the agent-side wiring is untested.
• The retain/recall proof used one bank (probe2) and one fact. It proves the stack runs and round-trips. It does not prove recall quality, multi-agent attribution, or the speaker-attested grounding that Echo's §6 analysis is about.
• Pulse→LAB was not probed from Pulse's side tonight — the Pulse tunnels (18301/18305/18313) were not up in this session. The LAN bind is measured from LAB; the far side is Echo's already-closed step 1.
• Cost was not measured. Every write calls the opencode-go gateway. The observed call was ~3,030 in / ~447 out tokens, but I did not price a real workload.
• HERMES_HOME was not pointed at the pg0 instance; the DB path in instance.json is pg0's default under ~. If a non-default path is wanted, that is a config decision, not a discovered one.