LAB connectivity + control-plane audit — 2026-09-27 (Tam)

All findings reproduced with live probes. Corrections to my earlier migration plan incorporated (Sol/Young review).

1. No tailnet on LAB — confirmed

• tailscale binary absent (as user lab); tailscale: command not found.
• LAB interfaces: lo, wlp0s20f3 192.168.4.60/22, IPv6 only on the rest. Zero 100.x addrs.
• LAB is reachable from laptop only on LAN 192.168.4.60 (wireless — flaps; brd 192.168.7.255 /22).
• Laptop DOES have tailscale0 (100.85.164.30) but LAB doesn't → no tailnet leg to LAB at all.
• Away-from-home leg to LAB is currently nonexistent. The mesh ports LAB binds are LAN-only: ss -ltn on LAB shows 8300-8307 + 8432 bound to 192.168.4.60 (not 0.0.0.0), plus 22, 4090 (opencode), 6333/6334 (qdrant) on 0.0.0.0, and reverse-tunnel listeners 8442/8443 on 127.0.0.1.

2. SSH user is lab, not sol — username bug, no sudo needed

• ~/.ssh/config Host lab → HostName 192.168.4.60, User lab, IdentityFile ~/.ssh/lab_ctrl_key.
• Verified: ssh -o BatchMode=yes lab → OK user=lab host=LAB ip=192.168.4.60, up 17h56m.
• id as lab: uid 1001, in groups adm, sudo, dip, plugdev (sudo group membership exists but passwordless sudo is NOT configured — sudo -n true → "a password is required").
• No sol user required. My key was already installed.

3. The laptop tunnel picture (corrected twice — final truth)

Two distinct tunnels, both pointing at the OLD cloud box ubuntu@100.124.146.127 (tailnet IP; still up as a wireguard/tunnel interface on the laptop, so the old box is NOT dead):

• autossh A (pid 1008068, key ~/.ssh/id_ed25519): -L 18300-18307 → old-box 8300-8307, + 19902/18442, plus -L 8300-8307 → old-box 8300-8307 (LOCAL-HIJACK: old-box agent ports squat on laptop loopback — dangerous).
• autossh B (pid 1008069, key laptop_ctrl_reverse): -R 9222→laptop-ssh, -R 8443→laptop-8441, -R 9903→9901, -R 5902, -R 8765, -R 9904→192.168.4.28:8446 — plus a SEPARATE manually-spawned ssh (pid 1225072, PPID = user systemd, NOT a declared unit) carrying the -L 1830x and -L 8432/8430/8320/8091/8433/8434 forwards.
• Liveness: the old-box stack is alive through the tunnel: laptop :8432/tasks (via the manual ssh) returns JSON; :18300-18307 respond HTTP 405 (A2A servers, alive — earlier "000" was wrong); :8430 CostGov 200; :8433/reports 200.
• Unit drift (systemd): laptop-tunnel.service = system unit not found; user units exist and run: laptop-reverse-tunnel.service (autossh B), eva-a2a-tunnel.service, and a laptop-tunnel.service (running, description "Laptop Consolidated Tunnel — single SSH"). Service names in fleet notes are ambiguous — pin systemctl --user cat laptop-tunnel.service.
• socat: a socat (pid 1399) is listening on laptop :9901; laptop :8441/:8447 are real python A2A servers (eva-a2a/server.py pid 1401, tam-a2a/server.py pid 87802) answering JSON-RPC (empty POST → proper -32601) — both alive.

4. Registry split-brain — quantified (fresh evidence)

Pulled task ID sets from both boards:

• Tunnel-path board (old box via laptop :8432): 100 tasks; LAB board (192.168.4.60:8432): 100 tasks; 89 shared, 11 only-old, 11 only-LAB.
• Only-old includes my task-83bf0aec, the task-2a00d648 migration-directive copy, task-0f36620c.
• Only-LAB includes task-12a7bca6, task-5edc1a4d (migration directive copy), task-fc043640 (split-brain reconciliation plan — Sol's verdict is posted there), task-2ad56118 (probe for Ace).
• This is the split-brain Sol's task-fc043640 describes: prod registry receives tunnel-path writes, LAB receives LAN/tailnet writes. That task says EXECUTE IN ORDER: freeze prod writes → snapshot prod solos.db → reconcile tasks+reports into LAB → verify → repoint laptop-tunnel writers to LAB → port governance (cron 8436 first, CostGov 8430, discovery 8433, Conductor 8320) → prod warm 48h → web tier last.
• LAB :8436 cron registry and :8430 CostGov: 000 — not present on LAB. Governance still only lives on old box via tunnel.

5. Vault state on LAB

• lab already has a vault tree at /home/lab/.solos/docs/vault/solos/research/tam/ with 49 .md files (tech/... etc.).
• Tam's lab_ctrl_key is NOT authorized there (ssh -i lab_ctrl_key lab@... → Permission denied; the working lane uses a different key/agent identity). Syncing to LAB needs either key installation by whoever holds the lab auth (Young's laptop id_ed25519 works) or the scp-to-old-box lane until reconciliation.
• sync_vault.sh (on /home/eva/tam_vault_mirror/) SRC=/home/eva/.solos/docs/Research/tam/, RHOST=ubuntu@100.124.146.127, RBASE=/home/ubuntu/.solos/docs/vault/solos/research/tam — still targets OLD BOX (correct choice until registry-of-record is declared; writes land wherever the old box's path points).
• Write-lane test to old box: ssh -i tam_vault_key ubuntu@100.124.146.127 'echo X' returns forced-command banner "tam vault access is write-only (scp/mkdir into vault)" — lane intact (scp writes only, exec refused).

6. Implications for the Hermes→LAB migration plan

• Sol's "sol-user blocker" dissolved (username bug), but the real blocker moved: LAB has no tailnet + governance services (cron 8436, CostGov 8430, Conductor 8320) are not on LAB. Migration sequencing must follow task-fc043640: reconcile registries first, port governance to LAB second, then move agents.
• Away-from-home access path today: laptop's own tailscale → old box → laptop reverse tunnel (9222). After LAB becomes home, needs either (a) tailscale install on LAB (needs Young sudo once, or tailscaled in rootless mode), (b) a persistent laptop→LAB forward via the laptop's existing tailnet membership, or (c) LAB behind the old box nginx domains. Until then: LAB = LAN-only; moving agents there today strands them off-home.
• 8300-8307 loopback-hijack (autossh A forwards laptop loopback 8300-8307 to old box) will collide with any future local/LAB service on those ports — flag for cleanup at repoint.
• 48h warm-rollback requirement + laptop is the only tunnel holder: while agents run on LAB off-home there is no path in. Keep agents on laptop until LAB leg exists, or bring tailscale to LAB as step 0.

7. What I changed (nothing destructive)

• No service edits, no key moves, no registry writes. Added this audit note to the vault (queued for sync). Board reply posted to task-fc043640 (LAB) + task-83bf0aec (old board) so both boards carry the audit.