Cron & Vault-Sync Inventory Audit — LAB vs Laptop (2026-09-27)
Cron & Vault-Sync Inventory Audit — LAB vs Laptop (2026-09-27)
Joint review with Eva for Young. Every claim below is verified by direct command output on the named host, not inferred. Three claims in the incoming inventory are wrong and one live silent failure is running right now.
🔴 Headline: the 30m vault sync is green and pointed at the wrong box
/home/eva/tam_vault_mirror/sync_vault.sh line 9:
RHOST="ubuntu@100.124.146.127" # OLD prod box
RBASE="$RHOST:/home/ubuntu/.solos/docs/vault/solos/research/tam"
Canonical LAB is lab@192.168.4.60:/home/lab/SolOS/docs/vault/solos/research/tam.
The job reports success every 30 minutes while writing to a legacy host:
2026-09-27 13:38:04 AEST OK pushed=1
Consequence — the canonical vault is behind. Full 30-file sha pull-back, local vs LAB:
| Result | Count |
|---|---|
| Match on LAB | 28 |
| Missing on LAB | 2 |
Missing from canonical LAB, present on old prod:
tech/2026-09-27-lab-connectivity-audit.md(local sha3a3ec36269a6…, mtime 12:34)tech/2026-09-27-lab-cutover-phase2-review.md(local sha73b3a433c9f0…, mtime 10:39)
So the newest two notes — the very ones that document the cutover — are absent from the source of truth. This is the exact failure mode Young called unacceptable: green status, no delivery. It has been running since the 09-27 cutover.
⚠️ Correction 1 — LAB's 4 timers are USER-level, not system
systemctl list-timers (system scope) shows 20 unit files, all stock Ubuntu
(sysstat, apt-daily, fstrim, logrotate, fwupd, e2scrub, man-db, motd-news…).
Zero SolOS timers in system scope.
All SolOS automation is in systemctl --user list-timers, enabled via
loginctl Linger=yes for lab (so it survives logout/reboot). Checking system
scope alone reports "nothing here" — which is likely how a wrong picture formed.
| User timer | Interval | Script | Result |
|---|---|---|---|
lab-eva-heartbeat.timer | 5 min (OnBootSec=2min) | runtime/bin/eva-heartbeat.sh | success, NRestarts=0 |
lab-eve-heartbeat.timer | 5 min (Persistent) | runtime/bin/eve-heartbeat.sh | success, NRestarts=0 |
lab-opencode-keypool.timer | 6 h (Persistent) | runtime/bin/opencode-keypool-refresh.sh | success, never re-run since boot |
launchpadlib-cache-clean.timer | 1 d | inline find ~/.launchpadlib/… -mtime +30 -delete | success |
lab has no crontab; /etc/cron.d holds only distro files (anacron,
e2scrub_all, sysstat, .placeholder). sudo -n is not available to this key, so
root/other-user crontabs were not enumerated — residual unknown, low risk.
⚠️ Correction 2 — there is no "snap cache clean" timer
The 4th user timer is launchpadlib-cache-clean — it prunes the Launchpadlib
API cache (find /home/lab/.launchpadlib/api.launchpad.net/cache -mtime +30 -delete), 1/day. Unrelated to snap.
A stock snap.firmware-updater.firmware-notifier.timer (hourly) does exist and is
the likely source of the mix-up. It is Snap Inc. stock, not fleet automation.
Net effect: LAB genuinely has 3 SolOS timers, not 4. Eva's substantive claim ("no content/publish crons") is ✅ confirmed — all three timers are heartbeat/keypool only.
⚠️ Correction 3 — the central cron registry does not exist
Fleet rule says scheduled jobs live on the central registry (:8436) and never as silent local-only schedules. There is no registry to live on:
http://192.168.4.60:8436/→ HTTP 000, no listenerhttp://192.168.4.60:8430/→ HTTP 000, no listener- LAB
ss -ltnfleet ports: 8300-8307, 8432, 8442, 8443, 8447. No 8436, no 8430. - Laptop holds
127.0.0.1:8430only — a loopback listener, no 8436.
So 0 of 16 jobs are centrally registered; every schedule is a local profile job. Confirms the standing note that governance ports are ABSENT on LAB. Whatever gets built needs the registry stood up first, or it will reproduce the same blind spot on LAB.
📋 Correction 4 — laptop cron counts (Eva's list undercounted)
Authoritative source: each profile's cron/jobs.json.
Eva — 14 defined, 11 enabled, 3 disabled (Eva reported 9 active):
| Job | Schedule | Enabled | Deliver |
|---|---|---|---|
| Market Watch (AU→HK→US) | 0 8,18 * * * | ✅ | telegram:8302448891 |
| Daily Market Briefing | 0 8 * * * | ✅ | telegram |
| Economic Watch — Rates & Housing | 0 10 * * 1 | ✅ | telegram |
| Evening Report — Scan + MA50 | 30 18 * * 1-5 | ✅ | telegram |
| ASX200 Scan | 30 17 * * 1-5 | ✅ | telegram |
| Content Watchdog (Pulse) | 0 10,16 * * 1-5 | ✅ | telegram |
| Portfolio Tracker DM | 0 8 * * 1-5 | ✅ | telegram |
| S32 Grid Bot (PAPER) | 35 17 * * 1-5 | ✅ | origin |
| SEMI Exit Alert ($40+) | every 15m | ✅ | origin |
| MINI Entry Alerts (Paper) | every 15m | ✅ | origin |
| MINI Desk Daily Status | 15 8 * * 1-5 | ✅ | origin |
| Watchlist — SEMI | 0 9 * * 1-5 | ⛔ disabled | telegram |
| MA50 Alerts (ASX) | 30 18 * * 1-5 | ⛔ disabled | telegram |
| BUB Price Alert ($0.105) | every 15m | ⛔ disabled | origin |
Two things Eva's list missed: the Economic Watch weekly job, and the whole
4-job paper/alert cluster (S32, SEMI exit, MINI entry, MINI status).
All four of those deliver to origin — in this CLI session that means no
message is ever delivered; output is written to cron/output/ and read by
nobody. Trading-alert jobs that alert nowhere.
Eve — 1 job: eve-daily-principle, 0 7 * * *, deliver=telegram.
Tam — 1 job: 305a8f2d68a3 tam-vault-sync, every 30m, deliver=local.
🔴 Second silent failure: eve-daily-principle has not run in 4 days
Its own incident record, still acked_at=None:
2026-09-27T07:00:57 RuntimeError: [drift_skip:silent] Skipped to prevent
unintended spend: global inference config drifted since this job was created
(provider 'openrouter' -> 'opencode-go'; model
'nvidia/nemotron-3.5-lightning:free' -> 'glm-5.3-flash'), and this job is
unpinned. No inference call was made.
Still skipping today. First seen 2026-09-23 → four missed daily runs. Fix is
one command: hermes cron edit 73fd3fff38a6 --provider opencode-go --model glm-5.3-flash. The alert fired once and then nothing — classic silent stop.
🧯 Unclosed incident debt
5 incidents across profiles, all acked_at=None, all unclosed:
- eva
bad0e4099b06+77f6d5273b4a—alerted, HTTP 400 provider"name" is not supported by this endpoint, Sep 17-19, still firing - tam
305a8f2d68a3×3 —detected/alerted, script exit 1, all Sep 21 (the 30m sync failure burst: 140 FAIL lines that day) - eve
73fd3fff38a6×2 —detected/alerted, drift_skip, Sep 23-27
Sync FAIL density by day, from vault_sync.log: 18 (09-10), 9 (09-15), 22
(09-16), 20 (09-20), 140 (09-21), 25 (09-22), then clean. Stable ~46-52 OK
runs/day since 09-23 (≈every 30 min, as expected).
🗺️ LAB vault state (canonical, for whoever builds)
- Path:
/home/lab/SolOS/docs/vault/solos/research/tam/ - Git repo, HEAD
267a6f1"docs: Pulse-laptop <-> LAB transport map (Sol 2026-09-27)" - Remotes:
origin=github.com/youngframeworks/solos-vault.git;prod=ubuntu@161.33.71.67:/home/ubuntu/.solos/docs/vault— 161.33.71.67 is not the host the sync script targets (100.124.146.127). Three hostnames for one fleet across one config. Worth pinning while the builder is in there. - Untracked in git, needs a commit:
solos/research/tam/2026-09-27-lab-access-lane-verification.md,solos/research/tam/tech/2026-09-27-lab-route-confirm.md, plus 3.obsidian/plugin dirs that should be gitignored. - Obsidian Local REST API (27124) is loopback-only on LAB → agent access needs
ssh -N -L 27924:127.0.0.1:27124+ HTTPS. Not exercised this pass.
✅ What the builder must fix (ordered)
- Re-point
sync_vault.shRBASE tolab@192.168.4.60+ canonical path; backfill the 2 missing notes; re-verify all 30 by sha pull-back. Do this first — it is a live gap in the source of truth. - Pin eve's
73fd3fff38a6provider/model to stop the drift-skip. - Re-deliver the 4 Eva jobs using
deliver=originonto a gateway-connected platform, or accept they are write-to-nowhere. - Stand up the :8436 cron registry (or declare the laptop profile the registry) and migrate all 16 jobs onto it with delivery target + failure signal + read-back, per the cron-discipline rule.
- Ack/close the 5 stale incidents so the queue means something.
- Add a 2-minute staleness watchdog on the canonical vault: compare newest
mtimeinsolos/research/tam/against local SRC and alert on drift. This audit found the drift by hand; nothing automated would have.
🎯 Tam's Read
The sync cron is the urgent item — it has been reporting OK pushed=1 every
30 minutes while writing to a dead prod host, leaving the two newest notes
missing from the canonical LAB vault, and LAB's registry port :8436 that all 16
jobs are supposed to live on does not exist, so every schedule on this fleet is
currently ungoverned and local-only.

