Production services lane redesign (2026-10-17)

1. Young's direction (2026-10-16)

"Prod redesigned as services lane: Docker containers (Medusa + POS + EVE + Hermes), Nuxt-based websites, per-tenant isolation per docker-systems-design."

Heart Fleet lives on all hosts (LAB, prod, Pulse). Prod is the public-facing services lane: customer hosting + serve + Heart Fleet workers. LAB = Brain (canonical vault, A2A gateway, opencode ARMS) — never serves publicly. Pulse = investment/cashflow.

2. Host roles (authoritative, host-topology-roles-2026-10-16)

Heart Fleet (portable workforce) runs on every host against that host's own opencode v2 ARMS. A2A endpoints per agent per host. opencode v2.0.27 verified on LAB + prod.

3. Production node — measured (docker-systems-prod-node, 2026-10-13)

Docker is already installed — removes the first deployment risk entirely.

4. Tenancy model — per-tenant Docker Compose project (docker-systems-design)

Per-tenant Compose project, on a per-tenant bridge network, with NO host port publication. nginx proxies to container IPs on the tenant network by container name. Port collisions become structurally impossible.

nginx (host, 80/443)
  │  proxy_pass http://medusa-<tenant>:9000   (container DNS on tenant network)
  ▼
docker network: tenant-<slug>  (bridge, isolated)
  ├── postgres:15-alpine      volume: pg-<slug>
  ├── redis:7-alpine          volume: redis-<slug>
  ├── medusa-server           WORKER_MODE=server, DISABLE_MEDUSA_ADMIN=false
  ├── medusa-worker           WORKER_MODE=worker, DISABLE_MEDUSA_ADMIN=true
  ├── storefront              Next.js (Nuxt/Next.js per tenant)
  └── eve-agent               Nitro (self-hosted)

nginx connects to tenant networks (docker network connect) and proxies by container name. Admin panels bind to the tailnet interface, not public. Only nginx (80/443) is public; every container publishes to 127.0.0.1 only; Postgres/Redis publish nothing.

Resource ceilings (prevent one tenant starving the node)

deploy:
  resources:
    limits:   { cpus: "1.0", memory: 2G }   # medusa-server
    # worker 0.5/1G · storefront 0.5/1G · pg 0.5/0.7G · redis 0.25/0.1G · eve 0.25/0.3G

Per-tenant ceiling ≈ 3.0 CPU / 5.1 GiB — ~4 tenants fit the measured budget with headroom (matches Tam's 4.8 GB/stack estimate). Oracle ARM free tier covers 2 OCPU + 12 GB always-on; our node runs 4/24 — excess is billable (A$40–60/mo if credits expire).

Four constraints

1. Builds are the real CPU risk — medusa build/Next.js builds spike on 4 shared vCPUs. Build once per template release; one shared image for all tenants; local registry container on the node; tenants pull. Never build per-tenant.
2. Disk is the long-term ceiling — 179 GB free; backups go offsite (node cannot be both host and backup target).
3. Existing services must not be disturbed — tenant vhosts added as new files in sites-enabled/, never editing the existing four (docs.youngle.au, pulse.youngle.au). TLS via existing letsencrypt pattern.
4. Tailscale is the private plane — tenant admin panels bind to tailnet by default; public storefront only through nginx.

5. Nuxt-based websites on prod

• docs.youngle.au — Docus (Nuxt 4 + Nuxt Content + Nuxt UI + Tailwind 4), auth-gated (401 by design). Runs the Vercel Docus template — our own vault pipeline on customer domain.
• pulse.youngle.au — Docus (same stack), live, public.
• yukonit.youngle.au — vhost → :8012 (existing site).
• Showcase/apex youngle.au — MISSING (serves nothing); P5 marketing/landing landing page is the gap to close.

All Nuxt sites are systemd services binding 127.0.0.1; nginx is the only public face. Tenant storefronts reuse the same pattern (Nuxt or Next.js per tenant template).

6. Per-tenant stack (one template, fleet-managed)

Base: fork medusajs/dtc-starter (MIT). One template repo, version-tagged; all tenants pull.

Per tenant:

• Postgres 15 (own container + volume) — backup cron → offsite.
• Redis 7 (own container).
• Medusa server (WORKER_MODE=server, admin on).
• Medusa worker (WORKER_MODE=worker, admin off).
• Storefront (Next.js or Nuxt, per template).
• EVE agent (self-hosted Nitro, eve build → eve start).
• Agilo POS (client tablets) — client-side, connects to Admin REST API over HTTPS.

Onboarding = one script: generate compose project + secrets (openssl rand), set domain in proxy, predeploy, create admin user, health-check /health. Nothing is "live" until /health AND /eve/v1/health return OK.

Multi-env: staging + production per client = two compose projects, same image tag, separate DBs/Redis. Cutover via cutover.sh (health-gated nginx upstream swap). Update pipeline: git tag → pull → predeploy (auto-migrate) → restart → health re-check. Rollback = prior tag (no schema rollback).

7. Secrets + isolation discipline

• .env.secrets.register per tenant, 0600, never in compose, never committed.
• Secrets generated via openssl rand -hex 32 (JWT, COOKIE, DB, REDIS, ADMIN TOKEN, EVE API KEY); publishable key read-scope, admin token write-scope HIL-gated.
• Per-tenant compose project namespacing (-p cib-<tenant> or dtc-<tenant>-<env>).
• Port register managed by onboarding script (.infra/ports.json), never hardcoded.
• ARM64 image policy — every image must be linux/arm64 (amd64 on ARM fails late looking like a config bug).

8. Heart Fleet on prod (agents-anywhere)

• Heart agents are portable: identity from registry, state = file tree, runtime = Hermes/OpenCode v2, presence via tunnels. Stand up = pull pack → enable unit → register port/tunnel → dashboard shows live.
• Prod already runs v3-heart-prod.service (active since 2026-10-12, 4 days, real work done: prod-e2e-proof, prod-heart-1 → done). Gap: no A2A listener on prod (83xx refuse) — a2a_server.py/serve.py exist in tree but are not installed as units. Sol (design) / Tam (ops) to close.
• Prod opencode v2.0.27 at /home/ubuntu/.opencode/bin/opencode (not on PATH; Heart unit's PATH includes it). Prod holds its own Console credentials → separate workspace billing from LAB (never summed).
• Prod capacity: 4 OCPU/24G/180G free — plenty for agent instances besides tenant stacks (~2 GB for 8 idle agents).

9. Pilot tenants (target: 2–3 on existing node)

• Young Le — youngle.au: Docs + Pulse (already delivered: docs.youngle.au live, pulse.youngle.au 200). Gap: apex landing page (showcase). Pilot 1 = our own content pipeline as a service (near-zero marginal cost, high showcase value).
• YukonIT — yukonit.com.au: KB + EVE + Medusa + StoreFront. YukonIT is an existing hosting business — we add to an operator who will judge us. Pilot 2 = full bundle (A$119 tier target). First tenant candidate.

Node capacity: ~4 full tenant stacks (4.8 GB each) or ~6 small tuned (3 GB each) on 23 Gi ARM. Pilot target 2–3 tenants alongside solos-docs/pulse. 5th tenant = new node (step-function in cost model).

10. Open items

1. OCI console check — trial credits vs PAYG, actual monthly burn (Mira).
2. youngle.au apex — DNS vs hosting (unverified HTTP 000).
3. YukonIT coexistence with their current platform (Sol integration question).
4. A2A listeners on prod — install a2a_server.py/serve.py units (Sol/Tam).
5. Heart agent placement — which Heart agent goes where first (Ace on prod first, as outage fix).
6. Wildcard vs per-domain TLS cert decision (Sol).
7. 7.1 GiB user.slice anomaly cleanup on prod (Min/Ace).
8. POS payments + offline mode (Sol feature spec; manual path first).
9. EVE beta hardening — pin @workflow/* per tenant (Sol spec; Ace builds).
10. MCP ruling — Young (OPEN-DECISIONS item 4).

11. Sources

• SolOS/hosts/host-topology-roles-2026-10-16.md (authoritative topology)
• docker-systems-design.md (Min/Tam, per-tenant isolation design)
• docker-systems-prod-node.md (Tam, measured prod node)
• COMMERCE-HOSTING-DESIGN-2026-10-13.md (Sol, tenant template + scripts)
• medusa-hosting-docker-study.md, medusa-nextjs-commerce-study.md, medusa-eve-integration-study.md, hermes-eve-cohosting-study.md (Tam, verified)
• business-plan-v2-aud-allowance.md, services-pricing-model-aud.md (Mira)
• 3in1-commerce-package-architecture.md, 3in1-feature-spec-and-prod-prep.md (Sol)
• vercel-platform-products-study.md (companion)
• SolOS/agents/agents-anywhere-design.md, SolOS/hosts/single-host-fleet-tradeoffs.md (Tam)

12. Register note

This doc is the architecture companion to 01-research/vercel-platform-products-study.md. Together they register the Vercel-template product landscape, MCP status, prod services-lane redesign, pilot mapping, and recommended tiers on the vault.