Anthropic Agents study (7 sources → adoption plan for SolOS)

Requested by: Young (2026-09-15): study with Echo, register to vault, Min publishes to docs.youngle.au. Tracker task-3c4000d6 (done). All sources fetched in full (curl+strip; web_extract 403-truncated). One source turned out to be a metadata error — flagged below.

Summary (3 bullets)

• Token budget is the dominant lever: token usage explains ~80% of outcome variance in research agents; single agents burn ~4× chat tokens, multi-agent ~15×. The 90.2% gain from Opus-4-lead + Sonnet-subagents over a single agent is real for broad research — and unaffordable for routine tasks without per-tier budgets.
• The most adoptable pattern for us is file-system-as-tool (Source 6): serving tool definitions via code files cut tool-discovery tokens ~150K → ~2K (98.7% reduction), compounding across a 12-agent fleet, with PII tokenization at the client giving deterministic security boundaries.
• Over-scaffolding is the trap: context-resets and sprint decomposition that were essential for Sonnet-4.5-class models became dead weight on Opus-4.x (one harness: $9 solo vs $200 without fixes). Audit scaffolding per model upgrade, one component at a time.

Key findings per source

1. Multi-agent research system (the flagship architecture piece)

• Orchestrator-worker pattern: Opus-4 lead plans and spawns, Sonnet-4 subagents execute parallel tool work. 90.2% improvement over single-agent Claude Opus 4 in breadth research; 150+ employees backed the architecture internally.
• Effort-scaling rules worth copying verbatim: 1 agent / 3–10 tool calls for simple fact-finding; 10+ subagents for complex multi-angle research. Effort-classification prompt sorts queries — don't always multi-agent.
• Parallel tool calling cut research time up to 90%; artifact passing (files, not conversation relay) prevents "game-of-telephone" corruption; checkpoint-resume for long research; "rainbow deployments" for riskless harness changes.
• Token economy: agents ~4× chat tokens; multi-agent ~15×. A single峡 ChatGPT-style interaction = 15× the tokens of a plain chat — CostGov implications are direct.
• Self-improvement nugget: a small tool-testing agent that rewrites bad tool descriptions cut completion time ~40% — cheap to run, fleet-multiplying.

2. Managed agents (the platform pattern)

• Virtualized session / harness / sandbox interfaces; session = durable append-only context log (getEvents/emitEvent), harness is stateless.
• Session spawning is cattle-not-pets: containers boot from image, die on completion; vault + proxy keep credentials out of agent containers.
• Reported p50 TTFT −60% / p95 −90% from lazy sandbox provisioning.

3. Harness design for long-running apps (GAN-style generator/evaluator)

• Two-agent loop with hard thresholds; Playwright MCP as the tool layer.
• Cost reality-check: solo Opus-4 run $9, V1 harness $200, V2 (tuned) $124.70 — the harness added a 22× cost bump that wasn't justified by task complexity, because something that helped (sprint contracts, context resets) became dead weight as the model improved. Scaffolding is a perishable asset, audit it every model hop.

4. Effective harnesses for long-running agents (the initializer/coder split)

• Initializer agent writes scaffolding (tests, TODOs, feature-ledger JSON); coding agent spins up fresh each session using the ledger as state — read-only except its own status field.
• Session-start verification ritual: prove the running app works before new work; git commit + progress note as mandatory exit criteria (not "I think it's done").
• Feature ledger in JSON, checked atomically, acts as a shared memory that survives context loss.

5. Tool-use context engineering (cookbook)

• Map your bottleneck to the primitive: clear tool-results first (cheapest), compaction second (extends context), memory third — all first-party API supported, no custom orchestration required.
• Never rely on compaction to transmit clean state — that's a harness design smell.

6. Code execution with MCP

• Present tools as a file tree of typed code APIs rather than inlined tool schemas → model reads definitions progressively; 98.7% token cut.
• Agents write intermediate results to workspace files; proven code gets saved as skills (SKILL.md + scripts) — a capability library that compounds.
• PII tokenization at the MCP client — sensitive flow tool→tool without entering the model context.
• Trade-off: requires real sandboxing/resource limits or it's worse than direct tool calls.

7. arXiv 2501.09223v2 — verification verdict

• Not a multi-agent paper. It's "Foundations of Large Language Models" (Xiao & Zhu, NiuTrans) — an LLM textbook. The task metadata ("Mobile Agent E / LLMV multi-agent") was wrong; do not build fleet decisions on it. Kept in the note as a verification-diligence precedent.

Cross-source synthesis for SolOS (adopt vs not)

ADOPT (in priority order):

1. Per-tier token budgets in the router (mapped to CostGov): simple fact-find = 1 agent/3-10 calls; complex = multi-agent with caps.
2. Filesystem/artifact handoffs, not conversational relay — integrates with our central registry naturally.
3. Durable append-only session log per agent; stateless harness; resume-from-last-event.
4. External skeptical evaluators with hard thresholds — catches our own judge bias and stub/claim drift.
5. Immutable task ledgers + session-start verification ritual from Source 4 (for any long-running cron/brief jobs).
6. Context primitives by bottleneck: clear-tool-results → compact → memory; first-party only.
7. MCP-as-code-filesystem + PII tokenization compounds across the fleet.
8. LLM-judge evals + retest harness assumptions each model upgrade (Sources 1, 2, 3 all said it; Opus-4.5-4.6 breaks scaffolding assumptions).

DO NOT:

• Multi-agent-fan-out low-value or sequential work — 15× token burn kills ROI; route single-agent through CostGov.
• Over-scaffold against current frontier models (audit per model upgrade).
• Trust agents' self-reported completion (Sources 3, 4) — external verification mandatory.
• Run agent-generated code without sandbox/limits.

Echo's contribution

Pending via shared workspace file (lane intermittent; breaker state open as of 2026-09-15 22:35 AEST). Her security-angle commentary (sandboxing, least-privilege, eval attack surface) merges into this note with credit if/when it lands.

Mira's Review (2026-09-15) — cost-lens co-authored section

Reviewer: Mira (CostGov owner), full study read. Verbatim key points integrated with credit.

📊 1. OpEx analysis — the numbers for CostGov

• Token usage = ~80% of outcome variance — routing budget is well spent only if shape matches task. Mira's rule: fan-out must be justified, not default; 15× burn for single-source lookup is pure waste.
• The $9 → $200 → $124.70 harness example = scaffolding is a perishable OpEx asset; every model upgrade needs a one-line audit in the budget cycle — dead scaffolding is silent 20×+ inflation. Added: "scaffolding audit" as a standing line item per model-hop.
• File-system-as-tools (Source 6, 150K→2K tokens, −98.7%): highest-ROI single adoption — we're already halfway there (registry skill corpus, SKILL.md library).
• Effort-tiered budgets map directly onto CostGov tier checks as budget caps per effort class — no new infra needed.

🛡 2. Risk & compliance additions

• Self-reported completion is not evidence (Sources 3–4) → enforce external verify as exit criteria on long-running jobs (aligns with our "show logs" discipline).
• PII tokenization at the MCP client pairs with per-citizen token rotation already queued in the hardening lane (Mira's Tam-infra review).
• arXiv mislabel verification catch = the right verification precedent: acting on a wrong source isn't free; metadata diligence is.

⏱ 3. Adoption verdict & timeline (Mira)

Mira's verdict: adopt items 1–3 now (near-zero effort, direct CostGov wins); item 4 gated on the sandbox/token-rotation hardening lane. Fan-out discipline is the biggest single cost lever in this note. ✅ Recommended for CostGov adoption directive, Tam credited as researcher.

Sources

All 7 sources fetched in full; URLs in frontmatter. Independent verification of the arXiv topic flagged inline.