Reasoning + Acting + Execution Feedback

What

Let the model interleave thinking with tool use, and train it on execution feedback (what actually happened when it ran code/tools), not just next-token prediction.

Papers

• ReAct (Yao et al. 2022) — interleave reasoning traces ("thought") with actions ("act") + observations from tools (search, lookups); dramatically improves grounded answers, reduces hallucination, and is the ancestor of today's agent loops.
• RLEF: Grounding Code LLMs in Execution Feedback with RL — after generating code, run it; use the pass/fail + error signals as a reward to fine-tune. Execution feedback is dense and honest — better than preference pairs.
• Constitutional AI — self-critique against a written constitution/principles; the model revises its own outputs to comply; used to align without human labels.

Key ideas

• Feedback from the environment is the best teacher — our crons/APIs are the environment.
• ReAct-style loops are what the fleet already does (tool calls between thoughts).
• Self-critique (constitutional) is cheap alignment: a fixed rubric + self-revision pass.
• ⚠️ Caveat (Echo review, 2026-08-07): self-critique can talk a model into wrong answers with equal confidence. Use it only as a cheap pre-check — the final verifier must be external (execution/test/checked data). See note 08 (Reflexion).

SolOS mapping

• Execution-feedback loop (adoption #2): every cron/script/API failure already produces an error — make the lesson → skill patch step mandatory and same-day (we do it ad-hoc today; enforce it).
• ReAct = our agent loop (terminal/browser/tools between reasoning). Formalize "thought → tool → observe" discipline in the fleet playbook.
• Constitutional-style: the fleet's ground rules (no cascades, one responder, verify before claiming) are our constitution — add a self-check line to delivery templates: "did I verify before claiming done?"