The Reflection Loop

What

The act → observe → reflect → retry loop — the direct ancestor of our execution-feedback adoption. Added to the reading list on Echo's review (2026-08-07).

Papers

• Reflexion (Shinn et al. 2023, arXiv 2303.11366) — after a failed attempt, the agent writes a natural-language reflection (what went wrong, how to fix) and retries with it in context. Verbal reinforcement: no weight updates, just persistent self-feedback. Big gains on coding (HumanEval) + decision tasks.
• Quiet-STaR (Zelikman et al. 2024, 2403.09629) — extends STaR beyond math: the model learns to think (generate rationales) on any text and use the rationales to improve next-token prediction; the "STaR for everything" step.
• Self-Rewarding Models (Yuan et al., 2401.01335) — models that both generate AND evaluate their own outputs, then fine-tune on the self-rewards — a fully self-improving loop (with the caveats below).

Key ideas

• Reflection is cheap (no retraining) and works by giving the model its own failure narrative.
• Quiet-STaR makes reasoning an implicit habit rather than a prompted technique.
• Self-evaluation loops are powerful but need external grounding to stay honest.

⚠️ The caveat (from Echo's review)

Models that self-critique can talk themselves into wrong answers with equal confidence. Self-verification without external grounding has known failure modes — a reflection loop needs a ground truth (tests, execution results, checked data). Reflexion works because execution feedback is real; pure self-critique (Constitutional-style) can rationalize errors.

SolOS mapping

• Our execution-feedback → skill loop is Reflexion at the fleet level: failures produce real feedback (cron errors, test failures, API checks), and the "reflection" is the lesson written into the skill. Keep the ground-truth rule: reflections only from execution outcomes, never vibes.
• Note 03's Constitutional AI recommendation is now qualified: use self-critique only as a cheap pre-check, never as the final verifier — pair it with a real verifier pass (note 02).