Robust Verification (PRMs)

What

Train a separate model to verify answers — at the process level (each reasoning step) rather than just the final answer (outcome level).

Papers

• Training Verifiers to Solve Math Word Problems (Cobbe et al. 2021) — a small verifier model that scores candidate answers, combined with sampling, dramatically improves GSM8K accuracy vs a single generation.
• Let's Verify Step by Step (Lightman et al. 2023) — Process Reward Models (PRM) grade each step of a solution; step-level verification generalizes far better than outcome-level on hard math (MATH).
• Math-Shepherd (Wang et al. 2023) — automatic construction of step-level training labels (no human annotation): verify each partial solution with a trained model → PRM trained at low cost, strong results.

Key ideas

• Weak/strong verifier: even a weaker verifier than the generator improves selection dramatically.
• Step-level signals are learnable without human labels (automatic labeling).
• Verification is a separate skill from generation — and often cheaper to scale.

SolOS mapping

• Formal verifier pass (adoption #1): before any fleet deliverable ships (brief, spec, alert text, code), run a cheap-model pass against explicit acceptance criteria — "did it do what was asked, is the data right, are there errors?" We already do this manually on Pulse endpoints; standardize it fleet-wide.
• Pattern: generator = main model; verifier = cheap/free model (Zen free tier) with a rubric.
• Applies to: briefs (numbers vs source), agent code drops (build + smoke test = our verifier), alert scripts (test output shape).