Agon: Two Models Grading Each Other Replace Process Labels

Agon is a reinforcement learning framework where two models grade each other's work — no process labels, no reward model, just a drafter and a grader trading roles until reasoning improves. That trade alone is worth paying attention to, because the expensive part of training reasoning models has always been labeling the reasoning itself.

How the Two-Model Loop Works

The setup is simple enough to reason about before you trust it. Two competing models alternate between drafting solutions and grading each other's attempts. The grading signal happens implicitly during training — nobody writes out step-by-step process labels, and there's no separately trained reward model scoring outputs. The models' disagreement does the work.

At inference time, the framework deploys as a two-stage cascade: one model drafts, the other answers after reading that draft. That's a deployable shape, not just a training trick, which matters more than it sounds — most RL results die the moment you try to ship them.

What the Numbers Actually Show

On mathematical and coding benchmarks, Agon doubles GRPO's pass@1, and delivers roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base model. The gains replicate across Qwen3.5, Gemma 4, and competitive programming code — replication across model families is the part I'd trust most here, since single-model RL wins routinely fail to transfer.

The stated next step is letting models reason together in latent space. That's ambitious, but it's also where I'd temper expectations: latent-space collaboration between two models is a much harder systems problem than alternating roles on text.

Where This Reinforcement Learning Framework Gets Annoying in Practice

The interesting question isn't whether the benchmark numbers hold — it's what happens when you're the one integrating this. A two-stage cascade means two models to host, two inference paths to monitor, and latency that scales with both stages instead of one. If your grader drifts or gets weaker than your drafter over time, you've built a feedback loop with no obvious failure alarm.

There's also an open question about domain coverage. The evidence here is math and competitive programming — domains where correctness is checkable even implicitly. Whether implicit grading holds up on messier tasks where 'good reasoning' is contested is exactly what the source doesn't tell us.

My read: this is the missing piece for teams who want reasoning gains without hiring annotators to write process labels — if cascade latency and grader stability don't get in the way. The part worth trying yourself is the inference-time cascade first; you can test whether a second model reading a draft actually improves your outputs before committing to any training run at all.