Moving Beyond Outcome-Only Rewards: Why Path Penalties Matter for Production Agents

Standard Reinforcement Learning from Verifiable Rewards (RLVR) has a massive blind spot: it only cares about the finish line. In a sandbox, that’s fine. In production? It’s a liability. If your agent completes a data migration but ignores auth protocols or works outside of business hours, that "success" is worthless. Deployability depends on the path taken, not just the final state. When you're building for the real world, a successful resolution is necessary but insufficient; you need to ensure the agent respects outcome-neutral constraints every step of the way. In fact, outcome-only training can violate these constraints on nearly every episode. The fix? A "penalize the path, reward the outcome" approach that hits high success rates with near-zero violations.

Why Outcome-Only Training Fails in Production

Real-world agentic environments are high-stakes and often irreversible. Standard RLVR is blind to path constraints, which means it wastes expensive rollouts on trajectories that reach the goal by breaking your rules. To make an agent actually deployable, you have to account for the "how" as much as the "what." If an agent takes a shortcut through a restricted directory to finish a job, the reward is moot because the deployment is a failure. We need a dense signal that guides the agent toward compliant behavior without forcing us to manually label every single intermediate step as "good" or "bad.

Verifiable Penalties vs. The Progress Potential Trap

The solution is to penalize the path while rewarding the outcome. This provides a reliable, dense signal for outcome-neutral constraints. Here’s the engineering win: verifiable penalties are way more reliable than progress potentials. Progress potentials are a nightmare to engineer because they require you to define what "partial progress" looks like—a moving target in complex environments. Verifiable penalties, however, are universally available and detectable. By identifying specific, prohibited actions—like hitting a restricted API or acting outside of permitted hours—and applying a penalty, you give the agent a constant feedback loop. It learns the boundaries efficiently from few examples, staying within the lines while optimizing for the final objective.

From Reward Engineering to Constraint Modeling

The real engineering shift here isn't just identifying the goal; it's the exhaustive mapping of the "illegal" state space. When you move from outcome-only rewards to path penalties, the burden of work shifts from reward engineering to constraint modeling. You have to define every possible way the agent could "fail" without actually failing the task. If your penalty set is too thin, the agent will find a loophole; if it's too dense or poorly designed, you risk hitting the "inaction trap," where the agent becomes too paralyzed by potential penalties to attempt the task at all. For those of us building in production, this means we need to stop asking "how do I tell the agent what to do?" and start asking "how do I strictly define what it cannot do?" It turns RL from a search for the "best" path into a rigorous exercise in boundary definition.