Stop Training Agents to Just 'Get the Job Done'

If you're building agents for anything beyond a sandbox, 'success' is a moving target that includes how the agent got there. A customer service bot that closes a ticket by violating privacy protocols or a system-admin agent that fixes a server by bypassing authentication are massive failures, even if the final state is technically "resolved." In production, the path is everything.

The RLVR Bottleneck in Production

The current hype around Reinforcement Learning from Verifiable Rewards (RLVR) hits a major wall when you move into environments with irreversible costs or strict operational constraints. RLVR is great at optimizing for a final goal, but it's often blind to the wreckage left behind. In a real-world stack, this approach wastes huge amounts of compute on "all-fail" groups—trajectories where the agent makes an irreversible mistake in step two but keeps spinning until step fifty.

If your agent is only rewarded for the final success, it has zero incentive to avoid "bad moves" that are cheaply detectable but don't technically prevent a (hypothetical) success. In production, you need to respect business hours, auth limits, and rate caps. If you only train for the outcome, your agent will violate these constraints nearly every time because they aren't penalized until the final check—if they are penalized at all.

The Recipe: Penalize the Path, Reward the Outcome

To fix this, we need to move toward a "penalize the path, reward the outcome" recipe. The goal is to treat outcome-neutral constraints (like staying within operating hours) as something to be penalized in real-time, while using verifiable progress potentials to provide a denser reward signal.

But here is the nuance: a dense signal is only useful if it provides reachable variance that the outcome reward lacks. If every successful path looks identical to the reward function, a dense signal won't help the agent learn the "right" way. By penalizing bad moves and rewarding progress, you give the agent a much clearer gradient to follow. This method has already shown success in proxies like software repair and system administration, where a successful resolution is necessary but fundamentally insufficient on its own.

Engineering for Production Safety

The real story here isn't just about making agents smarter; it's about making them "safe-by-default" in messy, high-stakes environments. As builders, the most interesting challenge isn't the RL math—it's the engineering of cheap detection.

If you can't cheaply detect a "bad move" in your specific stack, this recipe becomes a lot harder to implement. However, for any system where you can define clear "no-go" zones (like an API's rate limits or a database's write permissions), this is the blueprint. You want to build agents that don't just solve the problem, but solve it without getting your team fired.