Evaluating Temporal Preservation in Conversational Image Editing

OCCUR-Bench introduces a diagnostic benchmark specifically for evaluating temporal preservation in conversational image editing. While many models claim to handle image manipulation well, they often fail at maintaining consistency when an object is hidden and then revealed, a common scenario in multi-turn interactions.

The Gap in Temporal Consistency

Most current image editing evaluations focus on single-turn success—the ability to change a color or add an object in one go. However, in a conversational setting, users expect the model to remember what was behind a box or what an object looked like before it was obscured. The part worth being skeptical of in current results is that many benchmarks don't test these occlusion-and-revelation scenarios effectively. OCCUR-Bench addresses this by providing diverse scenarios with historical visual references, forcing the model to prove it actually understands the underlying scene geometry rather than just generating a plausible next frame.

Explicit Preservation via ReSpec

To bridge this gap, the researchers proposed ReSpec, a training-free framework that pairs restoration-aware instructions with historical visual references. In practice, this means the system moves from making implicit preservation a lucky byproduct of the model's weights to making it an explicit part of the prompt logic. Instead of hoping the model remembers a hidden chair, ReSpec provides the historical visual context needed to reconstruct it accurately.

What This Means for Production

The experiments show that ReSpec improves restoration fidelity and temporal consistency on the OCCUR-Bench. From a practitioner's perspective, this is significant because it offers a way to improve model behavior without the massive overhead of retraining or fine-tuning. It suggests that the bottleneck for consistent conversational editing might not be a lack of raw capacity in the models, but rather a lack of structured guidance on how to utilize historical context. If we can make preservation explicit through smarter instruction pairing, we can reduce the 'hallucination' of new details when objects reappear.