arXiv 2610.23542
Execution Before Memory: Diagnosing Memory Failures in Video World Models
AI19 upvotes
Authors: Weiyang Jin, Jiwen Yu, Sihan Xu, Yuwei Niu, Xihui Liu
Overview
The authors introduce MIST, a memory test that separates whether a scene was set, whether the requested change was carried out, and whether the final frame shows the result. They then propose M3-Agent, which recursively revises the details of the instruction so the frozen model carries out the requested change.
Introduction
Current memory tests for video world models fold several steps into one score, so a low result is easily misread as forgetting even when the required state change never happened.
Results
Relative to the baseline, M3-Agent more than triples the MIST reveal score, showing that the main bottleneck is failing to carry out the instructed change. It also raises WRBench's T score by 26.1% and MBench-T's U score by 52.7%.
Conclusion
Memory evaluations should first check whether the change was executed, because current baselines understate what the frozen model can do.