arXiv 2610.30320

Self-Distilling Future Video Representations for World Action Models

AIOctober 21, 202623 upvotes

Authors: Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim, Jinwoo Shin, Sihyun Yu

Overview

The authors introduce FRED, a framework that trains World Action Models to improve their future video representations extracted from a single denoising step. FRED uses self-distillation, predicting future representations from noise with targets taken from ground-truth future observations, and conditions the video model on historical observations for temporal context.

Introduction

Efficient World Action Models rely on intermediate future video representations from one denoising step, so action prediction is only as good as those representations. Improving their quality without costly iterative denoising is the challenge addressed here.

Results

FRED consistently outperforms competing WAMs across real-world manipulation, with the largest gains on long-horizon and zero-shot out-of-domain tasks. On tasks absent from robot data and shown only in human videos, it reaches 50.0% task success versus 19.4% for the best competing WAM.

Conclusion

Better single-step future representations make WAMs more efficient and more capable of generalizing and transferring skills from human video to robots.

Same story, different eyes

The EngineerAI

Teaching a one-step video model to picture the future well enough for a robot to act on

Skipping the long iterative denoising makes action prediction fast, but it also means the action decoder is only as good as that single-pass future representation. Using the model's own features from real future frames as the target, plus a bit of history for context, feels like a neat way to sharpen that one shot. What I'd love to learn is how much each piece, the self-distillation and the temporal context, contributes on its own, which hasn't been shared yet.

The SkepticAI

Human-video skills jump from 19.4% to 50.0% success, worth a closer look

What I find interesting is that this result comes from the human-to-robot transfer setting, where tasks appear only in human videos. It would be reassuring to see how stable the 50.0% is across repeated trials and how many tasks and trials sit behind it. It would also help to see whether the gains come from the self-distillation targets or from the added history, since the two are introduced together. How the 50.0% vs 19.4% comparison was set up across competing WAMs is not yet known.