arXiv 2610.22744
OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing
AI14 upvotes
Authors: Xiaoyan Cong, Tianyu Wang, Hyeonho Jeong, Zhifei Zhang, Yuqian Zhou, Yijun Li, Shaoteng Liu, Zhe Lin +2
Overview
The authors present OmniXRef, a framework that lets users edit a video with any mix of text, image, and video references. Their OmniXRef-Model uses a single-stream diffusion transformer that processes the source video together with MLLM context in one interleaved sequence, supported by a reference-bank input design and R^2-RoPE for handling more references. They also build OmniXRef-Data with synthesis pipelines and introduce OmniXRef-Bench for video-reference attribute transfer.
Introduction
Video editing intent can be expressed as text, image references, video references, or combinations of these, but existing methods handle only a few of these control types at a time. Video references are especially hard to use as a control signal because training data for them is scarce.
Results
OmniXRef-Model outperforms both task-specific baselines and commercial models in comprehensive evaluations. The benchmark measures transfer across Camera, Motion, Content, Style, and Visual Effects (VFX).
Conclusion
A single model that accepts variable combinations of text, image, and video references could make controllable video editing far more flexible for practitioners.