arXiv 2610.28698
Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds
AI46 upvotes
Authors: Kuan Zhang, Yukun Chen, Zhihao Yang, Yue Su, Xiangnan Wu, Zirong Chen, Run Luo, Jinkun Hou +3
Overview
The authors run a controlled study of continued pre-training (CPT) for embodied game agents, varying data volume, data diversity, and model size across 120 configurations. They evaluate with validation losses at three levels and with gameplay benchmarks, from worlds seen in training to unseen ones, and fit an empirical scaling law.
Introduction
Scaling data, diversity, and model capacity has helped embodied learning in physical settings, but it is unclear whether the same gains carry over to virtual worlds that differ in visual style, dynamics, controls, and objectives.
Results
Diversity improves transfer to related worlds, with limited gains in distant worlds, and the models are competitive with Minecraft specialists and frontier models in 3D worlds, performing best in real-time settings. Training shifts from a broad cross-world interface phase to a later world phase where gains narrow to in-corpus and related worlds.
Conclusion
Continued pre-training alone leaves persistent gaps in distant worlds, so building generalist embodied agents will likely require richer training stages beyond CPT.