arXiv 2610.26828

Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

AIOctober 19, 202664 upvotes

Authors: Zhaoyang Wang, Qianhui Wu, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao +4

Overview

The authors present WildAWM, a scalable pipeline that synthesizes controllable, executable computer-use environments and tasks spanning GUI, MCP service, and command-line interfaces. The pipeline is powered entirely by open-source models and adds execution-based quality gates for solvability, difficulty, and verifier reliability.

Introduction

Training agents through interaction in real computer systems is hard to scale and risks security problems. Existing approaches usually cover only a single interaction interface.

Results

The pipeline yields 341 long-horizon environment-task pairs, with GPT-6-Astra solving only 60% of them. Two Qwen models trained only in these synthetic worlds show substantial gains across 8 out-of-distribution benchmarks covering web, claw, and computer-use agents.

Conclusion

This shows that open-source-generated synthetic computer-use worlds can train agents that generalize to unseen real computer-use tasks, reducing reliance on proprietary models and risky live systems.

Same story, different eyes

The EngineerAI
One synthetic world where agents click, call tools, and run code
What stands out to me as an engineer is that a single environment supports GUI, MCP services, and command line together, so an agent can mix seeing, calling tools, and running code the way a person would. Making all three interfaces behave consistently inside one runnable world sounds like the hardest part to build. I also like the quality gates: running different agents, cross-checking judges, and testing whether verdicts hold up are practical ways to keep tasks solvable yet hard. How well this holds up as the environments scale far beyond 341 pairs is not yet known.
The SkepticAI
Three quality gates could tell us if synthetic worlds are truly trainable
What I like here is that solvability, difficulty, and verifier reliability are each checked by actually running agents, using mixed rollouts, cross-judge agreement, and counterfactual probes. If the verifiers hold up when a solution is subtly altered, the gains on the 8 outside benchmarks would be easier to trust. It would also help to see how much each gate contributes on its own, and how the two Qwen models compare with untrained baselines. Exact scores for the benchmark gains are not yet shared.
The FounderAI
Open-source models can now build the practice worlds agents learn in
From a founder's lens, the interesting gap is that training environments are becoming something you can generate, not something you have to license or build by hand. A good entry point right now is a narrow vertical: pick one workflow, like invoicing or clinic scheduling, and synthesize GUI, tool, and command-line environments with verifiers for just that, since the code is public and runs on open models. The window is timely because the same open-model approach lowers the cost for everyone, so the lasting edge likely sits in domain-specific tasks and trustworthy verifiers rather than in the generator itself. How well this transfers to messy, company-specific software is not yet known.