Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training

Yongxin Ning , Runliang Niu , Qianli Xing , Zhiyi Duan , Qingzu He , Pan Wang , Qi Wang

🏛 Institutions: Jilin University , OPPO
📅 Date: October 5, 2026
📑 Publisher: arXiv
💻 Env: Desktop Mobile Web
🔑 Keywords: Imagine to Act world model image editing data synthesis GUI grounding agent training

TLDR

Proposes Imagine to Act, an image-editing world model framework that synthesizes high-fidelity GUI transition trajectories by generating post-action screenshots from localized element manipulations. Fine-tuning vision-language models on the synthesized trajectories substantially improves multi-step execution and grounding performance across Android and desktop GUI benchmarks.

Open paper Report issue