Scaling GUI Agents with Visual State Transitions

State Transition Pretraining jointly learns inverse dynamics and forward GUI dynamics from visual state transitions before instruction-conditioned trajectory fine-tuning. The paper evaluates the resulting models on desktop and mobile agent benchmarks and releases its official implementation.

• Right Screen, Wrong Transition: World Models as Verifiers for GUI Agents
  October 8, 2026 · arXiv
• Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training
  October 5, 2026 · arXiv
• Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  October 19, 2026 · arXiv
• Beyond Next-Screen Prediction: The Decision Value of GUI World Models
  October 17, 2026 · arXiv
• AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
  October 1, 2026 · arXiv
• GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  August 30, 2026 · arXiv
• WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning
  August 27, 2026 · arXiv
• AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents
  August 6, 2026 · arXiv
• SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction
  July 17, 2026 · arXiv
• How Mobile World Model Guides GUI Agents?
  May 11, 2026 · arXiv
• UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics
  February 11, 2026 · arXiv
• Code2World: A GUI World Model via Renderable Code Generation
  February 10, 2026 · arXiv
• MobileDreamer: Generative Sketch World Model for GUI Agent
  January 7, 2026 · arXiv
• MobileWorldBench: Towards Semantic World Modeling For Mobile Agents
  December 16, 2025 · arXiv
• Unlocking Smarter Device Control: Foresighted Planning with a World Model-Driven Code Execution Approach
  May 22, 2025 · Findings of EMNLP 2025
• UIBert: Learning Generic Multimodal Representations for UI Understanding
  July 29, 2021 · IJCAI 2021
• Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training
  October 22, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• Adaptive-Group Process Rewards for GUI Agents
  October 13, 2026 · arXiv
• PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agents
  October 12, 2026 · arXiv
• Latent World Action Model for GUI Agents
  October 9, 2026 · arXiv
• Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents
  October 7, 2026 · arXiv
• nanoMuse: An Open-Source Personal Agent for Every Device You Own
  October 6, 2026 · arXiv