The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models
Dongsheng Liu , Chao Jin , Wenkui Yang , Hejin Wang , Junwei Yang , Zeren Zhang , Ziwei Chen , Huaibo Huang , Jie Cao , Ran He
- 🏛 Institutions
- University of Chinese Academy of Sciences , CASIA , Huawei Noah's Ark Lab
- 📅 Date
- September 26, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
Identifies state aliasing in GUI world models, where the visible interface omits transition-relevant environment state so identical observations can lead to different valid futures. Introduces StateAliasBench to isolate it and a lightweight predictive-state recovery that augments frozen world models, restoring state-sensitive prediction and improving agents on AndroidWorld.
Related papers (24)
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv
- PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience RetrievalSeptember 29, 2026 · arXiv
- AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal AgentsOctober 25, 2026 · arXiv
- GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent EnvironmentsAugust 30, 2026 · arXiv
- Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial EnvironmentsAugust 25, 2026 · arXiv
- MobileWorldBench: Towards Semantic World Modeling For Mobile AgentsDecember 16, 2025 · arXiv
- AndroidWorld: A Dynamic Benchmarking Environment for Autonomous AgentsMay 23, 2024 · ICLR 2025 (Poster)
- IGG: A Benchmark for Interactive GUI Grounding under Visibility ConstraintsOctober 23, 2026 · arXiv
- Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent TrainingOctober 22, 2026 · arXiv
- PointerBench: A Diagnostic Benchmark for Fine-Grained GUI GroundingOctober 22, 2026 · arXiv
- ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI AgentsOctober 19, 2026 · arXiv
- Agent World Model in the Wild: Computer-Use Agent Environment SynthesisOctober 19, 2026 · arXiv
- MMA-SafetyBench: A Benchmark for Multimodal Agent Safety EvaluationOctober 14, 2026 · arXiv
- cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use AgentsSeptember 30, 2026 · arXiv
- LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual PuzzlesSeptember 28, 2026 · arXiv
- ScreenHaystack: Finding Blind Zones in GUI GroundingSeptember 25, 2026 · EMNLP 2026
- VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgetsSeptember 11, 2026 · arXiv
- FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?September 7, 2026 · arXiv
- AutoGUI-v2: A Comprehensive Multi-Modal GUI Functionality Understanding BenchmarkApril 27, 2026 · arXiv
- GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding ModelsApril 15, 2026 · arXiv
- CocoaBench: Evaluating Unified Digital Agents in the WildApril 13, 2026 · arXiv
- What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI ReasoningApril 8, 2026 · Findings of ACL 2026
- GUIDE: Interpretable GUI Agent Evaluation via Hierarchical DiagnosisApril 6, 2026 · arXiv
- GUIDE: A Benchmark for Understanding and Assisting Users in Open-Ended GUI TasksMarch 26, 2026 · CVPR 2026