R-WoM: Retrieval-augmented World Model For Computer-use Agents
Kai Mei , Jiang Guo , Shuaichen Chang , Mingwen Dong , Dongkyu Lee , Xing Niu , Jiarong Jiang
- 🏛 Institutions
- Rutgers University , AWS Agentic AI
- 📅 Date
- October 13, 2025
- 📑 Publisher
- ICLR 2026 (Poster)
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
This paper tests whether LLMs can act as world models for computer-use agents and finds that simulation quality degrades sharply on full-procedure planning even when short-range prediction remains reasonable. R-WoM addresses this by grounding simulated rollouts with retrieved up-to-date tutorials, improving performance on OSWorld and WebArena, especially on longer-horizon tasks.
Related papers (24)
- Agent World Model in the Wild: Computer-Use Agent Environment SynthesisOctober 19, 2026 · arXiv
- Right Screen, Wrong Transition: World Models as Verifiers for GUI AgentsOctober 8, 2026 · arXiv
- AutoGUIWorld: Image Generators as Visual World Models for GUI AgentOctober 1, 2026 · arXiv
- WebWorld: A Large-Scale World Model for Web Agent TrainingFebruary 16, 2026 · arXiv
- DynaWeb: Model-Based Reinforcement Learning of Web AgentsJanuary 29, 2026 · arXiv
- Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent TrainingOctober 22, 2026 · arXiv
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv
- PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience RetrievalSeptember 29, 2026 · arXiv
- The GUI Is Not the State: Diagnosing State Aliasing in GUI World ModelsSeptember 26, 2026 · arXiv
- Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use AgentsFebruary 3, 2026 · arXiv
- SafePred: A Predictive Guardrail for Computer-Using Agents via World ModelsFebruary 2, 2026 · arXiv
- Agentic Reward Modeling: Verifying GUI Agent via Online Proactive InteractionJanuary 31, 2026 · arXiv
- BEAP-Agent: Backtrackable Execution and Adaptive Planning for GUI AgentsJanuary 29, 2026 · arXiv
- From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy AssimilationJanuary 9, 2026 · arXiv
- Just Do It!? Computer-Use Agents Exhibit Blind Goal-DirectednessOctober 2, 2025 · ICLR 2026 (Poster)
- Agent S2: A Compositional Generalist-Specialist Framework for Computer Use AgentsApril 1, 2025 · COLM 2025
- Back to Basics: A Skill-Centric Approach to Generalist Web AgentsOctober 26, 2026 · arXiv
- Evolving Agent InterfacesOctober 20, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer UseOctober 15, 2026 · arXiv
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use AgentsOctober 15, 2026 · arXiv
- Why Generate a Click? Plug-in Decision Heads for Web AgentsOctober 14, 2026 · arXiv
- Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web AgentsOctober 14, 2026 · arXiv