ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
Hanyu Lai , Xiao Liu , Yanxiao Zhao , Han Xu , Hanchen Zhang , Bohao Jing , Yanyu Ren , Shuntian Yao , Yuxiao Dong , Jie Tang
- 🏛 Institutions
- Tsinghua , Zhipu , University of Chinese Academy of Sciences
- 📅 Date
- August 19, 2025
- 📑 Publisher
- ICLR 2026 (Poster)
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
ComputerRL is a desktop-agent training framework that combines direct GUI interaction with programmatic APIs and scales online RL through a distributed infrastructure over thousands of parallel virtual desktops. Its Entropulse schedule alternates RL and supervised fine-tuning to stabilize long training runs, and the resulting GLM-ComputerRL-9B reaches 48.9% on OSWorld.
Related papers (24)
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use AgentsSeptember 29, 2026 · arXiv
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- AutoGUIWorld: Image Generators as Visual World Models for GUI AgentOctober 1, 2026 · arXiv
- MintAct: A Unified Visual Agent for Digital EnvironmentsSeptember 18, 2026 · arXiv
- IntentScore: Intent-Conditioned Action Evaluation for Computer-Use AgentsApril 6, 2026 · arXiv
- EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic ExperienceJanuary 22, 2026 · arXiv
- STEP: Success-Rate-Aware Trajectory-Efficient Policy OptimizationNovember 17, 2025 · Findings of ACL 2026
- Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data CurationSeptember 28, 2025 · arXiv
- DPO Learning with LLMs-Judge Signal for Computer Use AgentsJune 3, 2025 · arXiv
- CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAsSeptember 25, 2026 · arXiv
- ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI AgentsApril 13, 2026 · arXiv
- From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy AssimilationJanuary 9, 2026 · arXiv
- UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time GroundingJuly 29, 2025 · CVPR 2026 Findings
- AgentCPM‑GUI: Building Mobile‑Use Agents with Reinforcement Fine‑TuningJune 2, 2025 · EMNLP 2025 System Demonstrations
- AutoWebGLM: A Large Language Model-based Web Navigating AgentApril 4, 2024 · KDD 2024
- Evolving Agent InterfacesOctober 20, 2026 · arXiv
- VeriCUA: Learning Proactive Verification for Reliable GUI Agent RewardsOctober 19, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv
- Agent World Model in the Wild: Computer-Use Agent Environment SynthesisOctober 19, 2026 · arXiv
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer UseOctober 15, 2026 · arXiv
- LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use AgentsOctober 15, 2026 · arXiv
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon AgentsOctober 13, 2026 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv