ZeroGUI: Automating Online GUI Learning at Zero Human Cost

Chenyu Yang , Shiqian Su , Shi Liu , Xuan Dong , Yue Yu , Weijie Su , Xuehui Wang , Zhaoyang Liu , Jinguo Zhu , Hao Li , Wenhai Wang , Yu Qiao , Xizhou Zhu , Jifeng Dai

🏛 Institutions: Shanghai AI Laboratory , Tsinghua , SJTU , HKUST , CUHK
📅 Date: May 29, 2025
📑 Publisher: arXiv
💻 Env: Desktop Mobile
🔑 Keywords: reinforcement learning model-free online learning automatic task generation automatic reward estimation test-time adaptation ZeroGUI

TLDR

ZeroGUI studies how to train GUI agents online without human labels instead of relying on static offline supervision. It uses VLMs to generate tasks, estimate rewards, and support two-stage online reinforcement learning, improving both desktop and mobile GUI agents on OSWorld and AndroidLab.

Open paper Code Report issue

Related papers (24)

Generalization in Online Reinforcement Learning for Mobile Agents

March 8, 2026 · arXiv
MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments

October 14, 2026 · arXiv
Adaptive-Group Process Rewards for GUI Agents

October 13, 2026 · arXiv
MintAct: A Unified Visual Agent for Digital Environments

September 18, 2026 · arXiv
STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization

November 17, 2025 · Findings of ACL 2026
Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation

September 28, 2025 · arXiv
GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents

April 14, 2025 · arXiv
Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps

October 29, 2026 · arXiv
VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards

October 19, 2026 · arXiv
OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents

October 15, 2026 · arXiv
HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents

September 29, 2026 · arXiv
One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents

September 28, 2026 · arXiv
CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning

September 26, 2026 · arXiv
WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning

August 27, 2026 · arXiv
Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments

August 25, 2026 · arXiv
GSAR: Goal-State-Anchor Rewards for Mobile GUI Agents with Self-Evolving Data Synthesis

August 24, 2026 · arXiv
CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications

August 12, 2026 · arXiv
SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents

July 25, 2026 · arXiv
SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

July 13, 2026 · arXiv
MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

May 25, 2026 · arXiv
ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

April 13, 2026 · arXiv
Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions

April 8, 2026 · arXiv
Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction

April 7, 2026 · ACL 2026
UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experience

March 25, 2026 · arXiv