ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay

ARPO studies end-to-end reinforcement learning for GUI agents in long-horizon desktop environments where sparse rewards and rollout cost make optimization difficult. It augments GRPO with replayed successful experience and task selection, establishing a stronger OSWorld training baseline than prior policy-optimization approaches.

• GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents
  April 14, 2025 · arXiv
• Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
  October 29, 2026 · arXiv
• SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents
  September 29, 2026 · arXiv
• Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
  August 25, 2026 · arXiv
• Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agents
  August 22, 2026 · arXiv
• Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction
  April 7, 2026 · ACL 2026
• Generalization in Online Reinforcement Learning for Mobile Agents
  March 8, 2026 · arXiv
• CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
  March 3, 2026 · arXiv
• AgentCPM‑GUI: Building Mobile‑Use Agents with Reinforcement Fine‑Tuning
  June 2, 2025 · EMNLP 2025 System Demonstrations
• UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning
  March 27, 2025 · arXiv
• VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards
  October 19, 2026 · arXiv
• OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
  October 15, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• Adaptive-Group Process Rewards for GUI Agents
  October 13, 2026 · arXiv
• ComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents
  September 30, 2026 · arXiv
• HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
  September 29, 2026 · arXiv
• CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning
  September 26, 2026 · arXiv
• MintAct: A Unified Visual Agent for Digital Environments
  September 18, 2026 · arXiv
• SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents
  July 25, 2026 · arXiv
• SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL
  July 13, 2026 · arXiv
• OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  January 28, 2026 · arXiv
• STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization
  November 17, 2025 · Findings of ACL 2026
• Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
  September 28, 2025 · arXiv
• ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
  August 19, 2025 · ICLR 2026 (Poster)