ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replay

ARPO studies end-to-end reinforcement learning for GUI agents in long-horizon desktop environments where sparse rewards and rollout cost make optimization difficult. It augments GRPO with replayed successful experience and task selection, establishing a stronger OSWorld training baseline than prior policy-optimization approaches.

• GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents
  April 14, 2025 · arXiv
• Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
  August 25, 2026 · arXiv
• Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agents
  August 22, 2026 · arXiv
• Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction
  April 7, 2026 · ACL 2026
• Generalization in Online Reinforcement Learning for Mobile Agents
  March 8, 2026 · arXiv
• CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning
  March 3, 2026 · arXiv
• AgentCPM‑GUI: Building Mobile‑Use Agents with Reinforcement Fine‑Tuning
  June 2, 2025 · EMNLP 2025 System Demonstrations
• UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning
  March 27, 2025 · arXiv
• SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents
  July 25, 2026 · arXiv
• SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL
  July 13, 2026 · arXiv
• OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  January 28, 2026 · arXiv
• STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization
  November 17, 2025 · Findings of ACL 2026
• Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
  September 28, 2025 · arXiv
• ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
  August 19, 2025 · ICLR 2026 (Poster)
• DPO Learning with LLMs-Judge Signal for Computer Use Agents
  June 3, 2025 · arXiv
• ZeroGUI: Automating Online GUI Learning at Zero Human Cost
  May 29, 2025 · arXiv
• A Data-Driven Approach for Learning to Control Computers
  February 16, 2022 · ICML 2022
• WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning
  August 27, 2026 · arXiv
• Task-Adaptive Rubrics for GUI Reward Modeling
  August 25, 2026 · arXiv
• GSAR: Goal-State-Anchor Rewards for Mobile GUI Agents with Self-Evolving Data Synthesis
  August 24, 2026 · arXiv
• CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications
  August 12, 2026 · arXiv
• GUI-C²: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning
  May 29, 2026 · arXiv
• MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
  May 25, 2026 · arXiv
• LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
  May 8, 2026 · arXiv