STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization
Yuhan Chen , Yuxuan Liu , Long Zhang , Pengzhi Gao , Jian Luan , Wei Liu
- 🏛 Institutions
- MiLM Plus , Xiaomi , Renmin University of China , Wuhan University
- 📅 Date
- November 17, 2025
- 📑 Publisher
- Findings of ACL 2026
- 💻 Env
- Desktop Mobile
- 🔑 Keywords
TLDR
STEP improves the efficiency of online GUI-agent reinforcement learning by using per-task success rates to resample difficult tasks and decomposing trajectories into step-level training samples. Its success-rate-weighted advantages and step-level GRPO augmentation improve sample efficiency and training stability over trajectory-level GRPO on OSWorld and AndroidWorld.
Related papers (24)
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data CurationSeptember 28, 2025 · arXiv
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use AgentsSeptember 29, 2026 · arXiv
- Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial EnvironmentsAugust 25, 2026 · arXiv
- UI-Voyager: A Self-Evolving GUI Agent Learning via Failed ExperienceMarch 25, 2026 · arXiv
- Adaptive Milestone Reward for GUI AgentsFebruary 12, 2026 · arXiv
- ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use AgentsAugust 19, 2025 · ICLR 2026 (Poster)
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Adaptive-Group Process Rewards for GUI AgentsOctober 13, 2026 · arXiv
- Latent World Action Model for GUI AgentsOctober 9, 2026 · arXiv
- MintAct: A Unified Visual Agent for Digital EnvironmentsSeptember 18, 2026 · arXiv
- Agentic Reward Modeling: Verifying GUI Agent via Online Proactive InteractionJanuary 31, 2026 · arXiv
- From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy AssimilationJanuary 9, 2026 · arXiv
- ZeroGUI: Automating Online GUI Learning at Zero Human CostMay 29, 2025 · arXiv
- GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI AgentsApril 14, 2025 · arXiv
- Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS AppsOctober 29, 2026 · arXiv
- AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal AgentsOctober 25, 2026 · arXiv
- Evolving Agent InterfacesOctober 20, 2026 · arXiv
- VeriCUA: Learning Proactive Verification for Reliable GUI Agent RewardsOctober 19, 2026 · arXiv
- ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI AgentsOctober 19, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv
- Agent World Model in the Wild: Computer-Use Agent Environment SynthesisOctober 19, 2026 · arXiv
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv