STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization

Yuhan Chen , Yuxuan Liu , Long Zhang , Pengzhi Gao , Jian Luan , Wei Liu

🏛 Institutions: MiLM Plus , Xiaomi , Renmin University of China , Wuhan University
📅 Date: November 17, 2025
📑 Publisher: Findings of ACL 2026
💻 Env: Desktop Mobile
🔑 Keywords: reinforcement learning sample efficiency trajectory optimization step-level optimization OSWorld AndroidWorld STEP

TLDR

STEP improves the efficiency of online GUI-agent reinforcement learning by using per-task success rates to resample difficult tasks and decomposing trajectories into step-level training samples. Its success-rate-weighted advantages and step-level GRPO augmentation improve sample efficiency and training stability over trajectory-level GRPO on OSWorld and AndroidWorld.

Open paper Publisher Report issue