UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time Grounding

Shuquan Lian, Yuhang Wu, Jia Ma, Yifan Ding, Zihan Song, Bingqi Chen, Xiawu Zheng, Hui Li, Rongrong Ji

🏛 Institutions: Xiamen University
📅 Date: July 29, 2025
📑 Publisher: CVPR 2026 Findings
💻 Env: General GUI
🔑 Keywords: model reinforcement learning GUI grounding continuous reward cropping-based resampling decomposed grounding ScreenSpot-pro UI-AGILE

TLDR

UI-AGILE improves GUI agents through a continuous reward function that incentivizes high-precision grounding, a cropping-based resampling strategy for data efficiency, and decomposed grounding with selection for inference-time accuracy on high-resolution displays. It achieves 23% grounding accuracy improvement over baselines on ScreenSpot-Pro.

Open paper arXiv Edit on GitHub Report issue