UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning

Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, Hongsheng Li

🏛 Institutions: vivo AI Lab, MMLab, CUHK
📅 Date: March 27, 2025
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: dataset reinforcement learning rule-based action reward GRPO UI-R1-3B AndroidControl ScreenSpot-pro

TLDR

UI-R1 studies whether rule-based reinforcement learning can improve efficient GUI action prediction for multimodal mobile agents. It trains UI-R1-3B with GRPO on a curated set of 136 challenging mobile tasks using a rule-based action reward, and reports gains on ScreenSpot, ScreenSpot-Pro, and AndroidControl over the base model.

Open paper arXiv Edit on GitHub Report issue