UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning

Zhengxi Lu , Yuxiang Chai , Yaxuan Guo , Xi Yin , Liang Liu , Hao Wang , Han Xiao , Shuai Ren , Guanjing Xiong , Hongsheng Li

🏛 Institutions: vivo AI Lab , MMLab , CUHK
📅 Date: March 27, 2025
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: dataset reinforcement learning rule-based action reward GRPO UI-R1-3B AndroidControl ScreenSpot-pro

TLDR

UI-R1 studies whether rule-based reinforcement learning can improve efficient GUI action prediction for multimodal mobile agents. It trains UI-R1-3B with GRPO on a curated set of 136 challenging mobile tasks using a rule-based action reward, and reports gains on ScreenSpot, ScreenSpot-Pro, and AndroidControl over the base model.

Open paper arXiv Report issue