GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning

Weitai Kang , Bin Lei , Gaowen Liu , Caiwen Ding , Yan Yan

🏛 Institutions: University of Illinois Chicago , University of Minnesota , Cisco Research
📅 Date: August 6, 2025
📑 Publisher: ICLR 2026 (Poster)
💻 Env: Desktop Mobile Web
🔑 Keywords: GUI visual grounding reinforcement fine-tuning Adversarial KL Factor ScreenSpot GuirlVG

TLDR

GuirlVG studies how to make reinforcement fine-tuning work for GUI visual grounding instead of naively applying standard rule-based RL. It systematically tunes reward design, prediction format, and training setup, adds an Adversarial KL Factor for stabilization, and reports stronger ScreenSpot-family results with only 5.2K training samples.

Open paper arXiv Report issue