Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding

Yizhou Liu , Fei Tang , Yuchen Yan , Zhengxi Lu , Songqin Nong , Tao Jiang , Wenhao Xu , Wenqi Zhang , Weiming Lu , Jun Xiao , Yongliang Shen

🏛 Institutions: ZJU , Ant Group
📅 Date: September 14, 2026
📑 Publisher: arXiv
💻 Env: General GUI
🔑 Keywords: GUI grounding test-time adaptation negative learning CANL ScreenSpot-Pro

TLDR

This work introduces a label-free test-time training paradigm for GUI grounding built on two observations: coordinate-token confidence indicates correctness better than full-sequence confidence, and in sparse GUI coordinate spaces negative samples give more reliable signal than potentially noisy pseudo-positives. The resulting Confidence-Anchored Negative Learning reaches 92.1% on ScreenSpot-V2 and 33.8% on ScreenSpot-Pro, an 8.9-point absolute gain over the base model.

Open paper Report issue