Recurrent Latent Visual Search for GUI Grounding

Kaiyu Wu , Beichen Zheng , Weiyao Huang , Keze Wang

🏛 Institutions: Sun Yat-sen University
📅 Date: October 4, 2026
📑 Publisher: arXiv
💻 Env: Web Mobile
🔑 Keywords: GUI grounding visual search recurrent latent VLM ScreenSpot

TLDR

Proposes Recurrent Latent Visual Search (RLVS) for GUI grounding, addressing single-step grounding failures on crowded screenshots by iteratively refining latent target representations across recurrent search steps. RLVS improves grounding accuracy across desktop, web, and mobile interfaces without relying on explicit bounding box prompts.

Open paper Report issue