Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI Grounding
Zhiyuan Jiang , Shenghao Xie , Wenyi Li , Wenqiang Zu , Peihang Li , Jiahao Qiu , Siqi Pei , Lei Ma , Tiejun Huang , Mengdi Wang , Shilong Liu
- 🏛 Institutions
- Xi’an Jiaotong University , Princeton , PKU , University of Chinese Academy of Sciences , HKU , Michigan State University
- 📅 Date
- December 5, 2025
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
This paper studies zooming as a test-time prior for GUI grounding and proposes ZoomClick, which decides when to zoom, how far to zoom, and when to return to the original view during localization. It also introduces GUIZoom-Bench and reports stronger grounding results across several mainstream benchmarks.
Related papers (24)
- UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI GroundingApril 15, 2026 · arXiv
- Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface ElementsMarch 15, 2026 · arXiv
- Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal FusionFebruary 6, 2026 · arXiv
- MVP: Multiple View Prediction Improves GUI GroundingDecember 9, 2025 · arXiv
- MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearchOctober 13, 2026 · arXiv
- ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and SearchMay 21, 2025 · arXiv
- Improved GUI Grounding via Iterative NarrowingNovember 18, 2024 · arXiv
- Right Click, Wrong Box: Grading GUI Grounding by What a Click DoesOctober 27, 2026 · arXiv
- Learning to Look Beyond Screens: Transferable Visual Observation from GUI GroundingOctober 27, 2026 · arXiv
- AdaGround: Token-Normalized Adaptive Zooming with Verifier-in-the-Loop Refinement for GUI GroundingOctober 27, 2026 · arXiv
- PreView: Enhancing GUI Grounding with Preprocessed Views and CuesOctober 24, 2026 · arXiv
- IGG: A Benchmark for Interactive GUI Grounding under Visibility ConstraintsOctober 23, 2026 · arXiv
- PointerBench: A Diagnostic Benchmark for Fine-Grained GUI GroundingOctober 22, 2026 · arXiv
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv
- GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement LearningOctober 13, 2026 · arXiv
- SCA: Spatial Credit Assignment for Reinforcement Learning of GUI AgentsSeptember 29, 2026 · arXiv
- Relevance Does Not Imply Applicability: Experience Activation for Personal GUI AgentsSeptember 27, 2026 · arXiv
- ScreenHaystack: Finding Blind Zones in GUI GroundingSeptember 25, 2026 · EMNLP 2026
- How Many Pixels Is a Digit Worth? Place-Aware Coordinate Entropy for GUI Agent Confidence EstimationSeptember 21, 2026 · EMNLP 2026
- RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop SelectionSeptember 16, 2026 · ACM MM 2026
- Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI GroundingSeptember 14, 2026 · arXiv
- GUI-Primitives: Diagnosing Spatial Reasoning Failures in Vision-Language GUI GroundingAugust 22, 2026 · arXiv
- Hallucination-Free GUI Grounding via Regression-Free Layout-Aware MatchingAugust 10, 2026 · arXiv
- GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMsAugust 4, 2026 · arXiv