GUI-Spotlight: Adaptive Iterative Focus Refinement for Enhanced GUI Visual Grounding
Bin Lei , Nuo Xu , Ali Payani , Mingyi Hong , Chunhua Liao , Yu Cao , Caiwen Ding
- 🏛 Institutions
- University of Minnesota , Cisco Research , Lawrence Livermore National Labs
- 📅 Date
- October 5, 2025
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
GUI-Spotlight is a GUI grounding model that performs image-grounded reasoning by iteratively invoking specialized tools to narrow attention to the relevant screen region. Trained with only 18.5K examples, it reaches 52.8% accuracy on ScreenSpot-Pro, outperforming prior 7B grounding models trained on much larger datasets.
Related papers (24)
- POINTS-GUI-G: GUI-Grounding JourneyFebruary 6, 2026 · arXiv
- UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time GroundingJuly 29, 2025 · CVPR 2026 Findings
- ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer UseApril 4, 2025 · ACM Multimedia 2025
- Right Click, Wrong Box: Grading GUI Grounding by What a Click DoesOctober 27, 2026 · arXiv
- Learning to Look Beyond Screens: Transferable Visual Observation from GUI GroundingOctober 27, 2026 · arXiv
- AdaGround: Token-Normalized Adaptive Zooming with Verifier-in-the-Loop Refinement for GUI GroundingOctober 27, 2026 · arXiv
- PreView: Enhancing GUI Grounding with Preprocessed Views and CuesOctober 24, 2026 · arXiv
- IGG: A Benchmark for Interactive GUI Grounding under Visibility ConstraintsOctober 23, 2026 · arXiv
- PointerBench: A Diagnostic Benchmark for Fine-Grained GUI GroundingOctober 22, 2026 · arXiv
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv
- GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement LearningOctober 13, 2026 · arXiv
- SCA: Spatial Credit Assignment for Reinforcement Learning of GUI AgentsSeptember 29, 2026 · arXiv
- ScreenHaystack: Finding Blind Zones in GUI GroundingSeptember 25, 2026 · EMNLP 2026
- How Many Pixels Is a Digit Worth? Place-Aware Coordinate Entropy for GUI Agent Confidence EstimationSeptember 21, 2026 · EMNLP 2026
- RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop SelectionSeptember 16, 2026 · ACM MM 2026
- Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI GroundingSeptember 14, 2026 · arXiv
- GUI-Primitives: Diagnosing Spatial Reasoning Failures in Vision-Language GUI GroundingAugust 22, 2026 · arXiv
- Hallucination-Free GUI Grounding via Regression-Free Layout-Aware MatchingAugust 10, 2026 · arXiv
- GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMsAugust 4, 2026 · arXiv
- GUI-C²: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement LearningMay 29, 2026 · arXiv
- UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI GroundingApril 15, 2026 · arXiv
- GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding ModelsApril 15, 2026 · arXiv
- See, Point, Refine: Multi-Turn Approach to GUI Grounding with Visual FeedbackApril 14, 2026 · arXiv
- What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI ReasoningApril 8, 2026 · Findings of ACL 2026