ScreenHaystack: Finding Blind Zones in GUI Grounding

Chenyue Li , Xiaoxiao Sun , Yubo Deng , Qinlin Zhao , Serena Yeung-Levy , Yuhui Zhang

🏛 Institutions: Stanford
📅 Date: September 25, 2026
📑 Publisher: EMNLP 2026
💻 Env: General GUI
🔑 Keywords: ScreenHaystack benchmark GUI grounding ScreenSpot-Pro blind zones

TLDR

ScreenHaystack is a needle-in-a-haystack benchmark that relocates controlled target icons across high-resolution GUI backgrounds, revealing blind zones where grounding accuracy drops sharply for models including Qwen3-VL, UI-TARS, GTA, and UI-Venus. The blind zones transfer to ScreenSpot-Pro, and augmenting training data in them improves accuracy.

Open paper Code Dataset Report issue