Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding

This paper introduces the Screen Point-and-Read task, where a model must explain the region indicated by a user point on a GUI screenshot, and proposes the Tree-of-Lens agent to solve it. It also releases the ScreenPR benchmark across mobile, web, and operating-system GUIs plus the ASHL dataset for hierarchical screen-region detection.

• GUIGuard: Toward a General Framework for Privacy-Preserving GUI Agents
  January 26, 2026 · arXiv
• Beyond Clicking: A Step Towards Generalist GUI Grounding via Text Dragging
  November 7, 2025 · arXiv
• Scaling Computer‑Use Grounding via User Interface Decomposition and Synthesis
  May 19, 2025 · NeurIPS 2025 Datasets and Benchmarks Track (Spotlight)
• UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesis
  April 15, 2025 · Findings of ACL 2025
• SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models
  May 30, 2023 · NeurIPS 2023
• WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark
  April 13, 2026 · arXiv
• Gym-Anything: Turn any Software into an Agent Environment
  April 7, 2026 · arXiv
• WebArena-Infinity: Generating Browser Environments with Verifiable Tasks at Scale
  March 2026 · Blog Post
• PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent
  March 31, 2026 · arXiv
• SecAgent: Efficient Mobile GUI Agent with Semantic Context
  March 9, 2026 · arXiv
• WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces
  March 5, 2026 · arXiv
• Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization
  February 24, 2026 · arXiv
• AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the Wild
  February 12, 2026 · arXiv
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
  February 9, 2026 · arXiv
• MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments
  February 3, 2026 · arXiv
• SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis
  January 26, 2026 · arXiv
• SMAN-Bench: A Cross-System Benchmark for Mobile Agents under Single- and Multi-path, Ambiguous, and Noisy Tasks
  January 26, 2026 · ICLR 2026 (Poster)
• ShowUI-π: Flow-based Generative Models as GUI Dexterous Hands
  December 31, 2025 · arXiv
• MobileWorldBench: Towards Semantic World Modeling For Mobile Agents
  December 16, 2025 · arXiv
• GUI-360: A Comprehensive Dataset and Benchmark for Computer-Using Agents
  November 6, 2025 · arXiv
• Web-CogReasoner: Towards Knowledge-Induced Cognitive Reasoning for Web Agents
  August 3, 2025 · ICLR 2026 (Poster)
• NaturalGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory Dataset
  August 2, 2025 · arXiv
• FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM Agents
  June 9, 2025 · ICLR 2026 (Poster)
• Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
  May 21, 2025 · NeurIPS 2025 (Spotlight)