On the Robustness of GUI Grounding Models Against Image Attacks

This paper benchmarks the robustness of GUI grounding models under natural noise, untargeted attacks, and targeted attacks across mobile, desktop, and web interfaces. It finds that current models such as UGround remain highly sensitive to adversarial perturbations and low-resolution conditions, exposing a major reliability gap for practical GUI use.

• Constructing Challenging Browser-Use Tasks by Controlled Environment Interventions
  September 20, 2026 · arXiv
• RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
  September 18, 2026 · arXiv
• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks
  December 18, 2025 · arXiv
• OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models
  December 18, 2025 · arXiv
• Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents
  October 7, 2024 · ICLR 2025 (Oral)
• GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
  June 16, 2024 · ICLR 2025 (Poster)
• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
  January 17, 2024 · ACL 2024
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence
  October 5, 2026 · arXiv
• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  September 9, 2026 · arXiv
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  August 25, 2026 · arXiv
• GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models
  April 15, 2026 · arXiv
• PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents
  March 9, 2026 · arXiv
• NaturalGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory Dataset
  August 2, 2025 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)
• RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
  May 28, 2025 · ICLR 2026 (Oral)
• WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
  February 12, 2025 · arXiv
• CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
  July 1, 2024 · Findings of ACL 2025
• GUI Action Narrator: Where and When Did That Action Take Place?
  June 19, 2024 · arXiv
• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
  April 11, 2024 · NeurIPS 2024 Datasets and Benchmarks Track
• OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web
  February 29, 2024 · ECCV 2024 (Poster)
• AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
  October 29, 2026 · arXiv