On the Robustness of GUI Grounding Models Against Image Attacks

This paper benchmarks the robustness of GUI grounding models under natural noise, untargeted attacks, and targeted attacks across mobile, desktop, and web interfaces. It finds that current models such as UGround remain highly sensitive to adversarial perturbations and low-resolution conditions, exposing a major reliability gap for practical GUI use.

• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks
  December 18, 2025 · arXiv
• OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models
  December 18, 2025 · arXiv
• Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents
  October 7, 2024 · ICLR 2025 (Oral)
• GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
  June 16, 2024 · ICLR 2025 (Poster)
• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
  January 17, 2024 · ACL 2024
• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  September 9, 2026 · arXiv
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  August 25, 2026 · arXiv
• GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models
  April 15, 2026 · arXiv
• PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents
  March 9, 2026 · arXiv
• NaturalGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory Dataset
  August 2, 2025 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)
• RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
  May 28, 2025 · ICLR 2026 (Oral)
• WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
  February 12, 2025 · arXiv
• CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
  July 1, 2024 · Findings of ACL 2025
• GUI Action Narrator: Where and When Did That Action Take Place?
  June 19, 2024 · arXiv
• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
  April 11, 2024 · NeurIPS 2024 Datasets and Benchmarks Track
• OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web
  February 29, 2024 · ECCV 2024 (Poster)
• VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets
  September 11, 2026 · arXiv
• FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?
  September 7, 2026 · arXiv
• APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
  September 7, 2026 · arXiv
• ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults
  September 4, 2026 · arXiv
• GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  August 30, 2026 · arXiv
• Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
  August 25, 2026 · arXiv