macOSWorld: A Multilingual Interactive Benchmark for GUI Agents

macOSWorld is the first interactive benchmark for GUI agents on macOS, covering 202 multilingual tasks across 30 applications and a dedicated safety subset for deception attacks. The evaluation shows large performance gaps between proprietary and open-source agents, substantial multilingual degradation, and unresolved safety weaknesses on macOS-specific workflows.

• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
  April 12, 2026 · arXiv
• When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents
  February 9, 2026 · arXiv
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
  February 9, 2026 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation
  April 10, 2026 · arXiv
• LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios
  February 3, 2026 · arXiv