macOSWorld: A Multilingual Interactive Benchmark for GUI Agents

macOSWorld is the first interactive benchmark for GUI agents on macOS, covering 202 multilingual tasks across 30 applications and a dedicated safety subset for deception attacks. The evaluation shows large performance gaps between proprietary and open-source agents, substantial multilingual degradation, and unresolved safety weaknesses on macOS-specific workflows.

• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
  April 12, 2026 · arXiv
• When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents
  February 9, 2026 · arXiv
• When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents
  February 9, 2026 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation
  April 10, 2026 · arXiv
• LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios
  February 3, 2026 · arXiv
• WebTrap Park: An Automated Platform for Systematic Security Evaluation of Web Agents
  January 13, 2026 · arXiv
• It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents
  December 29, 2025 · arXiv
• DECEPTICON: How Dark Patterns Manipulate Web Agents
  December 28, 2025 · arXiv
• MPR-GUI: Benchmarking and Enhancing Multilingual Perception and Reasoning in GUI Agents
  November 30, 2025 · arXiv
• Investigating the Impact of Dark Patterns on LLM-Based Web Agents
  October 20, 2025 · IEEE S&P 2026
• MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control
  October 23, 2024 · arXiv
• Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents
  October 11, 2024 · arXiv
• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
  October 9, 2024 · ICLR 2026 (Poster)
• Dissecting Adversarial Robustness of Multimodal LM Agents
  June 18, 2024 · ICLR 2025 (Poster)
• Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
  June 9, 2026 · arXiv
• WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments
  April 30, 2026 · arXiv
• HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks
  April 10, 2026 · arXiv
• Preference Redirection via Attention Concentration: An Attack on Computer Use Agents
  April 9, 2026 · arXiv
• Gym-Anything: Turn any Software into an Agent Environment
  April 7, 2026 · arXiv
• HippoCamp: Benchmarking Contextual Agents on Personal Computers
  April 1, 2026 · arXiv
• PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents
  March 9, 2026 · arXiv
• OSExpert: Computer-Use Agents Learning Professional Skills via Exploration
  March 9, 2026 · arXiv
• OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks
  January 28, 2026 · arXiv