Attacking Vision-Language Computer Agents via Pop-ups

Shows that vision-language computer agents can be reliably distracted by adversarial pop-ups that human users would typically ignore. On OSWorld and VisualWebArena, these pop-ups achieve high attack success rates and sharply reduce task completion, while simple defenses like warning prompts remain ineffective.

• Dissecting Adversarial Robustness of Multimodal LM Agents
  June 18, 2024 · ICLR 2025 (Poster)
• Control OSWorld: An AI Control Environment for GUI Computer Use Agents
  October 2, 2026 · arXiv
• Preference Redirection via Attention Concentration: An Attack on Computer Use Agents
  April 9, 2026 · arXiv
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
  January 14, 2026 · arXiv
• Genesis: Evolving Attack Strategies for LLM Web Agent Red-Teaming
  October 21, 2025 · ICME 2026
• Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agents
  October 13, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
  October 9, 2026 · arXiv
• Latent World Action Model for GUI Agents
  October 9, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• CUAWright: A Minimal Unified Interface for Digital Agents
  October 2, 2026 · arXiv
• Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
  August 6, 2025 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)
• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
  April 11, 2024 · NeurIPS 2024 Datasets and Benchmarks Track
• Evolving Agent Interfaces
  October 20, 2026 · arXiv
• CUA-Lite: An Open Platform for Computer Use Agents
  October 19, 2026 · arXiv
• Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  October 19, 2026 · arXiv
• OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use
  October 15, 2026 · arXiv
• OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
  October 15, 2026 · arXiv
• LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
  October 15, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
  October 1, 2026 · arXiv
• AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
  October 1, 2026 · arXiv
• AgentBreaker: Evaluating Context-Aware Indirect Prompt Injection Risks in Modern Web Agents
  October 1, 2026 · ISSTA 2026