Attacking Vision-Language Computer Agents via Pop-ups

Shows that vision-language computer agents can be reliably distracted by adversarial pop-ups that human users would typically ignore. On OSWorld and VisualWebArena, these pop-ups achieve high attack success rates and sharply reduce task completion, while simple defenses like warning prompts remain ineffective.

• Dissecting Adversarial Robustness of Multimodal LM Agents
  June 18, 2024 · ICLR 2025 (Poster)
• Preference Redirection via Attention Concentration: An Attack on Computer Use Agents
  April 9, 2026 · arXiv
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
  January 14, 2026 · arXiv
• Genesis: Evolving Attack Strategies for LLM Web Agent Red-Teaming
  October 21, 2025 · ICME 2026
• Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
  August 6, 2025 · arXiv
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
  May 31, 2025 · NeurIPS 2025 (Poster)