WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks

WebSP-Eval is the first framework evaluating web agents on user-facing website security and privacy tasks such as cookie preferences, privacy settings, and session revocation. Across 200 task instances on 28 websites, agents fail more than 45% on tasks with stateful UI elements like toggles and checkboxes.

• WebPII: Benchmarking Visual PII Detection for Computer-Use Agents
  March 18, 2026 · arXiv
• HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities
  October 14, 2025 · ICLR 2026 (Poster)
• RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
  May 28, 2025 · ICLR 2026 (Oral)
• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
  April 22, 2025 · NeurIPS 2025 (Poster)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents
  March 12, 2025 · NeurIPS 2025 Datasets and Benchmarks Track (Poster)
• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
  April 12, 2026 · arXiv
• VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents
  June 3, 2025 · ICLR 2026 (Poster)
• Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks
  April 27, 2026 · arXiv
• WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark
  April 13, 2026 · arXiv
• The Amazing Agent Race: Strong Tool Users, Weak Navigators
  April 11, 2026 · arXiv
• ClawBench: Can AI Agents Complete Everyday Online Tasks?
  April 9, 2026 · arXiv
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
  April 8, 2026 · arXiv
• The Art of Building Verifiers for Computer Use Agents
  April 5, 2026 · arXiv
• When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigation
  April 1, 2026 · arXiv
• WebArena-Infinity: Generating Browser Environments with Verifiable Tasks at Scale
  March 2026 · Blog Post
• Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
  March 27, 2026 · arXiv
• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing
  March 26, 2026 · arXiv
• Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos
  March 23, 2026 · CVPR 2026
• WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces
  March 5, 2026 · arXiv
• TimeWarp: Evaluating Web Agents by Revisiting the Past
  March 5, 2026 · arXiv
• Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks
  March 4, 2026 · arXiv
• Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History
  February 19, 2026 · arXiv
• PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents
  February 5, 2026 · arXiv
• WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents
  February 3, 2026 · arXiv