An Illusion of Progress? Assessing the Current State of Web Agents

This paper argues that reported web-agent progress is overstated once agents are evaluated on more realistic online tasks. It introduces Online-Mind2Web with 300 tasks across 136 live websites, pairs it with the WebJudge automatic evaluation method, and uses that setup to show a much weaker picture of current web-agent capability than prior benchmarks suggest.

• A Baseline Assessment for the Agentic Web Technology Stack
  October 9, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
  September 28, 2026 · arXiv
• ClawBench: Can AI Agents Complete Everyday Online Tasks?
  April 9, 2026 · arXiv
• Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos
  March 23, 2026 · CVPR 2026
• AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
  October 21, 2024 · EMNLP 2024 (Poster)
• PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas
  October 16, 2026 · arXiv
• cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
  September 30, 2026 · arXiv
• GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosis
  April 6, 2026 · arXiv
• MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments
  February 3, 2026 · arXiv
• AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
  October 29, 2026 · arXiv
• SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent Skills
  October 26, 2026 · arXiv
• AgentProofBench: Benchmarking Reusable Verifiers for Web Agents
  October 25, 2026 · arXiv
• SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents
  October 24, 2026 · arXiv
• LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents
  October 19, 2026 · arXiv
• Interaction-as-a-Judge
  October 13, 2026 · arXiv
• BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views
  October 12, 2026 · arXiv
• When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks
  October 9, 2026 · arXiv
• PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence
  October 5, 2026 · arXiv
• Securing Computer-Use Agents Against Branch Steering Attacks
  October 2, 2026 · NeurIPS 2026 Workshop on Agents in the Wild
• Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and Trajectories
  October 1, 2026 · NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
• AgentBreaker: Evaluating Context-Aware Indirect Prompt Injection Risks in Modern Web Agents
  October 1, 2026 · ISSTA 2026
• AdaptArena: Evaluating Test-Time Personalization of Web Agents
  September 29, 2026 · arXiv
• DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis
  September 26, 2026 · arXiv