WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

WebVoyager is an end-to-end multimodal web agent evaluated on a benchmark built from tasks over 15 live websites. The paper also introduces a GPT-4V-based automatic evaluation protocol and reports 85.3% agreement with human judgment.

• REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites
  April 15, 2025 · arXiv
• SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
  October 19, 2024 · ICLR 2025 (Spotlight)
• VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets
  September 11, 2026 · arXiv
• ComponentBench: Diagnosing Component-Level Failures in Computer-Use Agents
  August 18, 2026 · COLM 2026
• StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection
  August 6, 2026 · arXiv
• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding
  July 19, 2026 · arXiv
• WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation
  July 7, 2026 · arXiv
• Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks
  April 27, 2026 · arXiv
• WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark
  April 13, 2026 · arXiv
• The Amazing Agent Race: Strong Tool Users, Weak Navigators
  April 11, 2026 · arXiv
• ClawBench: Can AI Agents Complete Everyday Online Tasks?
  April 9, 2026 · arXiv
• GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
  April 8, 2026 · arXiv
• WebSP-Eval: Evaluating Web Agents on Website Security and Privacy Tasks
  April 7, 2026 · arXiv
• The Art of Building Verifiers for Computer Use Agents
  April 5, 2026 · arXiv
• When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigation
  April 1, 2026 · arXiv
• WebArena-Infinity: Generating Browser Environments with Verifiable Tasks at Scale
  March 2026 · Blog Post
• Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
  March 27, 2026 · arXiv
• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing
  March 26, 2026 · arXiv
• Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos
  March 23, 2026 · CVPR 2026
• WebPII: Benchmarking Visual PII Detection for Computer-Use Agents
  March 18, 2026 · arXiv
• WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces
  March 5, 2026 · arXiv
• TimeWarp: Evaluating Web Agents by Revisiting the Past
  March 5, 2026 · arXiv
• Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History
  February 19, 2026 · arXiv