BEARCUBS: A benchmark for computer-using web agents
Yixiao Song , Katherine Thai , Chau Minh Pham , Yapei Chang , Mazin Nadaf , Mohit Iyyer
- 🏛 Institutions
- UMass Amherst , UMD
- 📅 Date
- March 10, 2025
- 📑 Publisher
- COLM 2025
- 💻 Env
- Web
- 🔑 Keywords
TLDR
BEARCUBS is a benchmark of 111 information-seeking questions that require web agents to operate on live websites instead of static replicas. Its tasks force multimodal interactions such as video understanding and 3D navigation, and each question comes with a short answer and human-validated browsing trajectory for transparent evaluation.
Related papers (24)
- AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access BarriersOctober 29, 2026 · arXiv
- SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent SkillsOctober 26, 2026 · arXiv
- AgentProofBench: Benchmarking Reusable Verifiers for Web AgentsOctober 25, 2026 · arXiv
- SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web AgentsOctober 24, 2026 · arXiv
- LoWAgent: A Low-Resource Multilingual Benchmark for Web AgentsOctober 19, 2026 · arXiv
- BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser ViewsOctober 12, 2026 · arXiv
- When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded TasksOctober 9, 2026 · arXiv
- A Baseline Assessment for the Agentic Web Technology StackOctober 9, 2026 · arXiv
- AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use TasksOctober 8, 2026 · arXiv
- PlaySuite: A Large-Scale Benchmark for Interactive Visual IntelligenceOctober 5, 2026 · arXiv
- Securing Computer-Use Agents Against Branch Steering AttacksOctober 2, 2026 · NeurIPS 2026 Workshop on Agents in the Wild
- AdaptArena: Evaluating Test-Time Personalization of Web AgentsSeptember 29, 2026 · arXiv
- WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web AgentsSeptember 28, 2026 · arXiv
- DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard AnalysisSeptember 26, 2026 · arXiv
- AgentTell: Behavioural Side-Channel Leakage in Browser-Use AgentsSeptember 26, 2026 · arXiv
- The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying KnowledgeSeptember 24, 2026 · Findings of EMNLP 2026
- Invisible in Space, Visible in Time: Motion Vision CAPTCHA against GUI AgentsSeptember 23, 2026 · ACM MM 2026
- Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web AgentsSeptember 23, 2026 · arXiv
- CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned EnvironmentsSeptember 23, 2026 · arXiv
- Constructing Challenging Browser-Use Tasks by Controlled Environment InterventionsSeptember 20, 2026 · arXiv
- RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use AgentsSeptember 18, 2026 · arXiv
- ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise SoftwareSeptember 15, 2026 · arXiv
- VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgetsSeptember 11, 2026 · arXiv
- ComponentBench: Diagnosing Component-Level Failures in Computer-Use AgentsAugust 18, 2026 · COLM 2026