CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents

CUAAudit studies vision-language models as autonomous judges of desktop-agent task success from observable interactions alone. Across multiple operating-system benchmarks, it finds that even strong VLM auditors degrade on harder environments and disagree substantially with one another, highlighting limits of model-based auditing.

• PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas
  October 16, 2026 · arXiv
• Interaction-as-a-Judge
  October 13, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety
  September 28, 2026 · arXiv
• Who Verifies the Agent's Success Claim? Self-Report Leakage in GUI-Agent Completion Judging
  October 21, 2026 · arXiv
• Beyond Next-Screen Prediction: The Decision Value of GUI World Models
  October 17, 2026 · arXiv
• A Baseline Assessment for the Agentic Web Technology Stack
  October 9, 2026 · arXiv
• Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and Trajectories
  October 1, 2026 · NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
• cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
  September 30, 2026 · arXiv
• WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
  September 28, 2026 · arXiv
• Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systems
  April 9, 2026 · arXiv
• GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosis
  April 6, 2026 · arXiv
• MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments
  February 3, 2026 · arXiv
• WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation
  October 22, 2025 · NeurIPS 2025 Workshop on Multi-Turn Interactions in Large Language Models
• Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
  September 18, 2025 · NeurIPS 2025 Datasets & Benchmarks Track (Poster)
• You Don’t Know Until You Click: Automated GUI Testing for Production-Ready Software Evaluation
  August 17, 2025 · SEA @ NeurIPS 2025 (Poster)
• An Illusion of Progress? Assessing the Current State of Web Agents
  April 2, 2025 · COLM 2025
• GUI Agents: A Survey
  December 18, 2024 · Findings of ACL 2025
• OS-Wizard: Scaling Computer-Using Agents to Specialized Software via Knowledge-Guided Exploration
  October 24, 2026 · arXiv
• Evolving Agent Interfaces
  October 20, 2026 · arXiv
• CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents
  October 20, 2026 · arXiv
• VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards
  October 19, 2026 · arXiv
• Comparing Computer Use Behavior Across Multiple Levels of Abstraction
  October 19, 2026 · arXiv
• CUA-Lite: An Open Platform for Computer Use Agents
  October 19, 2026 · arXiv