CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents

CUAAudit studies vision-language models as autonomous judges of desktop-agent task success from observable interactions alone. Across multiple operating-system benchmarks, it finds that even strong VLM auditors degrade on harder environments and disagree substantially with one another, highlighting limits of model-based auditing.

• Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systems
  April 9, 2026 · arXiv
• GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosis
  April 6, 2026 · arXiv
• MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments
  February 3, 2026 · arXiv
• WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation
  October 22, 2025 · NeurIPS 2025 Workshop on Multi-Turn Interactions in Large Language Models
• Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
  September 18, 2025 · NeurIPS 2025 Datasets & Benchmarks Track (Poster)
• You Don’t Know Until You Click: Automated GUI Testing for Production-Ready Software Evaluation
  August 17, 2025 · SEA @ NeurIPS 2025 (Poster)
• An Illusion of Progress? Assessing the Current State of Web Agents
  April 2, 2025 · COLM 2025
• GUI Agents: A Survey
  December 18, 2024 · Findings of ACL 2025
• Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
  June 9, 2026 · arXiv
• A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy Reduction
  May 1, 2026 · arXiv
• WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments
  April 30, 2026 · arXiv
• VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation
  April 23, 2026 · arXiv
• The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
  April 12, 2026 · arXiv
• HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks
  April 10, 2026 · arXiv
• EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning
  April 10, 2026 · arXiv
• Preference Redirection via Attention Concentration: An Attack on Computer Use Agents
  April 9, 2026 · arXiv
• Gym-Anything: Turn any Software into an Agent Environment
  April 7, 2026 · arXiv
• IntentScore: Intent-Conditioned Action Evaluation for Computer-Use Agents
  April 6, 2026 · arXiv
• GPA: Learning GUI Process Automation from Demonstrations
  April 2, 2026 · arXiv
• HippoCamp: Benchmarking Contextual Agents on Personal Computers
  April 1, 2026 · arXiv
• GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation
  March 27, 2026 · arXiv
• CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
  March 25, 2026 · arXiv
• Video-Based Reward Modeling for Computer-Use Agents
  March 10, 2026 · arXiv
• SpecOps: A Fully Automated AI Agent Testing Framework in Real-World GUI Environments
  March 10, 2026 · ICSE 2026