A3: Android Agent Arena for Mobile GUI Agents with Essential-State Procedural Evaluation

A3 is a mobile GUI benchmark built from 100 tasks over 20 dynamic online Android apps to evaluate agents beyond static or offline settings. Its essential-state procedural evaluation uses MLLMs as reward models to verify both intermediate progress and final completion on real online apps.

• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• The Art of Building Verifiers for Computer Use Agents
  April 5, 2026 · arXiv
• CUARewardBench: A Benchmark for Evaluating Reward Models on Computer-using Agent
  October 21, 2025 · arXiv
• Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
  May 21, 2025 · NeurIPS 2025 (Spotlight)
• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  September 9, 2026 · arXiv
• APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
  September 7, 2026 · arXiv
• ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults
  September 4, 2026 · arXiv
• GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  August 30, 2026 · arXiv
• Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
  August 25, 2026 · arXiv
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  August 25, 2026 · arXiv
• GSAR: Goal-State-Anchor Rewards for Mobile GUI Agents with Self-Evolving Data Synthesis
  August 24, 2026 · arXiv
• Benchmarking General Mobile Assistants in Challenging Real-World Scenarios
  August 21, 2026 · arXiv
• Benchmarking Living-Screen-Native GUI Agents on Short-Video Platforms
  June 3, 2026 · arXiv
• Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation
  May 26, 2026 · ICML 2026 (Poster)
• AndroidDaily: A Verifiable Benchmark for Mobile GUI Agents on Real-World Closed-Source Applications
  May 26, 2026 · arXiv
• MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
  May 25, 2026 · arXiv
• ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
  May 24, 2026 · arXiv
• ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
  April 13, 2026 · arXiv
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation
  April 10, 2026 · arXiv
• KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation
  April 9, 2026 · arXiv
• Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions
  April 8, 2026 · arXiv
• Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction
  April 7, 2026 · ACL 2026
• Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants
  April 1, 2026 · arXiv
• PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent
  March 31, 2026 · arXiv