MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment

MobileBench-OL benchmarks mobile GUI agents on 1,080 online tasks from 80 Chinese apps. It extends evaluation beyond instruction following to long-horizon execution, reasoning and exploration, and robustness to real-world noise, and pairs the benchmark with an automatic evaluation pipeline that supports environment reset.

• ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
  May 24, 2026 · arXiv
• AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI Agents
  March 19, 2026 · arXiv
• MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments
  December 22, 2025 · arXiv
• Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
  June 9, 2026 · arXiv
• CocoaBench: Evaluating Unified Digital Agents in the Wild
  April 13, 2026 · arXiv
• HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks
  April 10, 2026 · arXiv
• ClawBench: Can AI Agents Complete Everyday Online Tasks?
  April 9, 2026 · arXiv
• Gym-Anything: Turn any Software into an Agent Environment
  April 7, 2026 · arXiv
• OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks
  January 28, 2026 · arXiv
• LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent
  January 26, 2026 · ICLR 2026 (Poster)
• AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
  October 21, 2024 · EMNLP 2024 (Poster)
• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  September 9, 2026 · arXiv
• APPSim-Bench: Bridging Real-world Apps and Reproducible Evaluation for Mobile GUI Agents
  September 7, 2026 · arXiv
• ElderBench: Benchmarking Autonomous Mobile Agents for Older Adults
  September 4, 2026 · arXiv
• GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
  August 30, 2026 · arXiv
• Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
  August 25, 2026 · arXiv
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  August 25, 2026 · arXiv
• Benchmarking General Mobile Assistants in Challenging Real-World Scenarios
  August 21, 2026 · arXiv
• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• Benchmarking Living-Screen-Native GUI Agents on Short-Video Platforms
  June 3, 2026 · arXiv
• Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation
  May 26, 2026 · ICML 2026 (Poster)
• AndroidDaily: A Verifiable Benchmark for Mobile GUI Agents on Real-World Closed-Source Applications
  May 26, 2026 · arXiv
• MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
  May 25, 2026 · arXiv
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation
  April 10, 2026 · arXiv