MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment

MobileBench-OL benchmarks mobile GUI agents on 1,080 online tasks from 80 Chinese apps. It extends evaluation beyond instruction following to long-horizon execution, reasoning and exploration, and robustness to real-world noise, and pairs the benchmark with an automatic evaluation pipeline that supports environment reset.

• ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
  May 24, 2026 · arXiv
• AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI Agents
  March 19, 2026 · arXiv
• MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments
  December 22, 2025 · arXiv
• CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents
  October 20, 2026 · arXiv
• PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval
  September 29, 2026 · arXiv
• LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles
  September 28, 2026 · arXiv
• The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge
  September 24, 2026 · Findings of EMNLP 2026
• Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
  June 9, 2026 · arXiv
• CocoaBench: Evaluating Unified Digital Agents in the Wild
  April 13, 2026 · arXiv
• HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks
  April 10, 2026 · arXiv
• ClawBench: Can AI Agents Complete Everyday Online Tasks?
  April 9, 2026 · arXiv
• Gym-Anything: Turn any Software into an Agent Environment
  April 7, 2026 · arXiv
• OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks
  January 28, 2026 · arXiv
• LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent
  January 26, 2026 · ICLR 2026 (Poster)
• AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
  October 21, 2024 · EMNLP 2024 (Poster)
• iPrefBench: A User-Centric Benchmark for Personalized Mobile Use Agents
  October 26, 2026 · arXiv
• AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal Agents
  October 25, 2026 · arXiv
• MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile Agents
  October 16, 2026 · arXiv
• MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection
  October 16, 2026 · arXiv
• Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI Navigation
  October 15, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• How GUI Agents Fail: GraceBench for Execution Stability Diagnosis
  October 14, 2026 · arXiv
• OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native Tools
  October 9, 2026 · arXiv
• Can Agents Work for Everyone? Cross-User Reliability for Mobile GUI Agents in Personalized User Interfaces
  October 6, 2026 · arXiv