AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents

Yifan Xu , Xiao Liu , Xueqiao Sun , Siyi Cheng , Hao Yu , Hanyu Lai , Shudan Zhang , Dan Zhang , Jie Tang , Yuxiao Dong

🏛 Institutions: Tsinghua , PKU , Zhipu
📅 Date: October 31, 2024
📑 Publisher: ACL 2025
💻 Env: Mobile
🔑 Keywords: benchmark dataset AndroidLab reproducible environment mobile agent training

TLDR

AndroidLab provides a reproducible Android agent environment plus a benchmark with predefined virtual devices, shared action spaces, and 138 tasks across nine apps. It also builds an Android Instruction dataset from that environment and shows that the resulting data materially improves both open LLM and VLM mobile agents.

Open paper Report issue