AppAgent: Multimodal Agents as Smartphone Users

Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, Gang Yu

🏛 Institutions: Tencent
📅 Date: December 21, 2023
📑 Publisher: CHI 2025
💻 Env: Mobile
🔑 Keywords: framework AppAgent knowledge base autonomous exploration human demonstrations tap-and-swipe control

TLDR

AppAgent is a smartphone-use agent that operates through a simple tap-and-swipe action space without backend app access. It learns app usage through autonomous exploration or human demonstrations, stores that knowledge in a reference document, and is evaluated on 50 tasks across 10 apps.

Open paper Edit on GitHub Report issue