On the Effects of Data Scale on UI Control Agents

Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, Oriana Riva

🏛 Institutions: Google DeepMind, Google
📅 Date: June 6, 2024
📑 Publisher: NeurIPS 2024 Datasets and Benchmarks Track
💻 Env: Mobile
🔑 Keywords: dataset AndroidControl data scaling fine-tuning

TLDR

Studies how UI-control agent performance scales with more fine-tuning data and releases AndroidControl, a dataset of over 15K demonstrations across 833 Android apps. The paper shows strong in-domain scaling trends while highlighting that out-of-domain generalization remains harder.

Open paper Edit on GitHub Report issue