On the Effects of Data Scale on UI Control Agents
Wei Li , William Bishop , Alice Li , Chris Rawles , Folawiyo Campbell-Ajala , Divya Tyamagundlu , Oriana Riva
- 🏛 Institutions
- Google DeepMind , Google
- 📅 Date
- June 6, 2024
- 📑 Publisher
- NeurIPS 2024 Datasets and Benchmarks Track
- 💻 Env
- Mobile
- 🔑 Keywords
TLDR
Studies how UI-control agent performance scales with more fine-tuning data and releases AndroidControl, a dataset of over 15K demonstrations across 833 Android apps. The paper shows strong in-domain scaling trends while highlighting that out-of-domain generalization remains harder.
Related papers (24)
- UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement LearningMarch 27, 2025 · arXiv
- Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web AgentsJuly 2025 · Findings of ACL 2025
- Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-CorrectionApril 7, 2026 · ACL 2026
- PSPA-Bench: A Personalized Benchmark for Smartphone GUI AgentMarch 31, 2026 · arXiv
- Video-Based Reward Modeling for Computer-Use AgentsMarch 10, 2026 · arXiv
- SecAgent: Efficient Mobile GUI Agent with Semantic ContextMarch 9, 2026 · arXiv
- Turing Test on Screen: A Benchmark for Mobile GUI Agent HumanizationFebruary 24, 2026 · arXiv
- AmbiBench: Benchmarking Mobile GUI Agents Beyond One-Shot Instructions in the WildFebruary 12, 2026 · arXiv
- MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic EnvironmentsFebruary 3, 2026 · arXiv
- MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge EvolutionJanuary 27, 2026 · arXiv
- SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe SynthesisJanuary 26, 2026 · arXiv
- SMAN-Bench: A Cross-System Benchmark for Mobile Agents under Single- and Multi-path, Ambiguous, and Noisy TasksJanuary 26, 2026 · ICLR 2026 (Poster)
- MobileWorldBench: Towards Semantic World Modeling For Mobile AgentsDecember 16, 2025 · arXiv
- NaturalGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory DatasetAugust 2, 2025 · arXiv
- FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM AgentsJune 9, 2025 · ICLR 2026 (Poster)
- UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI AgentsMay 27, 2025 · NeurIPS 2025 (Poster)
- BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking MechanismMay 27, 2025 · EMNLP 2025 (Oral)
- ScaleTrack: Scaling and back-tracking Automated GUI AgentsMay 1, 2025 · arXiv
- ReachAgent: Enhancing Mobile Agent via Page Reaching and OperationApril 30, 2025 · NAACL 2025 (Poster)
- MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficient Mobile Task AutomationApril 30, 2025 · NAACL 2025 (System Demonstrations)
- LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration BenchmarkApril 18, 2025 · arXiv
- SpiritSight Agent: Advanced GUI Agent with One LookMarch 5, 2025 · CVPR 2025 (Poster)
- AppVLM: A Lightweight Vision Language Model for Online App ControlFebruary 10, 2025 · arXiv
- ShowUI: One Vision-Language-Action Model for GUI Visual AgentNovember 26, 2024 · CVPR 2025 (Poster)