Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments

Guo Gan , Yilun Zhao , Cong Chen , Jinbiao Wei , Tingyu Song , Zheyuan Yang , Lin Fu , Hong Zhou

🏛 Institutions: ZJU , Yale University , University of Chinese Academy of Sciences , Tongji University
📅 Date: August 25, 2026
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: AnTrap benchmark reinforcement learning safety security GRPO AndroidWorld

TLDR

AnTrap is an AndroidWorld-based benchmark that injects controllable runtime anomalies, organized into four categories with ten subcategories, into Android GUI-agent trajectories. Testing 16 leading GUI agents reveals substantial robustness degradation under these anomalies, and adversarial RL training can address many single-step traps but struggles with deep contextual failures such as state deadlock.

Open paper Code Dataset Report issue