Mobile GUI Agents under Real-world Threats: Are We There Yet?

Guohong Liu , Jialei Ye , Jiacheng Liu , Wei Liu , Pengzhi Gao , Jian Luan , Yuanchun Li , Yunxin Liu

🏛 Institutions: AIR, Tsinghua , University of Electronic Science and Technology of China , PKU , MiLM Plus , Xiaomi
📅 Date: July 6, 2025
📑 Publisher: MobiSys 2026
💻 Env: Mobile
🔑 Keywords: safety security adversarial attacks third-party content AgentHazard

TLDR

AgentHazard evaluates mobile GUI agents when untrusted third-party content appears inside otherwise legitimate app regions, a threat missing from standard static-content benchmarks. Its instrumentation framework supports a dynamic environment with 122 tasks and a static dataset with more than 3,000 scenarios; evaluated agents show average misleading rates of 42.0% and 36.1% in the two settings.

Open paper arXiv Publisher Code Dataset Report issue