When Agents See Differently: Exposing UI Desynchronization Threats in Mobile Agents

Heng Li , Fulin Zhao , Zhe Geng , Zhiyuan Yao , Wei Yuan , Xiapu Luo

🏛 Institutions: PolyU , Huazhong University of Science and Technology
📅 Date: September 15, 2026
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: security UI desynchronization repackaging attack human oversight

TLDR

This paper identifies human-agent UI desynchronization, where one mobile UI state presents materially different information to a human viewer, bound by occlusion and luminance contrast, than to an agent consuming screenshots and accessibility metadata. An automated framework builds deployable repackaged APKs that exploit the gap without runtime adaptation, reaching 77.9% and 66.9% average misleading rates across five mobile-agent frameworks and three backbones on 546 tasks, while a 186-participant study finds the perturbations hard for people to notice.

Open paper Report issue