Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments

Yitong Zhang , Ximo Li , Liyi Cai , Jia Li

🏛 Institutions: Tsinghua , Beihang University , PKU
📅 Date: September 14, 2025
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: security environmental injection attack dynamic environments LLM-driven environment simulation attention black hole trigger optimization Chameleon

TLDR

This paper studies environmental injection attacks on GUI agents under dynamic web conditions where trigger position and surrounding context vary across pages and sessions. It shows prior attacks degrade sharply under this more realistic threat model, then proposes Chameleon with LLM-driven environment simulation and Attention Black Hole supervision to recover attack effectiveness across six websites and four LVLM agents.

Open paper arXiv Report issue