SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents

Zonghao Ying, Yangguang Shao, Jianle Gan, Gan Xu, Junjie Shen, Wenxin Zhang, Quanchen Zou, Junzheng Shi, Zhenfei Yin, Mingchuan Zhang, Aishan Liu, Xianglong Liu

🏛 Institutions: Beihang University, Institute of Information Engineering, CAS, China University of Petroleum (East China), Zhejiang University of Technology, University of Chinese Academy of Sciences, 360 AI Security Lab, University of Sydney, Henan University of Science and Technology, Zhongguancun Laboratory, Institute of Dataspace
📅 Date: October 11, 2025
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: benchmark security evaluation attack taxonomy failure analysis adversarial manipulation SecureWebArena

TLDR

SecureWebArena evaluates the security of LVLM-based web agents with six realistic simulated environments, 2,970 trajectories, and six attack vectors spanning both user-level and environment-level manipulations. Its multi-layered protocol separates failures in reasoning, behavior, and task outcome, and shows that all tested agents remain vulnerable to subtle adversarial attacks.

Open paper arXiv Edit on GitHub Report issue