HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities

Xiaoxue Ren , Penghao Jiang , Kaixin Li , Zhiyong Huang , Xiaoning Du , Jiaojiao Jiang , Zhenchang Xing , Jiamou Sun , Terry Yue Zhuo

🏛 Institutions: ZJU , University of New South Wales , NUS , Monash University , CSIRO’s Data61 , Australian National University
📅 Date: October 14, 2025
📑 Publisher: ICLR 2026 (Poster)
💻 Env: Web
🔑 Keywords: benchmark security web vulnerabilities CTF penetration testing HackWorld

TLDR

HackWorld uses a CTF-style setup over 36 vulnerable web applications spanning 11 frameworks and 7 languages to test whether computer-use agents can discover and exploit realistic web flaws through GUI interaction. Current agents achieve exploitation rates below 12% and often fail at multi-step attack planning and security-tool use.

Open paper Code Report issue