CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs

Zhenhao Zhang , Zhaoyu Fan , Haohan Ying , Jingwen Hu , Hancen Fan , Junhao Zhou , Zitian Chen , Linchao Zhu

🏛 Institutions: Columbia , ZJU , University of Rochester , UIUC
📅 Date: September 25, 2026
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: CaptchaArena CAPTCHA dataset model reinforcement learning

TLDR

CaptchaArena is a training dataset of 50K execution-verified interactive CAPTCHA puzzles across 20 types and 5 interaction modes, with screenshot-action trajectories, step-by-step reasoning annotations, and pixel-mask annotations. Supervised fine-tuning followed by reinforcement learning on it yields CaptchaAgent, a single 9B policy for all 20 types.

Open paper Code Report issue