CocoaBench: Evaluating Unified Digital Agents in the Wild

Shibo Hao , Zhining Zhang , Zhiqi Liang , Tianyang Liu , Yuheng Zha , Qiyue Gao , Jixuan Chen , Zilong Wang , Zhoujun Cheng , Haoxiang Zhang , Junli Wang , Hexi Jin , Boyuan Zheng , Kun Zhou , Yu Wang , Feng Yao , Licheng Liu , Yijiang Li , Zhifei Li , Zhengtao Han , Pracha Promthaw , Tommaso Cerruti , Xiaohan Fu , Ziqiao Ma , Jingbo Shang , Lianhui Qin , Julian McAuley , Eric P. Xing , Zhengzhong Liu , Rupesh Kumar Srivastava , Zhiting Hu

🏛 Institutions: UC San Diego , MBZUAI , UMich , UC Berkeley , ETH , University of Cambridge , Gray Swan AI
📅 Date: April 13, 2026
📑 Publisher: arXiv
💻 Env: General GUI
🔑 Keywords: benchmark unified digital agents long-horizon tasks visual grounding CocoaBench CocoaAgent

TLDR

CocoaBench evaluates unified digital agents on long-horizon tasks requiring flexible composition of vision, search, and coding. Tasks are specified by an instruction and an automatic evaluation function, enabling reliable scalable evaluation across agent infrastructures. The best-evaluated system reaches only 45.1%, exposing gaps in reasoning, tool use, and visual grounding.

Open paper arXiv Code Dataset Report issue