AdaptArena: Evaluating Test-Time Personalization of Web Agents

Dongchan Shin , Xing Han Lù , Jiaqi Deng , Jay Gala , Tomás Vergara Browne , Jaewon Moon , Fengyuan Liu , Alexandre Drouin , Siva Reddy , Alexandre Lacoste

🏛 Institutions: Mila , McGill University , ServiceNow Research , Laval University
📅 Date: September 29, 2026
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: AdaptArena benchmark personalization preference inference

TLDR

AdaptArena is a benchmark of 480 tasks for test-time personalization of web agents, where the agent must infer a latent user preference from a relevant historical trajectory. Oracle agents given the true preference succeed on 82.92% of tasks while evaluated agents reach at most 15.62%, and correct preference inference does not guarantee downstream execution success.

Open paper Code Report issue