DynaWeb: Model-Based Reinforcement Learning of Web Agents

Hang Ding, Peidong Liu, Junqiao Wang, Ziwei Ji, Meng Cao, Rongzhao Zhang, Lynn Ai, Eric Yang, Tianyu Shi, Lei Yu

🏛 Institutions: SJTU, Sichuan University, HKUST, McGill University, Shanghai AI Laboratory, Gradient, University of Toronto, Mila
📅 Date: January 29, 2026
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: model-based reinforcement learning world model imagined rollouts WebArena WebVoyager DynaWeb

TLDR

DynaWeb trains web agents with model-based reinforcement learning by learning a web world model that supports imagined rollouts, then interleaving those rollouts with real expert trajectories. This synthetic-environment training loop improves open-source web agents on both WebArena and WebVoyager.

Open paper arXiv Edit on GitHub Report issue