WebServ: A Browser-Server Environment for Efficient Training of Reinforcement Learning-based Web Agents at Scale

Yuxuan Lu , Jing Huang , Hui Liu , Jiri Gesi , Yan Han , Shihan Fu , Tianqi Zheng , Dakuo Wang

🏛 Institutions: Northeastern University , Amazon
📅 Date: October 17, 2025
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: reinforcement learning training environment browser-server architecture scalability WebServ

TLDR

WebServ is a full-stack environment for large-scale RL training of web agents that combines a compact browser-side interface with efficient isolated server-side state management. It is designed to make parallel rollouts practical, and the paper reports faster launches, lower storage cost, and strong WebArena performance.

Open paper arXiv Report issue