Tree Search for Language Model Agents

Jing Yu Koh , Stephen McAleer , Daniel Fried , Ruslan Salakhutdinov

🏛 Institutions: CMU
📅 Date: July 1, 2024
📑 Publisher: TMLR 2025
💻 Env: Web
🔑 Keywords: tree search best-first search value function test-time compute VisualWebArena

TLDR

This paper adds inference-time best-first tree search to language-model web agents by searching directly in the environment and guiding expansion with a model-based value function. On top of a GPT-4o baseline it reports a 39.7% relative gain on VisualWebArena and a 28.0% relative gain on WebArena, showing that web-agent performance scales with additional test-time search.

Open paper arXiv Report issue