ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning

Xiao Yu , Baolin Peng , Vineeth Vajipey , Hao Cheng , Michel Galley , Jianfeng Gao , Zhou Yu

🏛 Institutions: Columbia , MSR
📅 Date: October 2, 2024
📑 Publisher: ICLR 2025 (Poster)
💻 Env: Web
🔑 Keywords: Reflective-MCTS exploratory learning test-time search contrastive reflection VisualWebArena ExACT

TLDR

ExACT combines Reflective-MCTS test-time search with Exploratory Learning to teach web agents to explore, evaluate states, and backtrack. On VisualWebArena, the GPT-4o-based search agent improves substantially over prior methods, and the fine-tuned model recovers 87% of the search agent's performance while using much less inference compute.

Open paper Report issue