WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation

Yaoyao Qian , Yuanli Wang , Jinda Zhang , Yun Zong , Meixu Chen , Hanhan Zhou , Jindan Huang , Yifan Zeng , Xinyu Hu , Chan Hee Song , Danqing Zhang

🏛 Institutions: Northeastern University , Boston University , University of Victoria , University of Minnesota , George Washington University , Tufts University , Oregon State University , University of Texas at San Antonio , OSU , PathOnAI.org
📅 Date: October 22, 2025
📑 Publisher: NeurIPS 2025 Workshop on Multi-Turn Interactions in Large Language Models
💻 Env: Web
🔑 Keywords: evaluation trajectory analysis weighted action graph multi-path evaluation WebGraphEval

TLDR

WebGraphEval evaluates web agents by converting many interaction trajectories into a unified weighted action graph instead of scoring only final success or conformity to one reference path. This graph view highlights redundancy, inefficiency, and critical decision points across agents and benchmark runs.

Open paper arXiv Report issue