Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search Systems
Maria Movin, Claudia Hauff, Aron Henriksson, Panagiotis Papapetrou
- 🏛 Institutions
- Stockholm University, Spotify
- 📅 Date
- April 9, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Web
- 🔑 Keywords
TLDR
This paper presents a trace-level evaluation framework comparing human and GUI-agent behavior across task outcome, query formulation, and navigation in a production audio-streaming search application. With 39 participants and a state-of-the-art GUI agent on 10 multi-hop search tasks, the agent matches task success but follows search-centric, low-branching strategies versus humans' content-centric exploration.
Related papers
- OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI GroundingFebruary 25, 2026 · arXiv
- WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph RepresentationOctober 22, 2025 · NeurIPS 2025 Workshop on Multi-Turn Interactions in Large Language Models
- AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM AgentsApril 13, 2025 · arXiv
- An Illusion of Progress? Assessing the Current State of Web AgentsApril 2, 2025 · COLM 2025
- GUIDE: Interpretable GUI Agent Evaluation via Hierarchical DiagnosisApril 6, 2026 · arXiv
- CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use AgentsMarch 11, 2026 · HEAL @ CHI 2026 Workshop