BTesting with Interactive LLM Agents

Yuxuan Lu, Ting-Yao Hsu, Hansu Gu, Limeng Cui, Yaochen Xie, William Headden, Bingsheng Yao, Akash Veeragouni, Jiapeng Liu, Sreyashi Nag, Jessie Wang, Dakuo Wang

🏛 Institutions: Northeastern University, Pennsylvania State University, Amazon
📅 Date: April 13, 2025
📑 Publisher: arXiv
💻 Env: Web
🔑 Keywords: A/B testing user simulation persona simulation e-commerce AgentA/B

TLDR

AgentA/B uses interactive LLM agents with diverse personas to simulate user behavior on real webpages for scalable A/B testing. In a controlled Amazon shopping experiment with 1,000 agents, it studies whether agent populations can reproduce human-like interaction patterns for UI/UX evaluation.

Open paper arXiv Edit on GitHub Report issue