AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents
Yuxuan Lu , Ting-Yao Hsu , Hansu Gu , Limeng Cui , Yaochen Xie , William Headden , Bingsheng Yao , Akash Veeragouni , Jiapeng Liu , Sreyashi Nag , Jessie Wang , Dakuo Wang
- 🏛 Institutions
- Northeastern University , Pennsylvania State University , Amazon
- 📅 Date
- April 13, 2025
- 📑 Publisher
- arXiv
- 💻 Env
- Web
- 🔑 Keywords
TLDR
AgentA/B uses interactive LLM agents with diverse personas to simulate user behavior on real webpages for scalable A/B testing. In a controlled Amazon shopping experiment with 1,000 agents, it studies whether agent populations can reproduce human-like interaction patterns for UI/UX evaluation.
Related papers (24)
- Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search SystemsApril 9, 2026 · arXiv
- OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI GroundingFebruary 25, 2026 · arXiv
- What Does "Now" Mean to Your Computer-Use Agent? Stale Screenshots, Page Loads, and Which Check to Run Before a ClickOctober 29, 2026 · arXiv
- InterA11y: Interactive Web Accessibility Error Detection with LLM AgentsOctober 29, 2026 · arXiv
- AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access BarriersOctober 29, 2026 · arXiv
- WebAnything: From the Wild Web to Scalable Agent EnvironmentsOctober 27, 2026 · arXiv
- SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent SkillsOctober 26, 2026 · arXiv
- Back to Basics: A Skill-Centric Approach to Generalist Web AgentsOctober 26, 2026 · arXiv
- AgentProofBench: Benchmarking Reusable Verifiers for Web AgentsOctober 25, 2026 · arXiv
- SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web AgentsOctober 24, 2026 · arXiv
- Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option MemoryOctober 24, 2026 · arXiv
- Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web AgentsOctober 22, 2026 · arXiv
- Step-Aligned Script Generation with Verification and Recovery for Web Automation TestingOctober 19, 2026 · arXiv
- LoWAgent: A Low-Resource Multilingual Benchmark for Web AgentsOctober 19, 2026 · arXiv
- SPIDES: Information-Flow Control for Browser AgentsOctober 16, 2026 · arXiv
- What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web SemanticsOctober 15, 2026 · arXiv
- Why Generate a Click? Plug-in Decision Heads for Web AgentsOctober 14, 2026 · arXiv
- Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web AgentsOctober 14, 2026 · arXiv
- STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI AgentsOctober 14, 2026 · arXiv
- Towards Safe Autonomy for Computer UseOctober 13, 2026 · arXiv
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon AgentsOctober 13, 2026 · arXiv
- MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearchOctober 13, 2026 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Interaction-as-a-JudgeOctober 13, 2026 · arXiv