OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding
Wee Joe Tan , Zi Rui Lucas Lim , Shashank Durgad , Karim Obegi , Aiden Yiliu Li
- 🏛 Institutions
- Onflow AI
- 📅 Date
- February 25, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Web
- 🔑 Keywords
TLDR
OpenFlo simulates user behavior on websites with multimodal GUI grounding rather than DOM parsing, producing standardized UX reports that integrate the System Usability Scale, Single Ease Questions, and Think Aloud. Built on Avenir-Web, it pairs robust web interaction with simulated user behavior profiles for continuous, scalable usability testing.
Related papers (24)
- ClickAgent: Enhancing UI Location Capabilities of Autonomous AgentsOctober 9, 2024 · SIGDIAL 2025
- Step-Aligned Script Generation with Verification and Recovery for Web Automation TestingOctober 19, 2026 · arXiv
- What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web SemanticsOctober 15, 2026 · arXiv
- Why Generate a Click? Plug-in Decision Heads for Web AgentsOctober 14, 2026 · arXiv
- Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web AgentsOctober 14, 2026 · arXiv
- STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI AgentsOctober 14, 2026 · arXiv
- MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearchOctober 13, 2026 · arXiv
- Seg2Click: Learning Pixel-Level Action Masks for GUI Visual GroundingOctober 12, 2026 · arXiv
- ConClick: Element-Wise Contrastive Learning Improves GUI GroundingOctober 9, 2026 · arXiv
- Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent TrainingOctober 5, 2026 · arXiv
- Recurrent Latent Visual Search for GUI GroundingOctober 4, 2026 · arXiv
- MintAct: A Unified Visual Agent for Digital EnvironmentsSeptember 18, 2026 · arXiv
- AutoTailor: Automatic, User-Aligned Capability Selection and Adaptation for Web AgentsSeptember 11, 2026 · arXiv
- Lexical Coupling in GUI Element Grounding: Sentence Embeddings Track Labels across Mobile and WebAugust 22, 2026 · arXiv
- Same Outcomes, Different Journeys: A Trace-Level Framework for Comparing Human and GUI-Agent Behavior in Production Search SystemsApril 9, 2026 · arXiv
- WebTestPilot: Agentic End-to-End Web Testing against Natural Language Specification by Inferring Oracles with Symbolized GUI ElementsFebruary 12, 2026 · arXiv
- ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge EvolutionJanuary 12, 2026 · arXiv
- VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding TasksDecember 18, 2025 · arXiv
- WebATLAS: An LLM Agent with Experience-Driven Memory and Action SimulationOctober 26, 2025 · NeurIPS 2025 Workshop on Language Agents and World Models
- Surfer 2: The Next Generation of Cross-Platform Computer Use AgentsOctober 22, 2025 · arXiv
- PolySkill: Learning Generalizable Skills Through Polymorphic AbstractionOctober 17, 2025 · arXiv
- ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and SearchMay 21, 2025 · arXiv
- ScaleTrack: Scaling and back-tracking Automated GUI AgentsMay 1, 2025 · arXiv
- CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web NavigationApril 30, 2025 · NAACL 2025 (System Demonstrations)