GUIPilot: A Consistency-Based Mobile GUI Testing Approach for Detecting Application-Specific Bugs

Ruofan Liu , Xiwen Teoh , Yun Lin , Guanjie Chen , Ruofei Ren , Denys Poshyvanyk , Jin Song Dong

🏛 Institutions: SJTU , NUS , William & Mary
📅 Date: June 9, 2025
📑 Publisher: ISSTA 2025
💻 Env: Mobile
🔑 Keywords: GUIPilot GUI testing model-based testing visual grounding application-specific bugs

TLDR

GUIPilot detects application-specific mobile GUI bugs by checking both screen-level and process-level consistency between design mock-ups and the implemented app. It solves a widget alignment problem to find layout inconsistencies and uses a vision-language model with visual prompts to execute and validate specified GUI transitions, reaching 94.5% precision and 99.6% recall on 80 apps and 160 mock-ups and surfacing nine confirmed bugs in a trading app with 32 million users.

Open paper Publisher Code Report issue