Context-Aware Workflow Decomposition for Automated Mobile UI Annotation Using Multimodal Large Language Models

Athar Parvez , Muhammad Jawad Mufti , Muqaddas Gull , Omar Hammad

🏛 Institutions: Unknown
📅 Date: June 1, 2026
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: UI annotation mobile UI understanding workflow decomposition MUIAnno

TLDR

This paper studies automated mobile UI annotation for UI understanding, accessibility, testing, dataset construction, and GUI agents. It decomposes annotation into context-aware stages with structured prompts and schema-constrained outputs, improving precision on expert-annotated MUIAnno mobile screens.

Open paper arXiv Report issue