CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications

Linqiang Guo , Li Gu , Zihuan Jiang , Zhixiang Chi , Siobhan Reid , Ziqiang Wang , Yuanhao Yu , Wei Liu , Yang Wang , Tse-Hsun (Peter) Chen

🏛 Institutions: Concordia University , Mila , University of Toronto , McMaster University
📅 Date: August 12, 2026
📑 Publisher: arXiv
💻 Env: Mobile
🔑 Keywords: CoAdapt-GUI test-time adaptation workflow context GRPO AndroidWorld-Generalization

TLDR

Mobile GUI agents stay brittle on applications absent from source training, so CoAdapt-GUI performs test-time adaptation that jointly updates a structured workflow context and the policy from the agent's own target-app rollouts and rewards. The workflow context retains transferable procedures, failure modes, and verification rules while excluding app-bound source details, and policy adaptation uses a task-context-matched group-relative objective over a LoRA adapter on a frozen VLM. It reaches 45.0% on AndroidWorld-Generalization against 37.5% for the reported Policy-Only TTA baseline, and lifts AndroidWorld Plus from 38.6% to 52.9%.

Open paper Report issue