Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction

Yan Zhang , Pei Fu , Daiqing Wu , Huawen Shen , Ruoceng Zhang , Shaojie Zhang , Jiahui Yang , Yu Zhou , Can Ma , Zhenbo Luo , Jian Luan

🏛 Institutions: Institute of Information Engineering , Xiaomi , University of Chinese Academy of Sciences , Nankai University
📅 Date: September 22, 2026
📑 Publisher: EMNLP 2026
💻 Env: Mobile Web
🔑 Keywords: MaP masked trajectory prediction multi-task model

TLDR

MaP (Masked Trajectory Prediction) casts multi-turn GUI interaction as a trajectory and defines training objectives by masking and predicting its components, unifying step-wise decision-making, state-action alignment, and long-horizon planning under one objective. A role-aware adapter routes tokens to specialised representation spaces, and MaP outperforms direct mixture training on five GUI navigation benchmarks.

Open paper Report issue