PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval

Bin Kang , Jiarui Ouyang , Li Jiang , Bin Chen , Zhuotao Tian

🏛 Institutions: University of Chinese Academy of Sciences , HKUST , Harbin Institute of Technology , CUHK , Shenzhen Loop Area Institute
📅 Date: September 29, 2026
📑 Publisher: arXiv
💻 Env: General GUI
🔑 Keywords: PrecogUI benchmark InterfereBench world model memory long-horizon tasks

TLDR

PrecogUI moves GUI agents from reactive to proactive decision-making with an experience pool of anomaly and success patterns, a simulator that forecasts the next symbolic UI layout for a candidate action, and a controller that fuses both with closed-loop error correction. It also builds InterfereBench for long-horizon tasks with strong disturbances, where PrecogUI surpasses prior methods.

Open paper Report issue