UFO2: The Desktop AgentOS

Chaoyun Zhang , He Huang , Chiming Ni , Jian Mu , Si Qin , Shilin He , Lu Wang , Fangkai Yang , Pu Zhao , Chao Du , Liqun Li , Yu Kang , Zhao Jiang , Suzhen Zheng , Rujia Wang , Jiaxu Qian , Minghua Ma , Jian-Guang Lou , Qingwei Lin , Saravan Rajmohan , Dongmei Zhang

🏛 Institutions: Microsoft , ZJU-UIUC Institute , NJU , PKU
📅 Date: April 20, 2025
📑 Publisher: arXiv
💻 Env: Desktop
🔑 Keywords: framework hybrid GUI-API control multi-agent speculative execution PiP virtual desktop UFO2

TLDR

UFO2 presents a Windows AgentOS that pairs a coordinating HostAgent with specialized AppAgents for individual applications. Its main system ideas are a unified GUI-API action layer, hybrid UIA-plus-vision perception, speculative multi-action execution, and a picture-in-picture virtual desktop that lets users and the agent operate concurrently.

Open paper arXiv Report issue