PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital World

Yanheng He , Jiahe Jin , Shijie Xia , Jiadi Su , Runze Fan , Haoyang Zou , Xiangkun Hu , Pengfei Liu

🏛 Institutions: SJTU , GAIR
📅 Date: December 23, 2024
📑 Publisher: arXiv
💻 Env: Desktop
🔑 Keywords: framework PC agent human cognition transfer PC tracker cognition completion multi-agent system

TLDR

PC Agent studies how to transfer human cognitive processes into desktop agents for complex digital work rather than short isolated tasks. It introduces PC Tracker for collecting cognitive interaction traces, a two-stage cognition-completion pipeline, and a planning-plus-grounding multi-agent system, showing promising results on long PowerPoint workflows with limited data.

Open paper arXiv Code Report issue