OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrations
Yixiong Xiao , Lang An , Hucheng Yang , Pinxue Ma , Yongquan Chen , Jingjia Cao , Yusai Zhao , Ting Wang , Ting Liu , Siqi Bao , Jingbo Zhou , Hua Wu
- 🏛 Institutions
- Baidu, Inc. , Ningxia Electric Power Engineering Co., Ltd.
- 📅 Date
- September 2, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
OmegaUse-SOP is a human-in-the-loop system that turns demonstrations of professional computer use into standard operating procedures for GUI agents, targeting domain-specific workflows with implicit knowledge and software-specific conventions.
Related papers (24)
- AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human DemonstrationsNovember 24, 2024 · ACL 2025
- AppAgent: Multimodal Agents as Smartphone UsersDecember 21, 2023 · CHI 2025
- CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI AgentsSeptember 4, 2026 · arXiv
- ASIL: Replacing Screenshot-and-Click with Structured State and Semantic ActionsAugust 27, 2026 · arXiv
- VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI AutomationApril 23, 2026 · arXiv
- EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience LearningApril 10, 2026 · arXiv
- ShowUI-Aloha: Human-Taught GUI AgentJanuary 12, 2026 · arXiv
- Surfer 2: The Next Generation of Cross-Platform Computer Use AgentsOctober 22, 2025 · arXiv
- BIMgent: Towards Autonomous Building Modeling via Computer-use AgentsJune 8, 2025 · ICML 2025 Workshop on Computer-use Agents
- LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOSMay 24, 2025 · arXiv
- UFO2: The Desktop AgentOSApril 20, 2025 · arXiv
- WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting PointFebruary 12, 2025 · arXiv
- PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital WorldDecember 23, 2024 · arXiv
- AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer AssistantOctober 24, 2024 · Findings of ACL 2025
- Agent S: An Open Agentic Framework that Uses Computers Like a HumanOctober 10, 2024 · ICLR 2025 (Poster)
- AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based AgentsSeptember 26, 2024 · ACL 2025
- OS-Copilot: Towards Generalist Computer Agents with Self-ImprovementFebruary 12, 2024 · LLMAgents @ ICLR 2024
- Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer ControlJune 13, 2023 · ICLR 2024 (Poster)
- Improving Proficiency and Efficiency of Android GUI Agents via Self-Generating Tool ActionsSeptember 6, 2026 · arXiv
- Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual AgentsAugust 28, 2026 · arXiv
- AOHP: An Open-Source OS-Level Agent Harness for Personalized, Efficient and Secure InteractionJune 22, 2026 · arXiv
- MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent ResearchMay 25, 2026 · arXiv
- ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental SynthesisMay 24, 2026 · arXiv
- ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI AgentsApril 13, 2026 · arXiv