Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
Saaket Agashe , Kyle Wong , Vincent Tu , Jiachen Yang , Ang Li , Xin Eric Wang
- 🏛 Institutions
- Simular Research
- 📅 Date
- April 1, 2025
- 📑 Publisher
- COLM 2025
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
Agent S2 is a compositional generalist-specialist framework that splits computer-use responsibilities across specialized and generalist models rather than using a single monolithic agent. Its core methods are Mixture-of-Grounding for precise localization and Proactive Hierarchical Planning for long-horizon control, yielding strong gains on OSWorld, WindowsAgentArena, and AndroidWorld.
Related papers (24)
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer UseOctober 15, 2026 · arXiv
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use AgentsOctober 15, 2026 · arXiv
- GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness EvolutionOctober 1, 2026 · arXiv
- HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use AgentsSeptember 29, 2026 · arXiv
- RSIAgent: Autonomous Exploration for Recursive Self-improvement in New EnvironmentsSeptember 14, 2026 · arXiv
- Watch and Learn: Learning to Use Computers from Online VideosOctober 6, 2025 · CVPR 2026
- CoAct-1: Computer-using Multi-Agent System with Coding ActionsAugust 5, 2025 · ICLR 2026 (Poster)
- LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOSMay 24, 2025 · arXiv
- Focused Omni-Modal GUI AgentsOctober 19, 2026 · arXiv
- Agent World Model in the Wild: Computer-Use Agent Environment SynthesisOctober 19, 2026 · arXiv
- Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual AgentsAugust 28, 2026 · arXiv
- Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use AgentsFebruary 3, 2026 · arXiv
- Agentic Reward Modeling: Verifying GUI Agent via Online Proactive InteractionJanuary 31, 2026 · arXiv
- BEAP-Agent: Backtrackable Execution and Adaptive Planning for GUI AgentsJanuary 29, 2026 · arXiv
- LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI AgentJanuary 26, 2026 · ICLR 2026 (Poster)
- From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy AssimilationJanuary 9, 2026 · arXiv
- R-WoM: Retrieval-augmented World Model For Computer-use AgentsOctober 13, 2025 · ICLR 2026 (Poster)
- Just Do It!? Computer-Use Agents Exhibit Blind Goal-DirectednessOctober 2, 2025 · ICLR 2026 (Poster)
- SheetCopilot: Bringing Software Productivity to the Next Level through Large Language ModelsMay 30, 2023 · NeurIPS 2023
- Evolving Agent InterfacesOctober 20, 2026 · arXiv
- VeriCUA: Learning Proactive Verification for Reliable GUI Agent RewardsOctober 19, 2026 · arXiv
- Step-Aligned Script Generation with Verification and Recovery for Web Automation TestingOctober 19, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv