Watch and Learn: Learning to Use Computers from Online Videos

Watch & Learn converts Internet videos of human computer use into more than 53K executable UI trajectories by framing annotation as an inverse dynamics problem over consecutive screen states. The resulting data improves both general-purpose and specialized CUAs on OSWorld and yields state-of-the-art 7B-scale performance on WindowsAgentArena under the 15-step limit.

• HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
  September 29, 2026 · arXiv
• GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
  October 1, 2026 · arXiv
• CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
  March 25, 2026 · arXiv
• CoAct-1: Computer-using Multi-Agent System with Coding Actions
  August 5, 2025 · ICLR 2026 (Poster)
• VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos
  October 22, 2025 · arXiv
• Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
  April 1, 2025 · COLM 2025
• Evolving Agent Interfaces
  October 20, 2026 · arXiv
• CUA-Lite: An Open Platform for Computer Use Agents
  October 19, 2026 · arXiv
• Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  October 19, 2026 · arXiv
• OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use
  October 15, 2026 · arXiv
• OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
  October 15, 2026 · arXiv
• LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
  October 15, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agents
  October 13, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
  October 9, 2026 · arXiv
• Latent World Action Model for GUI Agents
  October 9, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• Control OSWorld: An AI Control Environment for GUI Computer Use Agents
  October 2, 2026 · arXiv
• CUAWright: A Minimal Unified Interface for Digital Agents
  October 2, 2026 · arXiv
• DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents
  October 1, 2026 · arXiv
• AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
  October 1, 2026 · arXiv
• Talk2Agent: Benchmarking Voice Interfaces for Text Agents
  September 30, 2026 · arXiv
• OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
  September 30, 2026 · arXiv