HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
Tongbo Chen , Junbo Niu , Zhengxi Lu , Niu Lian , Fei Tang , Yuchen Yan , Yike Hong , Yong Du , Yizhou Liu , Bofan Chen , Yongliang Shen
- 🏛 Institutions
- ZJU , PKU , Tsinghua
- 📅 Date
- September 29, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
HybridCUA trains computer-use agents to combine GUI interaction with the command line, using a data pipeline that produces GUI-only, CLI-only, and interleaved trajectories (HybridCUA-8K), followed by supervised fine-tuning and reinforcement learning with CLI-aware rewards. HybridCUA-9B reaches 53.6% on OSWorld, 14.8 points above its base model.
Related papers (24)
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- Watch and Learn: Learning to Use Computers from Online VideosOctober 6, 2025 · CVPR 2026
- ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use AgentsAugust 19, 2025 · ICLR 2026 (Poster)
- CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAsSeptember 25, 2026 · arXiv
- WebArena-Infinity: Generating Browser Environments with Verifiable Tasks at ScaleMarch 2026 · Blog Post
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness EvolutionOctober 1, 2026 · arXiv
- DeskForge: Dense Supervision from Desktop Environments for Computer-Use AgentsOctober 1, 2026 · arXiv
- AutoGUIWorld: Image Generators as Visual World Models for GUI AgentOctober 1, 2026 · arXiv
- MintAct: A Unified Visual Agent for Digital EnvironmentsSeptember 18, 2026 · arXiv
- IntentScore: Intent-Conditioned Action Evaluation for Computer-Use AgentsApril 6, 2026 · arXiv
- EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic ExperienceJanuary 22, 2026 · arXiv
- ShowUI-π: Flow-based Generative Models as GUI Dexterous HandsDecember 31, 2025 · arXiv
- STEP: Success-Rate-Aware Trajectory-Efficient Policy OptimizationNovember 17, 2025 · Findings of ACL 2026
- Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data CurationSeptember 28, 2025 · arXiv
- OpenCUA: Open Foundations for Computer-Use AgentsAugust 12, 2025 · NeurIPS 2025 (Spotlight)
- CoAct-1: Computer-using Multi-Agent System with Coding ActionsAugust 5, 2025 · ICLR 2026 (Poster)
- DPO Learning with LLMs-Judge Signal for Computer Use AgentsJune 3, 2025 · arXiv
- Efficient Agent Training for Computer UseMay 20, 2025 · ICLR 2026 (Poster)
- STEVE: A Step Verification Pipeline for Computer-use Agent TrainingMarch 16, 2025 · arXiv
- SpiritSight Agent: Advanced GUI Agent with One LookMarch 5, 2025 · CVPR 2025 (Poster)
- OS-ATLAS: A Foundation Action Model for Generalist GUI AgentsOctober 30, 2024 · ICLR 2025 (Spotlight)
- GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement LearningOctober 13, 2026 · arXiv
- ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI AgentsApril 13, 2026 · arXiv