CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
Prince Zizhuang Wang , Chenhao Liang , Zelong Xu , Aojie Yuan , Xiaolin Zhou , Haiyue Zhang , Yue Zhao , Xiyang Hu , Shuli Jiang
- 🏛 Institutions
- CMU , USC , University of Wisconsin-Madison , Arizona State University , AWS Agentic AI
- 📅 Date
- September 26, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
CUA-SWE is a benchmark, environment, and evaluation pipeline for software engineering with computer use, in which agents modify code, run commands, interact with the running application, and inspect visual feedback within one task. It spans four software engineering domains with deterministic task-specific tests, and includes tasks where required information is available only through the application's interface.
Related papers (24)
- EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI AgentsJanuary 25, 2026 · arXiv
- ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific WorkflowsMay 26, 2025 · ICLR 2026 (Poster)
- Programming with Pixels: Can Computer-Use Agents do Software Engineering?February 24, 2025 · ICLR 2026 (Poster)
- WebArena: A Realistic Web Environment for Building Autonomous AgentsJuly 25, 2023 · ICLR 2024 (Poster)
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsJuly 31, 2022 · NeurIPS 2022
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv
- PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse PersonasOctober 16, 2026 · arXiv
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software EvolutionOctober 11, 2026 · arXiv
- AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use TasksOctober 8, 2026 · arXiv
- PlaySuite: A Large-Scale Benchmark for Interactive Visual IntelligenceOctober 5, 2026 · arXiv
- MedImageOSWorld: Benchmarking GUI Agents for Medical Image ConsolesOctober 3, 2026 · arXiv
- Control OSWorld: An AI Control Environment for GUI Computer Use AgentsOctober 2, 2026 · arXiv
- Talk2Agent: Benchmarking Voice Interfaces for Text AgentsSeptember 30, 2026 · arXiv
- OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific SoftwareSeptember 30, 2026 · arXiv
- MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science WorkflowsSeptember 29, 2026 · arXiv
- EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?September 29, 2026 · arXiv
- OSWorld-Pro: Process-based Evaluation for Computer Use AgentsSeptember 21, 2026 · arXiv
- RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use AgentsSeptember 18, 2026 · arXiv
- CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided DesignSeptember 14, 2026 · arXiv
- CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real TimeSeptember 13, 2026 · arXiv
- JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task CompositionSeptember 9, 2026 · arXiv
- FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?September 7, 2026 · arXiv