Programming with Pixels: Can Computer-Use Agents do Software Engineering?

This paper introduces Programming with Pixels, a visual IDE environment for evaluating whether generalist computer-use agents can handle software engineering tasks rather than only simple desktop or web interactions. It also presents PwP-Bench, a benchmark spanning 15 software-engineering tasks across languages and modalities. The results show that purely visual computer-use agents lag behind specialist coding agents, but narrow text APIs such as file editing and bash dramatically narrow that gap.

• CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
  September 26, 2026 · arXiv
• CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents
  October 20, 2026 · arXiv
• CUA-Lite: An Open Platform for Computer Use Agents
  October 19, 2026 · arXiv
• PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas
  October 16, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv
• A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution
  October 11, 2026 · arXiv
• AgentHorizon: Evaluating Agentic Judges for Long-Horizon Computer-Use Tasks
  October 8, 2026 · arXiv
• PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence
  October 5, 2026 · arXiv
• MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles
  October 3, 2026 · arXiv
• Control OSWorld: An AI Control Environment for GUI Computer Use Agents
  October 2, 2026 · arXiv
• Talk2Agent: Benchmarking Voice Interfaces for Text Agents
  September 30, 2026 · arXiv
• OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
  September 30, 2026 · arXiv
• MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows
  September 29, 2026 · arXiv
• EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?
  September 29, 2026 · arXiv
• OSWorld-Pro: Process-based Evaluation for Computer Use Agents
  September 21, 2026 · arXiv
• RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
  September 18, 2026 · arXiv
• CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design
  September 14, 2026 · arXiv
• CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Time
  September 13, 2026 · arXiv
• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
  September 9, 2026 · arXiv
• FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?
  September 7, 2026 · arXiv
• ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices
  August 25, 2026 · arXiv
• CUADebug: Diagnosing and Repairing Computer-Use Agent Failures
  July 31, 2026 · arXiv
• OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
  July 30, 2026 · arXiv
• OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding
  July 29, 2026 · arXiv