Software Engineering for and with GUI Agent

This survey reviews 336 GUI-agent papers through a software-engineering lens, covering architectures, evaluation, lifecycle concerns, and deployment gaps. It identifies recurring weaknesses in recovery, safety enforcement, auditability, testing, and maintenance.

• Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
  September 2, 2026 · arXiv
• FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair
  March 18, 2026 · arXiv
• Where Not to Learn: Prior-Aligned Training with Subset-based Attribution Constraints for Reliable Decision-Making
  January 30, 2026 · arXiv
• A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?
  May 16, 2025 · arXiv
• A Survey on GUI Agents with Foundation Models Enhanced by Reinforcement Learning
  April 29, 2025 · arXiv
• Towards Trustworthy GUI Agents: A Survey
  March 30, 2025 · arXiv
• OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use
  December 20, 2024 · ACL 2025
• GUI Agents: A Survey
  December 18, 2024 · Findings of ACL 2025
• GUI Agents with Foundation Models: A Comprehensive Survey
  November 7, 2024 · arXiv
• CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering
  September 26, 2026 · arXiv
• LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects
  April 28, 2025 · TMLR 2025
• A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models
  March 30, 2025 · KDD 2025
• Programming with Pixels: Can Computer-Use Agents do Software Engineering?
  February 24, 2025 · ICLR 2026 (Poster)
• Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms
  November 17, 2024 · arXiv
• Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
  October 29, 2026 · arXiv
• Right Click, Wrong Box: Grading GUI Grounding by What a Click Does
  October 27, 2026 · arXiv
• Learning to Look Beyond Screens: Transferable Visual Observation from GUI Grounding
  October 27, 2026 · arXiv
• From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents
  October 27, 2026 · arXiv
• AdaGround: Token-Normalized Adaptive Zooming with Verifier-in-the-Loop Refinement for GUI Grounding
  October 27, 2026 · arXiv
• GUIFlow: Temporal Redundancy-Aware Incremental Inference for GUI Agents
  October 26, 2026 · arXiv
• PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
  October 24, 2026 · arXiv
• Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing
  October 24, 2026 · arXiv
• IGG: A Benchmark for Interactive GUI Grounding under Visibility Constraints
  October 23, 2026 · arXiv
• Correct Answers, Closed Gates: SolveBind for GUI Agents at the Verification Barrier
  October 23, 2026 · arXiv