Naive Visual Memory is Not Enough: A Failure-Mode Study of GUI Agents

This paper studies how visual memory affects GUI agents by classifying failures into cognitive failure, visual state misunderstanding, hidden operation blindness, and grounding error. It finds that full-image memory can reduce state-level failures while worsening action-level failures such as hidden operation blindness and grounding errors.

• BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks
  October 2, 2025 · arXiv
• Demo2Tutorial: From Human Experience to Multimodal Software Tutorials
  June 2, 2026 · arXiv
• STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models
  June 1, 2026 · arXiv
• GUI-C²: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning
  May 29, 2026 · arXiv
• MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents
  May 18, 2026 · arXiv
• Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretraining
  May 14, 2026 · arXiv
• Executable Agentic Memory for GUI Agent
  May 12, 2026 · arXiv
• LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
  May 8, 2026 · arXiv
• Step-level Optimization for Efficient Computer-use Agents
  April 29, 2026 · arXiv
• Training Computer Use Agents to Assess the Usability of Graphical User Interfaces
  April 28, 2026 · arXiv
• AutoGUI-v2: A Comprehensive Multi-Modal GUI Functionality Understanding Benchmark
  April 27, 2026 · arXiv
• Human-Guided Harm Recovery for Computer Use Agents
  April 20, 2026 · arXiv
• UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding
  April 15, 2026 · arXiv
• GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models
  April 15, 2026 · arXiv
• See, Point, Refine: Multi-Turn Approach to GUI Grounding with Visual Feedback
  April 14, 2026 · arXiv
• CocoaBench: Evaluating Unified Digital Agents in the Wild
  April 13, 2026 · arXiv
• Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
  April 9, 2026 · arXiv
• What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning
  April 8, 2026 · Findings of ACL 2026
• MAESTRO: Adapting GUIs and Guiding Navigation with User Preferences in Conversational Agents with GUIs
  April 7, 2026 · arXiv
• GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosis
  April 6, 2026 · arXiv
• AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
  April 3, 2026 · arXiv
• Terminal Agents Suffice for Enterprise Automation
  March 31, 2026 · arXiv
• Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
  March 27, 2026 · CVPR 2026
• Rethinking Token Pruning for Historical Screenshots in GUI Visual Agents: Semantic, Spatial, and Temporal Perspectives
  March 27, 2026 · arXiv