Focused Omni-Modal GUI Agents

Formulates Focused Perception to mitigate Sequential Distraction in omni-modal GUI agents by adaptively pruning transient streaming context across screenshots, audio, video, and device states to maintain action-sufficient representation.

• Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
  October 29, 2026 · arXiv
• From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents
  October 27, 2026 · arXiv
• ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI Agents
  October 19, 2026 · arXiv
• Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents
  August 28, 2026 · arXiv
• LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent
  January 26, 2026 · ICLR 2026 (Poster)
• Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
  April 1, 2025 · COLM 2025
• SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models
  May 30, 2023 · NeurIPS 2023
• Subgoal-aware Memory Graph for GUI Agent
  October 22, 2026 · arXiv
• VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards
  October 19, 2026 · arXiv
• Step-Aligned Script Generation with Verification and Recovery for Web Automation Testing
  October 19, 2026 · arXiv
• Adaptation of Computer Use Agents to New Software with User Manuals
  October 16, 2026 · arXiv
• What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics
  October 15, 2026 · arXiv
• OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use
  October 15, 2026 · arXiv
• OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
  October 15, 2026 · arXiv
• Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI Navigation
  October 15, 2026 · arXiv
• LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
  October 15, 2026 · arXiv
• Why Generate a Click? Plug-in Decision Heads for Web Agents
  October 14, 2026 · arXiv
• Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web Agents
  October 14, 2026 · arXiv
• Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution
  October 14, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• Adaptive-Group Process Rewards for GUI Agents
  October 13, 2026 · arXiv
• A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution
  October 11, 2026 · arXiv
• Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
  October 9, 2026 · arXiv
• Latent World Action Model for GUI Agents
  October 9, 2026 · arXiv