Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents

This work examines hybrid computer-use agents that can choose screenshots or text tools on OSWorld-MCP. It identifies an adoption gap in tool use and shows that training with a compatible post-tool observation policy can reduce image-context cost while improving the reported operating point.

• EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning
  April 10, 2026 · arXiv
• Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents
  February 15, 2026 · arXiv
• MAI-UI Technical Report: Real-World Centric Foundation GUI Agents
  December 26, 2025 · arXiv
• The Amazing Agent Race: Strong Tool Users, Weak Navigators
  April 11, 2026 · arXiv
• The Tool Illusion: Rethinking Tool Use in Web Agents
  April 3, 2026 · arXiv
• LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios
  February 3, 2026 · arXiv
• ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents
  January 30, 2026 · arXiv
• MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments
  December 22, 2025 · arXiv
• A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistants
  August 31, 2025 · arXiv
• TurkingBench: A Challenge Benchmark for Web Agents
  March 18, 2024 · NAACL 2025 (Oral)
• OS-Wizard: Scaling Computer-Using Agents to Specialized Software via Knowledge-Guided Exploration
  October 24, 2026 · arXiv
• Evolving Agent Interfaces
  October 20, 2026 · arXiv
• CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents
  October 20, 2026 · arXiv
• VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards
  October 19, 2026 · arXiv
• Comparing Computer Use Behavior Across Multiple Levels of Abstraction
  October 19, 2026 · arXiv
• CUA-Lite: An Open Platform for Computer Use Agents
  October 19, 2026 · arXiv
• Agent World Model in the Wild: Computer-Use Agent Environment Synthesis
  October 19, 2026 · arXiv
• PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas
  October 16, 2026 · arXiv
• Adaptation of Computer Use Agents to New Software with User Manuals
  October 16, 2026 · arXiv
• OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use
  October 15, 2026 · arXiv
• OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
  October 15, 2026 · arXiv
• LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
  October 15, 2026 · arXiv
• Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution
  October 14, 2026 · arXiv
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
  October 14, 2026 · arXiv