ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents

ToolTok treats GUI operations as paths over learnable tool tokens with semantic anchoring and curriculum learning. This makes GUI agents more efficient and generalizable, reaching competitive performance with far less training data than other post-training methods.

• CocoaBench: Evaluating Unified Digital Agents in the Wild
  April 13, 2026 · arXiv
• Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
  April 9, 2026 · arXiv
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation
  February 10, 2026 · arXiv
• iSHIFT: Lightweight Slow-Fast GUI Agent with Adaptive Perception
  December 26, 2025 · arXiv
• Visual Grounding for User Interfaces
  June 16, 2024 · NAACL 2024 Industry Track
• Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents
  August 4, 2026 · arXiv
• The Amazing Agent Race: Strong Tool Users, Weak Navigators
  April 11, 2026 · arXiv
• The Tool Illusion: Rethinking Tool Use in Web Agents
  April 3, 2026 · arXiv
• Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents
  February 15, 2026 · arXiv
• Learning with Challenges: Adaptive Difficulty-Aware Data Generation for Mobile GUI Agent Training
  January 30, 2026 · arXiv
• How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors
  January 29, 2026 · arXiv
• GUIPilot: A Consistency-Based Mobile GUI Testing Approach for Detecting Application-Specific Bugs
  June 9, 2025 · ISSTA 2025
• ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search
  May 21, 2025 · arXiv
• GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents
  April 14, 2025 · arXiv
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
  December 13, 2024 · arXiv
• Dual-View Visual Contextualization for Web Navigation
  February 6, 2024 · CVPR 2024 (Poster)
• VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets
  September 11, 2026 · arXiv
• TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents
  September 9, 2026 · arXiv
• FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?
  September 7, 2026 · arXiv
• Selective Knowledge Control for Continual GUI Agent Learning over Application Streams
  September 6, 2026 · arXiv
• AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents
  September 6, 2026 · arXiv
• From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents
  September 4, 2026 · arXiv
• Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents
  September 3, 2026 · arXiv
• Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization
  September 2, 2026 · arXiv