Dual-View Visual Contextualization for Web Navigation

This paper contextualizes each HTML element with its corresponding screenshot region and nearby elements, combining textual and visual features to represent webpage elements more informatively. It evaluates the approach on Mind2Web and reports consistent gains in cross-task, cross-website, and cross-domain settings.

• Seg2Click: Learning Pixel-Level Action Masks for GUI Visual Grounding
  October 12, 2026 · arXiv
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
  December 13, 2024 · arXiv
• Visual Grounding for User Interfaces
  June 16, 2024 · NAACL 2024 Industry Track
• Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web Agents
  October 14, 2026 · arXiv
• STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI Agents
  October 14, 2026 · arXiv
• MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search
  October 13, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks
  October 12, 2026 · arXiv
• When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks
  October 9, 2026 · arXiv
• Web$hell: A Unified Interface for Agentic Web Access
  October 9, 2026 · arXiv
• ConClick: Element-Wise Contrastive Learning Improves GUI Grounding
  October 9, 2026 · arXiv
• A Baseline Assessment for the Agentic Web Technology Stack
  October 9, 2026 · arXiv
• Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution
  October 7, 2026 · arXiv
• Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent Training
  October 5, 2026 · arXiv
• CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling
  October 5, 2026 · arXiv
• TeleTune: Evolving Agent Skills From Offline Telemetry
  October 4, 2026 · arXiv
• Recurrent Latent Visual Search for GUI Grounding
  October 4, 2026 · arXiv
• CUAWright: A Minimal Unified Interface for Digital Agents
  October 2, 2026 · arXiv
• GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions
  September 28, 2026 · arXiv
• DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis
  September 26, 2026 · arXiv
• MintAct: A Unified Visual Agent for Digital Environments
  September 18, 2026 · arXiv
• Lexical Coupling in GUI Element Grounding: Sentence Embeddings Track Labels across Mobile and Web
  August 22, 2026 · arXiv
• Enhancing Web Agents with a Hierarchical Memory Tree
  March 7, 2026 · arXiv
• OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding
  February 25, 2026 · arXiv