GPT-4V(ision) is a Generalist Web Agent, if Grounded

SeeAct studies GPT-4V as a generalist web agent and adds an online evaluation setup for running agents on live websites. It shows that GPT-4V is strong when grounding is handled manually, and identifies grounding as the main remaining bottleneck.

• Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web Agents
  October 14, 2026 · arXiv
• Step-Aligned Script Generation with Verification and Recovery for Web Automation Testing
  October 19, 2026 · arXiv
• What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics
  October 15, 2026 · arXiv
• Why Generate a Click? Plug-in Decision Heads for Web Agents
  October 14, 2026 · arXiv
• Learning How to Remember: Memory Modality Selection for GUI Agents
  October 13, 2026 · arXiv
• HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks
  October 12, 2026 · arXiv
• Web$hell: A Unified Interface for Agentic Web Access
  October 9, 2026 · arXiv
• CaptchaAgent: Verifiable Visual Reasoning for CAPTCHA Solving
  October 9, 2026 · arXiv
• A Baseline Assessment for the Agentic Web Technology Stack
  October 9, 2026 · arXiv
• CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling
  October 5, 2026 · arXiv
• TeleTune: Evolving Agent Skills From Offline Telemetry
  October 4, 2026 · arXiv
• CUAWright: A Minimal Unified Interface for Digital Agents
  October 2, 2026 · arXiv
• GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions
  September 28, 2026 · arXiv
• AutoTailor: Automatic, User-Aligned Capability Selection and Adaptation for Web Agents
  September 11, 2026 · arXiv
• Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective
  March 15, 2026 · arXiv
• Enhancing Web Agents with a Hierarchical Memory Tree
  March 7, 2026 · arXiv
• OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding
  February 25, 2026 · arXiv
• ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution
  January 12, 2026 · arXiv
• WebATLAS: An LLM Agent with Experience-Driven Memory and Action Simulation
  October 26, 2025 · NeurIPS 2025 Workshop on Language Agents and World Models
• Surfer 2: The Next Generation of Cross-Platform Computer Use Agents
  October 22, 2025 · arXiv
• PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction
  October 17, 2025 · arXiv
• CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation
  April 30, 2025 · NAACL 2025 (System Demonstrations)
• Infogent: An Agent-Based Framework for Web Information Aggregation
  April 29, 2025 · Findings of NAACL 2025
• SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
  April 9, 2025 · arXiv