MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
Tanmay Gupta , Piper Wolters , Zixian Ma , Peter Sushko , Rock Yuren Pang , Diego Llanes , Yue Yang , Taira Anderson , Boyuan Zheng , Zhongzheng Ren , Harsh Trivedi , Taylor Blanton , Caleb Ouellette , Winson Han , Ali Farhadi , Ranjay Krishna
- 🏛 Institutions
- AI2 , UW , UNC
- 📅 Date
- April 9, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Web
- 🔑 Keywords
TLDR
MolmoWeb is a family of fully open multimodal web agents (4B and 8B) trained on MolmoWebMix (100K+ synthetic trajectories and 30K+ human demonstrations). Operating as screenshot-only visual-language action policies without HTML or accessibility tree access, it achieves SOTA on WebVoyager, Online-Mind2Web, and DeepShop, outperforming larger closed models like GPT-4o.
Related papers (24)
- CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAsSeptember 25, 2026 · arXiv
- OpenCUA: Open Foundations for Computer-Use AgentsAugust 12, 2025 · NeurIPS 2025 (Spotlight)
- Web-Shepherd: Advancing PRMs for Reinforcing Web AgentsMay 21, 2025 · NeurIPS 2025 (Spotlight)
- SpiritSight Agent: Advanced GUI Agent with One LookMarch 5, 2025 · CVPR 2025 (Poster)
- ShowUI: One Vision-Language-Action Model for GUI Visual AgentNovember 26, 2024 · CVPR 2025 (Poster)
- OS-ATLAS: A Foundation Action Model for Generalist GUI AgentsOctober 30, 2024 · ICLR 2025 (Spotlight)
- CogAgent: A Visual Language Model for GUI AgentsDecember 14, 2023 · CVPR 2024 (Highlight)
- Multimodal Web Navigation with Instruction-Finetuned Foundation ModelsMay 19, 2023 · ICLR 2024
- DeskForge: Dense Supervision from Desktop Environments for Computer-Use AgentsOctober 1, 2026 · arXiv
- HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use AgentsSeptember 29, 2026 · arXiv
- SecAgent: Efficient Mobile GUI Agent with Semantic ContextMarch 9, 2026 · arXiv
- ShowUI-π: Flow-based Generative Models as GUI Dexterous HandsDecember 31, 2025 · arXiv
- UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI AgentsMay 27, 2025 · NeurIPS 2025 (Poster)
- Efficient Agent Training for Computer UseMay 20, 2025 · ICLR 2026 (Poster)
- STEVE: A Step Verification Pipeline for Computer-use Agent TrainingMarch 16, 2025 · arXiv
- Falcon-UI: Understanding GUI Before Following User InstructionsDecember 12, 2024 · arXiv
- Aguvis: Unified Pure Vision Agents for Autonomous GUI InteractionDecember 5, 2024 · ICML 2025 (Poster)
- MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI UnderstandingSeptember 23, 2024 · Findings of EMNLP 2024
- ScreenAI: A Vision-Language Model for UI and Infographics UnderstandingFebruary 7, 2024 · IJCAI 2024
- Spotlight: Mobile UI Understanding using Vision-Language Models with a FocusSeptember 29, 2022 · ICLR 2023 (Poster)
- Why Generate a Click? Plug-in Decision Heads for Web AgentsOctober 14, 2026 · arXiv
- From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMsSeptember 26, 2026 · arXiv
- Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory PredictionSeptember 22, 2026 · EMNLP 2026
- RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use AgentsSeptember 18, 2026 · arXiv