Harness Evolution Hits a Ceiling: When Weight Training Should Begin
Yuan Tian , Bing Hu , Hao Wang , Binghang Lu , Fang Wu
- 🏛 Institutions
- Independent Researcher , Purdue University , UC Berkeley , Stanford
- 📅 Date
- October 8, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Web
- 🔑 Keywords
TLDR
Analyzes the trade-off between self-evolving agent harnesses and weight training for long-horizon agents, showing that harness evolution fixes process failures (loops, budget exhaustion) while weight training is necessary for content failures. Validates the diagnosis rule across DeepPlanning and WebArena-Lite (+0.09 on 117 unseen tasks).
Related papers (24)
- Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and TrajectoriesOctober 1, 2026 · NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
- From Trajectories to Grounded Preferences: Process Preference Synthesis via Interaction Element Graphs for Web PRMsSeptember 26, 2026 · arXiv
- ComponentBench: Diagnosing Component-Level Failures in Computer-Use AgentsAugust 18, 2026 · COLM 2026
- Agentic Test-Time Scaling for WebAgentsFebruary 12, 2026 · arXiv
- WebATLAS: An LLM Agent with Experience-Driven Memory and Action SimulationOctober 26, 2025 · NeurIPS 2025 Workshop on Language Agents and World Models
- JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents AdaptationOctober 5, 2025 · arXiv
- WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement LearningMay 22, 2025 · EMNLP 2025 (Poster)
- Diagnosing Personalization Failures in Mobile GUI AgentsOctober 22, 2026 · arXiv
- What Does "Now" Mean to Your Computer-Use Agent? Stale Screenshots, Page Loads, and Which Check to Run Before a ClickOctober 29, 2026 · arXiv
- InterA11y: Interactive Web Accessibility Error Detection with LLM AgentsOctober 29, 2026 · arXiv
- AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access BarriersOctober 29, 2026 · arXiv
- WebAnything: From the Wild Web to Scalable Agent EnvironmentsOctober 27, 2026 · arXiv
- SkillLifeBench: Benchmarking Security Across the Lifecycle of Agent SkillsOctober 26, 2026 · arXiv
- Back to Basics: A Skill-Centric Approach to Generalist Web AgentsOctober 26, 2026 · arXiv
- AgentProofBench: Benchmarking Reusable Verifiers for Web AgentsOctober 25, 2026 · arXiv
- SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web AgentsOctober 24, 2026 · arXiv
- Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option MemoryOctober 24, 2026 · arXiv
- Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web AgentsOctober 22, 2026 · arXiv
- Step-Aligned Script Generation with Verification and Recovery for Web Automation TestingOctober 19, 2026 · arXiv
- LoWAgent: A Low-Resource Multilingual Benchmark for Web AgentsOctober 19, 2026 · arXiv
- SPIDES: Information-Flow Control for Browser AgentsOctober 16, 2026 · arXiv
- What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web SemanticsOctober 15, 2026 · arXiv
- Why Generate a Click? Plug-in Decision Heads for Web AgentsOctober 14, 2026 · arXiv
- Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web AgentsOctober 14, 2026 · arXiv