TeleTune: Evolving Agent Skills From Offline Telemetry
Justin Chih-Yao Chen , Elias Stengel-Eskin , Yan Chen , Pol Llado , Scott Counts , Mohit Bansal , Benjamin Van Durme , Harsh Jhamtani , Gaurav Verma
- 🏛 Institutions
- UNC , Microsoft , UT Austin
- 📅 Date
- October 4, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Web Desktop
- 🔑 Keywords
TLDR
Introduces TeleTune, a framework that induces reusable textual skill libraries for computer-use agents from offline user telemetry logs without goal labels or replayability. Guided by action-prediction error reductions on logged trajectories, TeleTune achieves 77.1% on WorkArena and 80.6% on Online-Mind2Web, outperforming Agent Workflow Memory by 6.7-7.7% while consuming up to 75x fewer tokens than live optimization.
Related papers (24)
- Back to Basics: A Skill-Centric Approach to Generalist Web AgentsOctober 26, 2026 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill InductionOctober 9, 2026 · arXiv
- CUAWright: A Minimal Unified Interface for Digital AgentsOctober 2, 2026 · arXiv
- Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web AgentsOctober 14, 2026 · arXiv
- HyWe: Unifying Search and Operation for Real-World Hybrid Web TasksOctober 12, 2026 · arXiv
- Web$hell: A Unified Interface for Agentic Web AccessOctober 9, 2026 · arXiv
- A Baseline Assessment for the Agentic Web Technology StackOctober 9, 2026 · arXiv
- CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time ScalingOctober 5, 2026 · arXiv
- AMBER: Training Long-Horizon Web Agents through Append-Only MemoryOctober 5, 2026 · arXiv
- GUITAR: Structured Failure Diagnosis of GUI Agents via State TransitionsSeptember 28, 2026 · arXiv
- EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic DataSeptember 17, 2026 · arXiv
- From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use AgentsSeptember 4, 2026 · arXiv
- Enhancing Web Agents with a Hierarchical Memory TreeMarch 7, 2026 · arXiv
- CUA-Skill: Develop Skills for Computer Using AgentJanuary 28, 2026 · arXiv
- Inducing Programmatic Skills for Agentic TasksApril 9, 2025 · COLM 2025
- From Grounding to Planning: Benchmarking Bottlenecks in Web AgentsSeptember 3, 2024 · ECAI 2025
- WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?March 11, 2024 · ICML 2024
- Dual-View Visual Contextualization for Web NavigationFebruary 6, 2024 · CVPR 2024 (Poster)
- GPT-4V(ision) is a Generalist Web Agent, if GroundedJanuary 3, 2024 · ICML 2024
- CogAgent: A Visual Language Model for GUI AgentsDecember 14, 2023 · CVPR 2024 (Highlight)
- Mind2Web: Towards a Generalist Agent for the WebJune 9, 2023 · NeurIPS 2023 Datasets and Benchmarks Track
- Towards Safe Autonomy for Computer UseOctober 13, 2026 · arXiv
- Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon AgentsOctober 13, 2026 · arXiv