Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale

Tianyue Ou , Frank F. Xu , Aman Madaan , Jiarui Liu , Robert Lo , Abishek Sridhar , Sudipta Sengupta , Dan Roth , Graham Neubig , Shuyan Zhou

🏛 Institutions: CMU , Amazon AWS AI , xAI
📅 Date: September 24, 2024
📑 Publisher: NeurIPS 2024
💻 Env: Web
🔑 Keywords: dataset synthetic demonstrations tutorial-to-demo synthesis indirect knowledge Synatra

TLDR

Synatra turns indirect knowledge sources such as online tutorials into direct demonstrations for digital agents and uses 100k such synthetic demonstrations to train a 7B web agent. The paper reports stronger results than comparably sized models on Mind2Web, MiniWoB++, and WebArena, while synthetic demonstrations cost about 3% as much as human-collected ones.

Open paper Report issue