LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles

Bingo Zhang , Haochuan Lu , Zongjie Li , Genjian Li , Ari Yu Zhang , Chaozheng Wang

🏛 Institutions: Vera Praxis , Tencent , HKUST , Independent Researcher , CUHK
📅 Date: September 28, 2026
📑 Publisher: arXiv
💻 Env: General GUI
🔑 Keywords: LongPuzzleBench benchmark long-horizon tasks puzzle games planning

TLDR

LongPuzzleBench evaluates GUI agents on 114 levels of six puzzle games played through native GUI actions, where a legal move can make the puzzle unsolvable several moves later. Seven of ten general-purpose agents solve nothing harder than Medium, and diagnostics show agents judge a move by its visible progress rather than the future options it leaves.

Open paper Code Report issue