GPT-6-astra leads LHTB 2.0 benchmark with 0.69 dense mean reward
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
LHTB 2.0 expands long-horizon terminal evaluation to 77 tasks across 13 domains, with three-hour runs and 30-minute verifier checkpoints. For production agent teams, the key shift is that dense progress scoring exposes execution reliability that binary pass/fail hides: on reconstruction tasks, models averaged 0.653 dense reward but only 0.221 binary success, so this benchmark is more useful for diagnosing where agents stall during extended workflows rather than just whether they eventually finish.
On reconstruction tasks, binary pass/fail verifiers score 0.221 while dense progress ladders score 0.653 — meaning outcome-only evals are throwing away roughly two-thirds of the real agent progress on long-horizon work. If you're measuring agents over multi-hour workflows with a final-state check, you're massively underestimating capability and getting no signal on where runs actually stall; adopt checkpointed process rewards (30-min verifier intervals over a 3-hour horizon) to see sustained-progress regressions your binary gates hide.