The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
In live, continuously-running environments, the smarter-but-slower model loses to a faster model because latency makes correct actions arrive too late—so inference speed, not just reasoning quality, determines task success. The practical payoff: a slow "teacher" can precompute reusable instructions offline that a fast "student" executes with zero teacher calls at runtime, roughly doubling scores (13.5→24.2) and transferring to other models without retuning—meaning you decouple deliberation from the hot path rather than paying reasoning latency per action.
GPT-6.1-sol beats faster Jev when the environment is paused, but loses when the game keeps running during inference. For production agents in live systems, raw reasoning quality is not enough: move expensive deliberation into offline/async teacher loops and deploy fast distilled policies or instructions at runtime, as TORSI raised Jev from 13.51 to 24.19 across all six game types without teacher calls during gameplay.