BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Across seven LLMs from 3B to 70B, larger models got better at accumulating evidence and inferring latent variables, but that improvement did not reliably translate into better downstream predictions. For production agents, this means a model may appear to “understand” the hidden state in a conversation yet still update decisions or forecasts non-Bayesianly, so multi-turn evaluation should score belief trajectories and action-relevant predictions, not just final answers.
Larger models (tested up to 70B) can approximate a rational Bayesian posterior when accumulating evidence over multi-turn conversations, but that improved latent inference does not translate into accurate downstream predictions—models infer the hidden state correctly yet fail to use it to update forecasts about the target outcome. If you're running multi-turn agents or assistants that track state across a conversation, don't assume correct internal belief tracking yields correct predictions or actions; evaluate the prediction step separately rather than trusting final-turn answers as a proxy for calibrated reasoning.