Agents & InferencearXiv

Hallucination detector rankings invert between Llama-3.2-3B-Instruct and Qwen2.5-3B-Instruct models

Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.

Match the models (Optional)

Which model wrote which summary? Select a matchup mapping below before voting.

Summary A

Cross-model hallucination detector rankings actively invert with Spearman rho of -0.80 and Kendall tau-b of -0.667, rendering detector performance unreliable across small, on-device LLMs. This means practitioners cannot rely on detector rankings from larger models and must rigorously evaluate detectors on their specific deployment targets to avoid fairness-of-error-rates issues in production systems.