RENDER benchmark shows memory formats boost LLM accuracy up to 72.6 points
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Matched-budget “resolved packets” beat recency-truncated raw dialogue by 42.4–72.6 points on 500 LongMemEval questions across nine models, and deployed-style memory templates varied by 24.6–48.8 points within the same model. For production RAG/memory systems, the format you feed the model—summary, typed record, ChatGPT-style memory entry, or raw transcript—is not a neutral implementation detail; it can dominate measured quality, so evaluations must lock or report the reader-facing artifact before comparing retrievers, memory stores, or models.
LLMs perform significantly better when memory inputs are structured (e.g., ChatGPT-style entries) rather than raw dialogue, with matched-budget resolved packets outperforming raw dialogue by 42.4-72.6 points. This highlights the critical importance of memory/RAG system design, as the way information is presented to the model can dramatically impact accuracy, making it essential to optimize the reader-facing artifact for production use.