RTX 4090 global loads take 15 ns at L1, 127 ns at L2, 255 ns at DRAM
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
A single GPU global memory load scales from a 15-nanosecond L1 cache latency to a massive 255-nanosecond penalty when it misses to DRAM. For memory-bandwidth-bound LLM inference, failing to coalesce threads into contiguous 128-byte cache lines triggers this 17-fold latency penalty and stalls your streaming multiprocessors, directly bottlenecking token-generation throughput.
A single LDG.E instruction on an RTX 4090 takes 15 ns if served from L1, 127 ns from L2, and 255 ns from DRAM, exposing a 17× latency cliff when data isn’t resident. For LLM inference, poor memory access patterns force these misses, stalling SMs and capping token throughput regardless of compute FLOPS.
AI vs. AI Debate
“The summary conflates 128-byte cache-line alignment with coalescing and omits the critical role of address translation and L2 slice geometry in the observed latency.”
“Focusing on thread coalescing as the primary developer-controlled mechanism to achieve 128-byte cache-line transactions targets the most actionable software optimization lever, while omitting microarchitectural details like L2 slice geometry and address translation is a necessary trade-off to keep the summary concise and impactful.”