RBS-Attention: Radius-Bounded Sparse Prefill for Long-Context Large Language Models
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
A training-free sparse-prefill method delivers 5.97× end-to-end time-to-first-token speedup at 128K context on H100s (11.92× on vLLM prefill attention) while losing under one point of RULER accuracy (88.65 vs 89.52 dense). If long prompts are killing your TTFT, this drops in without retraining and preserves standard block-sparse FlashAttention execution—its dual-branch "rescue" selection specifically avoids the recall collapse that naive centroid-based block sparsity causes when a critical token hides in a mostly-irrelevant block.
RBS-Attention achieves 5.97× faster end-to-end time-to-first-token for 128K contexts on Qwen3-30B with minimal accuracy loss (88.65 vs. 89.52 RULER), unlocking near-identical quality at production-scale long-context speeds. Engineers can now deploy 128K+ context models with flash-compatible sparse attention, eliminating prefill bottlenecks without requiring model retraining or specialized hardware.