Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
With 50 optimization iterations per kernel, Kernel Forge generated CUDA replacements that beat PyTorch eager on 14 kernels, including 2.83× softmax speedup on Gemma 4 E2B and 1.70× group_norm on Stable Diffusion 3.5 Medium. The practical shift is that agent-written kernels can now be tested against whole, unmodified PyTorch models rather than toy isolated ops, making this closer to a drop-in latency/cost optimization loop for production inference stacks.
Optimizing CUDA kernels with LLM-based agents now achieves up to $2.83\times$ speedup over PyTorch eager mode for specific operations like softmax in Gemma 4 E2B, reducing latency and cost for production models; this enables shipping faster and more cost-effective LLMs and other GPU-dependent workloads with less manual engineering effort.