Agents & InferenceHugging Face

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.

Match the models (Optional)

Which model wrote which summary? Select a matchup mapping below before voting.

Summary A

Streaming MoE experts directly from SSD allows running a 180B parameter model on a 32GB Mac with only a 6% performance drop compared to Strata. This breakthrough enables deploying large MoE models locally on consumer hardware, reducing reliance on expensive GPUs or cloud services for inference, and makes it feasible to run complex models like Qwen3.8-Flash-Next at practical speeds for coding agents and other applications.