Run a 428B Model on 150GB of Memory: Unsloth’s 2-Bit Quantization Preserves 91.5% Accuracy

MiniMax-M3.1-Flash can now run locally.

Unsloth released a GGUF quantized version on October 26: the 2-bit file is 143.9GB, retains 91.5% Top-1 accuracy, and is 45% smaller than the original. Unsloth lists the hardware requirement as 150GB of memory.

Background: MiniMax-M3.1-Flash is a 428B-parameter MoE (mixture-of-experts) multimodal model released by MiniMax this month, with 23B active parameters, a 1M context window, and targeting coding and Agent scenarios. MiniMax officially revealed that “Space Bunny,” which previously topped global leaderboards with 200T+ tokens, was an early version of it. The model weights were released on October 23.

Top-1 accuracy refers to the proportion of times the first token predicted by the quantized model matches the original model. 91.5% means the vast majority of outputs are no different from the original. Compressing the model to nearly half its size while still achieving this number is not easy.

In terms of performance, official benchmarks show Automation-Bench 62.2 points, DeepSWE 73.7 points, and BabyVision (with tools) 91.0 points, higher than peer-level DeepSeek-V4.1-Flash and Kimi-K3 on most items. API pricing is also aggressive: $0.15 per 1M input tokens, $0.60 output, and $0.015 for cached reads; the price is one-third of M3 and the speed is 3x.

The question becomes: without calling the API, can you run it yourself?

Unsloth’s answer is quantization. Quantization compresses model weights from high precision to low precision, reducing size at the cost of accuracy. M3.1 included QAT (quantization-aware training) and sparse attention during training, so even at 2-bit it can retain this accuracy—the model is naturally quantization-friendly.

Performance by quantization level:

The 8-bit version is a lossless conversion of the original safetensors. Unsloth’s quantization evaluation covers seven categories: coding, math, chat, tool calling, multilingual, reasoning, and Agent, to avoid overfitting to a single benchmark. On coding tasks, 2-bit still has 91.2% token agreement, and 4-bit reaches 97.6%.

Memory requirements: 1-bit 134GB, 2-bit 150GB, 3-bit 176-208GB, 4-bit 256GB, 8-bit 272GB. The deployment guide recommends reserving 160GB or more for 2-bit. Mac users with 128GB unified memory are just locked out.

There are two deployment paths. First, Unsloth Desktop: search for MiniMax-M3.1-Flash, download UD-Q2_K_XL, and run it with the recommended parameters. The command-line method is:

unsloth run --model unsloth/MiniMax-M3.1-Flash-GGUF:UD-Q2_K_XL \
    --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0 \
    --chat-template-kwargs '{"reasoning_effort":"high"}' \
    --host 127.0.0.1 --port 8080

Second, llama.cpp: compile PR #31192 branch, download the four shards, and run llama-server. Image input requires additionally downloading the 1.74GB vision projector mmproj-BF16.gguf. DSpark speculative decoding is also supported, but it requires loading an additional draft model.

The community’s reaction is realistic. One netizen said that 150GB of memory means adding a dedicated machine in the office, but for clients whose code cannot touch cloud APIs, this machine is already worth buying. Some asked whether a 64GB quantization could be released; others were concerned about how 2-bit performs in long-context Agent scenarios and on tool-calling JSON—after all, these scenarios do not only look at output token agreement.

A 428B-parameter model activates only 23B per token. MoE makes models bigger, and quantization lowers the barrier. For companies whose data cannot leave the internal network, the math already works.

Resources:

• GGUF: huggingface.co/unsloth/MiniMax-M3.1-Flash-GGUF
• Deployment guide: unsloth.ai/docs/models/minimax-m3.1-flash
• API docs: platform.minimax.io