The 748B Model Squeezed Into 64GB of RAM: The Part Unsloth's Tweet Didn't Mention

Unsloth's HuggingFace repo now has the GGUF for DeepSeek-V4.1-Flash-1017, with the quantization algorithm upgraded to Dynamic V3.1 (preview). The model has 748B parameters, 1M context, and an MIT license.

The tweet circulating on social media is very excited: it takes off in 64GB of RAM, inference is 3x faster, and it sweeps every benchmark.

After reading through Unsloth's guide, all three of these numbers need footnotes.

Start with 64GB. The docs say, verbatim: devices with 128GB or more of RAM can run it locally; 64GB can also run it, but slower. The two numbers differ by a factor of two, and the tweet picked the smaller one to mention.

What makes it possible to run a 748B model on consumer hardware is the Engram mechanism. Of DeepSeek-V4.1's 748B, 196B is Engram conditional memory, an n-gram lookup table. Unsloth puts this table on an SSD and loads it on demand via mmap; the table itself is 110.6GB to 202.8GB and does not occupy RAM. What actually needs to stay resident is the remaining 552B backbone weights.

So file size and RAM requirement are two different things. The smallest UD-IQ1_S downloads at 233GB, but needs only 128-145GB of RAM, because the 110.6GB Engram table stays on the SSD.

The 3x speedup in the tweet also has context. The docs say: the combination of Unsloth Desktop plus llama.cpp is 3x faster than other CPU/VRAM/ngram repos. DSpark speculative decoding alone contributes 1.3 to 1.5x decoding speedup, requires downloading an additional drafter module of about 8GB, and the versions must match.

The benchmark section, though, is solid.

This chart shows the average across six benchmarks. UD-Q2_K_XL scores 87.49, only 0.19 points behind the 508.4GB Q8 reference version at 87.68, and even overtakes it on AIME25, 97.50 vs. 97.08.

More interesting is the indirect sparring between Unsloth and ISTA-DASLab. ISTA's GSQ-RCO quantized version claims in its README that all bit widths achieve 100% accuracy on AIME25. Unsloth reproduced it, running 30 questions x 16 samples for each quantization, and concluded that GSQ-RCO retains 94.64% of the baseline, not 100%.

Unsloth also responded to the claim that KLD can be gamed, comparing across multiple held-out datasets, and concluded that its own UD V3.1 outperforms GSQ-RCO in all test regions.

There is an easily overlooked side effect: the more aggressive the quantization, the longer the model thinks. Unsloth tested at 32K context and found that lower-bit quantizations have a noticeably higher truncation rate. The docs recommend giving lower-bit versions a larger reasoning budget.

Hardware requirements, summarized:

There are two ways to run it. Unsloth Desktop is a GUI: searching for models, downloading, tuning parameters, and connecting tools all happen in one window. On the command line, use the llama.cpp PR they maintain (#30817); after building, add --load-mode mmap --lazy-mode on, and Engram loads from the SSD on demand.

The model page is here, and the full guide is here.

What's worth noting is not the 64GB number, but the combination of 1M context and the MIT license. Running a full 1-million-token context locally and throwing an entire codebase in at once--for teams whose data cannot leave the intranet, this is the scenario that can actually be deployed. As for whether 64GB can run it, the docs already make it clear: yes, slowly.

Don't be led astray by the tweet's exclamation points.