A 105GB model on a 16GB Mac: running is one thing, being usable is another

A 105 GB model and a 16 GB Mac—put those two numbers together and it usually means failure. Slotstream’s approach is to leave most of the model on the SSD and read only the experts needed for the current token into memory. It runs Qwen3.8-Flash-Next, a 125B-parameter open-source MoE model whose original 4-bit weights are about 105 GB.

MoE means that generating each token only passes through a subset of the expert networks, so you don’t have to compute all 125B parameters. Slotstream keeps shared weights in memory, reads experts from the SSD into cache, and leaves frequently used experts in RAM. The model hasn’t gotten smaller; it just accepts the premise that it doesn’t fit in memory. It’s like leaving your wardrobe in the hallway and taking out one piece of clothing at a time: you can get dressed, but changing clothes isn’t free.

The official numbers provide a reproducible set: on October 5, 2026, on a 48 GB M5 Pro with a 33 GB memory budget and the original 4-bit model, median generation speed across three workloads was 20.91 tok/s for a short prompt, 23.69 tok/s for a longer context, and 21.59 tok/s for completing a coding answer. Each workload was run three times, with no swap during measurement. Note that tok/s is generation speed; loading the model and processing the prompt are counted separately. In the comments, Michał Piszczek makes a more precise point: under the README’s 10 GB target, the first read at 2K is about 14 to 28 seconds, a 16K prompt takes about 54 seconds, while the 20-something tok/s corresponds to a 33 GB budget. How you choose the budget directly determines whether you have the patience to wait for the first token.

Context is another hard constraint. In automatic mode, a 48 GB machine gets a 32,768-token window, 64 GB gets 131,072, and only 96 GB or more reaches 262,144. Some commenters say 32k is unusable for agents; whether that’s true depends on the task, but long context requires more memory and time—there’s no avoiding it. The official docs also state that only one generation can run at a time, and connected apps share the same model.

The counterintuitive part is Macs with 96 GB or more. When the model can fit entirely in memory, Slotstream actually doesn’t optimize for such devices; other engines that keep the model resident in memory will be faster. It targets 16 to 64 GB machines—Macs that couldn’t fit the model in the first place. Author Carlos Galarza says he initially just wanted to run this model on his own Mac, but the standard loader exhausted memory before producing the first response. The project started from that failure.

Installation isn’t complicated, provided you have Apple Silicon, macOS 14 or later, at least 16 GB of memory, and about 110 GB of free SSD space. It won’t start on an 8 GB machine, and Windows, Linux, and Intel Macs are not supported. The first command in the terminal is:

curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh
slotstream doctor
slotstream run --prompt 'Why is the sky blue?'

doctor only checks memory and disk; it doesn’t load the model. The first run asks whether to download; the download may take several hours and can resume after interruption. After that it works offline, with no Python and no cloud account. The entire engine is a native Swift program running on Apple’s MLX and Metal. It can also use slotstream launch claude to connect Claude Code to the local model; Codex, Pi, opencode, and Hermes work the same way. Developers can integrate their own apps through Ollama, OpenAI-compatible, or Anthropic-compatible APIs, or the Swift library. Some commenters say the highlight is a single Swift binary with no Python. For a local tool, having one fewer runtime layer makes installation and idle behavior much simpler.

SSD lifespan is the most frequently asked question in the comments. Qwen AI Fans’ reply is: it’s mainly reads, not a write storm; frequently used experts stay in the in-memory cache, so the entire model isn’t dragged off the SSD for every token. But they haven’t published wear data. System swap and session caching can still add writes; automatic memory planning can reduce pressure, but machines with little memory or manually oversized settings can still swap frequently. The official project hasn’t provided long-term testing on this, so whether to buy in depends on your own tolerance for risk.

My assessment: Slotstream solves whether you can run a model when memory is insufficient; it does not solve whether long context and high concurrency are cheap. It suits offline, privacy-sensitive, lightweight agent, and single-person tinkering scenarios. If you want to use a 48 GB Mac to run a coding agent with 32k context, first accept the wait for the first token and the window limit. Someone in the comments mentions that Sushi achieves 512k context on a 48 GB M5 Pro and is faster; the source material doesn’t provide reproducible data, so it can only be treated as a lead. The measurement records on GitHub leave failed experiments in place too, which is more credible than just posting a single benchmark chart. The address is carloslfu/slotstream.