My dictation app was holding 1.9GB of memory to produce a transcript that a 181MB model produced identically. I only noticed because I ran out of room for something else.
Update, 2026-09-08. After two weeks of daily use I had the impression that small was a bit weaker on Chinese than the big model, which would undercut the conclusion below. So I recorded a second, harder clip and re-ran everything. The result is in the new section at the end. Short version: the "identical transcript" claim held for the first clip only, and on the second clip the big model was the one that dropped a word. One clip was never enough to say "no reason to exist", and I have softened that sentence.
The setup
PushSense runs whisper.cpp in-process with Metal. The app itself is 5.5 MB — a number I was quietly proud of, since the first version was Electron and weighed 200 MB.
The model was ggml-large-v3-turbo.bin: 1.6GB on disk. I picked it the way most people pick a model — it was the biggest one that was still fast, so presumably the best.
The same machine runs a 35B-parameter LLM locally, which eats 20–30GB. Those two facts didn't collide until I started paying attention to what stayed resident while I wasn't dictating.
Measuring instead of assuming
The first number that mattered wasn't file size. It was peak resident memory:
/usr/bin/time -l whisper-cli -m ggml-large-v3-turbo.bin -f sample.wav -l zh
→ 1,981,022,208 maximum resident set size1.98GB peak — 400MB more than the file itself, because inference allocates intermediate tensors on top of the weights.
Then I recorded 27.7 seconds of myself talking the way I actually talk — Mandarin with English work vocabulary mixed in, no language switching:
所以今天我们大概来讨论一下现在这个成果⋯⋯我很多时候都没有把 email 发给客户,客户有时候很多 customer support 他们都会来找我,所以其实真的很多时候你也应该去 follow up 的
Four English terms embedded in Mandarin: ready, email, customer support, follow up. That's the actual job.
Every model, same recording:
| Model | File | Peak RAM | Speed | Result |
|---|---|---|---|---|
| base | 141MB | 346MB | 0.47s | ✗ "custom support", "e-mail" |
| small-q5_1 | 181MB | 484MB | 1.11s | ✓ all four correct |
| turbo-q5_0 | 547MB | 695MB | 0.48s | ✓ all four correct |
| large-v3-turbo | 1,549MB | 1,889MB | 0.91s | ✓ all four correct |
| Breeze-ASR-25 q5_k | 1,031MB | 1,985MB | 1.49s | ✓ all four correct |
On this clip, the 181MB model and the 1.6GB model produced the same transcript. So did the 547MB one, character for character. (That stopped being true on a second clip, see the update below.)
The quality cliff is real, but it sits somewhere I didn't expect: between base and small, not between small and large. Base hears "customer support" as "custom support" and "email" as "e-mail". Small doesn't miss.
Why the memory is worse than it looks
I assumed the weights were memory-mapped, which would have made this a non-issue — mapped file pages are evictable, so the kernel reclaims them under pressure.
They aren't. whisper.cpp loads through std::ifstream into ggml_backend_alloc_ctx_tensors. I checked the source: zero occurrences of mmap across 9,293 lines, and issue #631 requesting it is still open.
So that 1.4GB is anonymous dirty memory. The kernel can't drop it. It comes straight out of whatever else you're running — in my case, a language model I care about more than my dictation tool.
Load-on-demand was the obvious escape hatch, so I measured that too: calling whisper_free does not return you to baseline. In-process footprint went 20MB → 1,774MB → 177MB. The allocator keeps a residue that scales with model size. Reloading is cheap (55ms for small-q5_1), so load-on-demand still works — just budget 80–180MB idle, not zero.
Two things I expected to work, and didn't
A model fine-tuned for exactly my problem. Whisper is architecturally bad at code-switching — it assigns one language token per 30-second window, and zero-shot Whisper posts 58.3% / 73.5% MER on the SEAME Mandarin-English benchmark. Breeze-ASR-25 is a Whisper-large-v2 fine-tune aimed straight at that, and on TTS-generated audio it looked like a clear win: it got "race condition" where turbo returned "Array's Condition", and "log" where turbo returned "Law".
On my actual voice, both models got all four terms right — and Breeze peaked at 1,985MB, higher than turbo, because quantization shrinks the file, not the runtime.
The lesson isn't about Breeze. It's that TTS is a bad proxy for speech. Synthesized audio enunciates jargon far more cleanly than a person does, which makes hard cases look easy and clean A/Bs look decisive. My tidy comparison was measuring the text-to-speech engine.
Apple's built-in recognizer. macOS ships SpeechTranscriber with on-device Chinese already installed — zero download, ~6MB in-process, 0.13s latency. On paper it deletes this entire problem.
It transcribed latency as "lentrensie", race condition as "raise fondition", and pull request as "pol reques" — deterministically, across three runs. Roughly 52% accuracy on developer terminology against whisper's 84%.
Zero memory is a great price for something you can't use.
What I changed
One line, effectively — the recommended model in the catalog:
static let recommended = all.first { $0.id == "ggml-small-q5_1" }!Same transcript. 484MB instead of 1,889MB. 1.4GB back for the thing I actually wanted the memory for.
Turbo stays in the list as q5_0 (547MB, not 1.6GB) for anyone who wants headroom on difficult audio. The 1.6GB f16 build is three times the memory of the q5_0 file, and on both clips I have measured so far the two produced the same words. (Original wording was "has no reason to exist". One clip does not support a sentence like that.)
The part I keep relearning
I chose a 1.6GB model because bigger sounded safer, and never checked. The check took twenty minutes and returned 1.4GB.
Every number above is from my own machine, against my own voice — which is the only reason I trust any of them. The one time I used synthetic audio, it pointed confidently at the wrong conclusion.
Caveats, stated plainly: the 84% / 52% jargon figures come from five synthesized clips (25 terms), so treat them as directional, not precise. The real recording contained business vocabulary, not the harder programming jargon (SwiftUI, rebase, memory leak) I still want to test. And Apple's customizedLanguage(modelConfiguration:) accepts a custom vocabulary, which could close its gap — I haven't tried it.
Update: a second, harder clip (2026-09-08)
The caveat above said the real recording had business vocabulary, not programming jargon. Two weeks of using small as the default left me with a nagging feeling that Chinese came out a little rougher than it had with turbo. Feelings are not measurements, so I recorded 36 seconds of the kind of thing I actually say to an agent: a complaint about a website homepage, in Mandarin, with famous, designer, website, SwiftUI dropped in, ending with an English sentence. Same machine, same whisper-cli, greedy decoding like the app uses, language set to Chinese.
| Model | Peak RAM | Time | Result |
|---|---|---|---|
| base | 363MB | 0.94s | ✗ "designer" → "迪士兰尔", "有没有" → "没有"; keeps "Swift UI" |
| small-q5_1 | 503MB | 0.77s | ✓ every content word, including "swift ui"; keeps every filler (那个, 来吧来来, 呃, "you you're") |
| turbo-q5_0 | 832MB | 1.60s | ✗ drops "SwiftUI" entirely: "具体代码用来吧,来测试一下"; fewer fillers |
| large-v3-turbo | 1,876MB | 2.01s | ✗ drops "SwiftUI" entirely: "具体代码用来测试一下"; cleanest output otherwise |
Beam search (-bs 5) and auto language detection did not change any of the four outputs.
So the transcripts are no longer identical, but the difference runs the other way from what I expected. Both turbo builds silently deleted the one programming term in the clip. Small kept it. What turbo did better was cosmetic: it dropped my stutters and filler syllables, which makes its output read cleaner. I suspect that is exactly what my two weeks of "small feels rougher" was picking up on. A transcript that includes every 那个 and 呃 looks worse than one that quietly omits a word you needed.
Two clips is still two clips. What I can say honestly now: on a business clip, small and turbo matched; on a developer clip, small was more faithful and turbo was more fluent. Neither clip shows small losing Chinese characters. If I find a clip where it does, it goes here. The default stays on small for now, and filler cleanup is a job for the optional LLM pass, not for a bigger acoustic model.
PushSense is free. A signed, notarized build is available to download.