Qwen3.5-9B
4bit / Q4_K_M
- Starting memory estimate
- 16 GB
For trying short tasks one at a time
9B 4-bit weights alone are about 4.5GB theoretically. Start with 16GB for short text.
Summarize PDFs and documents, find evidence in local files, and compare workflows for transcripts and meeting notes.
4bit / Q4_K_M
For trying short tasks one at a time
9B 4-bit weights alone are about 4.5GB theoretically. Start with 16GB for short text.
Summarize short documents, extract fields and organize meeting decisions. Start with one document converted to text.
PDF parsing is not automatic or guaranteed. Scans need OCR; cross-document questions need retrieval. Require source passages and check numbers and names.
Memory figures are editorial budgets including room for the OS and apps, not verified minimums. Text models assume 4-bit weights, 4K–8K input tokens and one model at a time. Long documents, images and concurrent apps need more. A3B does not mean the entire model fits in 3GB.
Assumes an Apple Silicon Mac and a compatible app: LM Studio or similar for text, Whisper apps for audio, and model-specific image/video tools. RAM alone does not determine speed. Start with a short input on your current Mac.
Short-context reading and information lookup are valued.
Multi-step workflows can accumulate errors. Evaluate document understanding separately from subsequent automation.
A qualitative synthesis of public discussions, not a representative survey. Reports include PCs and cloud setups, so they do not establish Mac speed or memory requirements. No individual profiles or anecdotes are reproduced.
On 2026-11-17, qwen3.8:27b on a Mac Studio (M5 Ultra, 96GB) answered five questions about one set of meeting minutes. It correctly declined to answer about an ad budget the minutes left undecided, got all five right, and took about 14 seconds. Only one short set of minutes has been tried; longer documents, reading PDFs directly, comparisons with other models, and the amount of memory used have not been measured yet.
Qwen3.5 9B · gpt-oss 20B · Gemma 4 26B · Qwen3.8 27B · Qwen3-Embedding 0.6B · Whisper
Check omissions, correct citations, abstention when there is no answer, and elapsed time on the same files. For audio, also count mistakes about decisions and assignees.
Use ordinary text search as a baseline. Keep retrieval, answering and speaker separation distinct. Candidate results are pending.
On 2026-11-07, Qwen3.6 35B-A3B (digest a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c) ran once on Mac Studio No. 2 (M5 Ultra, 96GB). It got 1 of 5 fixed document questions correct (accuracy 0.2). Only q1 returned readable JSON; q2 to q5 were cut off at the 512-token output limit and could not be read as JSON. Inference took 13.79 seconds; peak memory was not measured. Settings: temperature 0, seed 42, num_ctx 4096, num_predict 512. Input SHA-256: dfef80964941c6213f43d5d23dab07e94a21df7c2cedefeb1e095b901793547f. Run ID: cloud-97312596-b9ec-4969-a8f0-4cd8f5437865. Reruns under the same conditions, comparisons with a higher output limit, peak memory, and other models or languages have not been measured. Next, test under fixed conditions that separate output-limit cuts from answer accuracy.
Run: 20261108T194037-mac-llm-bench-f7f9 · Input SHA-256: dfef80964941c6213f43d5d23dab07e94a21df7c2cedefeb1e095b901793547f
Mac Studio No. 2 (96 GB); Qwen3.6 35B-A3B (digest a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c); temperature 0, seed 42, num_ctx 4096, num_predict 1024. Input SHA-256: dfef80964941c6213f43d5d23dab07e94a21df7c2cedefeb1e095b901793547f. fixture_commit 56209d577d033071f9d9773ee6fb1cfb1d0b7960.
Queue 20261108T194037-mac-llm-bench-f7f9. Independent grading scored 2/5 (accuracy 0.4) and failed. q1 and q2 returned readable correct JSON; q3 to q5 had format_error=invalid_json (done_reason=length, eval_count=1024). Adapter elapsed time was 21.861848 s. Peak memory is null with uninstrumented reason. This is not a passing result or proof for the full use case.
Cycle 1 (2026-11-07, same model and input, num_predict 512, run ID cloud-97312596-b9ec-4969-a8f0-4cd8f5437865, independent grade 1/5) remains as a separate attempt.
Raising num_predict to 1024 still cut off q3 to q5 at length, so the next cycle should measure under fixed settings that separate thinking tokens from the output JSON budget. Peak memory and other models or languages remain unmeasured.
Run: cloud-ff2bcee4-d23d-4c2a-9014-b3a92f239092 · Input SHA-256: dfef80964941c6213f43d5d23dab07e94a21df7c2cedefeb1e095b901793547f · fixture_commit: a3853f7394f07873040633633094823eccf27627
Mac Studio 1 · qwen3.8:27b · Ollama 0.34.3 · digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813 · temperature 0, seed 42, num_ctx 4096, num_predict 1024
On 2026-11-14, the fixed documents-qa-001 case ran once. Adapter elapsed time: 7.940749 seconds. All 5/5 response fields were empty strings with format_error=invalid_json and done_reason=stop. Independent grading failed: 0/5 (accuracy 0.0, passed=false). Peak memory is null (unmeasured because the adapter is uninstrumented). No rerun or manual repair was used to turn this into a success. This is not a Studio 2 measurement.
Cycle 1 (2026-11-07, num_predict 512, 1/5) and cycle 2 (2026-11-08, num_predict 1024, 2/5) remain separate attempts with Qwen3.6 35B-A3B on Mac Studio 2. The input SHA-256 is unchanged, but the machine and model differ, so this does not demonstrate a speed improvement. Unmeasured items and next steps in earlier sections describe those attempts at the time. The use case remains held, not a passing result.
The cause of the empty response is unverified. Next, investigate the separation of thinking output and answer JSON, and design a fixed-condition comparison. Peak-memory measurement and multilingual comparisons remain outstanding.
Run: 20261117T030017-honbu-local-first-5957 · Mini-queue: 20261117T030017-mac-llm-bench-1dd7 · Input SHA-256: dfef80964941c6213f43d5d23dab07e94a21df7c2cedefeb1e095b901793547f
Mac Studio 2 · qwen3.8:27b · Ollama 0.40.16 · digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813 · temperature 0, seed 42, num_ctx 4096, num_predict 1024
On 2026-11-17, the fixed documents-qa-001 case ran once on Mac Studio 2. Run ID: 20261117T030017-honbu-local-first-5957; mini-queue ID: 20261117T030017-mac-llm-bench-1dd7. qwen3.8:27b (Ollama 0.40.16, digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813), temperature 0, seed 42, num_ctx 4096, num_predict 1024. All five questions received responses; independent grading passed 5/5 (accuracy 1.0, passed=true). Elapsed time was 14.0 s (12.970658 s measured by the adapter). No manual fixes. Peak memory is unmeasured (null) because it is not instrumented. This is one case, not a pass for the full use case or results for other machines or languages.
The use-case status remains held. Next, check evidence matching and abstention on another document task under fixed conditions, and measure peak memory. Other models and languages remain unmeasured.
Run: 20261117T065546-mac-llm-bench-ec93 · Mini-queue: 20261117T065546-mac-llm-bench-ec93 · Input SHA-256: 5b79b558aa5b13fd69c7b52a34d7f4604885d66a8f5e72943ebfd433ce753835
Mac Studio 2 · qwen3.8:27b · Ollama 0.40.16 · digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813 · temperature 0, seed 42, num_ctx 4096, num_predict 1024
On 2026-11-17, the fixed documents-minutes-001 (minutes) case ran once on Mac Studio 2. Run ID: 20261117T065546-mac-llm-bench-ec93; mini-queue ID: 20261117T065546-mac-llm-bench-ec93. qwen3.8:27b (Ollama 0.40.16, digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813), temperature 0, seed 42, num_ctx 4096, num_predict 1024. All five questions received responses; the undecided advertising budget (q4) was correctly withheld (answerable: false, empty answer). Independent grading passed 5/5 (accuracy 1.0, passed=true). Elapsed time was 14.3 s (12.995193 s measured by the adapter). No manual fixes. Peak memory is unmeasured (null) because it is not instrumented. This is one case, not a pass for the full use case.
The use-case status remains held. Next steps require direct extraction from longer documents or PDFs, multi-model comparisons, and peak-memory instrumentation/measurement. Multilingual and other machines remain unmeasured.