Qwen3.5-9B
4bit / Q4_K_M
- Starting memory estimate
- 16 GB
For trying short tasks one at a time
9B 4-bit weights alone are about 4.5GB theoretically. Start with 16GB for short text.
Check publication criteria, route incoming messages, and decide whether a document is relevant to a question.
4bit / Q4_K_M
For trying short tasks one at a time
9B 4-bit weights alone are about 4.5GB theoretically. Start with 16GB for short text.
Classify messages or headlines and extract text into JSON. Define labels and schema; process small batches.
Allow an uncertain label. Validate both JSON syntax and field correctness; parseable output is not necessarily correct.
Memory figures are editorial budgets including room for the OS and apps, not verified minimums. Text models assume 4-bit weights, 4K–8K input tokens and one model at a time. Long documents, images and concurrent apps need more. A3B does not mean the entire model fits in 3GB.
Assumes an Apple Silicon Mac and a compatible app: LM Studio or similar for text, Whisper apps for audio, and model-specific image/video tools. RAM alone does not determine speed. Start with a short input on your current Mac.
Users find clear, simple instructions and information extraction useful.
Accurate descriptions do not necessarily translate into reliable file operations.
A qualitative synthesis of public discussions, not a representative survey. Reports include PCs and cloud setups, so they do not establish Mac speed or memory requirements. No individual profiles or anecdotes are reproduced.
On 2026-11-17, Qwen3.6 35B-A3B on a Mac Studio (M5 Ultra, 96GB) sorted eight inquiries (with its step-by-step thinking output turned off). It got 7 of 8 right but missed one that should have gone to a person for review, so it did not pass. It took about 3 seconds. Earlier attempts did not pass either. Next: improving that kind of judgment and comparing other models. The amount of memory used has not been measured yet.
Laya · Laya Multilingual · Laya-MLX · Qwen3.5 4B · Gemma 4
Use the same labeled data to measure false positives, misses, abstentions, speed and memory. Check how thresholds change the results.
Laya and small language models are candidates. Mac execution, thresholds and accuracy by language still need checking.
On 2026-11-07, MiniCPM5-2B (Q4_K_M, digest e680c6c9f9033490cb68df7c1145f733a3a1b377b387af0937251a70d3c71a14) ran once on a Mac mini. It got 4 of 8 fixed cases correct (accuracy 0.5); i001, i004, i005 and i006 were mismatches. Inference took 3.4 seconds (2.729335 seconds by the adapter); peak memory was not measured. Settings: temperature 0, seed 42, num_ctx 4096, num_predict 512. Input SHA-256: d5839ab5a81f9e0029e43db8c2939a6dd13609de6f1b94f7219e6197a54f01a3. Run ID: 20261107T111929-mac-llm-bench-dbaa. The initial record exited with code 1 because the grader received the wrong input location. After PR #53 fixed this, we regraded the same saved responses without inference; all six integrity checks passed and the 4/8 failure was confirmed. Fixing the runner does not mean classification quality passed. We did not rerun inference under the same conditions. Next, investigate the four mismatches and measure a proposed improvement separately under fixed conditions. Peak memory, abstention rates by threshold, and comparisons with other models or languages remain unmeasured.