Decisions and classification

Check publication criteria, route incoming messages, and decide whether a document is relevant to a question.

Selection guide

Models by task

Start with

Qwen3.5-9B

4bit / Q4_K_M

Starting memory estimate
16 GB

For trying short tasks one at a time

9B 4-bit weights alone are about 4.5GB theoretically. Start with 16GB for short text.

Tasks to try

Classify messages or headlines and extract text into JSON. Define labels and schema; process small batches.

Keep under review

Allow an uncertain label. Validate both JSON syntax and field correctness; parseable output is not necessarily correct.

Setup and limits

Getting started · Budget rationale

Memory figures are editorial budgets including room for the OS and apps, not verified minimums. Text models assume 4-bit weights, 4K–8K input tokens and one model at a time. Long documents, images and concurrent apps need more. A3B does not mean the entire model fits in 3GB.

Assumes an Apple Silicon Mac and a compatible app: LM Studio or similar for text, Whisper apps for audio, and model-specific image/video tools. RAM alone does not determine speed. Start with a short input on your current Mac.

User feedback overview

What users value

Users find clear, simple instructions and information extraction useful.

Recurring difficulties

Accurate descriptions do not necessarily translate into reliable file operations.

A qualitative synthesis of public discussions, not a representative survey. Reports include PCs and cloud setups, so they do not establish Mac speed or memory requirements. No individual profiles or anecdotes are reproduced.

Our tests and conditions

Candidates to test

On hold or being checked again A failed run is recorded; no passing result is available · Updated

On 2026-11-17, Qwen3.6 35B-A3B on a Mac Studio (M5 Ultra, 96GB) sorted eight inquiries (with its step-by-step thinking output turned off). It got 7 of 8 right but missed one that should have gone to a person for review, so it did not pass. It took about 3 seconds. Earlier attempts did not pass either. Next: improving that kind of judgment and comparing other models. The amount of memory used has not been measured yet.

Laya · Laya Multilingual · Laya-MLX · Qwen3.5 4B · Gemma 4

Test tasks and measures

Use the same labeled data to measure false positives, misses, abstentions, speed and memory. Check how thresholds change the results.

What we know and what remains open

Laya and small language models are candidates. Mac execution, thresholds and accuracy by language still need checking.

Before trying it on your Mac

On hold or being checked again

On 2026-11-07, MiniCPM5-2B (Q4_K_M, digest e680c6c9f9033490cb68df7c1145f733a3a1b377b387af0937251a70d3c71a14) ran once on a Mac mini. It got 4 of 8 fixed cases correct (accuracy 0.5); i001, i004, i005 and i006 were mismatches. Inference took 3.4 seconds (2.729335 seconds by the adapter); peak memory was not measured. Settings: temperature 0, seed 42, num_ctx 4096, num_predict 512. Input SHA-256: d5839ab5a81f9e0029e43db8c2939a6dd13609de6f1b94f7219e6197a54f01a3. Run ID: 20261107T111929-mac-llm-bench-dbaa. The initial record exited with code 1 because the grader received the wrong input location. After PR #53 fixed this, we regraded the same saved responses without inference; all six integrity checks passed and the 4/8 failure was confirmed. Fixing the runner does not mean classification quality passed. We did not rerun inference under the same conditions. Next, investigate the four mismatches and measure a proposed improvement separately under fixed conditions. Peak memory, abstention rates by threshold, and comparisons with other models or languages remain unmeasured.