Measurements by model
Measurements recorded on this site, grouped by model and Mac. Separate from task recommendations. Quantization, input and runtime affect results.
Go to task recommendations →Text models
Latest stored record for each Mac/model. Reply time is the median on short JCommonsenseQA questions; generation speed is a median from a separate generation task. They measure different workloads. Arrows indicate favorable direction, not an overall ranking.
How to read this table
Memory used is runtime-reported model memory, with estimates labeled; it is not total Mac peak memory or required RAM. Swap growth measures additional SSD-backed memory, not comfort. Accuracy includes correct/total counts; compare only compatible datasets and conditions.
28 records
Scroll horizontally; model names stay visible on the left.
| Model | Mac | Measured (JST) | Reply time s ↓ | Generation tok/s ↑ | Memory used GB ↓ | Swap growth MB ↓ | Knowledge JMMLU ↑ | Math MGSM ↑ | Conditions / detail |
|---|---|---|---|---|---|---|---|---|---|
| gemma3:27bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 0.38 | 23.18 | 17.52 | 0.2 | 76.0%152 / 200 | 91.0%91 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "6be2e2ba6843b9255820e2de2cbbe75cfdef0294ac7a531572e654d6e0eefe7d"
}Conditions / detail → |
| gemma3:27bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-17 | 0.25 | 46.98 | 16.76 | 0 | 74.5%149 / 200 | 93.0%93 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "6be2e2ba6843b9255820e2de2cbbe75cfdef0294ac7a531572e654d6e0eefe7d"
}Conditions / detail → |
| gemma4:12bQ4_K_MApple M6 · 16 GB | Apple M6 · 16 GB | 2026-10-24 | 26.32 | 16.25 | 8.13 | 226.1 | Not measured | Not measured | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.0",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "4eb23ef187e2c5462566d6a1d3bbbc2f1346d0b4327cbb66d58fffbcc9b2b05c"
}Conditions / detail → |
| gemma4:26bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 3.18 | 103.39 | 18.73Estimated | 0 | 85.0%170 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "001e5dafc3c77684c2307ebc6ab8e336e10c9b18eca52acf547d72fc83c3ca8c"
}Conditions / detail → |
| gemma4:26bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 2.1 | 159.46 | 18.73Estimated | 0 | 85.5%171 / 200 | 97.0%97 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"top_k": 64,
"top_p": 0.95,
"temperature": 1.0
}
},
"digest": "001e5dafc3c77684c2307ebc6ab8e336e10c9b18eca52acf547d72fc83c3ca8c"
}Conditions / detail → |
| glm-4.7-flash:latestQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 7.54 | 81.01 | 19.42 | 0 | 76.0%152 / 200 | 95.0%95 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"top_p": 0.95,
"min_p": 0.01,
"repeat_penalty": 1.0,
"temperature": 1.0
}
},
"digest": "4475827791a269b02c8ec49b1c3bc1abb5846bacf3fae015b75d33986322d8f6"
}Conditions / detail → |
| glm-4.7-flash:latestQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-17 | 5.79 | 101.83 | 19.54 | 0 | 82.5%165 / 200 | 95.0%95 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_p": 0.95,
"min_p": 0.01,
"repeat_penalty": 1.0
}
},
"digest": "4475827791a269b02c8ec49b1c3bc1abb5846bacf3fae015b75d33986322d8f6"
}Conditions / detail → |
| gpt-oss:120bMXFP4Apple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 1.88 | 114.79 | 64.33 | 0 | 82.0%164 / 200 | 99.0%99 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0
}
},
"digest": "714d86958b6e870a9320e78126ac1ce3477727a1cd9bf7194b8ace0d84c62fd8"
}Conditions / detail → |
| gpt-oss:20bMXFP4Apple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 1.77 | 89.87 | 12.84 | 0 | 83.0%166 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0
}
},
"digest": "f38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe"
}Conditions / detail → |
| gpt-oss:20bMXFP4Apple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-17 | 1.04 | 164.72 | 12.86 | 0 | 81.0%162 / 200 | 97.0%97 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0
}
},
"digest": "f38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe"
}Conditions / detail → |
| hf.co/openbmb/MiniCPM5-2B-GGUF:Q4_K_MQ4_K_MApple M6 · 16 GB | Apple M6 · 16 GB | 2026-10-12 | 2.88 | 81.55 | 2.05 | 0 | 70.0%140 / 200 | 90.0%90 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.0",
"sampling": {
"source": "generation_config.json(openbmb/MiniCPM5-2B)",
"options": {
"temperature": 1.0,
"top_p": 0.95
}
},
"digest": "e680c6c9f9033490cb68df7c1145f733a3a1b377b387af0937251a70d3c71a14"
}Conditions / detail → |
| MiniCPM5-2BQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 1.5 | 187.64 | 2.02 | 0 | 73.0%146 / 200 | 90.0%90 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "generation_config.json(openbmb/MiniCPM5-2B)",
"options": {
"temperature": 1.0,
"top_p": 0.95
}
},
"digest": "e680c6c9f9033490cb68df7c1145f733a3a1b377b387af0937251a70d3c71a14"
}Conditions / detail → |
| hf.co/openbmb/MiniCPM5-2B-GGUF:Q4_K_MQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 1.07 | 247.24 | 1.66 | 0 | 71.5%143 / 200 | 89.0%89 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "generation_config.json(openbmb/MiniCPM5-2B)",
"options": {
"temperature": 1.0,
"top_p": 0.95
}
},
"digest": "e680c6c9f9033490cb68df7c1145f733a3a1b377b387af0937251a70d3c71a14"
}Conditions / detail → |
| Hy-Memory-30B-A3BQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 0.1 | 110.39 | 19.22 | 0 | 65.5%131 / 200 | 88.0%88 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "なし(温度0)",
"options": {
"temperature": 0
}
},
"digest": "3d1cd167746ae18e21cbbb8f5200d524caa0d5239470e94b8caafc2931a15c46"
}Conditions / detail → |
| hf.co/tencent/Hy-Memory-30B-A3B-GGUF:Q4_K_MQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-22 | 0.08 | 142.01 | 18.89 | 0 | 68.5%137 / 200 | 90.0%90 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "なし(温度0)",
"options": {
"temperature": 0
}
},
"digest": "3d1cd167746ae18e21cbbb8f5200d524caa0d5239470e94b8caafc2931a15c46"
}Conditions / detail → |
| hf.co/unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_K_MQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-19 | 5.64 | 140.87 | 23.88 | 0 | 81.5%163 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "なし(温度0)",
"options": {
"temperature": 0
}
},
"digest": "ad2ef3b478c378987150cc477ce4ca3492f9c91572f36fa3c4f162c4796a9cf2"
}Conditions / detail → |
| muse-glimmer:30bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 15.01 | 24.85 | 17.58 | -283.1 | 83.0%166 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1"
}Conditions / detail → |
| muse-glimmer:30bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 7.52 | 51.22 | 15.8 | 0 | 84.0%168 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 64,
"top_p": 0.95
}
},
"digest": "de878ce33ad81d060001db1469a02eebe4d86f0ad58cfe52dc062fdcbe4464c1"
}Conditions / detail → |
| qwen3-coder-next:latestQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 0.12 | 107.19 | 52.05 | 0 | 76.5%153 / 200 | 95.0%95 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 40,
"top_p": 0.95
}
},
"digest": "ca06e9e4087c714d44355bf954099187890e63084b4a632b8e9956c4b9492074"
}Conditions / detail → |
| qwen3-coder:30bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 0.09 | 107.7 | 19.36 | 0 | 69.0%138 / 200 | 89.0%89 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8,
"repeat_penalty": 1.05
}
},
"digest": "06c1097efce0431c2045fe7b2e5108366e43bee1b4603a7aded8f21689e90bca"
}Conditions / detail → |
| qwen3-coder:30bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-17 | 0.07 | 160.48 | 19.37 | 0 | 70.5%141 / 200 | 94.0%94 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"repeat_penalty": 1.05,
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8
}
},
"digest": "06c1097efce0431c2045fe7b2e5108366e43bee1b4603a7aded8f21689e90bca"
}Conditions / detail → |
| qwen3-next:80bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 7.9 | 96.6 | 50.66 | 0 | 85.5%171 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"top_k": 20,
"top_p": 0.95,
"repeat_penalty": 1.0,
"temperature": 0.6
}
},
"digest": "b2ebb986e4e960bb3f2d636472a466b4027939c4d8d9a62fc82bed8ebfd1ae19"
}Conditions / detail → |
| qwen3.5:4bQ4_K_MApple M6 · 16 GB | Apple M6 · 16 GB | 2026-10-30 | 63.56 | 44.15 | 3.36 | 0 | Not measured | Not measured | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.0",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"presence_penalty": 1.5,
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95
}
},
"digest": "3c3adf637187c95955232acd3ed05b9ea45aeb2a5f953e631c38174a46b5b56a"
}Conditions / detail → |
| qwen3.5:9bnvfp4Apple M6 · 16 GB | Apple M6 · 16 GB | 2026-11-01 | 47.84 | 33.44 | 7.68 | 946 | Not measured | Not measured | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.40.13",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"min_p": 0.0,
"presence_penalty": 1.5,
"repeat_penalty": 1.0,
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95
}
},
"digest": "d63f830f28d515fefea546f213fca7a75e9dd5176ef51a6deabc952d1c99a911"
}Conditions / detail → |
| qwen3.6:35b-a3bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 8.52 | 125.71 | 22.27 | 2949.2 | 86.5%173 / 200 | 98.0%98 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"top_k": 20,
"top_p": 0.95,
"min_p": 0.0,
"presence_penalty": 1.5,
"repeat_penalty": 1.0,
"temperature": 1.0
}
},
"digest": "a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c"
}Conditions / detail → |
| qwen3.6:35b-a3bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 8.04 | 137.87 | 21.7 | 0 | 87.5%175 / 200 | 100.0%100 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95,
"min_p": 0.0,
"presence_penalty": 1.5,
"repeat_penalty": 1.0
}
},
"digest": "a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c"
}Conditions / detail → |
| qwen3.8:27bQ4_K_MApple M5 Max · 36 GB | Apple M5 Max · 36 GB | 2026-11-02 | 7.69 | 26.29 | 18.15 | 1208.5 | 86.5%173 / 200 | 100.0%100 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.2",
"Ollama": "0.34.3",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"top_k": 20,
"top_p": 0.95,
"min_p": 0.0,
"presence_penalty": 0.0,
"repeat_penalty": 1.0,
"temperature": 1.0
}
},
"digest": "aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813"
}Conditions / detail → |
| qwen3.8:27bQ4_K_MApple M5 Ultra · 96 GB | Apple M5 Ultra · 96 GB | 2026-10-18 | 3.79 | 55.14 | 16.43 | 0 | 85.5%171 / 200 | 100.0%100 / 100 | Conditions / detail{
"num_ctx": 8192,
"benchmark_ctx": 20480,
"runs": 3,
"temperature": 0,
"seed": 42,
"method_version": 1,
"datasets": {
"mgsm_ja": "MGSM-Rev2"
},
"benchmark_limits": {
"jmmlu": 16384,
"jcommonsenseqa": 4096,
"mgsm_ja": 16384
},
"macOS": "27.0.1",
"Ollama": "0.40.5",
"sampling": {
"source": "Ollama 公式タグ",
"options": {
"min_p": 0.0,
"presence_penalty": 0.0,
"repeat_penalty": 1.0,
"temperature": 1.0,
"top_k": 20,
"top_p": 0.95
}
},
"digest": "aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813"
}Conditions / detail → |
No matching measurements. Change the model or Mac filter. Missing measurements are not failures or zero scores.
How we test
Practical tests by model and task
Selected published records, not a controlled comparison of every model. Pass/fail refers to each task’s criteria, not overall model quality. Details retain settings and earlier records.
Scroll horizontally; model names stay visible on the left.
| Model | Task | Mac | Measured (JST) | Generation time (s) | Result | Conditions / detail |
|---|---|---|---|---|---|---|
| Qwen3.8 27B (Q4_K_M) | Repair one link | Mac Studio · M5 Ultra · 96GB | 2026-11-17 | 5.34 | Passed · 21/21 | Conditions / detail → |
| mlx-community/whisper-large-v3-turbo | Japanese speech recognition | Mac mini | 2026-11-06 | 18.17 | Failed · CER 10.5% | Conditions / detail → |
| mlx-community/whisper-large-v3-mlx | Japanese speech recognition | Mac mini | 2026-11-08 | 8.68 | Failed · CER 26.3% | Conditions / detail → |
| qwen3.6:35b-a3b | Normalize numerals in an existing transcript (no ASR) | Mac Studio · M5 Ultra · 96GB | 2026-11-15 | 8.36 | Passed | Conditions / detail → |
| qwen3.8:27b | Extract five answers and evidence from minutes | Mac Studio · M5 Ultra · 96GB | 2026-11-17 | 13.00 | Passed | Conditions / detail → |
Image and video models
Images and video are different tasks from text. Image timing is for subsequent images; video timing is per clip. Memory is the peak reported by each backend, not a directly comparable metric across implementations.
Scroll horizontally; model names stay visible on the left.
| Model | Image and video models | Mac | Measured (JST) | Generation time (s) | Peak memory (GB) | Conditions / detail |
|---|---|---|---|---|---|---|
| Z-Image Turbo | Image | Apple M5 Max · 36 GB | 2026-09-24 | 19.7 | 18.51 | Conditions / detail8bit · v0.20.0 {
"width": 1024,
"height": 1024,
"seeds": [
1,
2
],
"mlx_cache_limit_gb": 2
}Conditions / detail → |
| FLUX.2 [klein] 4B | Image | Apple M5 Max · 36 GB | 2026-09-24 | 5 | 16.67 | Conditions / detail8bit · v0.20.0 {
"width": 1024,
"height": 1024,
"seeds": [
1,
2
],
"mlx_cache_limit_gb": 2
}Conditions / detail → |
| ERNIE-Image Turbo | Image | Apple M5 Max · 36 GB | 2026-09-24 | 22.8 | 21.63 | Conditions / detail8bit · v0.20.0 {
"width": 1024,
"height": 1024,
"seeds": [
1,
2
],
"mlx_cache_limit_gb": 2
}Conditions / detail → |
| Qwen-Image | Image | Apple M5 Max · 36 GB | 2026-09-24 | 127 | 37.3 | Conditions / detail4bit · v0.20.0 {
"width": 1024,
"height": 1024,
"seeds": [
1,
2
],
"mlx_cache_limit_gb": 2
}Conditions / detail → |
| Wan 2.2 TI2V-5B | Video | Apple M5 Max · 36 GB | 2026-09-24 | 852.7 | 45.72 | Conditions / detail— · {
"width": 1280,
"height": 704,
"frames": 81,
"fps": 24,
"seed": 1,
"mlx_cache_limit_gb": 2,
"seconds": 3.4
}Conditions / detail → |