Demonstrated in this run
- Fix the one specified link
- Keep the other three files unchanged
- Pass the checks without manual repair
MAC × AI · PRACTICAL TEST
One broken link, repaired.
We asked AI to correct a link’s filename in a practice site. The repair and its work report passed every check.
Full use case still under reviewOne task, one run, on a practice site. Whether AI can handle site development as a whole is still unverified.
Mac Studio · M5 Ultra · 96GB
2026-11-17 · 1 task · 1 run
Test feature changes, bug fixes and site checks using a repository and its tests.
4bit / Q4_K_M
For trying short tasks one at a time
30B 4-bit weights alone are about 15GB theoretically. Add format overhead, context cache and OS headroom. Start with short tasks one at a time at 32GB.
Try HTML/CSS components, form scaffolding, small bug fixes and test drafts. Specify the files and scope.
Review architecture, authentication, payments and autonomous multi-file changes. Editing files and checking a browser require a compatible editor or agent, not just a model.
Memory figures are editorial budgets including room for the OS and apps, not verified minimums. Text models assume 4-bit weights, 4K–8K input tokens and one model at a time. Long documents, images and concurrent apps need more. A3B does not mean the entire model fits in 3GB.
Assumes an Apple Silicon Mac and a compatible app: LM Studio or similar for text, Whisper apps for audio, and model-specific image/video tools. RAM alone does not determine speed. Start with a short input on your current Mac.
Well-scoped code generation and scaffolding receive positive feedback.
Architecture, long agent runs and unintended file edits draw criticism. Results vary with setup and tooling.
A qualitative synthesis of public discussions, not a representative survey. Reports include PCs and cloud setups, so they do not establish Mac speed or memory requirements. No individual profiles or anecdotes are reproduced.
Measurement explained here: 2026-11-17 · linkfix-nested-002
On 2026-11-17, Qwen3.8 27B (Q4_K_M) on a Mac Studio (M5 Ultra, 96GB) was given one task: fix a broken link in a practice website. It passed all 21 grading checks in about 5 seconds with no manual fixes. Only this one task has been tried; comparisons with other tasks and models, and the amount of memory used, have not been measured yet.
Qwen3-Coder 30B · Qwen3.8 27B · Gemma 4 · gpt-oss · Hermes Agent · goose
Give each setup the same change request. Record passing tests, unrelated changes, elapsed time and manual fixes. Compare the model and the agent separately.
Agent comparisons and practice tasks are available. Quality, time and manual fixes for site changes are a separate test.
On 2026-11-14, Qwen3.8 27B (Q4_K_M, digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813) ran the fixed linkfix-typo-001 case once on Mac Studio 2 for cycle 4. Queue job 20261114T184141-mac-llm-bench-dc22, inference took 4.807 s (5.6 s in the queue), repaired one broken link (/guid.html to /guide.html) in a synthetic site, and passed all 22 independent checks on the saved workspace (status: passed, 0 manual fixes). It did not modify a real site. Peak memory was unmeasured (null) because it is not instrumented. One case does not establish performance for the full use case. The separate 2026-11-09 Studio 1 pass (8.514 s), 2026-11-08 Studio 1 pass (8.517 s), and 2026-11-06 failure (MiniCPM5-2B, 5.206 s, 9/17) remain recorded as separate attempts; status remains held. Settings: temperature 0, seed 42, num_ctx 8192, num_predict 4096. Input SHA-256: 22930c6d8b7dc704db8e1e3c34aba571a80cf39ac86444e66c662b0b6a42df2d. Run ID: 20261114T184141-mac-llm-bench-dc22. Next: measure another task and model, compare multiple cases, and instrument peak memory.
Run: 20261117T042150-mac-llm-bench-a069 · Input SHA-256: 6fc4f605433462379b23e167e5865cb00a4e94e6cbe4eb122dbee952b851540d
Mac Studio 2 · qwen3.8:27b Q4_K_M · Ollama 0.40.16 · digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813 · temperature 0, seed 42, num_ctx 8192, num_predict 4096
On 2026-11-17, fixed case linkfix-nested-002 ran once on Mac Studio 2 with qwen3.8:27b Q4_K_M (Ollama 0.40.16; digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813). Inference took 5.33705 s and the mini-queue job took 6.3 s. It repaired one relative link in a synthetic site (setup-mac.html → mac-setup.html). All 21 independent checker items were true, with zero unresolved links, the other three files unchanged, and no manual fixes. No real site was changed. Peak memory is unmeasured (null) because it is not instrumented. Input SHA-256: 6fc4f605433462379b23e167e5865cb00a4e94e6cbe4eb122dbee952b851540d. This does not establish performance for the full use case or a recommendation.
The use case remains on hold. The next cycle should measure another task or model, compare multiple cases, and add peak-memory measurement.