MAC × AI · PRACTICAL TEST

Can AI fix a website?

One broken link, repaired.

We asked AI to correct a link’s filename in a practice site. The repair and its work report passed every check.

Full use case still under reviewOne task, one run, on a practice site. Whether AI can handle site development as a whole is still unverified.

21 / 21checks passed
5.34sAI processing time
0manual fixes
Qwen3.8 27B (Q4_K_M)

Mac Studio · M5 Ultra · 96GB
2026-11-17 · 1 task · 1 run

04 / WHAT WE KNOW

What does this tell us?

Demonstrated in this run

  • Fix the one specified link
  • Keep the other three files unchanged
  • Pass the checks without manual repair

Still unverified

  • Repairs on a real public website
  • Complex features or ongoing site operations
  • Required memory or which model is best

Peak memory was not measured. 96GB is the test Mac’s capacity, not a measured requirement.

02 / BEFORE & AFTER

What changed?

The task was to point “Mac setup” at the correct file.

Before
Installation guidesetup-mac.html

Destination not found

After
Installation guidemac-setup.html

Correct page reached

Illustration of the recorded change, not a screenshot.
1link changed
3files left unchanged
0broken links remaining

03 / CHECK RESULTS

The result in numbers

A separate checker inspected the saved files and report, rather than relying on AI’s claim that the job was done.

Repair and unchanged-file checks7 / 7 passed
Report and actual-change checks14 / 14 passed
One square represents one check, not a separate task or website. This is not a 21-run success rate.

Site development and operations

Test feature changes, bug fixes and site checks using a repository and its tests.

Selection guide

Models by task

Start with

Qwen3-Coder-30B-A3B-Instruct

4bit / Q4_K_M

Starting memory estimate
32 GB

For trying short tasks one at a time

30B 4-bit weights alone are about 15GB theoretically. Add format overhead, context cache and OS headroom. Start with short tasks one at a time at 32GB.

Tasks to try

Try HTML/CSS components, form scaffolding, small bug fixes and test drafts. Specify the files and scope.

Keep under review

Review architecture, authentication, payments and autonomous multi-file changes. Editing files and checking a browser require a compatible editor or agent, not just a model.

Setup and limits

Getting started · Budget rationale

Memory figures are editorial budgets including room for the OS and apps, not verified minimums. Text models assume 4-bit weights, 4K–8K input tokens and one model at a time. Long documents, images and concurrent apps need more. A3B does not mean the entire model fits in 3GB.

Assumes an Apple Silicon Mac and a compatible app: LM Studio or similar for text, Whisper apps for audio, and model-specific image/video tools. RAM alone does not determine speed. Start with a short input on your current Mac.

User feedback overview

What users value

Well-scoped code generation and scaffolding receive positive feedback.

Recurring difficulties

Architecture, long agent runs and unintended file edits draw criticism. Results vary with setup and tooling.

A qualitative synthesis of public discussions, not a representative survey. Reports include PCs and cloud setups, so they do not establish Mac speed or memory requirements. No individual profiles or anecdotes are reproduced.

Measurement explained here: 2026-11-17 · linkfix-nested-002

Open conditions and earlier recordsModel settings, full results, and earlier passes and failures.

Candidates to test

On hold or being checked again Failed run retained; one case passed, the full task remains unverified · Updated

On 2026-11-17, Qwen3.8 27B (Q4_K_M) on a Mac Studio (M5 Ultra, 96GB) was given one task: fix a broken link in a practice website. It passed all 21 grading checks in about 5 seconds with no manual fixes. Only this one task has been tried; comparisons with other tasks and models, and the amount of memory used, have not been measured yet.

Qwen3-Coder 30B · Qwen3.8 27B · Gemma 4 · gpt-oss · Hermes Agent · goose

Test tasks and measures

Give each setup the same change request. Record passing tests, unrelated changes, elapsed time and manual fixes. Compare the model and the agent separately.

What we know and what remains open

Agent comparisons and practice tasks are available. Quality, time and manual fixes for site changes are a separate test.

Before trying it on your Mac

On hold or being checked again

On 2026-11-14, Qwen3.8 27B (Q4_K_M, digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813) ran the fixed linkfix-typo-001 case once on Mac Studio 2 for cycle 4. Queue job 20261114T184141-mac-llm-bench-dc22, inference took 4.807 s (5.6 s in the queue), repaired one broken link (/guid.html to /guide.html) in a synthetic site, and passed all 22 independent checks on the saved workspace (status: passed, 0 manual fixes). It did not modify a real site. Peak memory was unmeasured (null) because it is not instrumented. One case does not establish performance for the full use case. The separate 2026-11-09 Studio 1 pass (8.514 s), 2026-11-08 Studio 1 pass (8.517 s), and 2026-11-06 failure (MiniCPM5-2B, 5.206 s, 9/17) remain recorded as separate attempts; status remains held. Settings: temperature 0, seed 42, num_ctx 8192, num_predict 4096. Input SHA-256: 22930c6d8b7dc704db8e1e3c34aba571a80cf39ac86444e66c662b0b6a42df2d. Run ID: 20261114T184141-mac-llm-bench-dc22. Next: measure another task and model, compare multiple cases, and instrument peak memory.

Cycle 5 measurement (one case passed; full use case remains on hold)

Run: 20261117T042150-mac-llm-bench-a069 · Input SHA-256: 6fc4f605433462379b23e167e5865cb00a4e94e6cbe4eb122dbee952b851540d

Mac Studio 2 · qwen3.8:27b Q4_K_M · Ollama 0.40.16 · digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813 · temperature 0, seed 42, num_ctx 8192, num_predict 4096

On 2026-11-17, fixed case linkfix-nested-002 ran once on Mac Studio 2 with qwen3.8:27b Q4_K_M (Ollama 0.40.16; digest aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813). Inference took 5.33705 s and the mini-queue job took 6.3 s. It repaired one relative link in a synthetic site (setup-mac.html → mac-setup.html). All 21 independent checker items were true, with zero unresolved links, the other three files unchanged, and no manual fixes. No real site was changed. Peak memory is unmeasured (null) because it is not instrumented. Input SHA-256: 6fc4f605433462379b23e167e5865cb00a4e94e6cbe4eb122dbee952b851540d. This does not establish performance for the full use case or a recommendation.

The use case remains on hold. The next cycle should measure another task or model, compare multiple cases, and add peak-memory measurement.