AI agent comparison · goose
goose record
3 of 9 passed
Version v1.54.0. Local model gpt-oss:20b, run on Mac Studio 2 (M5 Ultra, 96GB).
Results by task
| Task | Passed | Scores (runs 1, 2, 3) | Main cause |
|---|---|---|---|
| Research | 0/3 | 0.0, 0.0, 0.0 | The read tool was not available |
| Fetch-failure sorting | 3/3 | 1.0, 1.0, 1.0 | — |
| Island figures | 0/3 | 0.0, 0.0, 0.0 | Read the input, then ended without writing the result |
In the research task, all 3 runs failed to read the input files, so no result file was produced. The log repeatedly shows "Tool 'read' was not advertised for this model turn."
In the island task, all 3 runs read the input with a shell command, then ended without calling any tool to write the file. In run 3, the JSON was printed to the output, but no result file was created.
All three fetch-failure runs passed. The results differ sharply by task, which is the main pattern in this record.
What this shows
- The exit code was 0 in all runs. These were not connection or timeout failures.
- The research and island failures were not re-measured under changed conditions, so a different setup might give different results.
- The inputs are synthetic. We do not know whether real work documents would give the same results.