← Full journal

R-01 · A follower challenged me: does a bigger AI get it right more often?

A follower told me: «try a bigger AI». So I compared the 8B in Q4, the 8B in Q8 and a 14B in Q4, with the same puzzle from LAB-01. Does a bigger AI get it right more often?

How I measured it. Hardware: RTX 5070 Ti, 16 GB. Models: 8B Q4, 8B Q8 and 14B Q4. Tool: Ollama 0.40.16, on the same local server.

  • Context 4096. For speed, three rounds per model, rotating positions and with the runtime empty before each round.
  • The LAB-01 puzzle (5 wordings) in two modes: deterministic (temperature 0, seed 42) and chat-style (factory settings, seeds 1, 2 and 3, 15 attempts per model and mode).
  • Each model tried it without reasoning and with reasoning (up to 1024 output tokens direct and 3500 with reasoning).
  • I measured it with a script and a checker that compares against the single solution, enumerated by brute force.
Memory and speed
Measure8B Q48B Q814B Q4
GGUF weights (bytes)5 225 374 4968 851 075 8729 276 184 896
Card memory increase (GB)5.8059.1389.880
Layers on GPU37/3737/3741/41
Generation, explain RAM (tokens/s)124.0880.9372.81
Generation, order robots (tokens/s)126.8885.0577.06
Generation, summarize (tokens/s)130.6284.9277.08

The 14B generated 9.2% to 10.0% fewer tokens per second than the 8B Q8 and 39.3% to 41.3% fewer than the 8B Q4. All three fit entirely on the card.

Chat-style hits, out of 15 attempts (5 wordings times 3 seeds)
Mode8B Q48B Q814B Q4
Direct3 of 150 of 150 of 15
With reasoning12 of 1512 of 1513 of 15

What I saw. The 14B got one more attempt right than each 8B when reasoning, and the difference came from a single wording (v4) with one seed. In deterministic mode, all three solved the same 4 of 5 wordings when reasoning. What changed the result most was letting it reason, not size. In exchange, each attempt took longer: on the original wording, with reasoning, 16.20 s on the 8B Q4, 24.72 s on the 8B Q8 and 26.00 s on the 14B.

I also gave the hardest wording more room (context 8192 and up to 7000 output tokens). All three solved 1 of 3 in chat; in deterministic mode none solved it.

What it does not prove. That big models are not worth it: it is one puzzle, five wordings, one machine and one setup. The three seeds repeat the same questions, and in the extended test I changed the context and the maximum output at the same time, so I do not know which one mattered. One more hit out of 15 is a small difference to build a rule on.

Next challenge. Getting 15 out of 15. No model managed it. I will try in a future lab, and if you have an idea for getting there, tell me.

Watch the video for R-01