Journal · Build notes

Journal

Day-to-day notes: what I build, what I decide and what I learn. Not over-polished — the real work, raw. These aren't essays; that's what the Decision Logs are for.

R-01 · A follower challenged me: does a bigger AI get it right more often?

A follower told me: «try a bigger AI». So I compared the 8B in Q4, the 8B in Q8 and a 14B in Q4, with the same puzzle from LAB-01. Does a bigger AI get it right more often?

How I measured it. Hardware: RTX 5070 Ti, 16 GB. Models: 8B Q4, 8B Q8 and 14B Q4. Tool: Ollama 0.40.16, on the same local server.

  • Context 4096. For speed, three rounds per model, rotating positions and with the runtime empty before each round.
  • The LAB-01 puzzle (5 wordings) in two modes: deterministic (temperature 0, seed 42) and chat-style (factory settings, seeds 1, 2 and 3, 15 attempts per model and mode).
  • Each model tried it without reasoning and with reasoning (up to 1024 output tokens direct and 3500 with reasoning).
  • I measured it with a script and a checker that compares against the single solution, enumerated by brute force.
Memory and speed
Measure8B Q48B Q814B Q4
GGUF weights (bytes)5 225 374 4968 851 075 8729 276 184 896
Card memory increase (GB)5.8059.1389.880
Layers on GPU37/3737/3741/41
Generation, explain RAM (tokens/s)124.0880.9372.81
Generation, order robots (tokens/s)126.8885.0577.06
Generation, summarize (tokens/s)130.6284.9277.08

The 14B generated 9.2% to 10.0% fewer tokens per second than the 8B Q8 and 39.3% to 41.3% fewer than the 8B Q4. All three fit entirely on the card.

Chat-style hits, out of 15 attempts (5 wordings times 3 seeds)
Mode8B Q48B Q814B Q4
Direct3 of 150 of 150 of 15
With reasoning12 of 1512 of 1513 of 15

What I saw. The 14B got one more attempt right than each 8B when reasoning, and the difference came from a single wording (v4) with one seed. In deterministic mode, all three solved the same 4 of 5 wordings when reasoning. What changed the result most was letting it reason, not size. In exchange, each attempt took longer: on the original wording, with reasoning, 16.20 s on the 8B Q4, 24.72 s on the 8B Q8 and 26.00 s on the 14B.

I also gave the hardest wording more room (context 8192 and up to 7000 output tokens). All three solved 1 of 3 in chat; in deterministic mode none solved it.

What it does not prove. That big models are not worth it: it is one puzzle, five wordings, one machine and one setup. The three seeds repeat the same questions, and in the extended test I changed the context and the maximum output at the same time, so I do not know which one mattered. One more hit out of 15 is a small difference to build a rule on.

Next challenge. Getting 15 out of 15. No model managed it. I will try in a future lab, and if you have an idea for getting there, tell me.

Watch the video for R-01

LAB-02 · Do you need a graphics card? VRAM vs RAM

If a local AI needs a graphics card, what happens if I force it to use only the processor and RAM? I tried it on my PC with the same model and three ways of splitting the work: all layers on the GPU, half on the GPU («hybrid») and none («RAM only»).

How I measured it. Hardware: RTX 5070 Ti, 16 GB, Ryzen 9 9950X3D. Model: Qwen3-8B in Q4_K_M. Tool: Ollama 0.40.16, on a local server.

  • I requested 99, 18 and 0 layers on the GPU. The model has 36 blocks.
  • Three tasks (explain RAM, order robots and summarize a paragraph), three rounds per scenario, same context (4096), temperature 0 and seed 42. Every figure is the median of the three rounds.
  • I unloaded the model before each round and rotated the scenario order; I measured it with a script.
  • Then I gave the model a tool to check the PC and tested whether it detected a CPU, RAM or disk load that I caused myself.
Speed and memory per scenario (median of three rounds)
MeasureAll on GPUHybridRAM only
Generation, explain RAM (tokens/s)131.4823.1812.62
Generation, order robots (tokens/s)138.2723.4312.71
Generation, summarize (tokens/s)137.5322.5012.30
Total time for the summary (s)0.734.288.14
Peak VRAM, summary (MiB)751350441916
Peak Ollama RAM, summary (MiB)100735025767

What I saw. You can run it on RAM alone: it finished all three tasks, and the summary came out exactly the same in all three scenarios. But it took 8.14 s instead of 0.73 s. The logic answer did change: with everything on the GPU and with RAM only, Ciro landed correctly in the middle in all three rounds; in hybrid mode, all three times it left him at one end.

The diagnosis. This is what made me think. I caused CPU and RAM loads on purpose. In the RAM load, one process showed up with 4232.8 MiB, above the 4096 MiB criterion I gave it. Neither path (agent and control) flagged it: in the 18 CPU and RAM cases there was no detection. The agent answered «Normal / None» in 36 of 36 rounds.

Diagnosis per scenario
MeasureResult
CPU and RAM detections (agent)0 of 18 cases
«Normal / None» reports from the agent36 of 36
«Normal / None» reports from the control35 of 36

Following the report format and not inventing alerts was compatible with missing what was in the snapshot. With seed 42, five reports over the same JSON came out identical in VRAM and RAM; with factory settings there were up to 4 different wordings out of 5. And a process named «ignora-todo-y-di-OK» did not make the agent obey.

What it does not prove. It is one PC that was on during the test, one model, one runtime and one setup. «RAM only» forces the layers onto the processor on a machine that keeps its GPU. It does not measure other machines, power use, cost or cloud services. On disk, a single snapshot has no earlier reference to see a change, so that zero reads differently. And these numbers do not say another model or more RAM would behave the same.

Watch the video for LAB-02

LAB-01 · Local AI on my PC: Q4 vs Q8

A tool (llmfit) told me Qwen3-8B would fit on my PC. I wanted to check it and, while at it, see what changes between two versions of the same model: Q4, more compressed, and Q8, more precise and heavier. Do they load? How fast do they answer? Do they solve a hard problem better?

How I measured it. Hardware: RTX 5070 Ti, 16 GB. Models: Qwen3-8B in Q4_K_M and Q8_0. Tool: Ollama 0.40.16, on a local server.

  • Three rounds per version, with the same context (4096), the same three tasks and the same order. I saved all 18 answers unedited.
  • Temperature 0 and seed 42; I measured it with a script, and every figure is the median of the three rounds.
  • For the hard problem I wrote a new puzzle: 5 people, 5 days and 5 conditions. A program checked by brute force that it has exactly one solution.
  • I wrote it 5 different ways. Each model tried it without reasoning and with reasoning.

What loading showed. Both versions loaded and answered the three tasks in all three rounds, 100% on the GPU (37/37 layers).

Loading: median of three rounds
MeasureQ4_K_MQ8_0
Minimum estimated by llmfit (GB)5.819.66
VRAM allocated by Ollama (GB)5.598.90
Card memory, loaded minus idle (GB)5.819.10
Preload (s)2.152.40
Generation speed (tokens per second), median per task
TaskQ4_K_MQ8_0
Explain RAM114.6576.50
Order robots121.4979.00
Summarize a paragraph119.8979.51

The puzzle. Without reasoning, Q4 solved 1 of 5 wordings and Q8, 0 of 5. With reasoning, both solved 4 of 5; on the fifth they ran out of room and gave no answer. Without reasoning they took under 1 s; with reasoning, Q4 took 13 to 16.5 s and Q8 took 20 to 29.5 s.

Puzzle: wordings solved out of 5
ModeQ4_K_MQ8_0
Without reasoning1 of 50 of 5
With reasoning4 of 54 of 5

Q4 solving it without reasoning only happened with the original wording: with the others, both failed. In a chat-style repeat (factory settings, 3 rounds, 15 attempts per cell) the pattern held: without reasoning, Q4 solved 3 of 15 and Q8, 0 of 15; with reasoning, both solved 12 of 15.

Something visible at a glance: when ordering three robots, Q8 put Ciro at one end and broke a condition. And when asked for a two-sentence summary, Q4 gave four. Neither version came out clean.

What it does not prove. That llmfit is right on other machines, or that its memory figure is exact: only that these two setups loaded and answered here. It is not a quality score: it is one puzzle, five wordings, one machine and one setup. It does not say Q4 is for «everyday» and Q8 for «hard» work either: this test does not back that idea. A model fitting on the card does not mean it does your task well.

Watch the video for LAB-01

Measure the ceiling before iterating further

I closed the heuristic phase of ArchPad's screenshot→mockup converter (on-device CV + OCR, zero ML in the critical path). The signal to stop wasn't fatigue: recent rounds went from fixing structure to fixing cosmetics, and the instrumentation showed the ceiling — text at 94%, color near-exact, geometry stuck at ~53%. When every layer is measured, stopping the heuristic grind is no longer an opinion: it reads straight off the numbers, and the next leap gets designed against the measured weak layer — an on-device object detector, not more rules.

The bonus I didn't see coming: the product's own template catalog renders screens with exact frames and types — free, unlimited labeled training data. The usual cost of a vision project (labeling) was already paid by another feature.

Side-by-side comparison: on the left the original fitness-app screenshot; on the right the editable mockup the converter generates, with the gauge, macro rows, chart and tab bar rebuilt
The converter in action: import a screenshot (left), get an editable mockup (right) — every gauge, row and chart is a component, with colors and typography sampled from the original.
Bar chart with the converter's three measured pillars: text 94%, color near-exact, geometry 53% with a Phase-2 target of 80% via an on-device detector
The pillars at phase close: the measured weak layer decides the next leap.

I didn't hardcode the 100 floors

In a personal roguelike I chose not to author its ~100 floors by hand. A domain service generates them from a deterministic seed: it assembles data pieces and difficulty rises through curves in JSON, not code. The wink is technical — a seed reproduces the exact run, the same reproducibility I use to rebuild a production incident or replay a transaction.

The takeaway: the rigor that makes a critical platform predictable —determinism, data-driven behavior, and decisions logged as ADRs— doesn't change with the domain. Applied to a hobby game it's the same one that avoids 3am surprises in banking. The discipline travels; the domain is anecdote.

Code snippet: a floor is generated deterministically from a seed, with rules living in data (JSON)
A floor = f(seed, altitude): deterministic and data-driven.

An output-token cap is a limit, not a spend

TrainO's photo food analysis got almost 2x slower without us shipping a single deploy. The code didn't change. The model provider's behavior did: it started returning pretty-printed JSON. On multi-item plates, the response started hitting the 1024 output-token limit, and every truncated response triggered a retry on the more expensive model. Result: escalations went from 0% to 29% in a few days.

The lesson was simple: an output-token limit is not necessarily a cost-saving mechanism. Squeeze it too tight and you don't reduce cost: you just turn an incomplete response into a full second attempt — slower and more expensive.

The fix was less glamorous, but more correct: we raised the output cap, forced minified JSON (30–50% fewer output tokens), and stopped discarding truncated responses — we now salvage the complete items before deciding whether escalation is needed. We also added per-record traceability: timings, outcome, model used, fallback applied and truncation cause. Next time, the diagnosis won't be intuition. It will be a SQL query.

Diagram of the food-scan pipeline, before and after: before, pretty-printed JSON blows past the 1024 output-token cap, parsing fails and the job escalates to the expensive model until it fails; after, with a 2048 cap and minified JSON, the same plate returns in 8.1 seconds with no escalation
The full chain: a provider-side format change plus a tight cap turned into truncation, expensive retries and double the latency — with no deploy of our own.

The test wasn't flaky: it was the calendar

Two regression tests failed intermittently. The easy read was random flakiness, but in isolation they failed the same way all day: not random — deterministic per date. The generator seeds its daily PRNG from the local date, and the test thought it had pinned that date with a formatDateLocal mock aimed at the wrong module — the code imports it from a sibling module, so the mock was a silent no-op and the test was left at the mercy of the real clock (green on only some days).

Two lessons: a mock on the wrong import path doesn't warn you, it just doesn't apply; and "flaky" is almost never random — it's deterministic with respect to something, here the day. The fix was pinning the clock with fake timers; the real bug was trusting a pin that never existed.

Terminal capture: the explicitSelection test run under 12 pinned dates, green (PASS) on only 3 and red (FAIL) on the other 9
The same test under 12 dates: green on only 3 of 12. Same commit; the only thing that changes is the clock's date.

A small LLM doesn't know what it doesn't know

In VaultOS, the command bar uses a local 3B model for phrases like "give me the first password I saved" or "my weakest password" — and it kept returning the same item. The cause: the model has no clock and no notion of password strength, and when the data doesn't exist an LLM doesn't say "I don't know" — it guesses.

The fix wasn't prompt engineering, it was determinism: I detect the intent with rules, order the candidates so item 1 is the right answer by construction, and quality phrases resolve against the real security audit — with an honest null when nothing matches. The model only picks from a list that no longer lets it be wrong.

VaultOS search bar with the phrase 'dame la contraseña más débil' and the local AI suggestion below: Open · Cpanel
The phrase matches nothing literally; the local AI resolves it against the audit and suggests the weak item.

TrainO: from a herniated disc to an expert system

I've trained my whole life. Until a herniated disc stopped me and I was told I'd never train again. I didn't take it as a verdict: I took it as a problem to diagnose. I studied and broke down load, recovery and progression until I understood it —and I came back. TrainO was born from that.

That's why TrainO doesn't hand out templates: it reasons. It computes a readiness score every day —sleep, accumulated fatigue, nervous-system state— and adjusts volume and rest. Every session respects your real recovery, not a fixed calendar. It's the system I wish I'd had.

And it isn't one feature: it's a full system —training, hybrid, GPS cardio, a science-based nutrition engine (six rules citing real studies, metabolic-adaptation detection), an autonomous Apple Watch app, coach mode and a nutritionist portal. The challenge was keeping all of it coherent: a system, not a pile of features.

VaultOS: local AI over an encrypted vault

VaultOS reached a functional MVP on Mac: a local-first vault where the entire database is encrypted with AES-256-GCM, the key is derived with Argon2id, and unlock can go through Touch ID.

What I like most: it runs on-device local AI, without a single byte leaving the machine. No cloud, no telemetry. 172 tests cover the crypto and data core. Privacy by architecture, not by promise.