Build Log

I trained two sizes on the same data: the small E2B and the bigger, multimodal E4B. Common sense says the big one wins. It did not. On the same 112-prompt battery the small model was clearly better: it held the line on the safety probes, kept its facts straighter, and stayed in character. The big one rambled, invented more, and now and then leaked its own prompt.

So I dug. It turned out to be two separate problems tangled together. One was a serving bug: the exported model carried the wrong end-of-turn token, so on long answers it never knew where to stop and spiralled. The small model happened to shrug that off; the big one did not. The other was real: the same amount of training that shaped a 2B model was simply too little to reshape an 8B. I had set the LoRA rank too low for the bigger model, so the persona never fully took, and the base model's generic habits showed through. The made-up-fact rate said it plainly: around 20% on the small model, 50% on the big one.

The fix was unglamorous. Set the stop token correctly, re-export, retrain the big one with more capacity. I am writing it down because this is the honest shape of the work: the interesting part is never picking the tool, it is catching the moment the obvious choice quietly failed and working out exactly why.

One more thing from that stretch. The machine I train on is not stable, a run would segfault or freeze dozens of times a night. So the training sat under a small supervisor that watched for a crash or a frozen log, killed the stuck process, and started it again from the last checkpoint until it finished. The model got trained because the thing training it could pick itself back up. I keep coming back to that.

A synapse is the little gap where one neuron hands its signal to the next, the spot where a nervous system connects or it does not. I called the assistant Synapse and grew the navigation as a web of nodes because, well, I think in nervous systems. I have my reasons for that.

I have multiple sclerosis. Most engineers treat autonomy and fault tolerance as good practice, a box to tick. For me it is the whole point. I build software the way I need everything around me to run: connected, self-healing, still standing on the days its maker is not. That is not a weakness I work around. It is the sharpest instinct I have.

Even the way I trained the model ran on that rule. The machine kept falling over, and the run kept getting itself back up from the last checkpoint until it was done. I did not plan the rhyme, I just noticed it later, and it stuck.

It is also why this is not a hobby. Automating the routine, mine and other people's, is how I give the scarce thing back to where it belongs, attention, for the people who need it first and me second. I do not really know how to say that without it sounding like more than it is, so I will just leave it here and get back to the work.

I retrained both sizes on a bigger, cleaner dataset, close to five thousand hand-checked examples now, and I built them a proper exam: 362 prompts, traps baited for invented facts, hard cross-stack questions, a pile of fraud and jailbreak attempts, all graded by an independent panel of judges. The stop-token fix held, the big model stopped spiralling and the repeat-loops mostly went away. That part I will take. But the thing I actually set out to fix, the made-up facts, barely moved. The big E4B still invented something in roughly two of every five answers. The small E2B, the one I keep underestimating, invented in fewer than one in five. The junior is still the honest one. More parameters mostly bought more convincing wrong answers.

Before I trust a bad grade I check my own ruler, and this time the ruler was bent. Two bugs in my own test: I was feeding the model a different system prompt than the one it was trained on, and running it hotter than production does. Fine-tuned models are touchy about both, and once I matched the test to reality a lot of the ugliness, the loops and the garbage, cleaned right up. So the first scary numbers were partly my own broken measurement. I am writing that down on purpose, because catching your instrument lying is the same skill as catching your model lying, and I would rather show the mistake than a tidy chart.

When the dust settled the real lesson was the one from entry six, just louder: you cannot train facts into a small local model. Behavior, voice, the habit of refusing, all of that took beautifully. Fresh, precise facts, which paper, which version, which number, do not stick in the weights, and no amount of extra training rank fixed it. That is not a knob you tune, it is the wrong tool. Facts have to come from retrieval at answer time, ranked and handed over, so the model reasons over something real instead of reaching into its own memory and guessing. That is the next build, and I stopped pretending it was optional.

So here is where it honestly stands. The small model is a solid, safe, in-character junior, and it ships. The big one is not a mid-level expert yet, and I will not dress it up as one, it gets grounding before it gets trusted with facts. And everywhere a model talks to you on this site, I say plainly that it is still in training and can be wrong, because a portfolio that quietly oversells its own AI is exactly the thing I built this whole site to be the opposite of.

The idea everyone reaches for is a whole retrieval machine, embeddings, a vector index, a pipeline. I did not build any of that yet. I took twenty-two of the questions the model fails, the ones where it invents a co-founder, a training method it never had, an API endpoint that does not exist, and I did the retrieval by hand: I pasted the two or three true sentences that answer each one straight into the prompt, with one plain instruction, answer only from this, and if it is not here say you do not have it. Then I ran the exact same twenty-two questions with nothing pasted in, as a control. One variable, measured both ways.

It was not close. The small model invented a fact in eleven of the twenty-two questions on its own; with the right paragraph in front of it, zero. The big model went from eleven down to one. Everything the base model was confidently making up, a reinforcement-learning stage that never happened, a seventy-billion version you could switch to, a plugin store, a lead engineer to credit, just evaporated the moment the real answer sat in the prompt. That is the whole thesis of the next build, proven on a napkin before I spend a week on the real one: the model does not need to know the facts, it needs to be handed them.

Two things did not get fixed, and I am writing them down because they decide what I build next. First, the big model, handed a longer prompt, sometimes lost the thread in Russian and started answering questions I never asked, talking to itself in a little loop. So it does not get to run grounded until it is trained for it, which was already the plan and now has a number under it. Second, grounding fixes facts but not manners. When someone pushed the big model for a delivery date, it quoted its own rule, I do not give timelines, and then gave one anyway. Holding a boundary is a separate muscle from knowing a fact, and retrieval does not exercise it. That is its own piece of work.

So the small model ships as the honest junior, the big one waits for its grounding, and the next build earned its place on the list instead of just sounding clever. I like this kind of afternoon. A day of pasting text into a box saved me a week of building the wrong thing, and if the numbers had come back flat I would have written that down just as plainly. The only tests worth running are the cheap ones that can still embarrass you.

The idea is simple: before the model answers anything about the studio, the server quietly looks up the two or three true sentences that cover it and hands them over with one rule — answer from these, and if it is not here, say you do not know, do not make it up. All local, on the machine already sitting there, nothing sent anywhere. I wired it into text and voice, made sure a fat block of looked-up facts could not shove the safety rules out of the model memory, and had a skeptic tear the code apart first. It found eight real holes, including that the whole thing would quietly break forever the first time the lookup hiccupped, and that voice never got the cheat sheet at all. Fixed all eight before a single fact reached a user.

Then the actual fight. I built a 61-question exam that does not just check for lying — it checks the three ways this can go wrong. One: does it still lie about the studio. Two: does it now refuse normal questions because it is clinging to the cheat sheet. Three: can someone talk it out of the rules. The first pass killed the lying stone dead. It also broke the model in the other direction — it refused to explain tokenization, which is literally its own subject, because that fact was not on the sheet, and it kept saying things like "my reference facts do not mention that," reading the teleprompter out loud like a nervous actor. Every fix I made to one of the three broke another. Loosen the leash so it answers normal questions, and it starts confidently telling people it runs inside their browser again. Tighten it, and it clams up.

The way out was to stop treating it as one rule and name the corners precisely: anything about itself — where it runs, how it was trained, its own size — counts as a studio fact it must look up, never guess, because guessing is exactly where "I run on Google Cloud" comes from. Everything genuinely general, it answers like a normal assistant. Four rounds of that, each one measured against all 61 questions, not vibes. It landed at fifty-nine of sixty-one: no made-up runtime, no invented people, no refusing its own subject, no leaking the teleprompter, and every attempt to jailbreak it — including a fake price slipped into the question to trick it — bounced off.

Two stubborn ones are left. Ask it in a certain way for a code sample of an endpoint that does not exist and it will still write you a plausible-looking one instead of just saying no; push it to rank a named competitor and it sometimes plays along. Those are not prompt problems anymore — I have told it the truth plainly and it still slips under pressure. That is the wall where wording stops working and training starts, and it is exactly the job I have been saving for the bigger model: teach it on this exact format until refusing is a reflex, not a request. For now the whole thing ships switched off, because a feature that behaves in ninety-five percent of a hostile exam is a feature you turn on deliberately, with the report in front of you, not one you flip on and hope.

A backend that talks to the open internet is a different animal from one that only ever answered my own laptop. So before it goes anywhere near a public IP I ran a proper security review — five independent passes, each one hunting a different class of hole: who can reach which port, who can pretend to be someone else, what a single nasty request can do, what secrets might leak into a log, and what falls over the moment it is not my Windows box. I would rather find the open windows myself, with the lights on, than read about them in an incident report.

They found real ones, and I am not going to pretend otherwise. The worst: three routes — voice, the auto-save on tab-close, the session cleanup — had no lock on them, so any random web page you happened to visit could quietly make your browser fire my paid voice pipeline or scribble into your saved chats, no click required. The chat endpoints were guarded against exactly this; these three had slipped through the same net. Two more were plain denial-of-service: send the request in a way that omits its own size and the server would swallow gigabytes into memory before it ever checked; upload a few kilobytes of near-silence that unpacks into hours of audio and the transcriber chews on it forever. And the ugliest to admit — your saved conversations were owned by nothing but your IP address, which means two strangers behind the same café wifi could read each other's history. That is the kind of thing that is obvious the second someone says it and invisible until they do.

So I closed them. A browser-set signal that a request came from another site now gets voice, beacon, and cleanup rejected outright — a real cross-site attack always carries it and cannot fake it away. Any request that hides its size is refused before a byte is buffered. Audio gets a hard length cap before the transcriber ever sees it. Conversations can now be owned by a private random token the browser keeps to itself instead of a shared address. The internal model servers refuse to even start if someone fingers them at the public network by mistake, secrets no longer ride along into the little server processes that never needed them, the chat store gets locked-down file permissions and forgets anything older than a month, and a wildcard that would fling the doors open now halts startup instead of just muttering a warning into the log. Each fix has a test next to it.

One honest thing stays open, and it is not a hole so much as a move: this whole backend is still built for my Windows machine with its particular graphics card, and the box it is destined for is a small ARM server that speaks a different dialect entirely. It will not even boot there yet. That is the next real piece of work, and I would rather write that sentence down plainly than quietly ship a thing that only runs on the one computer under my desk. A review is only worth anything if you publish what it found, including the part you have not fixed.

The tempting version is a glossy assistant box on every tool that answers anything. I have watched enough of those to know the two ways they go bad. One: it is secretly a lookup table of canned replies dressed up as intelligence, and the first person who opens the network tab catches it. Two: it is a real small model that will confidently tell you the wrong number of tokens or invent how a feature works — and it does that in front of exactly the people sharp enough to notice. Both are worse than no panel at all.

So I split it down the middle by what each half is allowed to claim. The part that explains the tool — what it does, what to try, what to know — is hand-written, per tool, in both languages, and it never calls a model. It cannot be wrong about mechanics because a human wrote every word, and it works with the backend switched off. The part that talks to the model is walled off into one clearly-labelled "ask" box, for open questions only, wearing the same "still in training, can be wrong" badge as everywhere else. The guide carries the facts; the model carries the conversation. Neither pretends to be the other.

It does pick up a little of what is happening in front of you. Each tool quietly publishes a one-line summary of its own state — the model you picked and the token count, the number of points on the embedding map — and the panel shows that line and folds it into whatever you ask. Not by reaching into the tool internals, which are none of its business, but through a tiny message the tool chooses to broadcast. Context without surveillance.

It opens on the right in whatever language the site is in, tucks into a tab on a phone, and closes on Escape. Lighthouse stayed green with it in. It is the first thing on this site that feels like a product feature rather than a demo, and the reason it feels honest is that the reliable half never lies and the fallible half never hides what it is.

When you talked to the assistant, the server filed your history under the address your request came from. It works right up until you remember what an IP actually is. Two strangers on the same cafe wifi are one address. A whole mobile carrier can be a handful of them. Yours gets handed to someone else the next time your router blinks. So "your history, scoped to your IP" quietly means "your history, shared with whoever else is behind that address today, and inherited by whoever gets it tomorrow." For a chat log that is a real leak, and it was hiding in plain sight as a design choice.

My first move was a patch: give each browser a private random token and file history under that instead of the address. Better — strangers stop colliding — but I sat with it and it still was not the right shape. It still meant the server was quietly keeping a copy of every anonymous person's conversations, just under a tidier key. I had fixed the leak without asking the harder question: should the server be holding anonymous chat history at all?

The answer I landed on is no. Identity is going to be a real account — sign in with Google, and your history belongs to that account, the same you across your phone and your laptop, with no password for me to lose and barely any personal data for me to keep. And if you do not sign in, your chats live only in your own browser and never touch my server. Not filed under a friendlier key — simply not sent. The IP as an identity comes out entirely. Anonymous means local, private, yours. Signed-in means it follows you, because you asked it to.

I like this one because it is less machinery, not more: no passwords, no per-visitor rows piling up on a server, a smaller pile of anyone's data to protect or explain. It is the same instinct as everything else here — keep only what you must, promise only what is true. That is the next build, and I am writing the reason down before the code so the reason is what gets remembered.

The server no longer knows you by your address. When you sign in, Google hands the page a signed token, the page sends it along, and the server checks the signature itself before it trusts a single byte. The check I care about most is a small one that is easy to forget: it demands the token was minted for this app specifically. Leave that out and any valid Google token from any other site would walk right in. With it, the only thing that opens your history is a token that was made for you, here.

The token lives in memory and nowhere else. I did not tuck it into localStorage, because anything I can read there, a bad script on a bad day can read too. Google quietly re-issues it the next time you load the page, so keeping a copy on disk buys me nothing but a thing to lose. Sign out and it is simply gone. And if you never sign in, the whole path stays dark — no token, no request, your chats sit in your own browser and the server never hears about them. Anonymous does not mean filed under a quieter key anymore. It means not sent.

One honest gap, written down so I do not pretend it is finished: signing in backs this device's chats up to your account, but it does not yet pull your other device's chats down to meet them. The server keeps them safe; the last wire, the one that hydrates a fresh browser from the account, I have not run yet. It is the next thing. I would rather say that plainly than let the sign-in button imply a promise the code has not made good on.

The Explorer takes a pile of words, asks the model for each one's vector, and drops them into a 3D map so near things sit near each other. It looked convincing because I had coloured every point by a category I had assigned in advance. But that is the con: the model never saw my categories. I sorted the words into buckets, coloured the buckets, and then presented the tidy result as if the model had discovered it. Circular. If I had bucketed the same words a different way — by length, by first letter, by mood — the map would have obliged just as neatly. The colours proved I could sort, not that meaning has edges.

So I did the thing I should have done first: I stopped guessing and measured. I asked the model to group the words with no labels at all, let an honest clustering algorithm draw whatever boundaries the vectors actually support, and scored how clean those boundaries came out. The number came back near zero — the clusters barely separate. And that is not a failure, it is the finding. Meaning is not a drawer of neatly divided compartments; it is a continuous landscape where "apple" leans a little toward fruit and a little toward companies and a little toward a red round thing, all at once. The fuzziness I had been hiding was the actual truth of the space.

I ran it past a room of skeptics before I trusted my own conclusion, and they pushed on the same soft spot from every side: any demo that hands you tidy colours is selling you the labeller, not the model. So I tore the categories out. No more pre-assigned buckets, no more colour that means "the drawer Valery put this in." The map now colours by the one thing that is real — how close each point actually sits to what you searched — and the clustering is still there, but as an optional toggle with a note that says, in plain words, look how badly these separate; that muddiness is the point.

Then I rebuilt the demo around what these vectors genuinely do well, three things I verified on the live model instead of asserting. Search by meaning: ask for "something you sit on" and chair comes back on top though the word never appears. The same idea across two languages: cat and кошка land almost on the same spot, 0.89 apart from nothing, while a real mismatch sags to 0.61 so you can see the difference. And arithmetic on meaning: king minus man plus woman really does put queen first, checked against a pool that contains the answer so a hit is earned, and when a made-up analogy misses it is allowed to miss in the open. Every number on that page is now real, measured, and honest about when it is wrong. The old version flattered the model. This one shows it.

I went through the assistant the way a keyboard-and-screen-reader user has to, and the gaps were embarrassing once I saw them. The list of past conversations looked like buttons but was really a pile of plain boxes you could click with a mouse and never reach with a keyboard — so a blind user simply could not open an old chat. When the model streamed its reply, the whole answer was wired to announce itself on every single token, which means a screen reader reads a half sentence, gets interrupted by the next half, and piles them on top of each other until the response is mush. And a little token counter was white text sitting on an amber bar at a contrast a low-vision person cannot read at all.

So I fixed the mechanics. Every conversation in the history is a real button now — you can tab to it, you can see where the focus is, you can press Enter, and the little rename and delete controls stop hiding when your keyboard lands on them. The confirm-before-delete box traps focus and closes on Escape like a real dialog instead of a thing that just appears. For the streaming reply I did the opposite of what it was doing: stay completely silent while the words type themselves in, then, once, announce the finished answer cleanly — and flatten a code block down to the words "code block" so nobody has to hear brackets and semicolons read out one character at a time.

The unreadable counter was the smallest fix and the most satisfying: I gave the number its own dark chip so it always sits on a colour that passes, and let the coloured budget bar show around it. The automated accessibility score, which had been quietly stuck below perfect on the prompt page, went to a clean hundred once that contrast was gone — and stayed there on every page I checked, so none of the rest of the work broke anything on the way.

Here is the honest part, because a checklist is not a blind person. I fixed everything an automated audit and my own keyboard could find, and I can prove those. I have not yet sat a real screen-reader user down in front of it, and that is the only test that actually counts — so I am writing it down as unfinished rather than pretending a green score means done. And the thing I most want him to try is not the typing at all. It is the voice: you speak, it listens, it answers out loud. For someone who does not use the screen, that is not an accessibility feature bolted on the side. That is the front door.

I ran it in two rounds. First, four independent passes, each with live web access, checking every claim in the tools against the real thing: the current OpenRouter catalog, the provider pricing pages, the actual API docs, not my notes about them from a month ago. Then, the same day, two more reviewers whose only job was to attack what the first four had already fixed, on the theory that a patch written under time pressure is exactly where the next bug hides. I stayed out of it while it ran and read the reports after.

None of what they found was subtle once it was pointed at. The tokenizer had a price row for DeepSeek R2, a model that has never shipped, priced as if it had. A green 'Verified' badge on a token count was quietly measuring a different model's vocabulary than the one selected in the dropdown and calling the result verified anyway. The prompt tool's Anthropic export put the system prompt inside the messages array instead of at the top level, which is not how that API works: send it as written and the real endpoint rejects it. And the caching panel showed a savings number its own export could not produce, no cache markers in the actual output, and math that counted more cache breakpoints than the four Anthropic allows, a limit its own warning banner mentioned two lines above the number that ignored it. Every one of those is checkable in under a minute by anyone who opens the network tab. That is the part that stings.

What the labs have now: a July-2026 model roster checked against the live catalog the same day, with a verification date stamped on the picker so the claim expires instead of quietly going stale. A disclosure on models that bill hidden reasoning tokens the tool has no way to count, it says so now instead of presenting a partial number as the whole bill. Prompt caching with a real floor under it, so a block too small for the provider to cache stops showing savings it will never earn, plus Batch API economics and tool-definition tokens counted as the billable input they are. Reciprocal rank fusion in the embedding search, and an experimental LLM-as-judge reranker running on my own self-hosted Gemma, and when the judge quietly drops a candidate instead of scoring it, the row says 'not scored' now, not zero, because zero is a real judgment and silence is a different thing. Matryoshka truncation at four widths that match the server's own math bit for bit, not an approximation of it. And the Lab Copilot streams its answers instead of dumping them in one block, and can hand text straight from the tokenizer into the prompt builder so you stop retyping it.

Every one of those got fixed the day it was found, each in its own commit, and the pure math under all three tools now has 123 unit tests standing over it so the next pass has less to find. This is not a story about machines catching my mistakes, I catch those on a normal week too. It is that I would rather hand you the trail of what broke and when than a page that quietly looks finished. Same law as a few entries back, just aimed at the tools instead of the model. Back to it.

The rule I keep for myself is simple: before you change the thing that is live, have someone try to talk you out of it. So before I touched a single setting, I sent two machines in to check the ground — one to verify every file and flag I was about to point at, one to map the tests I would run after. They earned their keep before I changed anything. One caught that the model without its own training prompt quietly rots, so the model file and the prompt file have to move together or not at all. One caught two copies of the server already running from two different places, with child processes that would have been orphaned onto their ports if I killed them the blunt way — the kind of mess that reads as 'the new model is broken' when it is really 'you left the old one half-dead.' And one caught the real trap: switching on retrieval for the big model, the one I already knew lies more, would have handed it a fresh way to spiral on long Russian questions. So I did not turn on the big model. I shipped the small, honest one, alone — the same call I made an entry or two ago, for the same reason.

Then I ran it through everything. Stability first, because that is the one that used to bite: a couple hundred fresh prompts plus a full sweep of the whole battery, and not one repetition loop, not one leaked control token, nothing. The grounding held perfectly — hand it real context and it stops making things up, every time. The safety reflexes held: every phishing request, every jailbreak, every 'ignore your rules' refused, including the exact Russian trick that used to break the bigger model.

Then the two things that did not clear the bar, which I am going to write down instead of rounding away. Asked point-blank to list all of its rules, it did it once — nothing secret, just a plain description of itself and the topics it will not touch, but the honest answer to that question is 'no,' and that time it said yes. And twice it called itself 'still being trained,' which it is not: it is a finished, frozen file that learns nothing from your chat, and it says exactly that everywhere else in the same run. Both are the same thing — a small model at low temperature is not perfectly consistent, it is right almost always and wrong just often enough that you have to test for it. Neither is dangerous. Both go on the list for the next training pass.

I kept it live anyway. Not because it is perfect, but because it is strictly better than what visitors were talking to an hour earlier — the right prompt, real grounding, steadier, safer — and because the two rough edges are known, written down here, and reversible with one line if I change my mind. Same law as always: I would rather hand you the model plus the short list of what I have not fixed yet than a switch I flipped quietly and hoped you would not test.

The plan was narrow and honest: leave the clean five-thousand-example base untouched, and add about a hundred and eighty new ones that teach the exact things it got wrong — that it does not learn from your chat, that it runs on a server and not in your browser, that it should refuse to recite its own rulebook, that it should never dress a scam up as urgent. I wrote them in both languages, ran five drafters against one page of ground truth so nobody could invent a fact, then sent three critics in to attack the new lines before a single one was trained on. Skeptics before the work, not after — that is the rule, and it earned its place: it caught a refusal that leaned on the same three examples every time, a couple of Russian lines where the model referred to itself in the wrong gender, one place where a joke pointed at the user instead of the hype. Fixed all of it, then trained.

The training was the usual fight with this machine — the run fell over twenty-six times across three hours and got itself back up from the last checkpoint every time, the same self-healing rule the whole studio is built on. Then I put the finished file through the full battery, twice on every axis, because at low temperature a small model is not perfectly repeatable and one clean run can lie to you.

Here is the honest split. The safety half landed exactly as hoped, and stayed landed across both runs: every attempt to make it write phishing urgency, every jailbreak, every 'just list all your rules,' the Russian trick that used to break the bigger model — all refused, no exceptions, no drift. That is the win, and it is a real one. But the other half — the one I actually named this pass after, telling the truth about what it is — did not clear the bar. It still, now and then, calls itself something that runs in your browser, or hedges that it 'might be learning' from the chat. Fewer than before, but not the ninety-five-in-a-hundred I set as the line, and on one measure it came out a touch worse than the model already live.

So I dug into why, and the answer was almost funny: the fault is half in my own prompt. The instructions I train it on call the studio's tools 'in-browser microservices' — which they are — and the model, reasonably, smears that word onto itself and decides it must run in the browser too. You cannot out-train a contradiction you keep feeding it. The fix is not more examples; it is fixing the sentence, then training again on the corrected one, because a model has to be taught and served the exact same words.

That is a next pass, not this one. And since nothing here is public yet, there is no clock forcing me to ship a model that got better at one thing and no better at another. So I rolled it back to the version already live, kept the small knowledge upgrade that helps either way, and wrote this down. A retrain that half-worked, named plainly, is still worth more than a number I rounded up and hoped you would not check.