We deleted our own speed improvement

We made our AI slower on purpose. Here is the story.

A change made every article about four seconds faster. It worked. It also broke two articles out of every six. We deleted it, and we wrote down why.

Four seconds is easy to sell. A broken article with your name on it is hard to take back.

I run Mastheads, an AI newsroom that writes and checks articles for websites. My own name sits on the sites it writes for. Every AI writing tool says it is faster, smarter, better. None of that can be checked from the outside. So if I were buying one, I would want to know one thing. What does the team do when a change works most of the time, but not always?

Here is what we did. It is a small story, and that is the point.

Four seconds

Mastheads takes three to eight minutes to write one article. Along the way it runs checks on the draft. Two of those checks used to run one after the other. We tried running them at the same time.

It worked. It saved about four seconds per article. It also broke two articles out of every six.

So we went back to the slow way. And we wrote down why.

Four seconds is not a feature. Going back is not a launch. I am telling you anyway. How a team handles a change like that tells you a lot. It tells you what will happen to your site later. A pricing page cannot tell you that.

The instructions are not the product

Here is what people get wrong about building with AI. They think the instructions are the product. They are not. An instruction to the AI model is a request. The model treats most of it as a suggestion.

Ask for a word count and you get about that. Ask for at most three links and sometimes you get five.

So the whole of Mastheads rests on one rule. We do not trust the model to follow instructions. We check its work in code and fix what it got wrong.

That rule gives you one promise you can hold us to. A link to a page we never fetched cannot appear in your published article. In plain words: if the model makes up a link, we remove it before your article is built. It never reaches your website, the API or your screen.

I want to be exact about that. It does not mean the model cannot make things up. It can. It means a made-up link cannot end up in your article. The small claim is the true one. The small claim is the one you get from us.

Why AI writing all sounds the same

This one is not a Mastheads problem. Everyone has it.

Every AI article opens the same way. It repeats the question. Then a line about the modern world. Then three subheadings. People blame lazy instructions. That is not it.

The model has no memory between articles. Every request is the first one it has ever seen. So "vary your openings" is an instruction it cannot follow. It has nothing to vary against.

We have the receipt, twice. The old version of Mastheads asked the model to take turns between six ways of opening an article. Articles still opened on a number. The new version measured it from the other side. Six of eight real articles opened by repeating the summary they were given.

So we stopped asking. We pick the opening in code, before the model writes a word.

That is a smaller claim than "our AI has more personality". It is also the only version that is true.

Three questions to ask anyone selling you an AI writer

You cannot read our code. So here is what to ask anyone selling this kind of tool. Us too.

Ask what happens when a check cannot run. Everyone has checks. Ask what happens when one of them fails to run. Ask if anyone wrote down the reason.

We decided ours one at a time. If we cannot find sources for a topic, we say no before we write a word. Nothing has been spent yet. If a finished article falls a little short, we do not throw it away. You paid for that work. If nobody has read an article, it does not go live on its own. Publishing is the step that can hurt you. And if we cannot read a customer's signed terms, we let them in anyway. Locking a paying customer out is worse.

Put simply: you are never charged for nothing, and we never publish something we could not check.

Ask how they test a change. When we change how we judge sources, we test it first. The test uses the same fixed set of pages every time. A change does not get in because it reads better.

The last one nearly went wrong. The new version rejected almost every page. In the test it produced an article with no sources at all. Because the pages were fixed, we could see it. It did not look like search results moving around. We fixed it and ran the test again before any customer saw it.

Ask what their tests look for. Our tests do not only check that the product works. One of them looks at every place where Mastheads can spend your money. If a new one appears with no guard on it, the build fails. It finds those places itself. Nobody has to keep a list.

That test once had a blind spot. It passed while checking almost nothing. We fixed it. Then we added a second check. The first one must find a minimum number of places. A test that checks it is really checking.

We have written more code to test Mastheads than to build it. There is more test code in the engine than there is engine: 25,274 lines of product against 27,220 lines of tests.

What this is worth to you

Nothing, directly. Nobody buys tests. You buy articles.

But you are handing a system your website's name. Sometimes the login to your website too. The difference between tools will not show in a demo. It shows up a year and a half in, on the day something is wrong. That day, you need to know if anyone will tell you.

So my pitch is not that we are faster. It is this. Mastheads was refined across more than 20,000 published articles. We measured the change that would have made it faster still. We deleted it, because it was not worth what it cost.

New accounts get ten articles over seven days, free, with no card. Run a batch and judge the output. This piece is about the part you cannot see in the output. That part is what matters on a bad day.