ferroman@systems: ~ — fish 3.7
← ~/blog

Engineering got faster, understanding did not

AI made writing code roughly ten times cheaper. Understanding code did not get cheaper at all. The bottleneck moved to QA capacity and product attention, and the only fix that scales is a compounding loop of automated regression.

Writing code got roughly ten times cheaper. Understanding code did not get cheaper at all. Everything that has gone wrong in our delivery this quarter is downstream of that one asymmetry.

Where the bottleneck went

It did not disappear, it moved. Velocity is now fast enough that a change can be built, reviewed and merged before anyone has had enough time to look at it and say something about it. The constraint used to be typing. Now the constraint is QA capacity and product attention, and neither of those compresses, because both of them are made of people understanding things.

You cannot AI your way through “does this actually do what we said it would do”. Somebody has to hold the intended behaviour in their head and compare it to the real behaviour. That is understanding work, and understanding work runs at human speed.

Coverage is a choice, not a completeness target

We should not try to cover everything. We should make sure the agents and the developers know the exact scenarios the product is supposed to serve — the vision of the product — and treat everything else as derivative of that. Right now we are not covering even those fully.

That is the part that scares me more than the speed. Speed with a clear target is a good problem. Speed without one is how you end up with a large amount of confidently produced work that nobody really understands. And the understanding gap will only grow from that point.

The only real fix is a reinforcement loop, not more hours

There are two ways to increase QA capacity: hire, or automate the regression. The second one compounds and the first one doesn’t.

The loop: the more automated tests exist, the more QA time is freed from repeating the regression by hand; the more time QA has, the earlier they can cover new features; the earlier they cover new features, the more the developers write tests as part of the work instead of after it; which frees more QA time. It is slow to start and it is the only thing that scales with the velocity we now have.

Hiring does not really close this. Hiring adds a person who has to be brought up to speed (and maintain understanding over time), by the people who are already the bottleneck.

The bumpy road

This is going to be uncomfortable for a while. We are effectively onboarding a new person every single day — that is what it feels like to work with agents that have no memory of yesterday’s context and no accumulated judgement about our system. The cost of that onboarding is paid by whoever knows the system and domain best, which is the same small group that is already the constraint.

The trade nobody wants to name

I hit the honest version of this on a piece of work I did myself in four days. If I had handed it to the team it would have taken two weeks minimum of cross-communication alone. So we are in a situation where we either rush it solo and it is fast, or we slow down and pull more people in — and pulling more people in is the thing that makes us faster next quarter and slower this one.

I do not think there is a clever answer. I think there is a decision, and it should be made deliberately rather than by default, and right now we are making it by default every time somebody picks up a ticket alone because it is quicker.

The camera week — automating what you don’t understand

A week of camera debugging on a client project: customers hand over RTSP URLs, some of them work and some of them don’t, and nobody knew which port to check per vendor. A week was spent poking at what was essentially a one-field problem, because the system had never been mapped.

The point is that you cannot automate what you don’t understand. Once it is understood, the tool is trivial; and the understanding outlives every tool you build on top of it.