Latency only matters where a human is waiting

Latency is mostly psychological quantity. When we fight for latency we are fighting for a person’s attention, and that is the only reason. Physically there is almost no reason to optimise it — the exception is when you are actually moving a lot of data, and then it is throughput you care about, not latency.

Once you accept that, the engineering decisions get much simpler, because you can ask a question that has an answer: is there a human waiting on this?

The perception ladder

Under about 30 ms is instant. Nobody perceives it as a wait; it feels like the thing happened when you touched it.

Under about 50 ms is comfortable. You notice there was a moment, and it does not cost you anything.

Above roughly 300 ms it is all just “slow”. This is the part people miss when they spend a sprint taking 800 ms down to 400 ms: the user experience did not improve, because both of those live in the same perceptual bucket. You did real work and bought nothing.

These numbers are not made up. The Doherty Threshold, from research Thomas Doherty did at IBM, puts a line at about 400 ms: below it, people do not just feel the speed — they start thinking faster and understanding the system better. Above it, the work gets done. Below it, it gets done better.

Aza Raskin’s “The Blink of an Eye” maps the same territory as a ladder: under about 100 ms the interface feels like it reacted; around one second you are aware of the delay, but the connection is kept; around ten seconds anxiety sets in and focus starts to leave; past thirty seconds people stop waiting and start thinking about leaving. A human blink lasts roughly 300–400 ms — which is where the name comes from. If the feedback takes longer than a blink, you stop feeling connected to the machine.

If the work takes three seconds, the network is noise

This is the argument I keep having. Somebody wants to move a service closer to the user, or shave a hop, on a request whose actual processing takes three seconds. The network contribution is a rounding error inside the thing the user is already waiting for. Optimise the three seconds or optimise nothing.

Where it genuinely matters

Games, where latency is the product. Video calls, where lag breaks the turn-taking that makes conversation possible. Shops, where the funnel is measurable and a slow page has a price tag attached. In those cases you spend whatever it takes.

Everywhere else, ask who is waiting. Very often the honest answer is that nothing is waiting except a queue consumer that does not care.

The car key example

You press the fob while you are still walking toward the car. The unlock could take a full second and you would never know, because you have not arrived yet. There is no human attention on the result at the moment it is produced. That is a whole category of operation, and most background processing lives in it.

People have thresholds, not curves

The other thing I keep seeing is that human tolerance is not a smooth function. Somebody does not experience 200 ms as twice as good as 400 ms. What they have is a threshold: “I should be warned within three minutes.” Below it, indifferent. Above it, angry. Doherty’s result was the same shape: below 400 ms, people think faster and comprehend better; above it, nothing measurable changes. So the design question is not “how fast can we make it”, it is “what is the threshold, and are we inside it”.

This is where SRE meets the business, and it is one of the few conversations where the business genuinely has the better instinct — they know where the human is. We know what it costs to get there.

Fix the route everyone waits on

Without that, what gets improved is decided by anecdote, or by some guy who decided it matters. It can be completely unimportant — you are doing nonsense, improving what does not need improving. Meanwhile there is one route that everybody goes through, it is super slow, and that is the one to fix: the permission check, because it really affects people. Users come, and everything is slow because of it.

Instead someone wants to make a table load two seconds faster. If it loads in 30 seconds, the user genuinely does not care whether it is 28. What really gets on their nerves is a page that could appear in 150 milliseconds appearing in three seconds. Observability with an SLO behind it shows what exactly is slow, why, and how often it is used, so it becomes clear what needs fixing and what does not.

The wait can be shaped, not just shortened

The head works this way, and nothing will change it. Look at the queues at stations: a long queue looks frightening. Make people walk along a path to get in and it has a smaller psychological effect than making them stand and wait. Walking is easier than waiting.

People complained that elevators were slow. So they put a panel in the elevator that lights up floor by floor. You feel you are going faster, because the buttons light up as you pass the floors, instead of standing in a box that tells you nothing for 30 seconds. Ding, ding, ding — the same 30 seconds, psychologically completely different. Elevator music exists for exactly the same reason: the elevators were slow. It is purely psychological and has nothing to do with the real speed.

That is why we add percentages to spinners. Anyone will tell you a spinner with percentages is better than a bare spinner, if they are accurate: show how much is left and you feel progress. A bare spinner turning for twenty seconds reads as “something has hung”. Put a progress line under it — even one that is completely made up, that just crawls along in roughly the time the work usually takes — and psychologically it becomes “almost done, I’ll wait”. We can simply estimate that it takes about ten seconds and draw that.

UX is a psychological story, and it plays a role in SRE too, because SRE is work with developers and with the business. Psychological factors matter there as well.