The spinner was lying

A spinner is a promise. It says: something is happening, it is happening for you, and it will be over soon. Ours span for eight seconds and only one of those three things was true.

The endpoint did real work — that part was honest. What it got wrong was who the work was for. Almost none of it was needed to answer the request. It was needed eventually, by a report someone would open tomorrow. We were making a person hold still for a job that had no interest in them.

Optimising was the wrong reflex

My first instinct was to profile it. Find the slow query, add the index, cache the expensive call. Good instincts, completely misapplied — there was no dumb mistake buried in there. The work genuinely took eight seconds, and shaving it to six would have changed nothing about the experience except the exact duration of the lie.

Two questions were being treated as one:

• How long does this work take?
• How long does a human have to sit there?

The first is a property of the work. The second is a property of your architecture. You can take the second to nearly zero without touching the first, and that is almost always the cheaper move.

Hand back a receipt, not a result

The rewrite was boring, which is the point. The endpoint stopped doing the work. It validated the input, wrote a job, handed back an identifier, and got out of the way. Workers picked the job up, spread the independent parts across themselves, coordinated through Redis, and wrote the result where the client could come find it.

python@router.post("/reports")
async def create_report(payload: ReportIn) -> ReportRef:
    report = await reports.create(payload, status="queued")
    await queue.enqueue("build_report", report.id)
    return ReportRef(id=report.id, status="queued")

The total work did not get faster. It got wider — the parts that never depended on each other stopped pretending they did — and, more importantly, it stopped happening in front of an audience.

What it costs

Worth being honest about the bill. You now run a queue, supervise workers, and own a job whose failure nobody is synchronously waiting to hear about. You need somewhere to put the result and a way for the client to learn it arrived. You have traded one simple slow thing for one fast complicated thing.

Above a few seconds that trade is nearly always worth it. Below a few hundred milliseconds it nearly never is. Knowing which side of that line you are on is the actual engineering.

Latency is not how long the work takes. It is how long you ask someone to believe you.