A check that can't say no isn't a check

For twenty days a cost dashboard reported our cloud bill accurately every morning. It was blind to 90% of the money.

Nothing was broken. The dashboard read Cloud Monitoring, summed the estimated cost of the Cloud Run services it enumerated per project, and reported the total. Cloud Run genuinely did cost about A$0.18 that period. The A$33.70 sitting in Cloud SQL never appeared — not because of a bug, a stale cache or a bad query, but because no possible reading of that instrument could have shown it.

The remedies in the bias literature are attitudes — and attitudes don’t survive the handoff

Daniel Pereira’s guide to cognitive biases (2022, ISBN 978-1-998007-39-4) sets out the standard account, built on Kahneman and Tversky’s work from the 1970s and 80s: confirmation bias is the tendency to seek out, favour and remember information that supports what you already believe, while discounting evidence that doesn’t. It runs consciously and unconsciously at the same time, and it’s self-reinforcing — each piece of agreeable evidence makes the next contrary one easier to discount.

The counter-measures the literature offers are consistent across sources: recognise that you’re susceptible, cultivate intellectual humility, diversify your information sources, assign someone to argue the other side, actively seek disconfirming evidence, update your beliefs as new evidence arrives.

Every one of those is correct. Every one of them is also a property of a person, in a room, at the moment of deciding.

That’s the gap worth naming. Most analysis inside a company like this one isn’t performed by a person in a room. It’s performed by a script that runs at 6am, a dashboard someone glances at, an agent that summarises, and it’s read three months later by someone who wasn’t present when the assumptions were set. Intellectual humility is not a property that survives being written to a JSON file. Devil’s advocacy doesn’t run on a schedule.

So the bias relocates. It moves out of the analyst and into the instrument — and once it’s there, it’s harder to see, because instruments don’t sound like opinions. They sound like facts.

The one question that transfers: could this have returned “no”?

Here is the version of the discipline that survives being handed to a tool, and the single question worth asking of any check, dashboard, test or agent output:

What is the complete set of results this check can produce — and does any of them contradict what I already believe?

If none does, the check isn’t evidence. It’s decoration. It will go on agreeing with you at any level of resolution, forever, and each agreement will feel like confirmation because that’s exactly what confirmation feels like from the inside.

This also explains the most common wrong reflex. When a number looks suspicious, the instinct is to get a better version of the same instrument — more granularity, longer history, a nicer chart. That refines the agreement. What’s needed instead is a source with a different failure mode: something that would break in a different place, so that the two disagreeing is informative.

Three times this happened to us

The cost model. The audit that finally answered the question didn’t come from a better dashboard. The founder asked whether the reported numbers reflected money actually spent, and the answer was sought somewhere else entirely — the billing reports themselves, read directly, across seven separate billing accounts that had never been consolidated. Real spend for 1–20 July 2026 came to A$37.24, of which A$33.70 (90.5%) was Cloud SQL on two apps — in a service category the estimate model had never queried. One of those lines had risen 959%. The estimate wasn’t lying. It answered the only question it was capable of answering.

Sun hours in a 3D garden. A landscape design tool needed to tell a user how much sunlight a given tree receives. The obvious implementation casts rays with the full obstacle set loaded — which means the tree’s own canopy blocks every ray it casts. Every tree in every garden came back shaded. The build was green, the console was clean, and the feature was confidently wrong. It surfaced only when someone sat with the output and asked whether the number was actually right.

The word “idempotent.” A generator that injects structured data into pages carried a guard so it would replace its own block rather than append a new one. The guard’s regex never matched its own marker, so every run appended another copy. The function was named after the property it lacked, and reading the code confirmed the belief the name created. What broke the tie was running it twice and comparing the output.

The shape is the same in all three. Nothing malfunctioned. Each artefact did exactly what it was built to do. The failure was that the set of results it could produce contained no disagreement.

Where this lives on the canvas: box R

The Business Solution Canvas has one cell that sits outside the three wings — R, risks and open assumptions.

Most teams fill a cell like that the way they’d fill a risk register: external things that could go wrong. A competitor moves, a regulation changes, a key person leaves. That’s a useful document, but it’s a different one. R’s job on this canvas points inward. It names which other cells are currently beliefs.

That matters because a completed canvas has a quietly dangerous property: every cell looks equally confident. A customer-validated M1 and a guessed M1 are set in the same typeface, at the same size, with the same authority. Nothing on the page distinguishes what was checked from what was assumed on a Tuesday and never revisited. R is where that symmetry gets broken deliberately, because the layout won’t break it for you.

How to write a line in box R

1. Name the cell and the belief, in one sentence. “M3 assumes infrastructure cost per active project stays under X.”
2. Write the disconfirming result. “We’d know we were wrong if ___.” If you can’t finish that sentence, you don’t have a check — you have a preference.
3. Name where that result would come from, preferring a source with a different failure mode over a higher-resolution version of the one you already trust.
4. Date the line. An undated assumption is a belief with tenure.
5. Give every cell a status — assumed, checked, or live — so the canvas stops presenting guesses and facts in identical type.

When this is overkill, and one honest limit

Not every cell earns this treatment. Thirteen lines in R is a canvas nobody will maintain, and an assumption ledger that isn’t maintained is worse than none, because it looks like diligence. Spend the effort on the cells that money, a launch date or a legal obligation hangs on. Leave the rest marked “assumed” and move on — that label is doing real work on its own.

The honest limit is bigger than the method. None of this detects bias; it only makes disagreement structurally possible. In all three cases above, the thing that actually caught the error was a person asking one question out loud — is this number right? The discipline shortens the distance between that question and a real answer. It does not ask the question for you, and nobody here has automated that part.

One thing worth keeping, though: an empty R doesn’t mean the model is confident. It means the model is unaudited.

The Business Solution Canvas is free under CC BY-SA at bixso.ai/bsc — the R cell is on it, waiting to be uncomfortable. If you’d rather work through your own model with someone, book a consult; the first one is free.