Coverage Is Not Health, and the Arithmetic Used to Say Otherwise

If you are evaluating monitoring for a portfolio, here is a question worth asking every vendor, and the answer tells you a great deal: what does your health score do with a unit you cannot reach?

We got this wrong, and it is worth writing down because the wrong answer is the intuitive one.

What the bug was

A unit we could not see sat inside the health denominator at a weight of zero. Which means our connectivity problem was arithmetically indistinguishable from the customer's failing compressor. Six dark units dragged a portfolio's score down exactly as hard as six broken ones.

A portfolio owner looking at that number would have concluded their equipment was in worse condition than it was, and the party responsible for the discrepancy was us.

Three outcomes, not two

The root cause is a modelling error that is easy to make. A monitoring product that only distinguishes something is wrong from nothing is wrong has nowhere to put the third case, and quietly reports it as one of the other two:

• We looked, and found a fault. A confirmed finding. Service needed.
• We looked, and found nothing. Good data, nothing wrong, baseline learned. Healthy.
• We could not look. No usable data at all. Not reporting.

The third is a fact about our monitoring, not about your equipment. Filing it under either of the first two misinforms you — and note that folding it into “healthy” flatters us while folding it into “faulty” flatters nobody. Both are wrong.

What it looks like now

Dark units leave the denominator and are reported beside it: 92% healthy · 6 units not reporting. A percentage always travels with a coverage count.

Two consequences follow, and both are deliberate. A unit that stays dark never escalates into an equipment fault, however long it is dark — because we still know nothing about the equipment. And it is still work, but it is our work: it belongs on an operations queue, never on your service queue.

Healthy has to be earned

The same principle applies at the other end. A unit reporting normally that has not yet learned its own baseline does not read as healthy. It reads as no faults found, which is the true and weaker claim — the safety and mechanical alerts are armed from day one, but slow-leak detection needs roughly three weeks of running data before it means anything.

Those units are excluded from the health score rather than counted as healthy, for the same reason as the dark ones. We have not finished looking.

Why this is a buying question

Because at portfolio scale, the number on the dashboard is the product. If it silently averages the vendor's blind spots into a statement about your assets, then the more units the vendor loses contact with, the worse your equipment appears — and you have no way to tell those two things apart from the outside.

“Your equipment is fine” and “we cannot see your equipment” have to be two different sentences. The first is a report on your assets. The second is a bill against us.

This post describes how our own alert engine is built. Where it states a figure from published research the source is linked; where it describes our engineering decisions, that is what it is, and we have labelled it rather than dressing it up as a finding.