Is Your KPI Dashboard Lying to You? An Audit Checklist

# Is Your KPI Dashboard Lying to You? An Audit Checklist

Most KPI dashboards don't lie outright. They do something more corrosive: they present numbers that are technically accurate and practically useless. The board nods, the team moves on, and no decision changes — because no decision was ever attached to the numbers in the first place.

This isn't a tooling problem. It's a thinking problem, and it survives every dashboard migration. Here's how to audit your way out of it.

The museum dashboard. Forty charts, updated monthly, reviewed never. Every metric was added because someone asked for it once; none were ever removed. The dashboard is a record of past curiosities, not current decisions.

The vanity gallery. Metrics that only move in one direction and flatter everyone: total registered users, cumulative revenue, social followers. They can't go down, they don't inform action, and they exist to be screenshotted.

The orphan metrics. Numbers with no owner and no threshold. "Churn is 4.2%." Is that good? Who decides what to do if it hits 6%? If nobody can answer, the metric is decoration.

The definition drift. "Active user" means something different in product, marketing, and the board deck — and the dashboard doesn't say which definition it's using. Two teams argue about the number; both are right about their own definition; the decision waits.

The lagging-only trap. Every metric describes what already happened: revenue, churn, NPS. Nothing on the dashboard predicts or can be influenced this week. The team is driving by rearview mirror.

Work through your dashboard metric by metric. For each one, demand answers to these five questions. A metric that can't answer all five doesn't belong on a leadership dashboard — it belongs in an analyst's notebook, or nowhere.

1. What decision does this number change? The only question that matters. Name the specific decision: "If this drops below X, we do Y." If you can't name one, the metric is a candidate for removal. Dashboards exist to trigger decisions, not to document the business.

2. Who owns it? Every metric needs a named owner — someone whose job includes moving it and explaining it. Shared ownership is no ownership. If the owner left six months ago and the metric stayed, that's your answer about its importance.

3. Is it leading, lagging, or both? Lagging metrics (revenue, churn, retention) tell you what happened. Leading metrics (trial activation rate, support ticket backlog, sales cycle length) tell you what's coming and can be acted on now. A healthy dashboard pairs them: every lagging outcome metric should have at least one leading indicator the team can influence this quarter. If your dashboard is all lagging, you're managing history.

4. Can it be recomputed? Ask the uncomfortable question: if two people computed this metric independently from the source data, would they get the same number? If the answer is "probably not" — undocumented filters, spreadsheet logic, a query only one person understands — the metric is a rumor with a chart. Tidal Arc's term for this is the recomputation test, and most dashboards fail it on at least a third of their metrics.

5. Is it a ratio, and is the denominator honest? Raw counts mislead; ratios inform. But the denominator is where manipulation hides — "conversion rate" against which population? "Churn" of whom, over what period, counting what as churned? Demand the denominator in the label or the footnote. A metric whose denominator you can't state is a metric you don't understand.

Three quick heuristics for spotting vanity metrics in the wild:

• It can't go down. Cumulative totals, all-time highs. If a metric has no possible bad news, it carries no information.
• Nobody's compensation or roadmap depends on it. Follow the incentives. Metrics that matter are tied to someone's decisions.
• It's precise about the wrong thing. "Average session duration: 4:37" — precise, and useless without knowing whether longer sessions mean engagement or confusion.

Expect to remove or demote 30–50% of a typical dashboard on first audit. That's normal, and it's the point. The usual outcome:

• Keep (5–10 metrics): pass all five questions. These are the leadership dashboard.
• Demote: useful for a specific team or investigation, but not a standing leadership metric. Move to team-level views.
• Fix: the decision is real but the definition is broken. Assign an owner, write the definition down, recompute from source.
• Remove: vanity, orphan, or duplicate. Archive, don't delete — someone will ask where it went.

Then write the definitions down. Every surviving metric gets a one-paragraph definition: what it measures, how it's computed, who owns it, and what decision it drives. This document — not the dashboard — is the actual KPI system.

A dashboard audit isn't really about dashboards. It's about whether the leadership team has agreed on what it's managing. The metrics are just where the disagreements become visible. Teams that do this exercise usually discover they were never arguing about the numbers — they were arguing about the strategy, through the numbers.

That's why it's worth doing properly, and why it's the kind of work that benefits from an outside eye: someone who will ask "what decision does this change?" and wait for the real answer.

---

Keep reading: an audit checklist only goes so far on its own — see two worked examples: the Executive Analytics & KPI Audit (a full eight-finding audit) and the KPI & North Star Framework (from 61 dashboards to 20 numbers). Or start a conversation: /contact/.

This article contains no client data and no case studies. The "30–50%" figure above is a heuristic from practice, stated as such — your mileage will vary by dashboard.