Four ways charity charts mislead (usually by accident)

Most misleading charts aren’t made by anyone trying to mislead. They’re made at 4pm on a Thursday, using the software’s defaults, for a meeting at nine the next morning.

That matters, because real decisions get made from these charts: which appeal gets the budget, whether retention is a worry, what goes in the trustee report. A chart that tells the wrong story by accident can still steer a decision.

The good news is that the same few problems turn up again and again, and each one is easy to spot once you know to look. Here are the four we see most often, followed by two short checklists: one for the people who build reports and one for the people who read them.

1. The squashed axis

Start the y-axis at 40% and a drop in retention from 45% to 42% looks like a cliff edge.

Sometimes zooming in is the right call. Three points of retention can be a lot of money. But it should be a decision, not something the software did for you.

If you read reports: check where the axis starts before you worry. If you build them: bars start at zero, always. If you zoom in on a line chart, make it obvious.

2. The average that describes nobody

An average gift of £45 can come from hundreds of £10 gifts and a couple of £1,000 ones. No actual supporter gives £45. A single legacy can move an average more than a whole year of appeals.

If you read reports: ask what a typical supporter looks like. If you build them: show the median, or gift bands such as under £10, £10 to £50 and over £50, alongside the average or instead of it.

3. Points versus per cent

Retention moving from 40% to 44% is up 4 percentage points. It is also up 10%. Both are true, and they sound very different in a trustee meeting.

If you read reports: when you hear “up 10%”, ask “10% of what?” If you build them: pick one, say which, and stick to it across the whole report.

4. The convenient window

Start a chart in April 2020 and almost anything looks like a recovery. Compare December with January and almost anything looks like a collapse.

If you read reports: ask what the chart would look like with a different start date. If you build them: compare like with like, such as this December against last December, and say why you chose the dates. If the story only holds for the exact dates on the chart, it isn’t much of a story.

Two checklists worth stealing

Print them, pin them up, or paste them into the front of your next report.

Before you send it (for data and insight teams)

  • The title says what the chart shows, not what it’s about: “Online donors lapse twice as fast as face-to-face”, not “Retention by channel”
  • I can name the decision this chart helps someone make
  • It’s clear what’s being counted: people, gifts or pounds
  • There’s something to compare against: last year, target or another channel
  • Bars start at zero, and any zoomed-in axis is obvious
  • Every percentage has its “out of how many” nearby
  • One thing is highlighted, and it’s the thing the title is about
  • One-offs, such as an emergency appeal, a large legacy or a CRM migration, are labelled where they happen
  • Someone outside the data team has read it and could explain it back to me

Before you nod along (for fundraisers and everyone else in the room)

  • I know whether this is people, gifts or money
  • I know what it’s being compared with, or I’ve asked
  • I’ve looked at where the axis starts and what dates it covers
  • For every percentage, I know “out of how many”
  • I can say the “so what” in one sentence, in my own words
  • I’m leaving with one question or one thing to try

Start with one report

You don’t need to fix every chart at once. Pick the report that goes round most often. Run it past the first checklist before it goes out next month, and ask a couple of the people who receive it to try the second.

If you’d like a hand reviewing what your organisation currently sends round, or building reports in Power BI or your CRM that answer the questions your teams actually ask, get in touch with the ADA team.