We often judge how common or likely something is by how easily examples of it come to mind. Psychologists Amos Tversky and Daniel Kahneman called this the availability heuristic. It is a handy shortcut, but it misleads us whenever some events are much easier to remember than others.
The examples that come to mind are a sample, but
they are not a
random sample.
They are chosen by what is dramatic, recent, or
emotionally striking, and that is a biased way to sample.
The remedy: before deciding that something is common
or rare, ask what the actual counts or rates are, rather
than relying on what you happen to remember.
A conclusion can only be as good as the data behind it. If the data that would have argued against a conclusion are left out, even honest arithmetic on what remains will mislead. This can be done on purpose ("cherry picking"), or it can happen without anyone intending it.
Reporting only the results that support your case.
Example: A fund manager advertises "up 40% over the
last three years," choosing a starting date right at a
market low. Starting the chart a year earlier, at the peak,
might show a loss.
Example: A researcher removes a few data points as
"outliers," but only the ones that weaken the result.
Sometimes the unfavorable data are discarded by the world
before we ever see them: we only get to look at the cases
that "survived."
Example: In World War II, the military studied the
bullet holes on bombers returning from missions, and
considered adding armor where the holes were most common.
The statistician Abraham Wald pointed out that these were
the planes that made it back. The places with no
holes, such as the engines, were where hits brought a plane
down. That is where the armor was needed.
Example: "They don't make them like they used to!"
The old buildings and appliances we still see are the ones
that lasted. The badly made ones were torn down or thrown
out long ago.
Example: Average returns of mutual funds look better
than they really were if the funds that did badly and
closed are left out of the average.
Studies that find a "significant" result are much more
likely to be published than studies that find nothing. The
studies that found nothing stay in a file drawer.
This matters because a test at the 5% significance level
will, by design, give a "significant" result about 1 time in
20 even when there is no real effect. Run 20 tests, report
only the one that "worked," and you have discarded the 19
pieces of unfavorable data that put it in context.
(See the z and t
tests page for what significance means.)
Cherry picking,
survivorship bias, and
publication bias
on Wikipedia.
xkcd's "Significant": multiple testing in one comic.