A common problem with evidence sampling is drawing conclusions from insufficient data. This is related to the generalisation fallacy.

To prove a theory, it is not enough to observe a couple of instances that seem to support it. If we want to know what percentage of the population take holidays abroad, we can’t find out by asking five people, calculating the percentage, and applying the result to the population as a whole. We need more data.

This raises the question: how much data is enough? At what point does a data-set become sufficiently large to draw conclusions from it?

Of course, having enough data is not a black-or-white affair; there is no magic number of observations which, when reached, means that any conclusion drawn is adequately supported. Rather, sufficiency of data is a matter of degree; the more evidence the better. The amount of confidence that we can have in an inference grows gradually as more evidence is brought in to support it.