It is quite often claimed that the ills that religion brings outweigh any good that it might do. This year, provocatively, The Guardian decided to mark Christmas by slapping precisely this claim across its front page. “Religion Does More Harm Than Good - Poll” was the headline as people across the country geared up to celebrate God becoming incarnate.

Of course, all polls have their problems. For a start, they are better as guides to public opinion than as guides to the facts (the appeal to popularity is a fallacy, remember). Then there are all of the difficulties involved in getting a representative sample of a sufficient size to be meaningful, and formulating questions that don’t have too much of a distorting effect on people’s answers.

The biggest problem with the article in The Guardian, however, is that the poll simply didn’t say what The Guardian said it did; they misrepresented the data.

The survey reported that 82% of those asked see religion as a source of tension. There is a big difference, however, between saying that a thing is a source of tension and saying that it does more harm than good. FA Cup finals are a source of tension. General elections are a source of tension. A-levels are a source of tension. That doesn’t mean that these things do more harm than good.

It may well be that most of the 82% who see religion as a source of tension also believe that tension to be trivial in comparison with the great goods that religion brings. That is, it is perfectly possible to answer “yes” to the question in the poll without endorsing The Guardian’s headline claim.

Thankfully, The Guardian didn’t make its case just on the 82% who see religion as a source of tension; they made a comparison with the 57% who said that religion is a force for good: “an overwhelming majority see religion as a cause of division and tension - greatly outnumbering the smaller majority who also believe that it can be a force for good.” (Note the subtle manipulation here; 57% said that religion is a force for good, but The Guardian weakened this to “can be a force for good.”)

The idea, then, is that because more people said that religion is a source of tension than said it is a force for good, the poll supports the idea that religion does more harm than good.

Presumably this is because to get these figures there must be some people who think that religion does some harm but no good (and so that it does more harm than good). But consider the figures carefully. What is the greatest possible number of people who did say that religion does harm but didn’t say it does good? 43%. A minority.

That means that for the poll to support the idea that religion does more harm than good, it must be assumed that a significant number of those who said that religion is both a source of tension and a force for good think that all things considered religion does more harm than good. Now that may be the case, but that’s a pretty strong assumption. Describing something as a “force for good” sounds a lot like saying “all things considered it does more good than harm”, whereas describing it as “a source of tension” sounds nothing like saying “all things considered it does more harm than good.”

The data simply don’t yield The Guardian’s headline claim. The Guardian, perhaps to try to cause a stir at Christmas, was putting words in people’s mouths.