Pollarise everyone

The tactics of Net Zero Watch, the shadowy group of Brexity climate change denying MPs and lobbyists hit a new low today.

The results of a new poll were released in the Sun on Wednesday 6th April. Strangely the paper didn’t mention who had commissioned it, but it soon became apparent that it was Net Zero Watch.

They claimed that the poll showed that:

By a margin of 44% to 36%, the public are in favour of lifting the controversial ban on shale gas extraction

This was suprising as it shows a strikingly different outcome from the polls conducted by the BEIS on a quarterly basis over the last few years, the last of which showed that only 17% of people supported fracking, while 45% opposed it. Importantly that survey also showed that 49% of people admitted that they know little or nothing about fracking. The series of polls has shown that those with greater awareness of fracking were significantly more likely to oppose than support it.

Against this background the Net Zero Watch commissioned poll asked the following question:

Shale gas is a natural gas found underground, extracted using a method known as hydraulic fracturing (fracking). There are large reserves of shale gas in parts of Northern England, particularly Lancashire, but there is currently a temporary ban on fracking in the UK. Some argue that in light of Russia’s invasion of Ukraine, the UK should meet its gas needs through using home-grown sources of energy, and that means lifting the temporary ban on the use of shale gas. Others say that lifting the temporary ban using shale gas would cause a significant environmental impact and argue that it will not have a significant impact on energy prices to UK consumers. Which of the following statements is closest to your view?

The UK should lift the temporary ban on shale gas extraction

The UK should not lift the temporary ban on shale gas extraction

Don’t know

No prizes for guessing which answer Net Zero Watch were hoping to get from the survey, by the way.

In market research, professional questionnaire designers are at pains to avoid weighted or leading questions. A leading question is a question asked in a way that is intended to produce a desired answer.

The wording of a research question can have a massive impact on the results of a survey. This example from https://cxl.com/blog/leading-questions/ shows this very clearly:

In cases of incurable disease, doctors should be allowed to “assist the patient to commit suicide”: 51% agree

In cases of incurable disease, doctors should be allowed to “end the patient’s life by some painless means”: 70% agree

So let’s consider the question asked in the Net Zero Watch survey, bearing in mind that a large proportion of respondents are likely to admit that they know little or nothing about the subject and are therefore more susceptible to manipulation.

The actual questions are actually quite straightforward. The respondent is asked to choose from 3 choices which are phrased in an objective and unweighted way. On their own they might be expected to result in a reasonable outcome, probably with a small majority answering Don’t Know.

The UK should lift the temporary ban on shale gas extraction
The UK should not lift the temporary ban on shale gas extraction
Don’t know

However, they are preceded by a preamble which “educates” the respondent and leads him/her towards a desired response

We can break the preamble down:

Shale gas is a natural gas found underground, extracted using a method known as hydraulic fracturing (fracking).

This is not overly contentious although the use of the word “natural” is perhaps not necessary and immediately brings a value judgement into the narrative, suggesting that fracking is “natural”

There are large reserves of shale gas in parts of Northern England, particularly Lancashire, but there is currently a temporary ban on fracking in the UK

Here the question designer sets up the apparent unreasonableness of the temporary ban given the “large reserves” which they claim lie under Northern England. No qualification or quantification of these reserves is provided. It is not needed. They just want to place the available bounty into opposition with the temporary ban. There is in fact no ban either – it is a moratorium, but ban is a stronger word and will of course elicit a more polarised response amongst those who are not aware of the distinction.

Now we move on to the final part of the set up

Some argue that in light of Russia’s invasion of Ukraine, the UK should meet its gas needs through using home-grown sources of energy, and that means lifting the temporary ban on the use of shale gas.

It is absolutely unnecessary to this sentence to preface it with “Some argue that in light of Russia’s invasion of Ukraine” but harnessing the tragic events unfolding in the Ukraine, but this does of course freight the question with all of the horror that we see unfolding daily. Equally the use of “home-grown” suggests a natural, harmless source of energy. It almost goes unnoticed that if “some argue that …the UK should meet its gas needs through using home-grown sources of energy” they are going to be very disappointed because even the fracking companies don’t claim to be able to provide energy security for the UK. The less well-informed may assume here, of course that this is though a reasonable proposition because they are being told it is. Instead we are presented with the false choice that if we want to meet (implicitly all of) our gas needs using domestic sources then the only answer is to remove the non-existent “ban” on fracking.

To set up the alternative answer we have a much less emotionally charged proposition

Others say that lifting the temporary ban using shale gas would cause a significant environmental impact and argue that it will not have a significant impact on energy prices to UK consumers.

Note that the environmental impact mentioned is a singular one, and the proposition does not have an equally weighted instruction at the end – “and that means keeping the temporary ban on the use of shale gas“.

The respondent is now primed to provide the answer that Net Zero Watch require.

In a way it is surprising that with such a loaded question they were only able to get a 44/36 win, although they were clearly able to convert enough of the Don’t Knows from other surveys.

Just for fun imagine if the preamble had been phrased like this

Shale gas is found underground and extracted using a method known as hydraulic fracturing (fracking). There are reserves of shale gas in parts of Northern England, particularly Lancashire, but there is currently a temporary ban on fracking in the UK as a result of the earthquakes caused when they fracked at Preston New Road in Lancashire. Some argue that the UK could meet some of its gas needs through using fracking. Others say that lifting the temporary ban using shale gas would cause significant environmental impacts and that it will not reduce energy prices to UK consumers. Which of the following statements is closest to your view?

Do you think the results might have been different?

Or maybe just compare the Net Zero Watch question with the one asked by the BEIS in their quarterly tracker:

From what you know, or have heard, about extracting shale gas to generate the UK’s heat
and electricity, do you support or oppose its use?

This one, when last used, resulted in 17% support and 44% opposition. As a lot of people seem unaware of the subject the “Don’t Know” and “Neither Support nor Oppose” was a relatively high 39%.