Welwyn Hatfield resident and market research professional Darren Lewis explains why the Council’s bin survey belongs in the garbage.
The Garden Waste Consultation survey is rubbish. Here’s why.
You may have heard about the council’s current garden waste consultation. Or you may not, as they haven’t actually written to us, the residents, to tell us about it. But if you do manage to find it, at least there is a survey for you to fill in.
And what a survey it is too.
I should declare my interest. I work in market research, surveys have been my job for 20 years. I can confidently say this garden consultation survey is among the worst I’ve seen. And believe me, I’ve seen some shockers.
If the council attempt to use the survey as evidence for their actions regarding the consultation their incompetence will be clear for all to see. This study won’t stand up to even the most basic scrutiny.
Why is it so bad? Let me give you the three most glaring examples.
The survey isn’t representative of residents
Having a representative survey is crucial. It’s a golden rule for any serious and official survey. Most surveys are completed by a representative sample of the whole target population. This isn’t an exciting topic, but it’s important. Let me explain why.
If a nightclub wanted to open up on Parkway, market research may be conducted to assess views about whether a nightclub would be desirable. If the survey only asked residents aged 65+ there might well be concerns or negative reactions.
But what if the survey was only conducted among residents aged 18-30? Reaction would most likely be more positive. So two groups of people with two very different reactions to the same survey, but neither of them are fully representative of all residents.
That’s why a representative sample is vital to get the true picture. I could go on, but I can already see a few of you nodding off at the back.
This survey makes no attempt to be representative of local residents. A survey that is not representative of the target population is biased. And as a result, it’s worthless.
The survey is open to abuse
On 7th November the WH Times reported that more than 3,000 people had completed the survey. I think it’s reasonable to assume this would mean more than 3,000 individuals, each completing the survey once.
The same person can complete this survey as many times as they like. All they need to do is take the survey, clear their browsing history, then take the survey again, clear their browsing history, and so on. Or, take the survey using different devices, such as their home PC, then their mobile, a work PC etc.
If someone had the time, or maybe wanted to influence the results in a certain way, they could take the survey over and over again.
This is because there is no way of checking who has taken the survey, or if they are even a resident of Welwyn Hatfield.
A serious, representative survey would invite people to take part, giving them a unique serial number or password, which maintains the anonymity of the respondent (always important), but allows the people running the survey to know it’s been completed. Once a person has completed the survey, they can’t take it again.
A survey that can be taken repeated number of times, by the same person, who doesn’t even have to belong to the intended target group, is biased. And, as a result, worthless.
The questions are poorly designed
The two issues I’ve discussed above are enough to make this survey impossible to be viewed as a credible exercise, but the poorly designed and leading questions top it off nicely.
Collection of the brown bin is currently paid for by council tax. But there is nowhere in this survey where residents (or whoever is taking the survey) can state that they want to maintain the current service, or give an opinion about how they will feel if the service is removed. This usually occurs if the survey organiser knows what the answer will be, but doesn’t want to hear it (or doesn’t care).
At best, this is poor design. At worst it is an attempt to cover up actual opinion. Surely understanding actual opinion is the point of a consultation?
By not providing all possible options in relation to the service, the questions lead the respondent to give answers that are better for the council. They could say, for example “70% of residents would prefer a subscription service”, ignoring the fact that many residents would give a different answer if it was included in the survey.
A survey with questions that lead people to give certain answers is biased. And, as a result?
Yes, you’ve guessed it. Worthless.
So, what should the council do? Scrap this ridiculous survey, stop trying to give false credibility to their plans, start a proper consultation with residents and allow all views to be heard.
Or, you know, just collect the brown bins and include it in the council tax.
Welwyn Hatfield Labour Councillors have tabled a motion calling on the council to scrap this idea now. You can attend the Full Council meeting and see the debate for yourself at the Council Chamber in Welwyn Garden City on 21st November at 7.30pm