The heart of the matter – are opinion polls history?

Over the weekend in Australia, the federal election results took everyone by surprise. In the lead-up to voting, the polls predicted an easy win for Labor with 51:49 on a two-party preferred vote.

With the country coming to terms with getting it so wrong, some lucky punters benefiting from the longshot odds and others getting paid out early on a Labor win (SportsBet paid out $5.2million!) one question I keep asking myself is how were the polls so far off base?

This is not an isolated incident when we look at Brexit, the US election and now the Australian election. So, is the opinion poll as we know it dead?

One of the major reasons for this polling inaccuracy is, without doubt, the automation of polling research. While technology is an amazing enabler, it cannot replace the human process of sounding out how a person really feels.

Robocalls are now commonplace in the collection of data through fixed telephone lines. Yes, landlines! These are cheap but have a low response rate, especially among younger voters. The previous system where people called and asked questions, while not perfect, seemed to provide a more accurate result. Human to human interaction is much more effective in eliciting honesty and creating a sense of importance in the questions being asked.

The lack of trust in automated systems is also an important factor.  It is very unlikely that anyone appreciates getting a phone call from a computer, and even less likely that they will trust it enough to give it their personal opinions.

I am a big believer that technology can be used to improve systems and processes, but it has to be a technology that works with the people it is trying to help. Obviously, cost-cutting was a contributing factor, but electoral polling clearly requires a rethink in terms of how technology is applied to surveys. Conversely, social media analysis is proving to be more accurate through monitoring keywords, trends and comments and may well be the way to go.  Perhaps this also raises broader questions around the rapidly wider use of bots and AI to deal with everyday issues and problems. We must never underestimate the power of human to human intuitive interaction to get to the heart of a matter. This is something that cannot be automated or replaced.