In the next few months millions of dollars will be spent on political market research – much of it on focus groups which take the form of group discussions.

Allegedly it allows political parties to identify what people are thinking and how to respond to that. It is not a consultative project but rather one often devoted to testing ideas which form the basis of slogans. It also won’t be much about policy unless it is to elicit ideas for attacks on opponents’ promises.

The focus group is a product of the distinctions between qualitative and quantitative research which date back to social sciences developments in the US and in Australia.

In Australia researchers such as University of Melbourne Professor A.F. ‘Foo” Davies, and Graham Little a pioneering political psychologist who was also at UoM, interviewed people at depth for the Images of Australia project. Hugh Mackay was also an Australian pioneer of such interviews which he used to develop quite detailed pictures of Australian society.

But over decades focus group research has evolved from that approach and Judith Brett’s new book, Doing Politics, has an important chapter (Writing Ordinary People’s Politics) which looks at the weaknesses of much political quantitative and qualitative research and outlines alternative approaches.

Graham Chant, former Principal at ChantLink consultants and before that an academic psychologist, published a paper (Qualitative Research: Issues & Techniques) psychology in 2012 which also described qualitative research methodological problems and recommended measures to address them.

Commenting recently on the paper he said: “While I have not been directly involved in Market Research now for at least three years, my observations prior to retiring is that usage/application of group discussions has progressively got worse. And this applies to Australia, but even more so to the USA.”

“The direction of group discussions in Australia has been negatively impacted by: group recruiting agencies having large data bases of people who are willing to attend groups (perhaps because they get paid or because they like the idea of participating). But this means that these people represent a biased set of respondents and do not necessarily reflect the broad Australian society (for instance, you could argue that wealthy people are less likely to participate as would be the case of those who hold more senior roles in business or the public service).

“The environment in which groups are conducted is now very much focussed on specialist group rooms that contain one-way screens, cameras and recording devices. When respondents arrive they sit outside in waiting rooms prior to being ushered into the group room. The environment is certainly not conducive to open discussions and I was certainly aware in the past that many groups had at least eight and often more participants.

“The original approach to groups was to conduct them in people’s homes or offices, environments which reflected something people were used to and encourage ‘real’ interactions. This is certainly no longer the case – it’s probably due to pressure from clients wanting to see the ‘real thing’ (as Russell Morris might say), and not necessarily trusting reporting from the market researcher,” he said.

Chants also raises questions about the now generally standard group of eight when in normal society real ‘groups’, rarely consist of more than four to six people.

Chant also warned that, even if groups are conducted according to the practices his paper recommends, the researcher still needs to beware of accepting the outputs as representative of a large population.

This is particularly a problem with commercial research which generally have results based on a limited number of groups although political attitudinal research usually draws on many more groups over longer periods of time.

But both commercial and political attitudinal research have a specific purpose. It is not consultation about what should be done but rather what can be done.

There is no doubt that political attitudinal research can be both excellent and valuable but given the secrecy surrounding it there is much about that it is worrying.

However, close observers can sometimes discern what the political parties are seeing in their attitudinal research by reverse engineering what they are saying by asking the perennial Lasswellian questions of: Who says what, which channel, to whom, with what effects, for what purpose and under what circumstances?

The questions are also a useful way to deconstruct articles by political reporters quoting sources – particularly those sources who insist on being quoted without attribution. In Australia the main source of this sort of stuff is the Prime Minister or the Prime Minister’s Office. Journalists continue to co-operate with these surreptitious efforts to distort reality because they don’t want to get taken off the drip.

The reporting of quantitative research is also problematic and it is not unusual to see a political journalist pontificating on a 0.5% percent change in a survey with a statistical error range of two to three percent. Because media outlets preference their own polling they tend to mention other polls so rarely do you see the detailed comparisons, aggregation and probability estimates such as those produced in the US by FiveThirtyEight.

Retrospective analysis – if based on actual research and not opinion – is often more robust and some of that has some implications for our views of the 2019 ‘miracle’ and possible 2022 outcomes.

For instance, if you tried to reach a consensus about the 2019 ‘miracle’ from media reporting you would conclude any or all of the following: the polling was wrong; voters didn’t trust Shorten; Morrison was a miracle worker; tradies swung from Labor to Liberals; coal miners hated Bob Brown; Labor’s policy platform was too big and threatening; and most Australians (whether they owned investment properties or shares or not) were terrified of losing their negative gearing rorts and their franking credits.

Yet some post-election Liberal Party analysis, discussed by Judith Brett in her essay on Scott Morrison’s Quiet Australians, suggested that the decisive factor was the vote swings of suburban working mothers in the 35 to 54 age group.

This doesn’t seem to have been discussed much anywhere – but if it’s right Scott Morrison could be in big trouble in 2022 as all the indications are that current Liberal Party support among females is the lowest in the Party’s history.

Indeed, election night might see Grace Tame, Brittany Higgins and a few former female Liberal MPs – among others – dancing on Morrison’s political grave along with lots of others.