Every day we are confronted with statistics and – worse – with media, political and pundit interpretations of what they mean.

For instance once a fortnight we see the Newspoll results and the thousands of words devoted to analysing one or two percentage point shifts in a sample with a plus or minus error range of around 2.5% and undisclosed assumptions about preference distributions. The Australian unemployment figures are searched for significance even though the methodology underlying them, and the failure to account for people who have given up looking for work, mean that the real unemployment rate is probably double the official one. The Roy Morgan Research estimates are probably a much better guide to the real situation than the official ones.

But how do you guard against being misled? Some things are easy – the million jobs claim trumpeted by the Turnbull Government merely need to be correlated with the increases in population due to natural growth and immigration. But others are more complex.

Tim Harford, who writes leaders and The Undercover Economist column in the Financial Times (as well has having published four books), gave some valuable advice on the subject in the FT and in longer form on his website.

Harford’s advice was prompted by the 2013 comment by Professor Harold Pollack of the University of Chicago that: “The best financial advice for most people would fit on an index card.” For the online generation puzzling as to what the card might be it was once was, and sometimes still is, an important research aid. As Harford said: “…would this work for statistics too? There are some obvious parallels. In each case common sense goes a long way; in each case dizzying numbers and impenetrable jargon loom; in each case there are stubborn technical details that matter; and, in each case there are sharp people with an incentive to lead us astray.”

So, Harford’s index card guide to statistics covers: observe your feelings – in other words acknowledge the emotional baggage you bring to the numbers; understand the claim – elucidating this by citing Douglas Adams’ supercomputer Deep Thought’s philosophy that “Once you know what the question actually is, you’ll know what the answer means”; is this a causal relationship – whether a causal claim is being made and whether it is justified; what’s been left out – are the axes on a graph designed to magnify the impact of the stats (which was the most useful lesson for its PR career the blog learnt from Psych 1 at university); get the backstory – why are you seeing it and where did it come from; put things in perspective – for instance by asking if a claim is actually a big or small number or comparing it with the historical trend ; beware ‘statistical significance’ –a number can be statistically significant while having no practical importance; embrace imprecision – use some rules of thumb to put the numbers in perspective; be curious – what does it mean, cui bono, what’s missing; treat surprises as a mystery rather than a threat. Harford quotes (although it’s possibly apocryphal) Isaac Asimov in this context: “The most exciting phrase in science isn’t ‘Eureka!, but ‘That’s funny.’”

A corollary of this is the way numbers and quantification get misused in performance measures – a problem unfolding daily in the banking Royal Commission. A recent book, The Tyranny of Metrics by Jerry Z. Muller (reviewed in Science 2 February 2018) illustrates this. The book ranges over the infamous fetishisation of body counts in the Vietnam War; the Campbell-Goodhart principle that reliance on measurement for behavioural incentives leads almost inevitably to corruption; and how metrics can only be indicators and which can lead to terrible consequences.

Of course one ought not forget that statistics and metrics can  sometimes be extremely valuable, sometimes extremely misleading and sometimes a very inconvenient reality for postmodern theorists. In his various books Ben Goldacre recounts dodgy studies, rigorous approaches and plain idiocy. One of his books is I Think You’ll Find It’s a Bit More Complicated Than That – a collection of columns and articles from The Guardian and other publications.

In that book he reproduces a 19 August 2006 Guardian article about the invaluable Cochrane Library – a resource which does much to hasten what Goldacre once dubbed as the transition from eminence-based to evidence-based medicine. The article also contains one of the great putdowns of all time.

Goldacre’s column addressed a paper attacking the Cochrane Library, entitled Deconstructing the Evidence-Based Discourse in Health Sciences: Truth, Power and Fascism, published in the International Journal of Health-based Science. Goldacre initially thought it was a Sokal-type hoax but this one is real. It’s objective: “The philosophical work of Deleuze and Guattari proves to be useful in showing how health sciences are colonised (territorialised) by an all-encompassing scientific research paradigm – that of post-positivism – but also and foremost in showing the process by which a dominant ideology comes to exclude alternative forms of knowledge, therefore acting as a fascist structure.” If you would like a good laugh, and some risible attempts to justify its publication, search the controversy out online. The journal editors also didn’t obey the first rule of crisis management – fess up and apologise quickly.

Goldacre describes the Cochrane Library which was founded as an organisation which produces systematic reviews of the published medical literature. It is named after a pioneering epidemiologist Archie Cochrane of whom Goldacre says: “After the war, and after working on miners’ lung disease Archie helped to aspire a democratising shift towards evidence-based medicine throughout the whole of medicine and, as a consequence, probably saved more lives than any doctor you know. Before that he was a prisoner of war in Nazi Germany…..And before that, in 1936, he dropped out of medical school and travelled to Spain to join the International Brigade where he fought genuinely totalitarian oppression, the fascists of General Franco with his own two hands. Now what did you do in your summer holidays?”

The blog is grateful for its friend John Spitzer for forwarding the Science Muller review.