Nate Silver, who correctly predicted the result in every State in the 2012 Presidential election, created some interest in the uses of probability theory based on Bayes’ theorem. But the alternative – speculation based on impressions – still seems to be the preferred way to approach problems such as predicting election outcomes.
Of course some psephologists have developed sophisticated data bases and are careful with the application of statistical sampling evidence to make assumptions about election outcomes. Others focus on the prediction markets which, while better than the assumptions of most people in parliamentary media galleries, are still less than perfect. Silver has gone back to sport – leaving the New York Times for the Disney Channel – and admitting that after his 2012 effort the only likely course in the future was down.
The whole question is put into focus by the fact that 2013 is the 300th anniversary of the posthumous publication of Jacob Bernoulli’s book The Art of Conjecture and, fittingly the International Year of Statistics. There were thinkers on probability and uncertainty before him. Professor Manfred Decker in the Bulletin of the American Mathematical Society (Volume 50 Number 3 July 2013 and available online) says: “The terms probability and expectation can be traced back to Cardano, Fermat and Pascal; apparently not to the science of the ancient world. The logical treatment of these notions was first developed by C.Huygens in 1656 who connected his theories to games of chance. Bernoulli goes one step further …(and) clearly connects probability to civil , moral and economic matters” through his law of large numbers. This whole development of probability theory and the associated question of risk assessment is well discussed, in popular form, in Peter L. Bernstein’s 1996 book Against the Odds: the remarkable story of risk.
A quick search suggests that there seem to be few people in Australia – and certainly not political commentators and pundits – pursuing this in terms of political analysis. It may be that some are pursuing it on the quiet, as the blog’s friend John Spitzer suggested, when sending the blog the Decker paper. It would be valuable in betting markets to say the least just as Silver’s interest outside his day job started in poker rather than politics. With Australian politics compulsory voting, ample opinion research findings and relatively limited numbers of voters ought to make the task somewhat easier. Instead, the most sophisticated media analysis seems to involve aggregation or averaging of polls. It may be that the levels of mathematical skills involved, or numeracy generally, are not common among the commentators. Equally mathematicians who might turn to the problem perhaps consider it trivial.
In the current Federal election, extrapolating probabilities from opinion poll research and the Silver FiveThirtyEight mode, suggests the probability of Tony Abbott winning is between 60% and 89% depending on which poll you believe. Now you might say that commentators are already reflecting that but the blog doubts it is based on detailed research and rather more on the zeitgeist or the belief that you don’t need a theory to tell you something that obvious. It may also be that the blog’s quick literature search was not thorough enough, although it is odd that the subject is not prominent, particularly considering all the publicity generated about Nate Silver, and the regularity with which something not obvious to political commentators turns out to be the reality .
Meanwhile, in another paper in the same issue of the journal as Decker’s article, Edward C. Waymire comments on the rich possibilities of what can be achieved in the area saying: “Beyond the richness inherent in the unique perspective that statistical and probabilistic thinking continue to inspire in purely mathematical developments, contemporary socio-political issues pertaining to risk assessment serve as striking reminders of the importance of proper communication of the scope of mathematical theories and applications in relation to the law. It is apparent that the future of risk assessment and uncertainty pertaining to increasingly sophisticated developments in genetics, cyber security, finance, communication systems, and beyond will only increase the challenges to the proper mathematical reckoning with uncertainty already experienced 300 years ago.”