“Expert Political Judgment”

by Philip E Tetlock (Princeton University Press 2005)

“There are more things in heaven and earth, Horatio, than are dreamt in your philosophy”. This was Hamlet’s admission that he was confused by complexity and had difficulty in coming to judgment. Hamlet’s solution was inexpert and created a new set of political problems.

“Expert Political Judgment” is an attempt to identify the characteristics of individuals who have the ability to analyse situations in depth and with accurate foresight so that their decisions are informed by expert political judgment. The author is a psychologist but has worked for many years with a range of specialists in different disciplines in order to distil the quintessence of expert political judgment, not only for the immediate need but sustainable into the longer term. The main focus of the book is on forecasting outcomes of particular situations and on identifying the specific techniques and mental attitudes which do so most successfully. Luck is recognised as a factor but is set aside as exogenous. The quest is for the mindset and toolkit which will optimise forecasting by “quantifying the unquantifiable”. For the mindset contrasts are drawn between “radical sceptics”, who expect nothing and “meliorists” who are open to seeking improved outcomes. Another facet of mindset is Isiah Berlin’s contrast between “hedgehogs” who “know one big thing” and “foxes” who “know many little things”. In the context of the book “hedgehogs” emerge as having fixed views, seeing issues as “black or white” and supremely self-confident. By contrast “foxes” are open-minded, flexible and self-critical. One key finding of the book is that “foxes” emerge as winners of most of the tests, yet “hedgehogs” are more focussed and willing to make tough decisions. In times of increasing uncertainty it would seem that fox-like characteristics are at a premium over those of “hedgehogs” in evaluation, though “hedgehog” confidence is needed to take action.

The book draws to a conclusion with a challenge: “Are we open-minded enough to acknowledge the limits of open-mindedness?” This chapter is a critique of scenario planning which the author sees as advising only that “anything is possible”. Too often those involved are over absorbed in inward looking details to build their stories, while an outside view is needed to provide a reality check. Tetlock fails to realise that scenario planning should be used as a means of guiding action not engendering endless debate.

Judgment seems to involve a metacognitive trade off between theory driven and imagination driven modes of thinking. Theory offers certainty and imagination helps to cope with uncertainty. The author sees the best long term predictor of good judgment to be a Socratic commitment by protagonists to thinking about how they think.

The book ends with an examination of objectivity and accountability. It seeks objective standards for assessing judgment whilst recognising that subjectivity colours most judgments. It offers a whole range of tools to help achieve a self correcting approach to judgment. It ends on a note of resignation: “There are ineradicable pockets of subjectivity in political judgment. But progress is not as hopeless as opponents of social science never tire of insisting”. Like so many of the “hedgehogs” Tetlock keeps quoting, he sees himself as “almost correct”, even though his quest has failed.

Adrian Davies
15 July 2007