Skip to main content

Questioning Optimism


I'm Adam Harris and I'm an experimental psychologist from University College London.

I am perhaps an unusual contributor to the Imperfect Cognitions blog as I have argued that cognitions might seem imperfect because of imperfections in prevalent methodologies, predominantly arising from a failure to appreciate the importance of understanding the appropriate normative basis of a task. Specifically, my work has suggested that the assumed ubiquity of optimism across our species is based on questionable evidence.

A prominent example of this work is presented in a paper I wrote with Ulrike Hahn (published in the Psychological Review) in which we demonstrated, through simulation, that rational agents could be labelled as optimistic on the prevalent, comparative method of testing unrealistic optimism. On this method, participants respond to the question "Compared with the average student of your age and sex, how likely are you to..." where future life events are inserted for the ellipsis.

Responses are typically provided on a -3 (much less likely than the average) to +3 (much more likely) scale, where a response of zero represents 'about the same as average'. The logic of the test rests upon the recognition that if participants are accurately reporting their chances, then the average of their responses should be the average. Consequently, any deviation from zero is taken as indicative of a systematic bias. The oft-observed result that average responses on this scale to negative events are significantly negative is taken as evidence that, on average, the members of the group underestimate their relative chances. Because we do not wish to experience negative events, such a result is taken as evidence of optimism.

In Harris and Hahn (2011), we demonstrated that 3 statistical artifacts could generate the oft-observed pattern of results from rational Bayesian agents who are, by definition, unbiased. Such a result raises questions over the interpretation of the same pattern of results observed in human participants. There is no longer any evidence for bias on these tests if the pattern of results are consistent with those of rational agents. Essentially, these tests fail the major pre-requisite for a satisfactory test of bias: Unbiased agents appear biased!  In ongoing (as yet unpublished) research, we have failed to identify any evidence for optimism after controlling for these confounding artifacts (my website will be updated when these results are published).

In addition to specifically raising concerns over the understanding of comparative unrealistic optimism, this work highlights, generally, the importance of understanding what participants’ responses represent and the appropriate normative standard for those responses. In unrealistic optimism research, participants’ responses represent their understanding of their own risk and the average person’s risk. Normatively, their own risk includes their estimate of the base rate as well as any individuating information they possess. This insight is a critical consideration when evaluating conclusions from any measure designed to assess bias in risk estimates about real-world events (see also, Harris et al., 2013)

Fortunately, there is also an initial, easy, check for an optimism bias that cannot be accounted for on statistical grounds. The implications of any statistical account in terms of researching optimism are opposite for events of opposite valence. If, for example, a statistical account predicts lower responses for negative events, it will also predict lower responses for comparable positive events. Because of the reversed desirability of positive and negative events, however, the same direction of effect that constitutes optimism for one valence would constitute pessimism for the opposing valence.

Thus, the inclusion of both positive and negative events can serve as a first-stage litmus test to identify a possible confounding influence of statistical artifacts. I therefore recommend that researchers routinely include both positive and negative events in their tests of optimism. In my own work, all such tests to date have observed the same direction of effect in each valence. This constitutes seeming optimism in one valence and pessimism in the other, thus failing to provide the conclusive evidence required for optimism.




Labels:

Popular posts from this blog

Delusions in the DSM 5

This post is by Lisa Bortolotti. How has the definition of delusions changed in the DSM 5? Here are some first impressions. In the DSM-IV (Glossary) delusions were defined as follows: Delusion. A false belief based on incorrect inference about external reality that is firmly sustained despite what almost everyone else believes and despite what constitutes incontrovertible and obvious proof or evidence to the contrary. The belief is not one ordinarily accepted by other members of the person's culture or subculture (e.g., it is not an article of religious faith). When a false belief involves a value judgment, it is regarded as a delusion only when the judgment is so extreme as to defy credibility.
Read more

Motivated Reasoning in Science

Today's post is by Josh May (University of Alabama, Birmingham). In this post, he talks about one of his papers published in Synthese and entitled " Bias in Science ". Josh May Much discussion of the replication crisis in science has focused on the social sciences, particularly psychology. A common narrative is that the social sciences are particularly susceptible to powerful biases, such as moral and political ideology. I argue instead for a parity thesis: all areas of science are subject to bias, through the general psychological mechanism of motivated reasoning . This provides a unified framework for understanding how values influence the entire scientific enterprise. The scientific process involves numerous decisions that can be influenced by one's values--including moral, political, and prudential values--which manifest as goals or motivations. A researcher wants badly, say, to publish in a prestigious journal in order to either advance her career or maintain ...
Read more

Models of Madness

In today's post John Read  (in the picture above) presents the recent book he co-authored with Jacqui Dillon , titled Models of Madness: Psychological, Social and Biological Approaches to Psychosis. My name is John Read. After 20 years working as a Clinical Psychologist and manager of mental health services in the UK and the USA, mostly with people experiencing psychosis, I joined the University of Auckland, New Zealand, in 1994. There I published over 100 papers in research journals, primarily on the relationship between adverse life events (e.g., child abuse/neglect, poverty etc.) and psychosis. I also research the negative effects of bio-genetic causal explanations on prejudice, and the role of the pharmaceutical industry in mental health. In February I moved to Melbourne and I now work at Swinburne University of Technology.  I am on the on the Executive Committee of the International Society for Psychological and Social Approaches to Psychosis and am the Editor...
Read more