Today's post is part 2 of a two part series from Steven Bland on his paper "Intellectual Humility and Humbling Environments", published in Review of Philosophy and Psychology.
 |
| Open science as a humbling environment |
In last week’s post, I argued that a lack of intellectual humility in individuals can have beneficial effects on individual learning and collective deliberation, but only in humbling environments. Designing, creating and sustaining such environments is one of the ways collectives can manifest intellectual humility.
Humbling environments have the following five features:
1. They elicit evaluable behavior.
2. They produce actionable feedback.
3. They afford multiple opportunities to solve similar problems.
4. They incentivize the pursuit of intellectual goods.
5. They are forgiving without being overly permissive.
Forecasting tournaments and open science are two prime examples of humbling environments.
1. They elicit evaluable behavior.
2. They produce actionable feedback.
3. They afford multiple opportunities to solve similar problems.
4. They incentivize the pursuit of intellectual goods.
5. They are forgiving without being overly permissive.
Forecasting tournaments and open science are two prime examples of humbling environments.
In forecasting tournaments, participants make daily predictions in response to questions about future events, such as: will the price of crude oil hit $90 a barrel within the next twelve months? (condition 3) Each participant’s performance is evaluated using Brier scores which measure the accuracy of their aggregate predictions (condition 1). These scores also serve as feedback that participants can use to improve their performance over time (condition 2). The tournament format encourages such improvement (condition 4), and the scoring system ensures that large mistakes aren’t ruinous, as they would be if logarithmic scoring rules were used (condition 5).
As its name suggests, open science requires its practitioners to be transparent about their methods and evidence, so that they can be evaluated by other scientists (condition 1). Traditional channels of peer review and replication, as well as newer practices of critical appraisal, such as automated error detection and computational reproducibility checks, generate feedback that can inform future scientific research (condition 2).
As its name suggests, open science requires its practitioners to be transparent about their methods and evidence, so that they can be evaluated by other scientists (condition 1). Traditional channels of peer review and replication, as well as newer practices of critical appraisal, such as automated error detection and computational reproducibility checks, generate feedback that can inform future scientific research (condition 2).
An increasing number of journals incentivize such practices by adopting, for example, the Transparency and Openness Promotion guidelines and registered report submission formats (condition 4). The competitive dynamics of science encourage scientists in the same field to take different approaches to outstanding problems, in the hopes of finding what others have missed (condition 3). Finally, scientific and academic establishments are forgiving when it comes to good faith errors, mistakes, and oversights – hence, the institution of tenure, for example – but not when it comes to fraudulent research (condition 5).
Both forecasting tournaments and open science foster intellectual progress: practiced forecasters generally produce more accurate predictions over time, and open scientific communities home in on increasingly accurate theories through the process of critical exchange. They do so not by attracting inordinately humble participants, but by creating environments that make manifest the intellectual limitations of their individuals and communities. This is what makes them humbling.
The process of learning to acknowledge and appreciate our intellectual limitations may be just as valuable as the virtue that results from this process. What it requires from us, collectively, is that we create the conditions in which this is likely to happen.
Both forecasting tournaments and open science foster intellectual progress: practiced forecasters generally produce more accurate predictions over time, and open scientific communities home in on increasingly accurate theories through the process of critical exchange. They do so not by attracting inordinately humble participants, but by creating environments that make manifest the intellectual limitations of their individuals and communities. This is what makes them humbling.
The process of learning to acknowledge and appreciate our intellectual limitations may be just as valuable as the virtue that results from this process. What it requires from us, collectively, is that we create the conditions in which this is likely to happen.