We sent out the survey form of our Rationality Checklist and over 250 of you responded (N=283 to be precise). We'll be sending out part two of four to our newsletter in two weeks, so sign up if you'd like to contribute data!
The first rationality habit we asked you about was Noticing Confusion:
When I see something odd - something that doesn't fit with what I'd ordinarily expect, given my other beliefs - I successfully notice, promote it to conscious attention and think "I notice that I am confused" or some equivalent thereof. (Example: You think that your flight is scheduled to depart on Thursday. On Tuesday, you get an email from Travelocity advising you to prepare for your flight “tomorrow”, which seems wrong. Do you successfully raise this anomaly to the level of conscious attention?)
This turned out to be a skill a lot of our respondents use frequently. When we asked you to remember the last time you consciously noticed confusion, here's what you answered:
And several of you had stories to share about how noticing confusion had helped you ask new questions and get curious:
Someone recently nerd-sniped me with the question of why there are two high tides per day on opposite sides of the Earth. I started with my cached answer (that the earth is more strongly drawn to the moon than is the water on the far side), but recognized that this predicts that the near and far tides should differ by no more than 1-2%, and that this was not actually the case. So I noticed that I am confused!
Or how noticing confusion broke you out of a rote pattern in time to catch an error:
At my work we have an off-site file warehouse with some two thousand banker boxes of files. One of my jobs is to cull any files that fall under certain criteria (age of case, age of client etc.). It involves a lot of paperwork and tracking to make sure I always know where every file is at all times. At one point, while moving a file from a box to a cull pile, I noticed that the destroy date for this file on my list of files to cull and the same date on my master list (which the cull list is derived from by hand, due to no access to custom database reports) did not match. My first impulse was to just go with the longer date, drop the file back into its box and move on, but they should have matched, and I was curious on what I messed up. It ended up that a bad copy-paste between lists offset the cull dates on one list by a few files, rendering the list useless and dangerous. I had to go back through the week's work to double check things and saved some files from being prematurely destroyed. It was extra work, but it stopped a much larger disaster. Now I have extra checks in place to verify lists are proper before they are used.
At our workshops, we work on training people to notice and welcome confusion, because it's a valuable signal that something is wrong with your map of the world. In addition to training a general awareness of confusion, we help participants learn about Bayes theorem or reframing techniques or miscalibrated aversions, so you can increase your sensitivity to confusion in areas normally occluded by cognitive bias.
Check out all the results and stories from out Rationality Checklist survey by browsing the 'Rationality Checklist' category