Another point in the argument against timed tests…
Thanks to the awesome Grace Chen, I saw the following article yesterday:
Here are the two key paragraphs from the post (emphasis mine):
Deborah Kelemen and her colleagues presented 80 scientists (including physicists, chemists and geographers) with 100 one-sentence statements and their task was to say if each one was true or false. Among the items were teleological statements about nature, such as “Trees produce oxygen so that animals can breathe”. Crucially, half the scientists had to answer under time pressure – just over 3 seconds for each statement – while the others had as long as they liked. There were also control groups of college students and the general public.
Overall, the scientists endorsed fewer of the teleological statements than the control groups (22 per cent vs. 50 per cent approx). No surprise there, given that mainstream science rejects the idea that inanimate objects have purpose, or that there is purposeful design in the natural world. But look at what happened under time pressure. When they were rushed, the scientists endorsed 29 per cent of teleological statements compared with 15 per cent endorsed by the un-rushed scientists. This is consistent with the idea that a tendency to endorse teleological beliefs lingers in the scientists’ minds. This unscientific thinking is usually suppressed, but time pressure undermines that conscious suppression.
So let me get this straight—when professional scientists, who have years of training in a discipline, are forced to assess whether or not scientific statements are true under time pressure, their performance decreases by half and they blow almost one third of the questions, essentially earning a 71%, were this a test in school? That is, professional scientists become C- scientists when exposed to pressured multiple choice testing.
And we think that having novice students with almost no training in a particular science discipline take multiple choice tests that are filled with distractor responses designed to elicit misconceptions is an accurate, fair or useful way to judge students’ scientific understanding?
This also ties in beautifully to the some of the points Michael Pershan was making in his masterful discussion last night of the “Failure(?) of the Math Mistakes Blog”, which you should make a point to listen to. In his talk, Michael talks about why students make mistakes on ideas that they already know, such as why students will say , and yet he knows that students know this isn’t true, as they can explain why when confronted with this mistake.
Michael points to a possible explanation for this discrepancy from Daniel Kahneman, author of Thinking Fast and Slow. Khaneman writes:
If a satisfactory answer to a hard question is not found quickly, [the
intuitive capacity] will find a related question that is easier and will answer it. I call the operation of answering one question in place of another substitution.
So the quick, rapidly responding part of our brain that often succumbs to misconceptions and incomplete heuristics will too often leap at answering a question intuitively and incorrectly when faced with time pressure, before the slower, reasoning brain has a chance to process and come to a thoughtful and often correct response to the question when given time to think. My recollection of Kahneman also is that he said that this leaping to the wrong answer can never be eliminated—it can only be reduced by giving ourselves more time to process, often by deliberately introducing mental speed bumps (like drawing a diagram) designed to slow us down and allow our reasoning brain time to process.
So what’s the point of timed tests again? Is it to simply remind our students of how error prone their brains are when operating under high stress and time pressure?