Skip to content

The NYT on learning: studying notes can be deceiving

January 21, 2011

Thanks to @abaconmoore for this great retweet:

The article, To Really Learn, Quit Studying and Take a Test Already, reports on new research findings reported in Science that students who take a test asking them to actively recall information retain more than those who simply “study” or make concept maps. But what is awesome about this study is they didn’t just measure how the students performed using the various study strategies, it also measured how the students thought they performed.

Here is the money quote from the NYT:

These other methods [rereading notes and concept mapping] not only are popular, the researchers reported; they also seem to give students the illusion that they know material better than they do.

In the experiments, the students were asked to predict how much they would remember a week after using one of the methods to learn the material. Those who took the test after reading the passage predicted they would remember less than the other students predicted — but the results were just the opposite.

This really gets me thinking, especially as I am going through my students’ mid-year evaluations and reading their requests for me to lecture more. My students love lectures (it’s the predominant teaching method used in many of their other classes); many of them describe good learning days as having pages of notes and feeling like they’ve absorbed everything from class. A lot of these comments show that I haven’t done a good job of showing my students how to take notes in this class, which relies on discussion and collaboration. But perhaps even more importantly, this article in the NYT gives me a great insight why my students like lectures and and notes so much. Re-reading notes feels like learning. When you do this, you can actually feel yourself back in the class, hearing the things the teacher said. But the problem is, you aren’t learning what you don’t know. You aren’t pushing yourself to master the things that aren’t in your notes, the questions, the new situations and problems. You’re wallowing in comfort.

This is the exact opposite of how physics class can feel for a lot of my students. We seek out our misconceptions and misunderstandings constantly. In fact, in can feel exactly like the feelings of the frequent testing group, quoted from the NYT article below:

“The struggle helps you learn, but it makes you feel like you’re not learning,” said Nate Kornell, a psychologist at Williams College. “You feel like: ‘I don’t know it that well. This is hard and I’m having trouble coming up with this information.’ ”

This reminds me of a good workout, and a wonderful quote tweeted by the by the incredible Kate Nowak, “Pain is weakness leaving the body,” borrowed from the Marines, which I tweaked a bit to say

Confusion is ignorance leaving the brain.

In reflection, I might prefer to put this in a less hard core, more positive light, but I think the connections to the Marines can be a good one. No one would ever think that you can get a good workout by re-reading your notes about workouts, listening to a lecture about how to do a bench press, or making flashcards about technique. The only way to get a workout is to do the work—life the weights, run the distance, swim the laps. And the way you know you’ve gotten a workout is when you feel the burn in your muscles after the workout. In this light, the job of the teacher is coach or trainer. It reminds me of this great post by Dan Meyer, my blogging inspiration, on the role of teacher as spotter.

The good teacher puts weight on the student’s intellectual bar and lets her struggle under that weight as long as possible, asking questions to help her cut through the confusion, just like the spotter shouts encouragement at the lifter.

.

Plans for change

I’m still thinking about all the ways in which I’m going to try to put these ideas to use. First, I’ve already written my students a blog post with this article which I hope will spark some conversation. I’m very curious to see how they will react, and if they can absorb the greater point of the article that real learning doesn’t come through comfort and absorption, it comes through struggle and failure. When I get back from leave, I think I will add it to my metacognition curriculum and have the students read it for homework.

I can’t deny my students’ desire to have notes, and I think they really can be valuable if they are used correctly, but I need to teach my students to do this. Even though I’m on leave, I’m trying to emphasize, via my sub, the need to stop and summarize big learning points. The incredible 20 minute pulse check project, started by a colleague of mine @jlgough , is a great initative. So is Question-Evidence-Conclusion Cluster note taking, which I learned from Cal Newport, who hit upon the advice of telling a story after you read or go to a class to enhance your learning two years ago, which sounds a lot like the findings in the NYT.

Students start by writing down a question, followed by a few pieces of evidence, and then finally a conclusion. This is exacly how much of our class discussion is organized, but I think since we sometimes don’t explicitly write this out, they don’t think it’s worth recording. Someday, I would love to find out how great English teachers teach students to carry on 50 minute discussions and have students come away with notes that carefully capture the most important questions and ideas of the discussion. Q/E/C note taking is simple, and it’s something I can do by just highlighting the process a bit more for my student (it also links nicely with my thoughts of emphasizing long chains of reasoning). Here’s an example

Q: What can we conclude about the momentum in a collision between two dynamics carts
E: We saw that the carts collided, and thus were exerting forces on each other for the exact same amount of time.
E: By newton’s third law, we know that the force of cart A on cart B must be equal and opposite to the force of cart B on cart A.
E: If we rearrange Newton’s second law, we get m\vec{v}_f-m\vec{v}_i=\vec{F}_{net}\Delta t, or \Delta \vec{p}=\vec{F}_{net}\Delta t.
E: Since the times are equal, and the net forces on each object is opposite, this must mean the changes in momentum of each object are opposite each other two.
C: When we add up the changes of the objects to get the total change for the system, the two opposite changes will cancel out, and the total momentum will not change

Finally, I want to continue to help my students see the connection between how they learn physics and how they learn a sport. If they can begin to see themselves more and more in the role they once had when they first held a baseball bat, or a lacrosse stick, barely knowing how to swing it, but still willing to devote hours to practicing (and failing), all of the joy of the game, I think they will be able to find the resolve necessary to reach some very deep understandings in my class.

10 Comments leave one →
  1. January 23, 2011 10:10 pm

    A fascinating study, and I loved “confusion is ignorance leaving the brain.” A wonderful point to bring to students’ attention is the idea that reading your notes makes you feel smart (because it’s familiar) while testing yourself makes you feel stupid (because it makes you realize what you don’t know). This train of thought depends on the assumption that if you’re “smart” enough, you just “get” things in a comfortable, effortless way; so, if learning is uncomfortable, it means you’re not “smart.” Unfortunately, I suspect that most students do hold that assumption, and this study may be a way to begin debunking it.

    I’m surprised, though, that the results are being interpreted as “throwing down the gauntlet” to constructivism, and concept-mapping in particular. To my knowledge, concept-mapping is not intended to produce recall; it is intended to help students practise finding relationships. Recalling those relationships is a different skill — and, for anything more complex than multiplication tables, not the most important skill.

    Why are we asking if concept-mapping is the gold standard of learning? Wouldn’t it make more sense to question whether recall is the gold standard of testing?

    As for testing, it always amazes me that my students don’t test themselves. They wait for me to do it. Ultimately, I suspect that we could expose the false dichotomy between constructivism and the results of this study by having students construct ways to test themselves.

  2. January 23, 2011 10:29 pm

    Great points! Some additional thoughts:

    A fascinating study, and I loved “confusion is ignorance leaving the brain.” A wonderful point to bring to students’ attention is the idea that reading your notes makes you feel smart (because it’s familiar) while testing yourself makes you feel stupid (because it makes you realize what you don’t know). This train of thought depends on the assumption that if you’re “smart” enough, you just “get” things in a comfortable, effortless way; so, if learning is uncomfortable, it means you’re not “smart.” Unfortunately, I suspect that most students do hold that assumption, and this study may be a way to begin debunking it.

    This is the fixed mindset at work. Much of my work in developing ametacognition curriculum has focused on emphasizing Carol Dweck’s Mindset research to my students and helping them to see the value of growth mindset.

    I’m surprised, though, that the results are being interpreted as “throwing down the gauntlet” to constructivism, and concept-mapping in particular. To my knowledge, concept-mapping is not intended to produce recall; it is intended to help students practise finding relationships. Recalling those relationships is a different skill — and, for anything more complex than multiplication tables, not the most important skill.

    Yes—I didn’t mean to devalue concept mapping. I think it is very useful in a modeling physics class like mine, where in order to succeed, students need to see the connections between all the various models (constant velocity, balanced forces, constant acceleration, unbalanced forces) they are learning.

    Why are we asking if concept-mapping is the gold standard of learning? Wouldn’t it make more sense to question whether recall is the gold standard of testing?

    This is a great question. I don’t fully see how people are beginning to argue that this article gives further weight to the “high stakes testing” movement.

    As for testing, it always amazes me that my students don’t test themselves. They wait for me to do it. Ultimately, I suspect that we could expose the false dichotomy between constructivism and the results of this study by having students construct ways to test themselves.

    Great idea! I’ve always wanted to work on this. I’ll try to do something with this in the near future and report back.

    • January 24, 2011 12:21 am

      I look forward to your results! To clarify, I wasn’t suggesting you were devaluing concept mapping — I was suggesting that the NY Times summary of the research (and some of the people they interview) seemed to. The “throwing down the gauntlet to constructivism” quote is from the Times. It seems silly that this is interpreted as “taking a test is better than studying.” More sensible to conclude that taking a test is part of studying.

      As for “high-stakes testing”… *sigh*. The study’s methodology clearly involves retesting students repeatedly on the same material. That inherently makes it LOW-stakes testing. It also means that the study measures, but is incapable of distinguishing, the benefit from things like “making a mental list of things you didn’t know,” “talking to other test-takers out in the hallway to swap answers,” “mulling over your answers on the drive home and improving them in your head,” etc. In a 1-shot test, you end up doing all of these things, but they don’t help you.

      Maybe the NY Times writers should be required to take a course to learn how to “construct the meaning” of a scientific study.😉

  3. February 2, 2011 7:25 pm

    To me, a test is self-limiting–that is, it only tests what is on that test. Although I have noticed that, while taking a test, I learn its content, I only learn its content. This is why I really like standards-based grading, which is far more open-ended and can go where the student takes it.

    • February 2, 2011 11:35 pm

      I’m totally with you on being a fan of standards based grading, and I’ve seen myself be able to ask open ended problems that simply present my students with a situation, and no question, and watch, impressed, as they go about breaking the problem down and finding all sorts of information. How do you find you are able to go beyond the test in your class?

      • February 4, 2011 1:00 pm

        I’m mostly able to “go beyond the test” through the way that Mr. Ludwig has designed his system, in which he gives the student a topic and then asks them to learn about it. He is available for questions, but the student only goes as deeply into the topic as they desire. I prefer to go to the molecular level on every main content area, while other students may only give a brief overview.

        • February 7, 2011 7:04 pm

          This sounds very interesting. Can you give me an example of what sort of topics you might get from your teacher? How does he (or you) decide what standards you must demonstrate in order to achieve a particular grade?

        • February 10, 2011 1:40 pm

          Well, he’s set up the system so that a student must achieve scores in each of the nine standards (which were, I believe, somewhat arbitrarily designated). These standards include (to name a few) Content (which is then broken into substandards such as Cellular Reproduction and DNA Structure), Research, Experimental Design, and so on. The overall “grade” is determined by the student’s lowest score (based on a 4, 3, 2, 1 scale, where “4” is above and beyond the expected performance, “3” is meeting expectations, and so on). So, if a student has all 4’s and 3’s, they would receive an A. All 3’s translates to a B, and the scale continues from there. If you would like to know more, I really recommend his blog.

Trackbacks

  1. Retrieval beats concept maps for studying « Gas station without pumps
  2. Will you join our global physics department? « Quantum Progress

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: