Skip to content

One way to not assess a 5th grader’s understanding of science—the CRCT.

July 15, 2011

Today, I had a fascinating conversation on Twitter with a number of colleagues both near and far. It started with this tweet by me to my colleague @adchempages:

He responded

I’ll pause here for a bit of commentary. For those that don’t know, the CRCT is the Criterion Referenced Competency Test taken by Georgia public school students as a measure of their comprehension of various state standards. This is the start of something I find wrong with the education system today. I don’t mean to pick on my colleague, since I would say he is like nearly every other parent out there, but I am troubled that parents simply accept CRCT scores as “evidence” that their students are progressing. I also know that I think it’s tied to our culture’s response to information overload and constant searching for short metrics that convey quality, but I think this phenomenon applied to other parts of education (college rankings, SAT scores, the challenge index ranking of high schools, etc) has had a negative impact on education, could CRCT scores be doing the same? Would education improve if parents insisted to know what is on the exam and exactly how their child performed, beyond the single number summary they are given?

So I then posted this:

I’ve included a copy of the CRCT practice test here so you can get an idea of its content:

View this document on Scribd

And then the twitter fun begins, with more people adding their thoughts:

This is one of my big points—I don’t think this test really is testing science—it’s testing whether or not the student has learned the names of a bunch of phenomena he/she does not understand “static electricity”,”current electricity,” etc.

Richard Feynman describes the value of this type of learning better than almost anyone else (be sure to listen to the whole video to hear a real example of scientific learning):

This point is certainly vaild, and I didn’t really address it in the conversation. I think my problem with this test isn’t so much that it presents ideas that aren’t really correct with a high school understanding of physics (though I dislike this), my problem is more that it seems to push students toward simply getting the name right, without any desire to test whether or not the student knows how that particular fact was discovered, or how to apply that fact in a new context.

In response to this tweet, yes, I would prefer a lab exam. I’d prefer giving a child a battery, bulb and a single wire, and having him/her light the bulb. Since this task routinely takes my students 20 minutes or more (and I doubt I have any kids who were ever in any danger of not passing the 5th grade science CRCT, I think this may indicate that these students never fully understood these ideas.

And this is something I am very curious about. How would my 9th grade students do on this CRCT practice exam?

I DM’d a link to the test to @BHS_Doyle, who is helping to set elementary school standards in New Jersey, and has written a number of wonderful posts on the topic here.
Here was his response.

ow–test is exactly the kind of stuff that kills science. Now, if the child had the battery/wires/etc/ in front when taking the test….

And this is the sort of exam that was involved in the CRCT cheating scandal at APS. While I find what those the teachers did to be involved unethical, I still have to look at this meaningless exam, and try to see things a bit from their perspective. Lots of research shows students cheat most where there is no connection between teacher and student, and students have no clear understaning of the intrinsic value of the assessment. Did APS teachers have this? My guess is no, and so the treshold for resorting to cheering became all the more closer.

So how would you win this back? Are there good standardized ways to measure learning in elementary school?

Frank Noschese informed me that not every elementary school standardized test needs to look like the one above. The state of New York administers a performance based standardized assessment to all of 5th graders. One example is having students determine what variables affect how far a golf ball moves a cup when it is rolled down a ramp.

19 Comments leave one →
  1. July 15, 2011 6:20 am

    John:

    I agree with @adchempages on this one (Q5 on the exam). Given the context in which 5th grade teachers teach, the answer (insulators) is appropriate and correct for 5th graders. I have watched the 4th and 5th grade teachers at Drew Charter teach. They do mostly hands on experimentation (like the circuit question and Q5) and students would come up with those answers. They might also assess them with a lab exam. Granted there might be a context in which those materials are not insulators, but they wouldn’t happen for a 5th grader in the classroom. It might also not be developmentally appropriate to take them to that level. It doesn’t mean we shouldn’t teach or assess them according to the level at which we teach the material. Since the CRCT is criterion-referenced, then the test is appropriate and OK if it asks them which are “normally” insulators. Maybe its the wording not be specific enough. They are “normally” insulators.

    Interesting!

    Bob

    • July 15, 2011 7:38 am

      Bob,
      I’m certainly willing to concede that a 5th grader need not “know” that all objects can conduct at a high enough potential difference, but I think my larger cirtique remains. More or less, this is a test of “can you find the right name” for this phenomenon/object, and I don’t really see this as testing science. In fact, I see it reinforcing a misunderstanding of science, that knowing the names of things is the primary goal of science, similar to Feynman’s criticism. It’s sad to me that things as interesting as the instruction you describe at Drew would be boiled down and judged by a test like this one.

      • July 17, 2011 8:06 am

        John,

        Seems to me it depends entirely on the type of teaching that preceded the test. If the teacher did some hands on labs that had students determine whether a substance was a conductor or an insulator (at the 4th or 5th grade level) and the experiment was designed as an inquiry lab, then I think the question is a good one and is not merely about “finding the right name.” However, if the teacher just had students do a worksheet on the topic then it probably is about just getting the right name. I do think there might be a better question format for the type of question, but I do think it is more about the teaching and preparation of the students for engaging in the concept.

        Bob

  2. July 15, 2011 7:14 am

    I think tests like these are actually disrespectful to children. It assumes that children cannot do the skills of science (exploration, reasoning, testing) and so reduces science to facts and categories. Just like how I know teachers who freak out when introducing a lab because the students didn’t come up with the “right” hypothesis — let them find out from their lab experience and discussion! I’d rather they learn the skill of examining evidence than knowing the “right” category for something.

  3. July 15, 2011 8:27 am

    My problem with this kind of test is not that insulator and conductor are relative terms. While a piece of plastic may be conductive in certain situations, I wouldn’t expect a 5th grader to know that. What I WOULD want a 5th grader to TRULY understand is that she will continue to refine her knowledge as she continues to grow up. That is, what she learns in the 5th grade is not the final scientific word, and she should expect to see her current models of the physical world broken and revised as she becomes older. So when she learns later (much later) that you can hook a large enough power supply to a plastic rod to get current to flow, she isn’t so much surprised as she is totally jazzed and excited.

    No standardized test that I know of measures that most important understanding!

    I look at this test and think “Ok, I know exactly what I want to do with my 5th grade science students. They will smash this test after taking my hands-on class!” It would be messy, loud, joyous and fun. And my little charges (pun! hehe) would really know something (not nearly everything! Matter & Interactions must wait!) about atoms and current and static electricity. I’d be frantically setting up experiments every morning, cleaning up (sort of… you know me) in the afternoon and grading in the evening. My students might find the test’s focus on vocabulary confusing after our class, but they would muddle through those questions and do very well.

    But I could also teach a very different class and get pretty good results. I could do lots of vocabulary, lots of rote crap that doesn’t lead to understanding, lots of drill and kill. The classroom would be neat, and people dropping by would see rows of children with their heads bent quietly over worksheets, or me performing a very clear and entertaining lecture. The assembly line would run smoothly, I’d be left at night with a manageable amount of grading. If I’m lucky, I have a Scantron and can give practice multiple choice tests (just 30 minutes!) over the weekend. I’ll get good test results and then send these kids off for a similar experience in middle school and then on to high school where the high-school-teacher-me will tear his hair out at the total lack of understanding that these kids have. That lack of understanding is not about the finer points to each of these questions on the sample test. It is about how science works.

    The Physics AP C is the same. You can either really learn some physics and do well on the test, or you can prep for the test, missing the physics along the way. The latter must be easier, because it’s what most teachers do. Just ask university physics professors.

  4. rhettallain permalink
    July 15, 2011 11:04 am

    I guess the real answer to the appropriateness of questions: depends on what you are trying to do.

    If you are trying to test and see if students know the results of someone else’s experiment, the insulator question is fine. I do not like these questions. They can be answered by google.

    If you are trying to test if students can interpret the results of experiments, you could show some setups with a battery, bulb and different materials. Show these set ups with different materials in them and ask the students to determine from the experiment if it acts like an insulator or conductor.

    Finally, maybe you want to see if they can create their own experiments. In this case, the question could be: design an experiment to test if pure water is a conductor or insulator at low potentials (really, low electric fields).

    • July 16, 2011 1:31 pm

      John, when you say, “I don’t mean to pick on my colleague, since I would say he is like nearly every other parent out there, but I am troubled that parents simply accept CRCT scores as “evidence” that their students are progressing.”, you misrepresent me a little.

      I am under ZERO illusions about what the CRCT *really* means, and my children’s ever increasing scores on those tests do not excite me in any meaningful way.

      HOWEVER, you should see the value (and accompanying fanfare) that was placed on the fact that my son’s third grade class at Mirror Lake Elementary School in Douglas County, had a 100% pass rate on the CRCT reading test. This was the first time that this had happened in the county for ANY elementary school, and it was heralded by parents, teachers and administration as a GIANT accomplishment. While you’ve got that kind of investment by three different groups, it’s going to be difficult to get away from it.

      • July 17, 2011 10:15 pm

        Adrain,
        Thanks for the clarification. I find your description of the fanfare for the passing of the CRCT at your child’s elementary school troubling, but as a product of the Douglas county school system not all that surprising. I guess what I’m thinking is what would happen if parents came together to demand change. What if you and other parents went to school board meetings to demand better tests? There are school districts where parents are providing considerable resistance to the spread of more shallow and mostly useless standardized testing in schools.

    • July 16, 2011 1:40 pm

      “If you are trying to test and see if students know the results of someone else’s experiment”.

      It’s always been my argument that’s EXACTLY what we ARE (and indeed SHOULD be) doing in high school! High school science is NOT ‘research’ where the answers are not known, it’s the laying of the fundamental facts that allow some people to move on to meaningful, ‘real’ science later on.

      By all means let’s change education so we CAN develop some of those ‘research’ skills at an earlier age (I’m on board and willing to do some of that), but let’s not destroy the whole process – learning some basic ‘facts’ from an engaging human who is also an experienced, contextual educator, is still a hugely important piece of exposure, and one that beats ‘Googling’ a fact, EVERY. SINGLE. TIME!

      • July 17, 2011 10:17 pm

        Adrian,
        I guess I keep coming back to—why can’t we give them a taste of meaningful science now, when they’re young and enthusiastic? Likely if they continue on in science, they’re going to need that enthusiasm to persist through a lot of hard (and sometimes boring) work.

  5. July 17, 2011 12:10 am

    @adchempages I get what you’re saying, but I think the “our school got 100% and it brought us great joy” example is a big reason why people are freaking out. It’s misplaced fanfare and it saddens me that this is the thing that brings us together.

    Learning the names of things is a part of science, but when we all celebrate only because we’ve learned the names of things, that’s a problem.

    Don’t get me wrong. I’m at a public school that’s very much under the gun. If we ever break the mythic 800 API mark there will be much rejoicing.

    And if you want to see a poor standardized test check out the California 8th grade social studies test. http://www.cde.ca.gov/ta/tg/sr/documents/cstrtqhss8.pdf

  6. Robert Bell permalink
    July 17, 2011 12:13 pm

    The question of to what depth we introduce concepts in Science in the lower age groups arises often and are not necessarily determined by age, but on the experience the pupils have gained in the past. If the underlying premise is that pupils develop to fruition, then we, in part, have the responsibility for creating the right environment and stimulus to enable this to happen. This learning environment should not be limited by dogma or even examination requirements. I do recognise the need for accountability, but releasing a child’s creative capacity and engaging him in his own learning is foremost in my mind; the curriculum is just a vehicle for achieving this and should not be the driving force. So the question of to what depth we should introduce a concept depends very much on the pupils themselves. The vital issue is that nothing that is formalised at an early stage should prove to be an obstacle to later development.

    • July 17, 2011 10:19 pm

      Robert,
      Great points—I doubt this sort of thinking played any sort of role in the design of the CRCT.

  7. July 17, 2011 4:29 pm

    I think it’s important for people to know some ideas without needing to look them up every time, and approximations like “insulator” can be helpful. What worries me about the test above is the high probability of false positives. A student who sees a balloon stuck to a wall and says “static electricity” has demonstrated no more knowledge than a student who called it “balloon-stick-to-wall-ishness.” A good assessment lets us know if we are pseudoteaching. This one doesn’t. Michael Doyle‘s assessment replaces multi-choice questions with a propane torch, and I think he gets better data…
    “I ask what comes out of the propane torch after the propane as the propane is burned. I consistently get two answers—fire and carbon dioxide. I never get water. I’ve asked hundreds of kids the question, with the equation sitting up on the board, and it’s like H2O is some mysterious stuff stuck to the equation just to make it balanced.”

    • July 17, 2011 10:20 pm

      I love Doyle’s writing on elementary ed, and I agree, that perhaps the most troubling question on the exam is the static electricity question. It’s hard for me to imagine how knowing the name of this phenomenon conveys any level of understanding when tested in a question like this.

  8. July 17, 2011 7:24 pm

    I suspect that if you really engage students in learning the science content deeply and authentically, and gradually engage them in discourse that makes contact with appropriate vocabulary they will do really well on this test. I think the real damage of this test is that someone or some school might teach to it. This test is mostly about vocabulary, which is unfortunate, because it sets a really low bar for students learning and achievement; and this might encourage or reinforce fact-based instruction. If I were teacher, I’d have no problem with this exam as something that comes down from above, because it wouldn’t effect anything I’d do. My assessments wouldn’t look anything like that.

    • July 17, 2011 10:22 pm

      Brian,
      You’re totally right, but in at-risk schools where CRCT scores have high stakes in terms of student promotion, teacher firing and school funding, I think it’s easy to see how tests like these debase science education as teachers do teach to the test, and sometimes, as we saw in the Atlanta schools cheating scandal, even worse. I would love to see the emphasis on vocabulary in both elementary and middle school greatly reduced.

  9. Jill permalink
    July 19, 2011 11:48 am

    Good opinion piece about standardized testing being used to rate schools.

    http://www.washingtonpost.com/blogs/answer-sheet/post/the-problem-with-how-school-quality-is-measured/2011/07/18/gIQA7depMI_blog.html#pagebreak

    • July 24, 2011 1:33 am

      Absolutely. It’s like Valerie Strauss and Jay Matthews live in two different worlds. One measures the quality of a school by dividing the number of AP tests given by the size of the senior class, and the other is actually out there looking to engage the larger community in a real discussion of what a school should do. Mark Phillips’s guest piece is a great example.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: