Skip to content

More on the SAT II

January 6, 2011

In a slightly random in interview with physicist Brian Greene in the NYT, he was asked the following question:

Do you think SAT scores define intelligence?
No. They define the capacity to answer questions on an SAT test.

How would you define intelligence?
Intelligence is the ability to take in information from the world and to find patterns in that information that allow you to organize your perceptions and understand the external world.

These two brilliant answers got me thinking about the physics SAT II. What does it measure? Certainly not “physics intelligence.” Is the answer as simple as “the ability to solve questions on the physics SAT II,” and if so, what does that mean?


I think more and more people are beginning to realize that the SAT doesn’t measure intelligence or aptitude (in fact, the name SAT, which once used to be an acronym for Scholastic Aptitude Test, was changed to the SAT (with no acronym) in 1993, when the College Board stopped pretending to give any meaning to the test via its name).

I’m more nervous about the Physics SAT II. I doubt more than 25% college admissions officers studied physics in either high school or college (nationally, 1/3 of high school graduates study physics, and the percentage is even lower in college), and I’m making an allowance that college admissions tends to draw larger percentage of humanities centered students. So this means most admissions officers have little or no experience with a physics, and therefore, they are highly inclined to turn to the SAT II as a measure of how well a student understands physics or is prepared for future study in science and engineering. If this test is a bad indicator of these things (I’ll make this case below), then we could be harming our ability to identify and encourage future scientists and engineers.

I am re-posting a very interesting post made by a former colleague, Mark Hammond, on the Matter and Interactions listserv. I’ve placed a few parenthetical comments in sqaure brackets [] to elaborate a bit.

On to Bruce’s question. The standardized test that causes us the most heartburn
is the SAT 2. No matter the sub-discipline (Biology, Chemistry or Physics), the
tests are forced marches through much memorized material. I’ll speak to the
Physics test. The list of topics is given below. The topics (or subtopics) that
appear in parentheses are those that I do NOT cover in my FIRST year course.

Mechanics

E&M (magnetic fields caused by currents, particles in B fields, electromagnetic
induction, circuits and circuit elements)

Waves

Kinetic Theory and Thermo (thermal expansion, thermal conductivity, heat
engines)

(Modern Physics)

I put all of Modern Physics in parentheses, because by “modern physics” the
College Board means a bunch of memorized details with no understanding… which
one conveys to students by asking them learn ABOUT physics, rather than learn
physics. My course is predicated on the atomic model of solids, gases and
liquids and is modern in that respect. Also, my students have an understanding
that our definitions of momentum (p=mv) and particle energy (E_k=0.5 mv^2) are
incomplete and need to be amended, per Einstein. That, I think, is a far more
responsible approach than talking about the Heisenberg Uncertainty Principle and
mass-energy equivalence without any deeper understanding.

I do not feel that I can reasonably cover any more material than I do (we do
have fewer class hours than the typical day or public school, as we have to let
the kids go home for longer-than-typical holidays). Our excellent results, year
in and year out, at getting students to shed their medieval preconceptions is a
direct function of a lot of repetition (in the form of using existing models
over and over again). These preconceptions do not leave easily, as the Ga. Tech
study shows [Georgia Tech recently conducted a study of its students’ understanding of basic Newtonian concepts using a Think-Aloud Study]. In fact, I don’t think it can all be done in a single year, and this is why I like having teacher well-versed in physics teaching chemistry so
that even the kids who don’t take my second year course get a constant barrage
of situations demanding a Newtonian approach the year immediately following
their sophomore physics class. I have the occasional student showing up in my
second year class who backslides into impetus theory upon meeting a new problem.
But I expect this and am ready to deal with it. I’m not saying none of my second
year students will backslide in college, but I think they are not prone to such
backsliding.

So how much of the entire test is missing from what I teach? From various
breakdowns I’ve seen of the weight given to each topic, I estimate my students
have not seen 20 to 25% of the material that they will be tested on. I have been
told by other teachers who teach deep and narrow courses similar to mine that
their experience has been that students score in the 600-650 range without extra
preparation. That is also my experience (my experience also includes being
pounded on by the parents of such kids who decided to take the test with no
extra preparation). The schools to which my students tend to apply (Columbia
Engineering, Carnegie Mellon (the Engineering program especially), other Ivies,
Davidson, Williams… yes they are an unimaginative bunch when it comes to putting
a college list together) consider a score below 700 to be worrisome. Here is the
rub. In the past several years, colleges have been telling us that they care
less about the SAT… and therefore are looking to the SAT 2’s as a better measure
of achievement. Thus most of our students NEED to sit for the SAT 2 in either
Physics or Chemistry, or, if they are applying to an engineering program, both.

You might ask, “How hard is it to put in the extra study on an individual basis
to get past the 700 mark?” It turns out that this is pretty hard without teacher
intervention/help/encouragement. Starting in April, I will be teaching seven
days a week (instead of the usual six) in order to help prepare students. Our
college counseling group has asked that I do this, apparently unaware of the
toll it takes on course preparation and/or marital bliss. As you can imagine,
this makes me grouchy. I’d be somewhat less grouchy if I were not merely
drilling memorized information into the students, but this is the best way to
get them ready for a memorization-based test over the course of 8 weekends. In
fact, I generally fail at the lecture-then-memorize scheme, as I can’t resist
getting out circuit elements, etc. and playing with them.

I have actually had some back and forth with Engineering College deans on the
subject of the SAT 2. Their response is “Well, we see a correlation between a
good SAT 2 score and success in our program.” End of argument. If our students
made such a statement, college professors would be all over them for assuming
causation where only correlation has been demonstrated. Ironic, isn’t it, that
college admissions staffs demand more critical thought from high schoolers than
they do of themselves? What of the student who has a firmer foundation in
science at the expense of all that memorized material? Is there room for that
student? Because this is such a sticky question, many high school physics
teachers pick a book and grind through a chapter a week, drilling the students
with practice SAT 2’s every weekend and not daring to look for true
understanding. This not only sends weaker students your way, it reinforces the
students’ preconceptions! “Hey, I passed physics with flying colors answering
from my gut, why should I change in college?”

I hope this explains somewhat the frustration I have with the current college
admissions situation. It is all the more irritating to me as we are getting more
and more students interested in pursuing science and engineering.

A final note. I had one student who totally drank my kool-aid and refused to
even take the AP after taking second year physics with me. Also, refused to take
the Physics SAT 2. It might have cost him admission to an “elite” school, as he
suffered one admissions disappointment after another (although he was one of our
top students). He landed, happily enough, in Temple’s Honors program. He asked
that I call the professor who would be placing him in physics, which I gladly
did as he was an excellent student (the fact that this professor had just
decided to test M&I in the coming year helped a lot!). Upon my recommendation
that this student be allowed to skip first semester physics (but not the second
semester), the professor agreed to meet with the student. During the meeting he
gave the student a test…the result of which was the student was placed in a
sophomore level E&M class! He took a junior level classical mechanics class the
next spring (by somehow skirting the pre-req’s and apologizing later… I hope he
learned that from me), earning the highest score in the class. “I just keep
going back to fundamental principles,” was his explanation for his success. He’s
on his way to a physics-math double major, when he started with the idea of
being a math major with a physics minor. I suspect his undergraduate institution
will not be an impediment to his graduate school selection. If I could get all
my students to be this brave, I wouldn’t need to write any of the above.

I think Mark has some great points here. Now my question is what could be a replacement for the SAT II to demonstrate to a harried college admissions officer that a student has a strong grasp of physics and is well prepared for college science study?

9 Comments leave one →
  1. January 7, 2011 1:59 am

    I think that many people will agree that the AP and SAT 2 tests are not ideal tests. The big problem is what to replace them with. Does anyone know how to design a test that tests understanding and can be administered on the scale of the AP and SAT 2 tests? I suspect that the CollegeBoard would be glad to offer such tests—they just don’t know how to produce them. Do you?

    • January 7, 2011 5:17 pm

      I don’t have a super satisfactory answer here. My thoughts are for students and teachers themselves to devise metrics that demonstrate understanding in a field that can be quickly and easily digested by an admissions officer I came across this great post on 21 things that will become obselete by 2020, and one of the thoughts it suggests is that standardized testing will fall away in favor of digital portfolios, and I could see this happening, so long as there were ways for admissions officers to quickly evaluate their quality? Crowdsourced reviews? Something like stackexchange? (This is an idea I’m planning to flesh out more in a blog post).

      I’m also curious what you think—what evidence in your field, presented by a student, would impress you and motivate you to contact the admissions office on that students behalf? What effect would that contact have in the admissions process?

      • January 8, 2011 1:44 am

        Dream on! I see practically 0 chance of admissions at state schools moving in the direction of portfolios. The enormous increase in applications to colleges has meant that faculty have essentially no part in evaluating student applicants. Admissions officers have almost no time to read the silly essays they have students write (I believe that something like 20% of the essays get read at state schools for the students on the borderline). The trend is definitely *away* from having content-knowledgeable faculty being involved in admissions.

        I looked at the “21 things that will become obsolete” and most of the predictions there are just plain stupid, in many cases pointing in the opposite direction from current trends.

        • January 8, 2011 7:33 am

          State schools are definitely a challenge, particularly with the impact that budget cuts are happening on campus. But this is not a universal. After Michigan’s point-based affirmative action policy was shot down in Gratz v. Bollinger, UM increased its admission staff to be able to conduct a holistic review of all of its applications. Similarly, UC researchers found that:

          Researchers from the University of California found that during the first year of comprehensive file review at the Irvine campus officials were able to conduct 15,000 full file reviews of applicants without significant increases in admissions expenditures and that the process identified at least 20% more qualified black, Chicano and American Indian students than simply admitting students on test scores and grades. The University of California has now begun this process statewide. In the next phase of our project’s work, we will seek to study the benefits and costs of these and other new admissions models in collaboration with universities.

          Also, when I graduated from high school in 94, the application to Georgia Tech was 1 page long, and had an optional essay. Today the application is something six times longer and requires an essay. I’m sure that many state schools, for a variety of reasons and budget constraints, will not move quickly toward holistic application review. But I think as technology improves, it will become easier and more cost effective to do holistic review, and so I wouldn’t be surprised to see things continue to move in that direction. In my original point, however, I said that the portfolio review would have to something that could be quickly and easily digested by admissions officers, otherwise it will never happen. I think there might be a way to pull this off via crowd-sourced reviews which I’m going to try to post soon, but it is a fantasy at this moment.

          I recognize there is a lot of wishful thinking here, but here’s a further thought. When I graduated from college (Duke) we had something like 8 physics majors from a total class of 2000, only one of whom was female. A few miles down the road in Davidson, NC, Davidson was graduating a class of 4 physics majors, out of a class of 470 (double the percentage of Duke), all of whom were female (and one of whom I married). I am aware tat the incentives at most major research universities don’t really reward departments for recruiting and retaining undergraduate majors, and perhaps the incentives are even negative, since excess time spend on undergrads might take away from research and graduate education. But small size of the physics majors and lack of female majors during my time at Duke actually became a significant point of concern for the department. Obviously, there are a ton of factors that go into creating physics, majors but if one of the pre-requisites for creating a physics major is enrolling students with some sort of physics interest/aptitude, and the SAT II is the default metric for using this, then it might very well be that the admissions office is doing a bad job of identifying prospective physics majors. With enough impetus, I could imagine a department that cared about having enough majors to make upper-level class enrollments deciding to get more involved in the admissions process. I do know that Davidson’s approach was very different—members of the department were more involved in the admissions process, inviting prospective students to tour the physics labs during visit back days, etc and they do not require the SAT II. Having a 100% female physics graduating class became quite a point of pride for the department.

          Do you have no influence over admissions decisions on your campus? If you were impressed by a high school senior and thought s/he might make make an excellent student in bioinformatics, is there a channel for you to communicate that to the admissions office? If so, what would it take to impress you to use this channel to communicate with the admissions office?

  2. January 8, 2011 11:39 am

    I have absolutely no influence over the admissions process
    as an individual faculty member. The only mechanism is through an
    Academic Senate committee that can rebalance slightly the
    admissions criteria. That takes years or decades to have any
    effect. For the past 2 decades the committee has been dominated by
    those who regard it as the sacred mission of the university to
    admit more minorities. All changes have been dedicated to that end,
    no matter what damage was caused to other reasonable goals (like
    getting more science students or getting students who are better
    able to handle college work). The UCI success was exactly of that
    type: admissions officers were given more power to ignore evidence
    of ability to do college work and pay more attention to factors
    correlated with minority status, and so managed to admit more
    minority students. That was the goal and they achieved it. The
    emphasis on diversity as the first goal of admissions, with
    excellence a distant second, has had several consequences for the
    university—some good, some bad. Different faculty have very
    different opinions about what the mission of the university is and
    whether the changes further that mission or make it harder. In
    fact, the university has many different missions, not all of which
    are completely compatible, so the problem is always one of balance.
    Currently, and for the next decade or so, the pendulum has swung
    towards greater inclusiveness in the admissions process. The UC
    system now promises to admit the top 9% of each high school, rather
    than the top 12.5% of the state. It doesn’t matter if some high
    school is graduating kids who are completely illiterate (and budget
    cuts for K-12 education have been severe enough that some high
    schools are coming close to that). The goal you want, paying more
    attention to factors that affect whether a student can do real
    academic work using more difficult to measure properties, is
    exactly the opposite of the trend that has been happening at UC.
    Admissions officers are never going to be able to tell good physics
    from bad physics in a portfolio (at least until physics departments
    are graduating so many excess physics students that they are
    willing to take jobs as admissions officers)—at best they’ll give
    points for “has a science essay”. This is not going to do a better
    job of selection of physics students than the SAT2 does, as flawed
    as the SAT2 is.

    • January 8, 2011 6:27 pm

      You’re absolutely right that the admissions office is never going to be able to evaluate, on its own, the quality of a prospective physics student, nor will it ever really be able to tell the exceptional violinist from the merely very good. This is why I think faculty involvement in the admissions process is essential. I mean the football coach sure is identifying and advocating for the students he wants on campus, right?

      • January 9, 2011 8:32 pm

        Yes, the football coach at most schools is identifying and advocating for players. But at those schools the coaches are paid much more for much less work than the faculty. Perhaps you could convince the faculty to go on paid junkets to science fairs around the country to spot new talent, if you reduced their workload to a coach’s workload and paid them like you paid coaches.

        Disclaimer: the university I teach at does not have a football team or much in the way of paid coaches. Intramural participatory sports are more important than interscholastic spectator sports. The students, faculty, and alumni are all happier to have it that way.

        • January 9, 2011 8:46 pm

          Yes, you’re right. But it would be a nice world if coaches had to spend all their time writing grants to fund their teams, while science professors got to travel the world recruiting talent for their labs.

Trackbacks

  1. The SAT II might be a roadblock for extraordinary accomplishment « Quantum Progress

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: