Information about the California Standards Test Part 2b

Part 1
Part 2

I haven't said anything about the students. This belongs in bold:

Do not make any placement decisions solely based on test scores. 

I know you do it. We do too. In fact, denying students an elective based on their state test scores is one of the joys of being in Program Improvement. But the state of California, perhaps in word but not deed, agrees. From the Post-Test Guide....and I quote:

Any comparison of groups between years should not be used for diagnostic, placement, or promotion or retention purposes. Decisions about promotion, retention, placement, or eligibility for special programs may use or include STAR Program results only in conjunction with multiple other measures including, but not limited to, locally administered tests, teacher recommendations, and grades. (page 13)

and then on page 14 they give almost the exact same quote directed towards individual students in a bolded and boxed callout:

Decisions about promotion, retention, placement, or eligibility for special programs may use or include CST or CMA results only in conjunction with multiple other measures including, but not limited to, locally administered tests, teacher recommendations, and grades.

WAIT!!! One more because this bears repeating (also on page 13)

While there may be a valid comparison to be made between students within a grade and content area, it is not valid to subtract a student’s or class’s scale score received one year in a given content area from the scale score received the previous year in the same content area in order to show growth. While the scale scores may look the same, they are independently scaled so that differences for the same students across years cannot be calculated using basic subtraction. 

Summary: You can't use test scores as the sole method for placement AND you can't use them to determine growth year to year. Wait....but what if a student scores Proficient one year and Basic the next? Shouldn't we put him in a second math class and catch him up? Noooooooooo.  Here's the easiest example why.

2nd grade - 62%
3rd grade - 65%
4th grade - 68%
5th grade - 60%
6th grade - 52%

What's this? This is the percent of students in California proficient in math in 2010.

Either: California sticks all of its best teachers in 4th grade. (unlikely). Approaching puberty makes students universally worse at standardized tests (plausible). The test is harder (that's what this study argues).

Here's the percentage of kids scoring Advanced:

2nd grade - 36%
3rd grade - 38%
4th grade - 42%
5th grade - 29%
6th grade - 23%

(7th and 8th grade we start tracking the kids more. At many schools, advanced 7th graders end up in Algebra and then Geometry in 8th. Some kids even take Algebra 2. A direct comparison is harder).

So a student drops from Proficient to Basic. Why? Hard to say. The test looks like it certainly got harder. With standard error and such he could have really been Basic last year. That's not to mention the inanity of deciding an entire year of coursework based on one 60ish question multiple choice test taken 80% of the way into the year. The test wasn't designed to make those kinds of inferences and you're left with too much doubt about the cause of the drop in score (or whether there even truly was a drop).

I didn't include anything about useful information about students because there isn't any. Well, there almost isn't any. You could probably tease out some information in conjunction with local assessments but really, what's the point? Right now I can tell you that Alondra got 9/13 correct in the "Functions and Rational Expressions" strand of her 7th grade math test. In order to figure out exactly what gaps she had, I need to give her a local diagnostic anyway. And that's really the whole point. If Alondra got an F in the class, I'm more concerned about that than her scoring Proficient. But for some reason, we think that because she scored Proficient on a single day of the year, that's more important than the other 181 days she was floundering. We let a single test override an entire year.

The hours that your school is going to spend dealing with this stuff could be spent doing something useful. You know, like standards-based grading.

Yeah I know. I came back to it. But really, isn't the complete and utter nonsense that is our current grading system a major reason for needing a standardized test in the first place? Right now we look and say, "Oh noes! Jason is basic this year! We've got to get him in extra math!" We have to do that because a B in Mr. X's class is not the same thing as a B in Mrs. Y's class. We don't trust that Jason getting a B in math means anything at all. But really, if we trusted the grades we could say, wait, Jason learned X, Y, and Z. He's fine. Or he still needs to learn Z but that's not going to require us to take him out of art or music the entire year.

If there's a moral here it's that we try to make the CST more than it is. The state of California does an awful job of communicating what the state test really does. I don't really think it's in the state's best interest to do a good job though. If more parents/students really knew what decisions were improperly being made off these score, the state would have a lot of 'splainin' to do. I also think from our end, because we have to devote so much time and energy towards it, we feel the need to justify that.

I've gotten some valuable information out of the test. I've revamped a couple of units. I've gone and visited a few really good science teachers and programs. That's fine. But you're not going to get much more than that. Invest your time and energy into something more meaningful. Oh and never place kids solely based on test scores. Just don't.



Full disclosure: I'm not against standardized testing. I find it useful to have a third party to calibrate my class against. I live in absolute terror that I'm teaching down to my kids. It helps me set a baseline. What I am against is the high stakes part. Even disregarding the punitive aspects of the test, the high stakes nature (and subsequent fear of cheating) has caused California to hide any useful information we might get out of these things that take up so much of our time.