My first assessment failure

...and certainly not my last.

Before I knew anything about standards-based grading, I only knew that the way that we graded stunk. So I developed my own system. It turns out there's a reason teachers who have been in the classroom for only two years usually don't develop their own systems. My system sucked.¹ I wanted to share it though to let you know what I learned from it and why the new system I use is much better.
I called it the Learning Matrix. Here's what a sample sheet looked like:
Learning Matrix

I based it off of two principals that still show up in my work with SBG:

1. Separate learning goals instead of one lump score.
2. Start with the end in mind and create a path to get there.

I decided to base it off of Bloom's taxonomy. Why Bloom's? I don't really remember at this point, other than we're always getting it thrown at us and maybe I figured I should use it. I recall at that time I used Costa's questions to create my question bank so I'm not sure why I didn't use that. Perhaps it just had a whole bunch of levels which I thought was cool. I don't know.

It worked somewhat similar to the skill checklist that many math bloggers use. The students had a set of questions or tasks that were specific to one of the grid boxes. Once or twice a week students would pull out the matrix and look at their grids. They'd select a box and find the accompanying assessment and work on it. The lower levels would involve definitions while the higher levels might be involve the completion of a mini-lab. After completion they'd bring it up to me and I'd give them a quick oral assessment. If they could satisfy my grilling I'd stamp it and they'd color in the box. There was also a pocket chart with the grid on it. Students would move an index card with their name on it to whatever they were currently working on. The idea was that a student could look for someone working on a higher level for help.

So what went wrong?

Ultimately the problem with this method was its basis on Bloom's. Bloom's is a classification system, not a learning progression. A learning progression lays out the steps needed to get to a learning goal. The current system I use creates a learning progression. How do I know this? Because I designed it that way. I started with my goal (in this case, the 3.0 concepts) and worked my way backwards. Let's say my 3.0 goal is for students to be able to calculate the average speed of an object. What do I need to know in order to do this? Some basic vocabulary, an idea of what motion is, some SI units, and how to measure stuff. That's the 2.0.²

I was waylaid by two other big things.

Number 1: I didn't teach the verbs. This wasn't necessarily a problem with my system so much as it was a problem with my teaching. I have a number of assessment pet peeves. Grading things that should be practice. Extra credit for non-academic stuff. Rubrics that focus on product guidelines instead of understanding. But gun-to-my-head I think I might say that my number one assessment pet peeve is grading something you don't teach. How in the world were my kids supposed to be able to "evaluate" when I didn't teach them how? I expected their mastery at one level to automatically gift them with the knowledge of how to do the next level. Even worse, I'd be surprised that they always got so badly stuck after starting a new level.

Now? I directly teach any higher order thinking that I will assess. If you're expected to "Compare/contrast speed, velocity, and acceleration," I'm teaching my kids how to make a comparison matrix. If I expect my kids to evaluate something, I'm teaching them how to create decision making criteria. Now, when a compare or evaluation question shows up on a test, they've got a set of skills to draw on. We need to directly teach thinking skills along with content knowledge. I used to think they just didn't get the content. They got the content. The thinking skill just got in the way.

Number 2: False equality. Not all topics need to be learned to equal depth. For the rest of their science lives, do all students need to understand the atomic theory? You bet. How about Archimedes Principle? Umm...not as important. The state of California dictates what I need to teach my students each year. That doesn't mean they all deserve equal time. In the new system, the atomic theory gets its own topic while Archimedes Principle falls under the topic Forces in Fluids. Some things (I'm looking at you polymers!) get demoted all the way down to just being vocabulary.


What's the take home message?
More than anything, that experience made me realize that this is going to be a long process. I felt like some sort of mad genius when I was designing the system. Two years into a formalized standards-based system, I'm still tweaking and learning. If you've been implementing a standards-based system you know what I'm talking about. If you're planning to, you'll know what to expect.

PS -  Don't forget to explicitly teach higher order thinking skills.


1: It still didn't suck as much as the way we regularly grade.
2: I could definitely be wrong in the setup of the learning progression. Perhaps learning doesn't occur in that order. However, my scales are fluid from year to year and I take solace in the fact that the research on learning progressions isn't all that solid.

Safety in numbers

Once a month I meet with a small group of teachers at my school to talk about assessment issues. Today I shared a post by Jerrid Kruse titled Grading: the myth of objectivity. I definitely recommend adding him to your Reader if you haven't already. Here's a relevant snippet:

I suggest we admit that our assessments are merely a judgement. Our judgement of students’ understanding and effort as measured by OUR measuring sticks not THE measuring sticks. I believe that once we admit the subjective nature of assessment, we will be free to work towards more authentic and hopefully accurate assessment means.

This eventually led us to a discussion on one of the essential elements of standards-based grading: professional judgment.

We tend to hide behind the shield of numbers. We enter in grades and weights and let our grading software do the rest. If a parent comes in to complain about a student's grade, we point to our gradebook and say, "They got 60 out of 100 on the last test and their overall grade is a 72%." Nobody can argue with the numbers. In fact, we're letting the numbers do the arguing for us.

That's the problem. The numbers protect us and give us authority. One of the strengths of SBG is that I'm required to defend every one of my grades. I can't hide behind the mean or median. I have to own my assessments and own every score that goes into my gradebook.

I am constantly re-evaluating my methods of assessment. I have to take into account the student's total body of work. I have to be completely clear on what it means to have mastered a standard. 

I understand the hesitation in switching to a SB system. It's scary to be so exposed. I gather and record information obsessively, at least in part because I live in fear of the day a parent kicks down my door, demands to know why his daughter got a C, and I can't defend my grade.

Professional judgment is a key pillar of SBG and perhaps the scariest. However, I firmly believe that true assessment reform cannot take place unless we as teachers are willing to take responsibility for the grade we record in our books.

Translating to a letter grade

I apologize for not posting anything assessment related in awhile even though this is ostensibly an assessment blog. A post by Edumacation snapped me back into things.

As you know, I separate my different standards into topics. At the end of the trimester I usually end up with 4 to 6 topics. These need to be combined into a single grade for our traditional A-F report card. At the end of the trimester I might have scores that look like this:

Motion	Graphing	Forces	Forces in Fluids
2.5	3.0	4.0	2.0

In my ideal world I could leave those as is. Throw some snazzy bullet graphs and charts onto the report card and send it on home.

I use a method called conjunctive scoring that I learned about in a Marzano book. The basic idea behind conjunctive scoring is that good scores can't make up for bad ones. No averages. This is how my scores are currently translated:

Grade	Lowest score	At least one
A	3.0	4.0
B	2.5	3
C	2.0
D	1.5
F	less than 1.5

So in my above example the student's final grade would be C because their lowest score is a 2.0.

Motion	Graphing	Forces	Forces in Fluids
2.5	3.0	2.5	3.0

This student would finish with a B because her lowest score is a 2.5 and she has at least one 3.

Against:
Clearly this is very different. There are no averages so it takes awhile for students to get it. You also really need to set aside time to circle back and allow students to go back and learn something they missed. This also requires a certain amount of balance in your topics. Obviously a killer topic can destroy grades. I would argue that some balance in the topics is something to strive for but there are ways to remedy this. The first year I tried SBG, the Motion topic killed kids because of all the math and because I just threw too much stuff into it. I tried to remedy this by separating out Graphing into a separate topic. That worked to chunk it a little more and helped focus my instruction.

For:
I chose this method precisely because high scores wouldn't make up for lower ones. One of the problems with averaging is that you can accumulate enough points that the rest of the course is irrelevant.We've all had students who realize they're so far into A territory that they can coast the last few weeks of school. I've (hopefully) distilled my many, many standards into a few big ideas. In my view, it's ALL important. I like that a student can have all 4s and one 2 and she'll need to work those last few weeks on that one topic.

If the ultimate purpose of grades is to communicate, then an A should be clearly defined. If an A means a student has mastered or surpassed the standards set for the class, it follows that they should master or surpass every single one.

Side bonuses:
We're in our last week of school before the trimester ends and I always notice it's very focusing for a student to take a look at their concept scores, zero in on the lowest, and start working away at it. It's a lot more nebulous to say, "You need 8% more for a C."

There's also this interesting psychological component. If you look at the first example, she only needs to bring up two scores and she can immediately move from a C to an A. Mentally, students feel this is easier to obtain than needing 14% more or 150 more points to move up two grades. I'm sure it goes back to topic scores being more focusing than percentages.

How did I arrive at that scale?
It came through negotiation. My scale was slightly tougher last year. For example, I allowed only one 2.5 for a B. This year, another teacher is trying standards-based grading and we decided on a common scale. I'm reasonably happy with the new scale. There's an interesting passage in the blog I linked to earlier about what an A or B should mean in comparison to meeting standards. Those are the same arguments we had. We thought of it in terms of our state tests. Anyone with an A or B should be advanced or proficient and a C student should have a reasonably good shot.

What about non-academic scores?
I actually don't use them, but I have no problem with them. Again, ideally you'd want to keep those separate. Most schools have a citizenship grade or something similar, if you redefine that you're all set.
If you can't, you'll need to weight them. You would arrive at a score for your academic topics using the conjunctive method. For non-academic scores, the conjunctive method can be used, but in this case I think averaging is the way to go. Going back to the killer topic idea, imagine your non-academic scores are organization, participation, behavior, and work completion. There's really no reason for any of those to kill the entire non-academic score. For your final grade you take your academic score, let's say 90%, and weight it against your 10% non-academic. There are classes that would have much higher non-academic weights (P.E. comes to mind) but for the most part, keep the academic portion high.

The take home is that it's possible to translate your standards-based scores into a traditional letter grade and I'm reasonably happy with the outcome. If translating to a final grade is holding you back, I think conjunctive scoring is a good solution.

I'm your father and you'll do it because I say so!

Remember when you were young and you'd get into an argument with your parents? I was a, let's say, special child in that regard. Eventually one or both of my parents would run out of reasons for doing something and resort to the "because I say so" argument. I have a vague recollection from Philosophy 10 that this is called argument from authority.

Points are our ultimate "because I say so." How many times have you as a student or teacher had this conversation:

Student: Why should I do this?
Teacher: Because it's worth X points.

When we don't do the work required to link our assignments with clear learning goals this is what we fall back on.

The key then to getting your students to complete their assignments is to create a clear link between "doing the work" and learning what you need to learn. Your new conversation should be:

Student: Why should I do this?
Teacher: Because it will help you learn X.

This is obviously a constant and ongoing conversation. I've mentioned it before in an older post but it's one of the crucial mindset shifts that needs to occur for your students to really buy in. You need to make their learning progress clear and relate it to the effort they've put in. Your classroom language is crucial here, which I'll post about in the future.

Here's one example I show my kids to help them link the work with the learning. I take a survey every trimester (I show last year's results in the beginning of the year). There are two questions:

1. Think of everything you were asked to do in science this trimester, what percentage do you think you completed?
2. What was your final trimester grade?

When I say everything I mean everything. I tell them to count exit slips, notes, turn to your partner types of activities, warm-ups, etc.,in addition to the traditional worksheets/lab stuff. I throw the results up on the screen the next day:

I remind them that I don't give them points for turning in work. What's going on in the graph is the students who are doing more work are learning more. I should probably print this graph out so I can just point to it every time the above conversation happens but paper has become quite the precious commodity at my school.

As a side note this is a good opportunity to do a mini-lesson on problems with self-report surveys. Generally, the A students tend to under-report how much work they've actually done. Some of them do it because they like to pretend they understood everything easily. Others don't count things like coming after school or getting help from friends during lunch. So while they may have only done 50% of everything in class, they more than made up for it with the extra time they spent outside of class that they don't think counts as work.

The F student results are also skewed because the super-duper F students don't even turn in the survey.  I've done this survey 7 times now and every time it ends up similar to that pattern above with A students doing the most, a nice even tapering by grade, and a huge drop off to F.

What is standards-based grading?

It's probably easiest to describe standards-based grading in comparison to traditional grading. Yesterday, as an introductory lab, I asked the kids to try to figure out how fast a toy car goes. My kids didn't have any real background knowledge yet, but I did provide them with a meter stick and stopwatch.



Traditional grading:
In traditional grading I might have made this lab worth 50 points. Usually in science we require a lab report and assign point values to each section of the lab report. For example, I might have made a correctly written hypothesis worth 10 points, the procedure worth 5 points, and graph and data table worth 15 points. In the end, a student would receive a grade based on their lab report and some sort of point amount for "getting the right answer" that s=d/t.

Problems:
There are many problems with this grading system and I will comment on them as I progress in this blog. I think the main problem is the misalignment of the purpose and the grading of the lab. The purpose of the lab was for students to attempt to figure out how to calculate the speed of a toy car and hopefully create a general rule or formula. What did I grade the students on? I graded them on how well they could follow a pre-assigned lab format. Imagine a student who was able to both calculate the speed of a toy car and figure out that s=d/t but for some reason decided not to number his procedure. On the other hand, you could have a student create the most beautiful data table in the world but completely miss the point of the lab. It's this misalignment of purpose and grade that is the primary failing of traditional grading.

Standards-based grading:
Here's where I always lose the holdouts on the teaching staff when I present standards-based grading to them. In standards-based grading, you only grade how well they have mastered a specific standard or topic. The grade completely ignores the format of the lab report¹. Their grade is aligned with the purpose of the lab. The grade should answer the question,"How well do they understand the concept and calculation of speed?"

The grade reflects their level of understanding on that specific topic or standard and nothing else. It seems obvious, but in practice it's a huge paradigm shift.


1. Well, you might actually have a separate standard for writing a lab report in which case you would give two scores, one for the content and one for the format.