SBG Implementation: Creating Assessments

Programming note: I'm breaking this post in to two parts. This one is on the test itself and the next will be on scoring and tracking progress.

Citation: I forgot to mention that I started off with the base system outlined in Classroom Assessment and Grading that Work and have spun it off a bit to better fit my context and style.

This is the second in a series on the process I went though implementing standards-based grading in my class. Part one here. If you haven't read that one yet, you should go back. You'll need to understand the 0-4 system before you read the rest.

Just as a refresher, here are the ground rules:

1. There is no canon for SBG.
2. Context is everything.¹

In case you're one of twelve people in the world that hasn't read it, here are Dan Meyer's templates. I take a topic-based approach versus his skills approach so our formatting is different and the meaning behind the 0-4 scale is different. However, his guiding principles are the same. Excerpted here:

1. It doesn't matter when you learn it as long as you learn it.
2. My assessment policy needs to direct my remediation of your skills.
3. My assessment policy needs incentivize your own remediation.

Adding a fourth: My assessment policy should do as much of the work as possible.
And fifth: The ultimate goal is self-assessment. Any assessment policy should help students to become better self-assessors.

Jargon alert: I use the term "standards" to refer to your in class learning goals, not your state standards. I use the term test and quiz interchangeably. I'm specifically referring to a written assessment that a student takes by him/herself

Step two in my adventures into standards-based grading: Creating Assessments
Before you get overly excited, this is not the post where you're going to learn how to write really rich and interesting problems. Look to your right. Most of those blogs devote themselves to that exact purpose. Grace has a good, short post on testing conceptual development. Personally, I like to use the rich problems for teaching, but not so much for written assessment. Context is everything here and I'd love for you to comment that I'm wrong about this. I have so many English learners that I find that I'm not really testing what I think I'm testing (h/t @optimizingke). My written test questions tend to be straight vanilla.

What I am going to write about today is test formatting. Not very glamorous, but it's one of those things that will make your transition much easier.

Tests exist so you and your students can tell where they are and what needs to be done next. The test itself should provide feedback.

There are many other ways to do this but tests have a couple of advantages. One: They're darn efficient. You can knock out a test in 15 minutes and get kids working on remediation in the same period. Two: There's a paper trail. It helps students to see how their thinking has progressed and to analyze the mistakes that they've made again and again (and again).

I definitely use multiple methods of assessment, and in fact it's necessary in any quality assessment system. However, written tests are the bread and butter for most teachers so that's what we're going to focus on here.

Here's a sample test. Most of my problems are drawn from sample state test questions, textbooks, and what we develop as a department:²
atomslg2

Specific features:
1. In the top right you'll see the topic name and which assessment number it is. They're going to reassess a lot. In my class that's usually a minimum 4-6 times for a single topic in class and even more if they want to come after school. You've got to have a way to keep them/you organized.

2. I wrote the learning goal on top. I'm actually planning to remove that. I think my original tracking sheets only had the learning goal on top and not the topic name. It was my way to help the kids match up the tracking sheet to the assessment. Now that I put the topic name on both, it's superfluous. You would think having the learning goal on the top would be like a cheat sheet to help my students answer the questions. Sadly, you'd be wrong. If anyone has a good reason for me to leave it, let me know.

3. After each test question, there's a letter that matches the question to the standard listed in their tracking sheet. Mostly, the reason for this is to help you and your students track their progress on each standard. There's a second plus. It serves as a self-check when you're writing your test questions. If you can't match up a question directly to a standard, or multiple standards, you need to edit your question, your standards, or both.

4. Questions are separated into 2.0, 3.0, and 4.0 questions. The front page consists of simple ideas, usually vocabulary or basic knowledge. The 3.0 questions are the more complex ideas. The 4.0 question has not been addressed in class but if they understand the complex ideas, they can apply those to arrive at a solution.

Most teachers I've observed cluster by problem type (multiple choice together, then matching, then short answer at the end.)

If you're clustering by problem type, please stop. It's unhelpful. You and your students should be able to identify some sort of pattern by looking at the results of the tests. That's really hard to do on tests like this. Unless your goal is to help students figure out what type of question they have the most difficulty with, I can't think of a good reason for doing this.

The second most common method is to cluster by standard. I like this method, but don't use it. I really like it for some purposes and designed our department benchmarks in this way. The strength is that there's a direct line between "what I missed on this test" and "what I need to work on." Remediation is really straightforward. Since I cluster my test questions by complexity, there's an extra step they need to work through to figure out what needs to be done next. It's not a difficult step, but some students will need more hand holding than I prefer.

So why do I choose to group my tests in this method? The most obvious is that this is how my grading scale works. It's really clear both to my students and to myself what score the test will receive. There's a better reason for choosing this method though.

I often come back to how I think using topics helps create connections across standards better than skill-based assessments and here's another example. In a skill-based assessment, students can easily see what specific standard they need help on, however because of the very format of the assessment (chunked by standards and often on separate pieces of paper) it is harder to look for patterns of errors and broad misconceptions. In a topic-based system, it is easier to see systematic errors.³

Imagine a student missing questions 7 and 9 on the test above. I could ask them to go and learn more about both states of matter and the kinetic molecular theory. In a skills-based assessment that would be my default, especially if those standards were assessed at separate times. In a topic-based system, when I read the answers together I might realize that the student has a knowledge gap that's affecting both questions. For example he/she might not understand the relationship between temperature and molecular motion, which is necessary to answer both questions.

Additionally, by clustering from simple to complex, I can quickly focus on a few common problem areas. If they're stuck on the first page, it's usually something like, "He has no idea what an atom is so he's just guessing at everything" or "She is confusing velocity and speed." If the difficulties occur on the 3.0 questions, I can rule out all of the simple stuff because I've seen that they've mastered that already.

This is totally possible to do with an assessment grouped by standard. I think the trade off of more direct remediation for making connections/ease of scoring is worth it.

All of this is a really long-winded way of saying that if you're going to implement a topic-based system, it makes sense to design your assessments by topic. Since you've taken the time to build a learning progression into your topic, keep that intact as well.

Sticking Points:
You can't figure out how to cram an entire topic into a single test without making it the SAT. You're aiming for frequent smaller assessments rather than a big one at the end of a unit. Don't feel the need to cram every last question in. Get to it next time. Notice that the second question only deals with the charge of a neutron. In other assessments I would have the properties of protons or electrons as well. Generally, if a student doesn't know the neutron is neutral, they're also not going to know the proton is positive and electron is negative. I don't feel the need to put all three on the same test. I just want to get an idea for what they know so I know what to teach next.

You've made this great topic scale but don't know how to write questions for it. I've found that commonly the problem isn't that I can't come up with a question for a standard, it's that the standard doesn't lend itself to being answered in a standard testing format. You just can't assess your state's oral language standards in writing. In your classroom, anything is an assessment. You've got your topic scales, get them to do whatever it is they need to do and evaluate it against those scales. My textbook's assessment software includes questions with pictures of a triple-beam balance and different arrows pointing at the numbers. You're supposed to then tell the mass of the object on the balance. I could do that. Or I could not be the laziest person ever and get out the triple beam balances and walk around as they determine the mass of different objects. For the most part, teachers seem to be pretty good about understanding that different modes of assessment are necessary. We're not as good at realizing that those other modes should be replacing written tests. Most teachers I know would start with having the actual triple beam balances out but then pull out the written tests later. Either they don't think it should count unless it's on a test or they're worried about CYA or they're just in the habit of putting everything on a test. Break the habit. Everything doesn't need to eventually show up in multiple choice format.

 
Next up: Scoring the assessment and tracking progress


1: Forgot to mention this last time. The book Mount Pleasant is about the high school my kids feed into. Don't buy it, but if you're in California you might want to check it out from your library and read what our current insurance commissioner and aspiring governor thinks of public schools. 
2: Teaser: After the implementation series is done, I'm going to start on the joys of common assessments. Most teachers I talk to hate them. If you do, you're doing it wrong. More likely, your school is doing it to you wrong.
3: It's easier for me at least. I don't have a good system, or even a mediocre system, for teaching students to look for patterns of errors and misconceptions.