SBG Implementation: Creating Assessments

Programming note: I'm breaking this post in to two parts. This one is on the test itself and the next will be on scoring and tracking progress.

Citation: I forgot to mention that I started off with the base system outlined in Classroom Assessment and Grading that Work and have spun it off a bit to better fit my context and style.

This is the second in a series on the process I went though implementing standards-based grading in my class. Part one here. If you haven't read that one yet, you should go back. You'll need to understand the 0-4 system before you read the rest.

Just as a refresher, here are the ground rules:

1. There is no canon for SBG.
2. Context is everything.¹

In case you're one of twelve people in the world that hasn't read it, here are Dan Meyer's templates. I take a topic-based approach versus his skills approach so our formatting is different and the meaning behind the 0-4 scale is different. However, his guiding principles are the same. Excerpted here:

1. It doesn't matter when you learn it as long as you learn it.
2. My assessment policy needs to direct my remediation of your skills.
3. My assessment policy needs incentivize your own remediation.

Adding a fourth: My assessment policy should do as much of the work as possible.
And fifth: The ultimate goal is self-assessment. Any assessment policy should help students to become better self-assessors.

Jargon alert: I use the term "standards" to refer to your in class learning goals, not your state standards. I use the term test and quiz interchangeably. I'm specifically referring to a written assessment that a student takes by him/herself

Step two in my adventures into standards-based grading: Creating Assessments
Before you get overly excited, this is not the post where you're going to learn how to write really rich and interesting problems. Look to your right. Most of those blogs devote themselves to that exact purpose. Grace has a good, short post on testing conceptual development. Personally, I like to use the rich problems for teaching, but not so much for written assessment. Context is everything here and I'd love for you to comment that I'm wrong about this. I have so many English learners that I find that I'm not really testing what I think I'm testing (h/t @optimizingke). My written test questions tend to be straight vanilla.

What I am going to write about today is test formatting. Not very glamorous, but it's one of those things that will make your transition much easier.

Tests exist so you and your students can tell where they are and what needs to be done next. The test itself should provide feedback.

There are many other ways to do this but tests have a couple of advantages. One: They're darn efficient. You can knock out a test in 15 minutes and get kids working on remediation in the same period. Two: There's a paper trail. It helps students to see how their thinking has progressed and to analyze the mistakes that they've made again and again (and again).

I definitely use multiple methods of assessment, and in fact it's necessary in any quality assessment system. However, written tests are the bread and butter for most teachers so that's what we're going to focus on here.

Here's a sample test. Most of my problems are drawn from sample state test questions, textbooks, and what we develop as a department:²
atomslg2

Specific features:
1. In the top right you'll see the topic name and which assessment number it is. They're going to reassess a lot. In my class that's usually a minimum 4-6 times for a single topic in class and even more if they want to come after school. You've got to have a way to keep them/you organized.

2. I wrote the learning goal on top. I'm actually planning to remove that. I think my original tracking sheets only had the learning goal on top and not the topic name. It was my way to help the kids match up the tracking sheet to the assessment. Now that I put the topic name on both, it's superfluous. You would think having the learning goal on the top would be like a cheat sheet to help my students answer the questions. Sadly, you'd be wrong. If anyone has a good reason for me to leave it, let me know.

3. After each test question, there's a letter that matches the question to the standard listed in their tracking sheet. Mostly, the reason for this is to help you and your students track their progress on each standard. There's a second plus. It serves as a self-check when you're writing your test questions. If you can't match up a question directly to a standard, or multiple standards, you need to edit your question, your standards, or both.

4. Questions are separated into 2.0, 3.0, and 4.0 questions. The front page consists of simple ideas, usually vocabulary or basic knowledge. The 3.0 questions are the more complex ideas. The 4.0 question has not been addressed in class but if they understand the complex ideas, they can apply those to arrive at a solution.

Most teachers I've observed cluster by problem type (multiple choice together, then matching, then short answer at the end.)

If you're clustering by problem type, please stop. It's unhelpful. You and your students should be able to identify some sort of pattern by looking at the results of the tests. That's really hard to do on tests like this. Unless your goal is to help students figure out what type of question they have the most difficulty with, I can't think of a good reason for doing this.

The second most common method is to cluster by standard. I like this method, but don't use it. I really like it for some purposes and designed our department benchmarks in this way. The strength is that there's a direct line between "what I missed on this test" and "what I need to work on." Remediation is really straightforward. Since I cluster my test questions by complexity, there's an extra step they need to work through to figure out what needs to be done next. It's not a difficult step, but some students will need more hand holding than I prefer.

So why do I choose to group my tests in this method? The most obvious is that this is how my grading scale works. It's really clear both to my students and to myself what score the test will receive. There's a better reason for choosing this method though.

I often come back to how I think using topics helps create connections across standards better than skill-based assessments and here's another example. In a skill-based assessment, students can easily see what specific standard they need help on, however because of the very format of the assessment (chunked by standards and often on separate pieces of paper) it is harder to look for patterns of errors and broad misconceptions. In a topic-based system, it is easier to see systematic errors.³

Imagine a student missing questions 7 and 9 on the test above. I could ask them to go and learn more about both states of matter and the kinetic molecular theory. In a skills-based assessment that would be my default, especially if those standards were assessed at separate times. In a topic-based system, when I read the answers together I might realize that the student has a knowledge gap that's affecting both questions. For example he/she might not understand the relationship between temperature and molecular motion, which is necessary to answer both questions.

Additionally, by clustering from simple to complex, I can quickly focus on a few common problem areas. If they're stuck on the first page, it's usually something like, "He has no idea what an atom is so he's just guessing at everything" or "She is confusing velocity and speed." If the difficulties occur on the 3.0 questions, I can rule out all of the simple stuff because I've seen that they've mastered that already.

This is totally possible to do with an assessment grouped by standard. I think the trade off of more direct remediation for making connections/ease of scoring is worth it.

All of this is a really long-winded way of saying that if you're going to implement a topic-based system, it makes sense to design your assessments by topic. Since you've taken the time to build a learning progression into your topic, keep that intact as well.

Sticking Points:
You can't figure out how to cram an entire topic into a single test without making it the SAT. You're aiming for frequent smaller assessments rather than a big one at the end of a unit. Don't feel the need to cram every last question in. Get to it next time. Notice that the second question only deals with the charge of a neutron. In other assessments I would have the properties of protons or electrons as well. Generally, if a student doesn't know the neutron is neutral, they're also not going to know the proton is positive and electron is negative. I don't feel the need to put all three on the same test. I just want to get an idea for what they know so I know what to teach next.

You've made this great topic scale but don't know how to write questions for it. I've found that commonly the problem isn't that I can't come up with a question for a standard, it's that the standard doesn't lend itself to being answered in a standard testing format. You just can't assess your state's oral language standards in writing. In your classroom, anything is an assessment. You've got your topic scales, get them to do whatever it is they need to do and evaluate it against those scales. My textbook's assessment software includes questions with pictures of a triple-beam balance and different arrows pointing at the numbers. You're supposed to then tell the mass of the object on the balance. I could do that. Or I could not be the laziest person ever and get out the triple beam balances and walk around as they determine the mass of different objects. For the most part, teachers seem to be pretty good about understanding that different modes of assessment are necessary. We're not as good at realizing that those other modes should be replacing written tests. Most teachers I know would start with having the actual triple beam balances out but then pull out the written tests later. Either they don't think it should count unless it's on a test or they're worried about CYA or they're just in the habit of putting everything on a test. Break the habit. Everything doesn't need to eventually show up in multiple choice format.

 
Next up: Scoring the assessment and tracking progress


1: Forgot to mention this last time. The book Mount Pleasant is about the high school my kids feed into. Don't buy it, but if you're in California you might want to check it out from your library and read what our current insurance commissioner and aspiring governor thinks of public schools. 
2: Teaser: After the implementation series is done, I'm going to start on the joys of common assessments. Most teachers I talk to hate them. If you do, you're doing it wrong. More likely, your school is doing it to you wrong.
3: It's easier for me at least. I don't have a good system, or even a mediocre system, for teaching students to look for patterns of errors and misconceptions.

SBG Implementation: Topic Scales

For those of you not active in the twitterverse, among the people I follow there's been an increasing interest in standards-based grading. Sam Shah called us an "inspiring ideological cult." I'm taking that as a compliment. 

With that in mind I'm going to explain my process for implementation.

Assumption: You already have a basic idea of what standards-based grading is. If not, go read every post by Shawn or Matt and come back.

Things to keep in mind:

1. There is no canon for SBG. It's a still new decades-old idea if that makes sense.
2. Context is everything. I've developed processes that fit my kids and style.

So here's my context: I teach 8th grade physical science in San Jose, California. That's intro physics, chemistry, and astronomy. Last year we had an API of 704 (800 is the goal in CA), with a similar schools rank of 4 and a statewide rank 3. 71% of my students are on free or reduced lunch and 44% are ELL.  We are 75% Hispanic. We border the East Hills which means we also have a group of kids that live in gated communities with swimming pools. I was originally the only SBGer at my school. This year there are two others (a 7th and 8th grade ELA teacher) and there are three more teachers (7th math, 6th math/sci, 7th ELA) that have adopted certain aspects of SBG.

What I've developed has been reasonably successful for my students, YMMV. Leave suggestions for improvement in the comments or send something through twitter or email.

Jargon alert: I'm going to use the word "standard" to refer to any unit of knowledge or skill you want to teach and assess. These are also variously referred to as learning goals or targets. A topic, also called a strand, is a collection of those standards that are grouped together in a meaningful way. Do not confuse standards-based grading with your state standards or standardization, although they probably will be related.

Step one in my adventure into standards-based grading: Building Topic Scales
Ok, maybe this isn't the first, first, first step, but it's the first meaningful one and most of us start here.

Even if you are not planning to implement standards-based grading, this is a valuable activity to do. You will become a better teacher just by going through this process.

For a quick review on why I chose to use topic scales, check out an older post.

What you're doing is setting levels of proficiency for each standard you hope to assess in your classroom. You are deciding on a target. My scoring system looks like this:
0 = No evidence of learning
1 = Can do most of the simple stuff with help
2 = Can do all of the simple stuff
3 = Can do all of the simple stuff and all of the complex stuff
4 = Can go beyond what was directly taught in class

I also have half points, like 2.5.

Group the standards into topics. This is a bit of vi vs. emacs. I'm not sure there's a right answer here.¹ The math peeps generally opt for a straight skills list. I've noted my disagreement with that and am sticking to it. The downside of topics is that it is slightly more difficult for a student to tell what he/she needs to remediate. There's an extra step that some kids have trouble negotiating. On the plus side, topics facilitate connections between skills. Most of us have a good sense about what to group into a topic. In content classes like science or history, your existing units will fit pretty well. There are interesting ways to group science and history beyond Energy and Ancient Egypt but I won't get into that now.  If you're ELA you can have Persuasive Writing, Genre or the have a topic for each of the Writing Traits if you use those. Math might have Order of Operations, Percent and Decimals, Fractions, Probability. Each topic should have one or two really big ideas. I think somewhere between 10 and 20 topics per year is manageable. Too few and they start to lack coherence. Too many become unmanageable.

Start with the 3. What are the complex ideas/concepts/skills that I want my students to know or be able to do? Ideally, this is a process you're sharing with your department. Probably, you're flying solo here as I was so I drew from a variety of sources. I started with my CA standards, framework, and looked at released test questions. I've recently found performance level descriptors (pdf) that I used to adjust my standards a little. I also took a look at the standards for the high school science classes and used my own experiences with college and laboratory science to some degree. Be specific about what you want your kids to learn. "Learn fractions" isn't that helpful. "Add/subtract fractions with two-digit denominators" is much better.

Your primary concern here is twofold: What do my students need to learn? and At what depth do they need to learn it? If you're looking for help on depth, Bloom's and Marzano/Kendall's taxonomies were fairly helpful, as was Webb's Depth of Knowledge (pdf).

Backwards plan the 2. At this point you should have somewhere between ten and twenty topics. In my Forces in Fluids scale, my 3.0 included determining the density of an object, using Archimedes principle, and understanding how to manipulate the buoyancy of an object. You'll need to break these standards into a learning progression. Ask yourself, What do my students need to know to be able to do this stuff? Well they needed some basic vocab: density, buoyant force, pressure. They needed some measuring skills and a couple of SI units. They needed to understand that things sunk if they were more dense than the liquid they were immersed in.  That became my 2.0. You are trying to build a natural progression up your scale. Notice that the things in the 2.0 are needed to learn the more complex items. I'm not bombarding them with 50 different vocabulary words. For each topic I try to limit the vocabulary and memorization to the specific items that they'll need to understand the concept.


Set your 4. The 4.0 exists for one reason. If you directly taught it in class, it's memorization. It doesn't matter if it's a name, date, or the causes of the Civil War. Whether it's a simple idea or a complex idea, it's still our little birdies upchucking facts onto their papers. So your question for 4.0 is, Where do I want them to go with this? In my classroom, this usually takes a few different forms.  Sometimes I ask them to connect different information together in news ways, such as the relationship between metals having free valence electrons and also being conductive. I teach them the facts necessary to make that connection. I also try to teach them how to connect information.² It's up to them to close the final gap. Other times I'll look at my high school standards and set the next natural step as the 4.0. In this case, I will spend a day or so directly teaching a few things. I violate my own don't-directly-teach-it rule here. Specifically, I wanted my kids to be exposed to balancing equations. You don't really need to know how to do it in 8th grade but our high school chem teachers spend forever on it. I didn't think it would be fair to require all students to learn how to balance equations so I compromised by setting it at the 4.0. I think of it as bonus knowledge. My other most common 4.0 style of question is just adding an extra half step onto an existing 3.0 level problem. The 8th grade standards for determining speed are very basic "You go 10 miles in two hours, what's your average speed?" problems. 4.0 might include finding an average of multiple trips or determining the speed you'll need to travel to obtain a certain average (I go 10 miles in two hours, if I want to maintain an average speed of 8 mph....). The higher levels on all of those taxonomies are helpful here as well.

Quick check: You have 10-20 topics. If you were to read the topic scales starting at the 2.0 and working your way up, it follows a natural and logical learning progression.  Your standards are grouped in a way that also makes sense. They're related and when students learn one standard in a topic, it helps them with the other standards at the same time.

Write out your assessments. I wrote out a pretty representative sample of all the questions I planned to put on my tests. It will be a back and forth process of writing problems based on your standards as well as revising your standards because you realize that's not quite what you wanted your students to do. I'm a huge-mungous believer that if you plan to assess it, you need to teach it. Look at your questions. If it states, "Analyze the effects of NAFTA on the economy and labor conditions in Mexico," then not only do you need to teach them about NAFTA, but you need to teach them how to analyze. Adjust your standards accordingly.

That's basically it. Look at your standards. Decide what proficient "looks like." Backwards plan how to get there. It took me a solid weekend to get a good rough draft. I used my first topic and immediately realized I had to revise things. Don't make 150 copies of all of your topic lists and laminate posters until you've got a few topics under your belt.

Science example: I have a topic called Atoms. The big idea is that all matter is made of atoms and that the existence of atoms can explain macroscopic phenomena. In the end, I want them to understand how temperature, pressure, and volume are related in a gas and the relationship between atomic motion, energy, and the state of matter. This is my 3. What do they need to know in order to get there? Well they've got to understand some basic vocabulary: matter, atom, solid, liquid, gas, pressure, temperature, volume. They've got to be able to differentiate between matter and non-matter. Since the 3.0 is centered around physical changes, they'll also need to know what properties of matter can and cannot be changed. They need to know the molecular motion of different states of matter. Those become my 2. For my 4, I ask them to explain certain phenomena that we haven't explicitly address in class. So we've talked about evaporation, but not condensation. We've learned previously why things float or sink, so I ask them to explain why a hot air balloon floats.

Non-science examples: Warning, my knowledge in these other areas kinda sucks so please don't flame me for getting some specific facts or terminology wrong.

Math example: For Kate Nowak's regression unit, Modeling or Regression would be a natural topic. The big idea might be that data can be modeled and then extrapolated using mathematical formulas. Kate might want her kids to be able to look at a graph and write a formula modeling the data. This is her 3.0. Going backwards from there, what do the kids need to be able to know in order to do that? Well it looks like they'll need to be able to identify different types of models. They need to be able to set up a graph. They need to be able to qualitatively explain what's going in the graph. They need to interpret different variables and constants. What would her 4.0 be? In our school we're asked to write across content areas so I naturally default to writing at this point. Perhaps she can give them a messier or incomplete data set and have them model it and justify why they chose that specific method. She may ask her students to create general rules for when to use each regression. She could ask them to identify specific examples of say, exponential growth, that weren't taught in class. Then gather some data and model it. 

Social Studies example: My big idea might be how the US government maintains a system of checks and balances using three branches of government.³ I want them to understand the historical roots for this system and how each system has served to enact change in their own way. What do my students need learn? They'd need to learn the basic functions of the three branches. They'd need to learn a few historical examples. 4.0? Students might be asked to contrast our system with a bicameral system. For higher level students you might use one of your historical examples and have them imagine how it might have played out differently with a different system of government.

Sticking points: The stuff that you'll struggle with.

You have a giant steaming pile of standards. You're going to end up with a lot of stuff. You MUST narrow the curriculum. If you are morally against omitting or skimming certain standards, leave a comment and I'll try to convince you. I'm going to assume we all understand the necessity. I usually don't omit standards entirely. However, I definitely underteach some. We have a fair amount of test score pressure on us. Instead of deleting, a few standards have been relegated to mere vocabulary and one or two just get mentioned and not assessed. There will always be a few one-offs because of testing but try to minimize them as much as possible.

Grain size. This is probably the big one for most of us. It's a struggle between being specific enough that you and the student can make a decision based on looking at assessment results and the simple management aspects of having to assess and track dozens of standards. This has been a constant tweaking process. So far, the most helpful thing has been analyzing student errors. Problem areas and hard to break misconceptions should be broken down a little bit more. When solving the mathematical problems, like rate, students would have trouble with using the correct units. I separated that out into its own standard. On the other hand, I lumped the characteristics of the planets into a single standard. Students came in with a lot of background knowledge so I didn't feel the need to have a separate standard for each planet.

The 4.0. Getting it "just right" ends up being tough. For some topics, it comes naturally to me. Others I struggle with.This is directly related to my content knowledge of that particular topic. My most common problem is getting overly excited with something cool and just blowing my students away. This isn't the time to have your students try to explain the double-slit experiment. History teachers really like counterfactual thinking questions. Those are super hard and you really have to be willing to devote a massive amount of time to teaching students how to do that. When you think 4.0, think half-a-step-up rather than a full level. This is definitely a context-dependent decision. You've got to know your kids and know what you teach and value.

Over reliance on verbs. The taxonomies are definitely a good guide, however simply adding in a "justify" or "predict" doesn't guarantee deeper conceptual understanding. Focus in on the big ideas and work backwards, don't try to build up the pyramid. I've made that mistake before.

Pro Tip 1: Don't go chronologically. Start with the topic you feel you've got down pat. It's really hard to create a progression when you only have a surface grasp of a certain topic. The physics and chemistry portions were pretty clear for me. The astronomy stuff I'm completely unhappy with even now.

Pro Tip 2: Be explicit with your performance standards. I've got students that can calculate the speed of an object on a test. I've got students that can figure out how fast they can skateboard. I want them to be able to do both. Write it out as two different standards and don't assume that just because they can do one, they can do the other. They can't and you won't know that unless you assess it directly.

Pro Tip 3: These scales aren't just for your tests/grades. They should guide everything you do in class. You should be able to directly link everything you do in class to something on your topic scales.

I hope this has been helpful. I'm working from memory here so I reserve the right to edit this page at anytime. If something is unclear, ask. If you've got a better method or something to add, I'd love to hear it.

Last second add: Scooped! Kate Nowak just blogged her checklist.



1: Unlike with vi vs. emacs. In that case, the correct answer is vi. I would like emacs if only it had a decent word processor. Ha!
2: I teach it pretty poorly, mainly with concept maps. If you've got a good strategy for helping students link disparate information together, let me know.
3: Wow. I'm really struggling with social studies. I guess I need to watch more History Channel.

Non-negotiable

Shawn had an interesting post he titled Can O'Worm: Eat It, Salary Schedule. I disagree with nearly everything he posted (scroll down to the comments), but I appreciate his willingness to go out on a limb and say something that might not be entirely popular.

In that spirit, here's my can o' worms:

Teacher autonomy is vastly overrated.

Before you go crazy, I will say that we should have more autonomy in many areas. Our schedule, our textbooks, our budget, our ability to choose between equally good methods of teaching something.¹

Here are three areas where we should NOT have autonomy.
1. Grades. You knew this was coming, seeing as how I'm assessment obsessed. It is 100% unequivocally NOT alright for an A in my class to mean something different than the teacher next door. Creating a clear definition of what a grade truly means is the essence of standards-based grading.

2. Pacing. Every same subject teacher should be teaching the same things at the same time. I'm not saying every day is exactly the same in every class, but on a topic/unit level, we should all be on the same pace. From a pedagogical standpoint, it makes collaboration and data analysis nearly impossible if I'm teaching about ancient Egypt while my neighbor is teaching about Greece. More importantly, there's a culture issue. Significant learning occurs...wait for it...outside of our classrooms. Maybe, just maybe, my students learn more in the hallways than at their desks. Having the same pacing creates a shared culture. Our kids should be able to talk about what they're learning, even incidentally. Even if it's to complain how bad Mr. Buell is at teaching.

Here's a not-entirely-imagined conversation:
Student 1: Mr. Buell SUX at teaching. I don't know get anything in his class. He just stood there and kept talking and talking about protons. WTF is a proton?
Student 2: Ms. W is kinda cool. A proton is that positive thingie in the nucleus......

There are strong teachers and there are weak teachers. Sometimes, our kids need to learn from someone else. Even if your school is entirely populated with master teachers, your style of teaching isn't going to be the perfect fit for every kid.

At our school, we have AVID and various remedial support classes. From everything those teachers tell me, it makes a HUGE difference when all the teachers of a subject are on the same track. Those of you who teach those classes, I'm sure you've run into the juggling act that is trying to help four different sets of kids learn four different things because their teachers can't agree on what to teach first.

3. Discipline. I'm known as a soft teacher. I break a lot of rules and let a lot of rules slide. My basic rule is that if you're learning and not hurting someone else's learning, pretty much anything goes. I've always been a "I should decide what is acceptable in my class" kind of teacher. You know what? I'm wrong. If I'm a hormonal 14-year old boy, I shouldn't have to learn 6 different sets of rules. I shouldn't have to learn that in my first period a raised hand means I just need to be quiet while in second period it means it means I need to turn and face the teacher and put my hands on my desk. My third period teacher marks us tardy if we're not in class while my fourth period requires that I have a paper and pencil out on my desk by the time the bell rings. It's ridiculous. If your school (like mine) has a lot of discipline problems, take a look at the varying levels of classroom norms a student has to negotiate throughout the day. Like with pacing, shared discipline creates a shared culture.

For me, number 3 is the hardest but I think most teachers generally have a problem with 1 and 2. I hate our rules. I feel like I'm fighting a flawed system by flouting the rules. I'm also undermining my fellow teachers and setting up my students to fail.²

If you're an English teacher and you're looking for a theme here, it's the idea of building a shared culture. If what I'm doing violates that shared culture, I shouldn't be allowed to do it. On the other hand, I should be allowed a whole bunch of autonomy within those confines.

As always, let me know what I'm wrong about and let me know what I forgot.

1: Notice the qualifier "equally good." If there's a better way and then there's my way, I don't get to choose my way.
2: That doesn't mean I won't work to change the rules. But until I do, we should all be following them.

My first assessment failure

...and certainly not my last.

Before I knew anything about standards-based grading, I only knew that the way that we graded stunk. So I developed my own system. It turns out there's a reason teachers who have been in the classroom for only two years usually don't develop their own systems. My system sucked.¹ I wanted to share it though to let you know what I learned from it and why the new system I use is much better.
I called it the Learning Matrix. Here's what a sample sheet looked like:
Learning Matrix

I based it off of two principals that still show up in my work with SBG:

1. Separate learning goals instead of one lump score.
2. Start with the end in mind and create a path to get there.

I decided to base it off of Bloom's taxonomy. Why Bloom's? I don't really remember at this point, other than we're always getting it thrown at us and maybe I figured I should use it. I recall at that time I used Costa's questions to create my question bank so I'm not sure why I didn't use that. Perhaps it just had a whole bunch of levels which I thought was cool. I don't know.

It worked somewhat similar to the skill checklist that many math bloggers use. The students had a set of questions or tasks that were specific to one of the grid boxes. Once or twice a week students would pull out the matrix and look at their grids. They'd select a box and find the accompanying assessment and work on it. The lower levels would involve definitions while the higher levels might be involve the completion of a mini-lab. After completion they'd bring it up to me and I'd give them a quick oral assessment. If they could satisfy my grilling I'd stamp it and they'd color in the box. There was also a pocket chart with the grid on it. Students would move an index card with their name on it to whatever they were currently working on. The idea was that a student could look for someone working on a higher level for help.

So what went wrong?

Ultimately the problem with this method was its basis on Bloom's. Bloom's is a classification system, not a learning progression. A learning progression lays out the steps needed to get to a learning goal. The current system I use creates a learning progression. How do I know this? Because I designed it that way. I started with my goal (in this case, the 3.0 concepts) and worked my way backwards. Let's say my 3.0 goal is for students to be able to calculate the average speed of an object. What do I need to know in order to do this? Some basic vocabulary, an idea of what motion is, some SI units, and how to measure stuff. That's the 2.0.²

I was waylaid by two other big things.

Number 1: I didn't teach the verbs. This wasn't necessarily a problem with my system so much as it was a problem with my teaching. I have a number of assessment pet peeves. Grading things that should be practice. Extra credit for non-academic stuff. Rubrics that focus on product guidelines instead of understanding. But gun-to-my-head I think I might say that my number one assessment pet peeve is grading something you don't teach. How in the world were my kids supposed to be able to "evaluate" when I didn't teach them how? I expected their mastery at one level to automatically gift them with the knowledge of how to do the next level. Even worse, I'd be surprised that they always got so badly stuck after starting a new level.

Now? I directly teach any higher order thinking that I will assess. If you're expected to "Compare/contrast speed, velocity, and acceleration," I'm teaching my kids how to make a comparison matrix. If I expect my kids to evaluate something, I'm teaching them how to create decision making criteria. Now, when a compare or evaluation question shows up on a test, they've got a set of skills to draw on. We need to directly teach thinking skills along with content knowledge. I used to think they just didn't get the content. They got the content. The thinking skill just got in the way.

Number 2: False equality. Not all topics need to be learned to equal depth. For the rest of their science lives, do all students need to understand the atomic theory? You bet. How about Archimedes Principle? Umm...not as important. The state of California dictates what I need to teach my students each year. That doesn't mean they all deserve equal time. In the new system, the atomic theory gets its own topic while Archimedes Principle falls under the topic Forces in Fluids. Some things (I'm looking at you polymers!) get demoted all the way down to just being vocabulary.


What's the take home message?
More than anything, that experience made me realize that this is going to be a long process. I felt like some sort of mad genius when I was designing the system. Two years into a formalized standards-based system, I'm still tweaking and learning. If you've been implementing a standards-based system you know what I'm talking about. If you're planning to, you'll know what to expect.

PS -  Don't forget to explicitly teach higher order thinking skills.


1: It still didn't suck as much as the way we regularly grade.
2: I could definitely be wrong in the setup of the learning progression. Perhaps learning doesn't occur in that order. However, my scales are fluid from year to year and I take solace in the fact that the research on learning progressions isn't all that solid.

Almost Formative

Karl posted about his assessment plan going back into the classroom. Matt responded. I'm continuing the conversation here. Like anyone else I ever link to, I like their sites more.¹ Go there first. Read everything. Come back.

I feel like I read this quote on Matt's site but I couldn't find it when I looked through the archives. Essentially the gist was that all conversations with teachers inevitably end up focused on the gradebook.

So here's the nit that I need to pick:

Allowing your kids to retake their tests does not automatically make it a formative assessment.

I'm not accusing Karl or Matt of making this assumption and both of them definitely understand the distinction. Along with not grading homework, this seems to be the thing that educators get hung up on as the way to call something formative. There is no single method that will make the class/teaching/assessments formative. Ungraded homework isn't formative. Retaking tests isn't formative. Exit slips, response systems, quickwrites, minute papers aren't formative. Standards-based grading isn't formative. Your entire class could consist of ungraded, infinitely retaken tests and it wouldn't be automatically formative. Formative assessment is the entire culture of your class.

What does that mean?

Your students should, at all times, be able to answer these essential questions.²

1. Where am I going?
2. Where am I now?
3. How can I close the gap?

Learning goals are pretty popular and many principals make posting them mandatory. In ELA classes at least, essay exemplars and such are pretty common. I think we could all do a better job (posting learning goals is not the same as communicating them) at this but at least we pay attention to question 1.

Question 2 we also pay some attention to. Where we usually go wrong is by not framing a student's current level in terms of the ultimate learning goal. To steal a simile from a previous post, it's like looking out of the window of your car to see where you are. It's nice to know your location, but really what you want to know is where you are compared to your destination and if you're going in the right direction.

Question 3 is where most of us fail. It's certainly my weakest area.

Typical almost formative teacher statement:

You scored a 75% on Adding and Subtracting Fractions. Learn it and come back after school and I'll replace your score.

 To parallel that statement, here's what the state tells my school:

You scored a 714 on last year's API. Increase it to 800 this year. Come back in May and I'll replace your score.

We know where we're going. We have a sense of where we are in relation to that goal. We just have no frikkin' idea how to get there. Part of the problem is that, like the state of California, we have no idea what is causing the gap. We can make assumptions based on previous experiences, but we're dealing with so many students/schools it's impossible to know the specific cause of every single problem that every single student/school encounters.

The larger, and less forgivable, reason is that it's simply easier to set goals and diagnose. Helping students/schools close the gap is the hard part.  

Telling a student to go learn something, without giving him/her the proper tools to do so, is the hallmark of the almost formative teacher.

How do we solve this?

As I said, this is certainly my weakest point so I'd love to hear what you say. The first step is always awareness. If you find yourself saying, "Go learn this" and the student gives you a confident nod and heads off with purpose, you're probably alright. Often they'll hesitate, stare at me for a second longer than they should, and trudge back to their seat. Those kids have no idea how to go out and learn whatever it is they need to learn.

Two things need to be directly addressed to solve the problem: metacognition and time.

The biggest help for my kids so far has been a double megadose of self-assessing and reflection. I try to directly teach a whole lot of different strategies. I'm not 100% at this, and perhaps it needs to go on my checklist, but I've tried to communicate the different methods we've used to learn different concepts. You know all those super cheesy teaching strategies you learn when you're in your credential program? Give one, get one. Think Pair Share. Jigsaw (my personal hell), Pair Coaching, etc. Those "How did you learn it?" questions we ask them are much more valuable when they can say learned really well when we used reciprocal teaching or rally coaching.³ I would like to get to the point where they have created their own master list of strategies that work best for them and they can just select one and go. Unfortunately, most of my kids are still stuck with "Find a kid who gets it and ask" or "Go look in the book." They'll never close the gap if they never learn how to learn.

The second necessity is time. I understand that at some point we all have to move on and say come after school. However, we also have to devote a substantial amount of class time to gap closing to break out of almost formative purgatory. Telling your kids to come after school usually only helps the kids who are motivated (self or externally). I tell many of my kids to come after school for help. Many can't. Others aren't interested.⁴ We need to schedule time in class. Put it into your curriculum map. I'm not talking about taking an extra day when the class bombed a test. Look at your yearly plan. Every 4-6 weeks insert a week of remediation/acceleration. Let the kids who are behind work on what they've fallen short on. Maybe they picked up a new strategy or maybe seeing the whole picture has made things more clear. Directly teach to targeted groups of kids. Teach them how to teach each other. Challenge the rest of the kids with something that will lead to equal amounts of elation and frustration.

I think it was Chris Lehmann who said that in schools, what we value privilege with time. If you value creating a culture where students are not punished for learning at different rates, you will privilege that with time.

Schedule time to teach them how to think. Schedule time to give them a chance to do so.






1: Liking a site is not equal to agreeing with everything, but they make me think. That's why we're here.
2:I first read it in Atkin, but were made famous by Stiggins. I don't know if Atkin got them from somewhere else.
3. I struggle with getting them to separate the things that helped them learn the best versus the things that they had the most fun with. These are often the same, but not always.
4:If TMAO were still blogging I like to think he'd point to our insistence that anyone who didn't get it the first time needs to come after school as a contributor to the achievement gap.

Stuck in the Middle

In my last post, I wrote about the books by Atul Gawande. I'm going to give an extended quote here, so if you're a representative of Henry Holt and Company, I apologize for trying to get people to read your books:

The hardest question for anyone who takes responsibility for what he or she does is, What if I turn out to be average? If we took all the surgeons at my level of experience, compared our results, and discovered that I am one of the worst, the answer would be easy: I'd turn in my scalpel. But what if I were a B-? Working as I do in a city that's mobbed with surgeons, how could I justify putting patients under the knife? I could tell myself, Someone's got to be average. If the bell curve is a fact, then so is the reality that most doctors are going to be average. There is no shame in being one of them, right?

Except, of course, there is. What is troubling is not just being average but settling for it. Everyone knows that average-ness is, for most of us, our fate. And in certain matters - looks, money, tennis - we would do well to accept this. But in your surgeon, your child's pediatrician, your police department, your local high school? When the stakes are our lives and the lives of our children, we want no one to settle for average. 

If you've ever heard any of the many version of Killing Me Softly With His Song, you know what I felt like reading this passage. I'm not worried about being a bad teacher. I'm worried about staying an average one.

Checklist Manifesto v1.0

A couple of months ago I read all of Atul Gawande's books. I highly recommend them. He's the kind of reflective practitioner I wish I was.

My personal favorite was Better, the section on Cystic Fibrosis had me pacing my room. It hit so close to home when I think about the weaknesses of my school and my teaching.

Just as I finished Checklist Manifesto, a post by the Science Goddess on checklists in education got me thinking about how I could use one.

In Checklist Manifesto, Gawande cites three different reasons for failure: the task is beyond our capacity, ignorance, and ineptitude. The first reason means we just can't do it. For example, telling me to fly by flapping my arms. The second reason means our knowledge is incomplete or missing for some reason. We just don't know how. If you told me to rebuild an engine, I would fail due to ignorance. The third reason is where checklists come in. We fail because we don't do what we know we are supposed to do. In the book, washing hands before interacting with patients is a primary example.

I've been working on a checklist to help me overcome my ineptitude.

Step one: Identify my own failures

I did this fairly informally. A few things I had in mind from the start and then I took 30 seconds at the end of each day to jot down whatever it was I forgot to do. I also did that at the end of the week and at the end of the unit I was working on. During lunch time, I asked a few teachers what were some things they knew they should do but sometimes forgot.

Step two: Prioritize

Turns out I'm pretty inept. I had a big list. I had to get it down to a manageable level. I'm not going through 30 things every day. The first thing I considered was bang for the buck. On my list, I wanted things that I felt would make a difference. I used a combination of research on hand and my gut. If I occasionally forgot to have my students turn in their notebooks on Friday, it wasn't the end of the world. On the other hand, I had to make sure I allowed for an extended writing period at least weekly.
The second thing I considered was my ability to execute the item. Again, I'm trying to solve my ineptitude, not my ignorance. I could have said something like,"Integrate one of the six writing traits into a lesson," However, I'd have to google the six traits to be able to even list them. There's no way I can execute that item. I also understand the awesome power of socratic seminar, but I have no idea how to do that either.

I also think it's unlikely that I'd do something 100% of the time if right now I'm only doing it occasionally. What I want out of this checklist is to move certain behaviors from 80% to 100%. Most doctors washed their hands most of the time. The checklist helped shift that behavior to all of the doctors washing their hands all of the time.


Step three: Build it

Gawande says there are Do-Confirm checklists and Read-Do checklists. Did I want to stop and check or do the checks as I go? I'm not going to carry around a checklist with me and checking items off. I decided to use it as a Do-Confirm checklist. I broke it into three sections: Topic, Weekly, Daily. The way it works is I plan my topic and then stop and confirm that I've included the items in my unit checklist. My weekly planning works the same way. My daily checklist has become more of a Read-Do. In the two weeks I've tried this, I've found that I've had the best success if I leave it by my laptop. Before class I take a quick glance to remind me. I often go to my laptop during class and I can take a peak throughout the day.

Step four: Evaluate and revise

Here's my list so far:
checklistv1.1

It's actually version 1.1. I made a revision already. My first checklist said Check for Understanding in place of Pause. Check for Understanding wasn't quite what I wanted. I wanted to make sure I stopped and quickly assessed how things were going, rather than just checking to make sure my students were understanding what was going on. Certainly Pause and Check for Understanding are related but I felt the latter made me focus too much on whether students were getting what I was doing, rather than whether what I was doing was at all valid. It's the difference between looking out of your window while driving and knowing where you are versus knowing you're on the right path to get to your destination. Pause reminds me to stop somewhere and make sure we're heading in the right direction.

My list also trends away from specific pedagogy and towards the non-academic. I'm good at science teacher stuff. I'm less good at teacher stuff. I've never been Harry Wong or Fred Jones or Rick Smith or whoever else I had to read when I was a new teacher. I can't even remember my own procedures, much less enforce them. I'm not a community builder either. This is one of my glaring weaknesses as a teacher. Being a single-subject teacher at a middle school makes this all the more obvious. Probably 80% of the teachers have a multiple-subject credential and about 50% have taught a self-contained class before. They're really good at the non-content specific stuff. I used to think they were wasting time with get-to-know you stuff and ice breakers. I was wrong.

The items that might need elaboration:
Tell a Story could be anything. It could a historical narrative on the development of the atomic theory but usually it's something like, "Two atoms walk into a bar....." I included this because of the stickiness of stories.
Extended writing or free response refers more to quickwrite type activities rather than lab reports or research papers.
Talk with Partner actually refers to the science teacher next door. Perhaps it shouldn't go on this list but I need reminding to do it. It falls into the category,"Stuff I know I should do but don't always do," so it went on the list.
Set Goals refers to my students setting a weekly academic goal (Learn how to differentiate between an acid and base) and a non-academic goal (Raise my hand to ask a question at least once). It's something new I've tried this year but I've been really bad about remembering it on Monday.
Names is squarely in non-academic territory. In the beginning of the year I'm really good about greeting my kids at the door, shaking hands, talking to them, and all that. By mid-year I'm scrambling to get the labs setup again and they get a good morning. At this point in the year I'm down to head nods. I want to say each kid's name every day. It doesn't have to be right away but by the end of the period, I want to have acknowledged every kid by name.
Closure is something I'm painfully bad at. I'm sure it relates to my procedure weakness but I often get caught by the bell.
Reflection and Feedback are both Friday things. Reflection relates to how well they've done at accomplishing the goals they set on Monday. Feedback is from the students on how the week went and what we should focus on next week.

As of now it's been 3 weeks of checklisting it. Except for closure, I'm pretty close to 100% on everything. It's a short time but I've caught myself a couple of times already. I'm not really sure what to do about closure since it's so dependent on other things going well. Perhaps I should concentrate more on quickening my transitions instead and that will give me the time I need for closure. On the other hand, the goal is still closure so maybe I should just leave it on there to remind me what I need to work on.

I'm looking for feedback on my checklist. I'm sure I'm forgetting a bunch of stuff.  

How can I improve on my checklist format/execution/content? What would you include on your checklist?