Almost Formative

Karl posted about his assessment plan going back into the classroom. Matt responded. I'm continuing the conversation here. Like anyone else I ever link to, I like their sites more.¹ Go there first. Read everything. Come back.

I feel like I read this quote on Matt's site but I couldn't find it when I looked through the archives. Essentially the gist was that all conversations with teachers inevitably end up focused on the gradebook.

So here's the nit that I need to pick:

Allowing your kids to retake their tests does not automatically make it a formative assessment.

I'm not accusing Karl or Matt of making this assumption and both of them definitely understand the distinction. Along with not grading homework, this seems to be the thing that educators get hung up on as the way to call something formative. There is no single method that will make the class/teaching/assessments formative. Ungraded homework isn't formative. Retaking tests isn't formative. Exit slips, response systems, quickwrites, minute papers aren't formative. Standards-based grading isn't formative. Your entire class could consist of ungraded, infinitely retaken tests and it wouldn't be automatically formative. Formative assessment is the entire culture of your class.

What does that mean?

Your students should, at all times, be able to answer these essential questions.²

1. Where am I going?
2. Where am I now?
3. How can I close the gap?

Learning goals are pretty popular and many principals make posting them mandatory. In ELA classes at least, essay exemplars and such are pretty common. I think we could all do a better job (posting learning goals is not the same as communicating them) at this but at least we pay attention to question 1.

Question 2 we also pay some attention to. Where we usually go wrong is by not framing a student's current level in terms of the ultimate learning goal. To steal a simile from a previous post, it's like looking out of the window of your car to see where you are. It's nice to know your location, but really what you want to know is where you are compared to your destination and if you're going in the right direction.

Question 3 is where most of us fail. It's certainly my weakest area.

Typical almost formative teacher statement:

You scored a 75% on Adding and Subtracting Fractions. Learn it and come back after school and I'll replace your score.

 To parallel that statement, here's what the state tells my school:

You scored a 714 on last year's API. Increase it to 800 this year. Come back in May and I'll replace your score.

We know where we're going. We have a sense of where we are in relation to that goal. We just have no frikkin' idea how to get there. Part of the problem is that, like the state of California, we have no idea what is causing the gap. We can make assumptions based on previous experiences, but we're dealing with so many students/schools it's impossible to know the specific cause of every single problem that every single student/school encounters.

The larger, and less forgivable, reason is that it's simply easier to set goals and diagnose. Helping students/schools close the gap is the hard part.  

Telling a student to go learn something, without giving him/her the proper tools to do so, is the hallmark of the almost formative teacher.

How do we solve this?

As I said, this is certainly my weakest point so I'd love to hear what you say. The first step is always awareness. If you find yourself saying, "Go learn this" and the student gives you a confident nod and heads off with purpose, you're probably alright. Often they'll hesitate, stare at me for a second longer than they should, and trudge back to their seat. Those kids have no idea how to go out and learn whatever it is they need to learn.

Two things need to be directly addressed to solve the problem: metacognition and time.

The biggest help for my kids so far has been a double megadose of self-assessing and reflection. I try to directly teach a whole lot of different strategies. I'm not 100% at this, and perhaps it needs to go on my checklist, but I've tried to communicate the different methods we've used to learn different concepts. You know all those super cheesy teaching strategies you learn when you're in your credential program? Give one, get one. Think Pair Share. Jigsaw (my personal hell), Pair Coaching, etc. Those "How did you learn it?" questions we ask them are much more valuable when they can say learned really well when we used reciprocal teaching or rally coaching.³ I would like to get to the point where they have created their own master list of strategies that work best for them and they can just select one and go. Unfortunately, most of my kids are still stuck with "Find a kid who gets it and ask" or "Go look in the book." They'll never close the gap if they never learn how to learn.

The second necessity is time. I understand that at some point we all have to move on and say come after school. However, we also have to devote a substantial amount of class time to gap closing to break out of almost formative purgatory. Telling your kids to come after school usually only helps the kids who are motivated (self or externally). I tell many of my kids to come after school for help. Many can't. Others aren't interested.⁴ We need to schedule time in class. Put it into your curriculum map. I'm not talking about taking an extra day when the class bombed a test. Look at your yearly plan. Every 4-6 weeks insert a week of remediation/acceleration. Let the kids who are behind work on what they've fallen short on. Maybe they picked up a new strategy or maybe seeing the whole picture has made things more clear. Directly teach to targeted groups of kids. Teach them how to teach each other. Challenge the rest of the kids with something that will lead to equal amounts of elation and frustration.

I think it was Chris Lehmann who said that in schools, what we value privilege with time. If you value creating a culture where students are not punished for learning at different rates, you will privilege that with time.

Schedule time to teach them how to think. Schedule time to give them a chance to do so.






1: Liking a site is not equal to agreeing with everything, but they make me think. That's why we're here.
2:I first read it in Atkin, but were made famous by Stiggins. I don't know if Atkin got them from somewhere else.
3. I struggle with getting them to separate the things that helped them learn the best versus the things that they had the most fun with. These are often the same, but not always.
4:If TMAO were still blogging I like to think he'd point to our insistence that anyone who didn't get it the first time needs to come after school as a contributor to the achievement gap.

Stuck in the Middle

In my last post, I wrote about the books by Atul Gawande. I'm going to give an extended quote here, so if you're a representative of Henry Holt and Company, I apologize for trying to get people to read your books:

The hardest question for anyone who takes responsibility for what he or she does is, What if I turn out to be average? If we took all the surgeons at my level of experience, compared our results, and discovered that I am one of the worst, the answer would be easy: I'd turn in my scalpel. But what if I were a B-? Working as I do in a city that's mobbed with surgeons, how could I justify putting patients under the knife? I could tell myself, Someone's got to be average. If the bell curve is a fact, then so is the reality that most doctors are going to be average. There is no shame in being one of them, right?

Except, of course, there is. What is troubling is not just being average but settling for it. Everyone knows that average-ness is, for most of us, our fate. And in certain matters - looks, money, tennis - we would do well to accept this. But in your surgeon, your child's pediatrician, your police department, your local high school? When the stakes are our lives and the lives of our children, we want no one to settle for average. 

If you've ever heard any of the many version of Killing Me Softly With His Song, you know what I felt like reading this passage. I'm not worried about being a bad teacher. I'm worried about staying an average one.

Checklist Manifesto v1.0

A couple of months ago I read all of Atul Gawande's books. I highly recommend them. He's the kind of reflective practitioner I wish I was.

My personal favorite was Better, the section on Cystic Fibrosis had me pacing my room. It hit so close to home when I think about the weaknesses of my school and my teaching.

Just as I finished Checklist Manifesto, a post by the Science Goddess on checklists in education got me thinking about how I could use one.

In Checklist Manifesto, Gawande cites three different reasons for failure: the task is beyond our capacity, ignorance, and ineptitude. The first reason means we just can't do it. For example, telling me to fly by flapping my arms. The second reason means our knowledge is incomplete or missing for some reason. We just don't know how. If you told me to rebuild an engine, I would fail due to ignorance. The third reason is where checklists come in. We fail because we don't do what we know we are supposed to do. In the book, washing hands before interacting with patients is a primary example.

I've been working on a checklist to help me overcome my ineptitude.

Step one: Identify my own failures

I did this fairly informally. A few things I had in mind from the start and then I took 30 seconds at the end of each day to jot down whatever it was I forgot to do. I also did that at the end of the week and at the end of the unit I was working on. During lunch time, I asked a few teachers what were some things they knew they should do but sometimes forgot.

Step two: Prioritize

Turns out I'm pretty inept. I had a big list. I had to get it down to a manageable level. I'm not going through 30 things every day. The first thing I considered was bang for the buck. On my list, I wanted things that I felt would make a difference. I used a combination of research on hand and my gut. If I occasionally forgot to have my students turn in their notebooks on Friday, it wasn't the end of the world. On the other hand, I had to make sure I allowed for an extended writing period at least weekly.
The second thing I considered was my ability to execute the item. Again, I'm trying to solve my ineptitude, not my ignorance. I could have said something like,"Integrate one of the six writing traits into a lesson," However, I'd have to google the six traits to be able to even list them. There's no way I can execute that item. I also understand the awesome power of socratic seminar, but I have no idea how to do that either.

I also think it's unlikely that I'd do something 100% of the time if right now I'm only doing it occasionally. What I want out of this checklist is to move certain behaviors from 80% to 100%. Most doctors washed their hands most of the time. The checklist helped shift that behavior to all of the doctors washing their hands all of the time.


Step three: Build it

Gawande says there are Do-Confirm checklists and Read-Do checklists. Did I want to stop and check or do the checks as I go? I'm not going to carry around a checklist with me and checking items off. I decided to use it as a Do-Confirm checklist. I broke it into three sections: Topic, Weekly, Daily. The way it works is I plan my topic and then stop and confirm that I've included the items in my unit checklist. My weekly planning works the same way. My daily checklist has become more of a Read-Do. In the two weeks I've tried this, I've found that I've had the best success if I leave it by my laptop. Before class I take a quick glance to remind me. I often go to my laptop during class and I can take a peak throughout the day.

Step four: Evaluate and revise

Here's my list so far:
checklistv1.1

It's actually version 1.1. I made a revision already. My first checklist said Check for Understanding in place of Pause. Check for Understanding wasn't quite what I wanted. I wanted to make sure I stopped and quickly assessed how things were going, rather than just checking to make sure my students were understanding what was going on. Certainly Pause and Check for Understanding are related but I felt the latter made me focus too much on whether students were getting what I was doing, rather than whether what I was doing was at all valid. It's the difference between looking out of your window while driving and knowing where you are versus knowing you're on the right path to get to your destination. Pause reminds me to stop somewhere and make sure we're heading in the right direction.

My list also trends away from specific pedagogy and towards the non-academic. I'm good at science teacher stuff. I'm less good at teacher stuff. I've never been Harry Wong or Fred Jones or Rick Smith or whoever else I had to read when I was a new teacher. I can't even remember my own procedures, much less enforce them. I'm not a community builder either. This is one of my glaring weaknesses as a teacher. Being a single-subject teacher at a middle school makes this all the more obvious. Probably 80% of the teachers have a multiple-subject credential and about 50% have taught a self-contained class before. They're really good at the non-content specific stuff. I used to think they were wasting time with get-to-know you stuff and ice breakers. I was wrong.

The items that might need elaboration:
Tell a Story could be anything. It could a historical narrative on the development of the atomic theory but usually it's something like, "Two atoms walk into a bar....." I included this because of the stickiness of stories.
Extended writing or free response refers more to quickwrite type activities rather than lab reports or research papers.
Talk with Partner actually refers to the science teacher next door. Perhaps it shouldn't go on this list but I need reminding to do it. It falls into the category,"Stuff I know I should do but don't always do," so it went on the list.
Set Goals refers to my students setting a weekly academic goal (Learn how to differentiate between an acid and base) and a non-academic goal (Raise my hand to ask a question at least once). It's something new I've tried this year but I've been really bad about remembering it on Monday.
Names is squarely in non-academic territory. In the beginning of the year I'm really good about greeting my kids at the door, shaking hands, talking to them, and all that. By mid-year I'm scrambling to get the labs setup again and they get a good morning. At this point in the year I'm down to head nods. I want to say each kid's name every day. It doesn't have to be right away but by the end of the period, I want to have acknowledged every kid by name.
Closure is something I'm painfully bad at. I'm sure it relates to my procedure weakness but I often get caught by the bell.
Reflection and Feedback are both Friday things. Reflection relates to how well they've done at accomplishing the goals they set on Monday. Feedback is from the students on how the week went and what we should focus on next week.

As of now it's been 3 weeks of checklisting it. Except for closure, I'm pretty close to 100% on everything. It's a short time but I've caught myself a couple of times already. I'm not really sure what to do about closure since it's so dependent on other things going well. Perhaps I should concentrate more on quickening my transitions instead and that will give me the time I need for closure. On the other hand, the goal is still closure so maybe I should just leave it on there to remind me what I need to work on.

I'm looking for feedback on my checklist. I'm sure I'm forgetting a bunch of stuff.  

How can I improve on my checklist format/execution/content? What would you include on your checklist?

SBG by David Cox

Questions?: The Process

This is a great example of an assessment cycle by David Cox, a math teacher in California. He's also planning to address my primary reservation about skill lists by developing projects that incorporate a group of skills.

Sorry for the lack of posts. I went on my school visits last week and I keep writing and trashing posts on it. It's hard to process all the great things I saw.

Professional judgment pt. 2

I was reading the last post and realized that if I were reading it I'd think, "Ok...so if he doesn't want me to just enter in grades, what do I do?"

We just had trimester grades due.

Me three years back:
Picture a giant stack of papers on the right. Today is the first day I'm looking at these papers.¹  I have my laptop open and three red pens. I grab a paper on the right. Check, check, check with my red pen. Enter something in the computer. It looks like a heist movie where they loop the surveillance feed. This takes hours. When I'm done with a class I look up and see what the final grades for my kids ended up being. I register surprise at a few of the grades. I then switch my class list to the next period and start over.

Me now:
Stack of papers still. Only this time they're spread out on the desk. I have a few recent tests out. Some labs. A few free responses and quick writes. I have some scribbled out notes from conversations I've had. A portfolio containing tracking sheets and self-selected work samples are in a folder. I've seen all or nearly all of the papers on my desk before. All of these belong to a single student and all of these are valuable data points. I decide if they have progressed (or regressed) from their last score and move on to the next kid.

The numbers still matter. The numbers inform your judgment. When we remove them from their context, they lose their utility.


1: I shudder to think how many missed learning opportunities I've accumulated over the years by amassing stacks of ungraded papers.

Safety in numbers

Once a month I meet with a small group of teachers at my school to talk about assessment issues. Today I shared a post by Jerrid Kruse titled Grading: the myth of objectivity. I definitely recommend adding him to your Reader if you haven't already. Here's a relevant snippet:

I suggest we admit that our assessments are merely a judgement. Our judgement of students’ understanding and effort as measured by OUR measuring sticks not THE measuring sticks. I believe that once we admit the subjective nature of assessment, we will be free to work towards more authentic and hopefully accurate assessment means.

This eventually led us to a discussion on one of the essential elements of standards-based grading: professional judgment.

We tend to hide behind the shield of numbers. We enter in grades and weights and let our grading software do the rest. If a parent comes in to complain about a student's grade, we point to our gradebook and say, "They got 60 out of 100 on the last test and their overall grade is a 72%." Nobody can argue with the numbers. In fact, we're letting the numbers do the arguing for us.

That's the problem. The numbers protect us and give us authority. One of the strengths of SBG is that I'm required to defend every one of my grades. I can't hide behind the mean or median. I have to own my assessments and own every score that goes into my gradebook.

I am constantly re-evaluating my methods of assessment. I have to take into account the student's total body of work. I have to be completely clear on what it means to have mastered a standard. 

I understand the hesitation in switching to a SB system. It's scary to be so exposed. I gather and record information obsessively, at least in part because I live in fear of the day a parent kicks down my door, demands to know why his daughter got a C, and I can't defend my grade.

Professional judgment is a key pillar of SBG and perhaps the scariest. However, I firmly believe that true assessment reform cannot take place unless we as teachers are willing to take responsibility for the grade we record in our books.

Translating to a letter grade

I apologize for not posting anything assessment related in awhile even though this is ostensibly an assessment blog. A post by Edumacation snapped me back into things.

As you know, I separate my different standards into topics. At the end of the trimester I usually end up with 4 to 6 topics. These need to be combined into a single grade for our traditional A-F report card. At the end of the trimester I might have scores that look like this:

Motion	Graphing	Forces	Forces in Fluids
2.5	3.0	4.0	2.0

In my ideal world I could leave those as is. Throw some snazzy bullet graphs and charts onto the report card and send it on home.

I use a method called conjunctive scoring that I learned about in a Marzano book. The basic idea behind conjunctive scoring is that good scores can't make up for bad ones. No averages. This is how my scores are currently translated:

Grade	Lowest score	At least one
A	3.0	4.0
B	2.5	3
C	2.0
D	1.5
F	less than 1.5

So in my above example the student's final grade would be C because their lowest score is a 2.0.

Motion	Graphing	Forces	Forces in Fluids
2.5	3.0	2.5	3.0

This student would finish with a B because her lowest score is a 2.5 and she has at least one 3.

Against:
Clearly this is very different. There are no averages so it takes awhile for students to get it. You also really need to set aside time to circle back and allow students to go back and learn something they missed. This also requires a certain amount of balance in your topics. Obviously a killer topic can destroy grades. I would argue that some balance in the topics is something to strive for but there are ways to remedy this. The first year I tried SBG, the Motion topic killed kids because of all the math and because I just threw too much stuff into it. I tried to remedy this by separating out Graphing into a separate topic. That worked to chunk it a little more and helped focus my instruction.

For:
I chose this method precisely because high scores wouldn't make up for lower ones. One of the problems with averaging is that you can accumulate enough points that the rest of the course is irrelevant.We've all had students who realize they're so far into A territory that they can coast the last few weeks of school. I've (hopefully) distilled my many, many standards into a few big ideas. In my view, it's ALL important. I like that a student can have all 4s and one 2 and she'll need to work those last few weeks on that one topic.

If the ultimate purpose of grades is to communicate, then an A should be clearly defined. If an A means a student has mastered or surpassed the standards set for the class, it follows that they should master or surpass every single one.

Side bonuses:
We're in our last week of school before the trimester ends and I always notice it's very focusing for a student to take a look at their concept scores, zero in on the lowest, and start working away at it. It's a lot more nebulous to say, "You need 8% more for a C."

There's also this interesting psychological component. If you look at the first example, she only needs to bring up two scores and she can immediately move from a C to an A. Mentally, students feel this is easier to obtain than needing 14% more or 150 more points to move up two grades. I'm sure it goes back to topic scores being more focusing than percentages.

How did I arrive at that scale?
It came through negotiation. My scale was slightly tougher last year. For example, I allowed only one 2.5 for a B. This year, another teacher is trying standards-based grading and we decided on a common scale. I'm reasonably happy with the new scale. There's an interesting passage in the blog I linked to earlier about what an A or B should mean in comparison to meeting standards. Those are the same arguments we had. We thought of it in terms of our state tests. Anyone with an A or B should be advanced or proficient and a C student should have a reasonably good shot.

What about non-academic scores?
I actually don't use them, but I have no problem with them. Again, ideally you'd want to keep those separate. Most schools have a citizenship grade or something similar, if you redefine that you're all set.
If you can't, you'll need to weight them. You would arrive at a score for your academic topics using the conjunctive method. For non-academic scores, the conjunctive method can be used, but in this case I think averaging is the way to go. Going back to the killer topic idea, imagine your non-academic scores are organization, participation, behavior, and work completion. There's really no reason for any of those to kill the entire non-academic score. For your final grade you take your academic score, let's say 90%, and weight it against your 10% non-academic. There are classes that would have much higher non-academic weights (P.E. comes to mind) but for the most part, keep the academic portion high.

The take home is that it's possible to translate your standards-based scores into a traditional letter grade and I'm reasonably happy with the outcome. If translating to a final grade is holding you back, I think conjunctive scoring is a good solution.