Nobody Fails...Almost

For three, almost four, glorious weeks I had no Fs. And then life got in the way. Let me explain.

One of the nice things about the benchmark is that it gave me what Robyn Jackson referred to as a red flag early warning system. I had a simple cutoff and everyone below that cutoff had automatic actions to take. I could definitely have done that with my standard assessments but I'm pretty relaxed about their progress. For better or for worse, I'm not the type of teacher that's constantly pressing kids to maintain a certain pace. The benchmark forced a deadline.

Because of the common assessment plan, I'd been doing lunch time and after school tutorials. The groups were small, 3-8, so I'd still had time to eat. All I'd been missing was the lunchtime socialization with other teachers. To paraphrase Peter Gibbons, I wouldn't say I'd been "missing" it, but that's another story.

The kids were coming until they learned whatever it was they needed to learn. Some of the kids made it out in the first fifteen minutes. Others spent nearly every lunch with me for two weeks.

What I realized midway through the tutorial sessions was that if they had learned it now, I should have been updating their grades. For some reason it didn't really occur to me that learning stuff from the benchmark wasn't any different from learning stuff from my class. It's the same stuff. Ok. I had to sneak in a little bit extra that wasn't on the benchmark, but it was basically the same stuff. I had forgotten one of the fundamental tenets of standards-based grading. It doesn't matter when you get it and it doesn't matter how you get it as long as you get it in the end. Once they made it out of tutorials, I updated their score to passing (2.0 in my case).

What was nice was that I didn't plan to do this so it wasn't like, "Come to tutorial and you'll pass the class." They came because I told threatened politely asked them to come.

And then?

In a comment, Bill Ferriter mentioned the problems with sustainability if planned intervention isn't a schoolwide thing.

Shortly after that comment, a couple of off-campus gang incidents made their way onto campus. The members of the smaller-in-number gang started hanging out in my classroom during brunch and lunch. They act cool, play music off my computer and pretend like they're setting up shop in a teacher's classroom, but truthfully I'm protection.

Lunch tutorials have been canceled for awhile and so tutorials just aren't available anymore for a good amount of kids. Not coincidentally, the kids who just can't come after school (safety reasons, have to babysit siblings, have to work, secretly live out of district) are the kids who are most in danger of failing.

This might not be a specific problem at your school but it's a good example of "something always comes up."

Like Bill said....frustrating. Systems to prevent kids from failing need to be built into the school day. We need to allocate time, resources, and people - from bell to bell - that we can use to catch our kids before they fail. Failing kids is easy.  Guaranteeing learning is not.


Postscript: Today I checked out It's Being Done and How It's Being Done [Amazon links] by Karin Chenoweth from the library. It's really interesting to read about different schools and what they're doing to guarantee an education for all. She's picked a variety of schools that have differing approaches.

Followup Post: Common Formative Assessments

So we did it. Was it worth it?

Recap: Students were given a benchmark. The benchmark was broken into 4 topics—Matter, States of Matter, Atoms, and Chemical and Physical Changes. The following week they spent Monday through Thursday in leveled classes. Monday was Matter, Tuesday was States of Matter, etc. They retested on Friday.


For all the classes, 294 kids, 26% were proficient in the first go and 43% in the second. 35% of the kids were in the lowest range (scoring 35% or less) in the first round and that went down to 18% in the second.

On the downside, only 2/3 of the kids made any growth which means that 1/3 of them were flat or went backwards.

I made 60% proficient and 75% advanced for this benchmark. It was somewhat arbitrary but I figured all students should, at minimum, know the first three questions for each topic.

All students who scored less than 60% have another level of intervention. For students scoring less than 50% on their benchmark they signed up for an after school or lunch tutorial this week based on their lowest topic score. They also signed up for an additional tutorial for any topic that they got 0 or 1 correct.

I'm signing them up on a week by week basis. They can exit out of tutorial sessions with a half-sheet quiz taken sometime before the next session. It's just two or three questions that are representative of the middle level questions. They'll need to get 100% to exit.

Students who scored 50% or 55% were given the option to instead do the relevant section from a workbook. They showed me they had it completed on Friday and took the exit quiz. Maybe 70% of those students took that option.

This week I had Lego League meetings twice and a staff meeting so I met with them twice after school and every day at lunch. It's been fast. 15 minutes at lunch and maybe 20 minutes after school (larger crowd). I'd say there's a 90% attendance rate with a few extra kids that just come with their friends. Talking with the absentees I'm fairly sure they're just honestly forgetting (especially the kids who sign up for lunch but I don't see them normally until after lunch) so I need to start sending notes out 4th period and before school gets out.

I have no idea why they're coming. I've made it pretty clear that the benchmark doesn't directly affect their grade, although if they're behind right now it will get more and more difficult to keep up. That's not to say they're excited about coming, but I think we often underestimate how much our students really do want to learn.

Other observations:

1. The test was noisier than I would have liked but I'm not too worried about it. I don't want to fall into the "must have more data" rabbit hole. It's super low stakes and the worst that happens is a student is placed in the wrong level for a single period or that they need to come for extra tutoring. I can live with that.
   
   
2. Clearly the skill-based lower level stuff had more bang for the buck. The earlier questions, which mainly were vocab and simple skills, made huge growth. The latter questions which required more conceptual understanding barely budged. It makes sense given the time constraints but I'm not quite sure what the best approach is here. Do we acknowledge the limitations and just target high growth skill-based stuff? Or do we take another stab at higher-level concepts and hoping it clicks for whatever reason? My natural inclinations says to always go higher-level but then again, perhaps needing a stronger base is what's holding back the higher-level stuff. I don't know yet.
   
   
3. The periods where we were able to have 3 teachers for 2 classes worth of students made a HUGE difference. Probably obvious, but the difference was big at every level of student for every topic. I'm putting out feelers to non-science teachers to come the next time we do this. I'll help in your class during my prep sometime in exchange for your help that week. Nobody has committed yet but a few have sounded interested.
   
   
4. Greg suggested using 1, 2, 3, 4 for the MC answers instead of A, B, C, D. This was super clutch. Data entry was blazing and it took me less than 30 minutes to enter in the scores for all the kids. Another teacher has this 10-key USB keyboard he uses with his laptop and says he was able to get it all done in less than 20 minutes.
   
   
5. I did pre/post class graphs and they're on the bulletin board. I don't know how I feel about it.
   
   
6. To sort classes, we wrote all our students names on index cards with topic scores down the right side. We just sorted them into piles each day.
   
   
7. Switch up what teacher takes the "high" kids. Students started thinking they were in the dumb class when they saw me every day. There was a fair amount of movement between levels though. I probably saw about 80% of the 8th graders at one point or another. I loved meeting new students. For me, that was one of the highlights.
   
   
8. Our secretaries were not fans because our attendance kept getting messed up. By the end we had a system but the first two days I marked a few kids absent who just wandered off into the wrong group.
   
   
9. We didn't get to collaborate beforehand at all. I was home with a sick kid on Thursday and Friday everyone is outtie as soon as the bell rings. I ended up doing the first day rosters myself over the weekend and emailing them. I'm pretty sure one of the teachers didn't think I was serious about this whole plan so it caught him off guard Monday morning. The first day we had some overlap in what had already been done in other classes. We sorted it out but next time I'm going to have us really talk it out.
   
   
10. Letting them sign up for their own tutoring session has been helpful. They don't mind coming with friends and they've been good about reminding each other.
    
    
11. I was the only one who looked at everyone's data. I'm not sure how to introduce the next stage of having us compare and figure out how to teach certain parts better. Seems touchy. Also, both teachers are probationary so I don't know how much me being their evaluator would be in the back of their heads.
    
    
12. Certain students are kept in different classes on purpose (usually rival gang reasons) so we were worried about mixing them up. We mainly lucked out. We cheated and put one kid in the wrong level. He ended up being suspended for the week anyway but it's a bridge we'll have to cross next time.   

A story I have to share: One of the frequent flyers to the office was 1 of 2 students in the school to score 95%. He first claimed he copied. When it was pointed out he couldn't copy and do better than the person he copied from he claimed to have just guessed. We told him it was nearly impossible for him to guess and get 19 out of 20 right. Finally, he told us to be quiet about it because he didn't want anyone to know he was smart. He got sent to the office later in the day and mentioned his high score to the vice-principal. Then he explained what we were learning about. Then he told her not to tell anyone. I've seen him twice since then and both times he's managed to mention his score.

I'm planning to keep this going for the year. The part-time teacher was really enthusiastic about her results (from 25% to 55% and the boy mentioned above) and so she's on board. The other teacher has been out sick this week so I didn't get a chance to talk with him about it. I'm not sure about his buy-in and, for better or worse, he's not the type to openly fight it.

There are some details we're going to change for next time but it's mainly procedural. We like the general format. The attendance issue was a big problem and we're going to try to keep better track of how we teach certain topics.

I'm also not sure what to do about the pre/post tests. I just gave the same one twice but gave zero feedback on their scores the first time. The dummy/smart kid class thing is something I've really tried to avoid so I was hoping by not letting them know their results they wouldn't be so focused on which class they were in. The SBG monster inside me is yelling to give them immediate feedback so they know what they need to actually focus on during that week. We framed each period as, "You're here because you probably need to work on...." or "You probably missed questions about...." but I'm not sure if that's enough. This is something I'll need to experiment with.

Overall I'd say it was worth the effort. Generally the results were positive and at least one teacher is enthusiastic about it. I tried some feedback forms from the kids but they were less than helpful. Their satisfaction basically tracked their improvement from pre/post. I should have seen that one coming and I should have given the feedback form to them before they got their results back.

If you've made it this far and have any advice or questions - comment, twitter, email.

Massive Post on Common Formative Assessments

This one's long. Even by my wordy standards. Skip to the The Prep and The Intervention if you just want to see what we're doing.

I'd like to say a huge base of research has led me to wanting to start common formative assessments (also called benchmarks or interim assessments), but I think the results are mixed at best. "Works when it's done right" can be applied to almost every ed reform I've ever seen. Really there were two factors.

1. My kids.
Unless you're Steve Poizner, you don't look at East San Jose and think, "Isn't this where they filmed The Wire?"¹ On the other hand, we've got our own issues. Many of my kids don't come in with a lot of background knowledge or outside support.² Thus, their academic success is almost entirely dependent on my abilities as a teacher. If they don't get something at the end of the year, it's because I couldn't teach it in the right way for them to get it. I'm not the best teacher for every student. I don't want them to fail because I wasn't the right fit.

2. My teachers.
It's my 6th year. Here are the teachers I've worked with in 8th grade science:

Year 1: Mr B and Mrs. D
Year 2: Mr. S and long-term subs
Year 3: Mr. S and Mr. F
Year 4: Mr. S and we couldn't fill the spot so we dropped it and loaded our classes.
Year 5: Long-term subs, including a two month period where I taught every single 8th grader on a rotating schedule.
Year 6: Mr. L and a teacher who teaches a single section of 8th sci during my prep.

Stability hasn't been our strong suit. By my second year I was the most experienced teacher, so if you've ever wondered why I spend so much time reading blogs and twitter, now you know. You are all my mentors.

Common assessments are my response to those two factors. I needed to allow my students not to be limited by my teaching abilities and I needed to create some sort of stability in my department. Oh wait. I'm supposed to say something about test scores. Yeah. That too. If my principal is reading this (Hi Diane!) I did it to help us get to an 800 API.

Note: In California we use the terms Advanced, Proficient, Basic, Below Basic, and Far Below Basic. I'm going to use those here because we have a reasonably shared definition of them, not because I agree with the terms.

Grace, Greg, Dan and David all helped me out on this directly or indirectly. Hooray again for twitter/blog mentors. If you've read any of the DuFour books it's obvious we borrowed heavily from the PLC model.

(Edit: I should also mention the book Building a Professional Learning Community at Work by Bill Ferriter and Parry Graham. I let my principal borrow it in the spring and never got it back and I can't remember what ideas I got from it)

The Prep:
We're giving them about every 6 weeks. We met the day before we started this cycle and will do so again at the beginning of the next one. We looked over the test, suggested some changes and agreed on specific scoring criteria for the two short answer questions.

We give the benchmark on a Tuesday. Tuesday and Wednesday afternoon we spend just punching in data into a spreadsheet that Greg sent me and then go home and take a look. On Thursday we meet and formulate a plan.  Share what worked. Share what didn't. Write out some lessons. All that good stuff.

Friday, kids get their results back broken down by standard.

The Intervention:
Monday the kids line up at our doors and are called into different rooms based on results. They get targeted instruction. Right now our current benchmark is on four standards (matter and its properties, states of matter, structure of atoms, chemical vs. physical changes). Only two of us are teaching at any one time so we are playing it by ear based on the results on how we want to separate the kids. As of now we anticipate four groups:

A very small group of kids who blow everything away. These kids we're planning to just set free during the week with whatever project they'd like. We figure some will want to perform their own investigations, some will want to serve as small group tutors, and some will want to create something (digital or something that goes boom or whoosh or boom then whoosh). Before you object, we're not ignoring them. They're going to be elbow deep in awesome.

Almost to just barely proficient kids will get their own teacher.

Basic and Below Basic will get their own teacher.

Far Below Basic will get the third teacher who normally would be on prep. We're all going to be working for free through our prep this week, the part-time teacher is going 4 unpaid periods per day. Yeah, she's a champ. If you're in the Bay Area, offer her a full-time job.

Tuesday, line them up and call names based on the next standard. Wednesday repeat. Thursday repeat. Friday, same benchmark a second time. We keep a graph of the class results on the bulletin board.

Kids who don't score at a proficient level on the second one come after school for small group help until they are caught up.³

Monday, back in our normal classes to start the cycle all over again.


The Discussion:
Pretty much everyone I know hates their benchmarks. I get it. The math teachers and English teachers (district-mandated) hate theirs too.

Here's the key: The three of us have 100% control over our common assessment. Our district and our admins don't touch it.

Benefits
We wrote our assessments. Yours suck because you bought them from Pearson and they don't align to the standards in your classroom or your state. I mean, they say they align to the state standards. But they don't ask the questions in the same way or at the same depth as your state test and there's just no way your class standards are in there so the information you get is pretty meaningless.

We all have hard and soft copies of the assessments we can refer to. How in the name of Zeus's butthole am I supposed to get any meaningful information if I'm just randomly guessing at what actually will be on the benchmark? Crazy idea: How about instead of guessing and surprises, I look at what's being assessed and target that. That way I can see if my methods were actually effective instead of seeing whether I guessed right.

No stakes. I'm not worried teachers will just give out answers or "teach to the test" because we own it. There's no admin looking at our test scores and hinting that we need to step it up because—even though you've got all the English language learners and we stick the trouble kids in your class because you're good with those kids—your test scores are 3% lower than the school average.

Fast turnaround. Our math and ELA department get their benchmark results anywhere from 4 to 6 weeks after taking it. Useless. We're going from test to plan in 48 hours.

Focusing on a few key ideas. Our math department benchmarks sometimes have 1 or 2 questions per standard. You can't get any useful information from that, especially on a multiple choice test. We went with four key standards tested with 20 multiple choice questions. Five for each standard with the level getting increasingly more difficult (i.e. The first question on states of matter is the easiest. The fifth question is the hardest). We added two short answer questions focusing on the two thinking skills we're emphasizing right now. Benchmark #2 will be about 80% current unit and 20% review. We're sticking with 5 questions per standard so some of our benchmarks are slightly longer or shorter depending on how many standards we've been working on for that cycle.

We can respond to conditions on the ground. One of the science teachers had some personal things come up and had to miss some time. He won't be able to teach what's in the benchmark by the date we had scheduled it. We moved the date back. 

Drawbacks
Without admin or district support, we're putting in a whole lot of extra time. No early release or paid subs. We have zero collaborative time built in to our school year. No fancy electronic scoring so it's all hand coded. I might need to just take a sick day and spend it entering in scores and looking at the data.

I'm not sure how sustainable this will be. On the other hand, if we can get all of our students across the stage at the end of the school year, it'll be worth it.

My required plug for standards-based grading.
With department-wide standards-based grading, the benchmark test itself is unnecessary. We would already have the necessary data in our gradebook and would be able to team up as needed. I get excited just thinking about how great it would be to have an ongoing stream of data to compare with other teachers. I don't have that right now. 

However, SBG still lends its own brand of awesome. Take a look at the three steps a student might go through. First, the student has an opportunity to learn in class. Second, in a targeted and leveled remediation. Third, after school. Three different levels. In standards-based grading, it doesn't matter when you learn it, as long as you learn it. Carlos drops major knowledge on the first benchmark and has an A in the class already. Brenda has some issues with states of matter, goes through the week of intervention and learns what she needs to learn. Brenda now has an A too. Mikey sleeps through both step 1 and step 2. Now, he's coming after school. Every week. With me. Special time. Mikey finally learns something because he's sick and tired of seeing my striped polo shirts every day. Mikey knows that it's not acceptable NOT to learn. Mikey also knows that I believe he can learn and that I won't give up on him. And yes, he's got his A. Mikey likey.

Am I making every kid come after school until they've got an A? Nope. No Ds or Fs would make me plenty happy. But you know what? If you've gone from an F to a C, all of a sudden, it's not too far to an A.

In case you're wondering how benchmark fits into the actual gradebook we've decided on two things:

1. The benchmark score will be reported but not computed into the grade. We're just going to manually input it into the comments section of the report card.
2. We all have our own system of grading (don't get me started) but we've agreed on a policy that if you pass the benchmark, you can't fail the class. I admit this one wasn't my idea and I worried that once you attach a grade you start worrying about cheating and I always hate using a single-assessment for anything permanent. However, it was pointed out that a student who can pass the benchmark demonstrates a minimum level of understanding and a few basic precautions can minimize cheating.

Here's a copy of the first benchmark in case you're curious. As an aside, writing the benchmark helped me empathize with the writers of the state tests. You kind of have to make it boring and vanilla. Whenever you introduce anything interesting, you have to worry about kids being freaked out by the strangeness and screwing with your data. It has to be straightforward and cut right to the point. Not to say that this is a good thing, but it was interesting for me to experience that.

Let me know if you have anything you think I should change/add/remove. What works for your school's benchmarks? What doesn't work? What would you change to make it work?




1: If you're not in California, Poizner ran in the GOP primary for governor. He wrote a book called Mount Pleasant which is the high school my kids feed into. It was his reflection on teaching (a single period, one semester) in the school. It was less than glowing. Our neighborhood does not smell like garbage and high school seniors aren't menacing anyone. On the flipside, everyone really does call the school Mount Pregnant. He's an outsider though so....not cool.
2: Just to be clear, the parents are supportive. They just often lack the ability, resources, or time to help their child at this point in their schooling. We often confuse unable with unwilling.
3: To answer Matt's question, reassessment is optional. Learning is not.