That's too hard to measure!

I often get caught up in arguments with my peers about evaluation. We'll start talking about ways of going beyond right/wrong or complete/not complete and get in to what is the true purpose of whatever they're teaching. Inevitably, one of my fellow teachers will say something like, "Well, you can't really measure that."

I was flipping through the book How To Measure Anything by Douglas Hubbard and came across this definition of measurement:

Measurement: A set of observations that reduce uncertainty where the result is expressed as a quantity.

After reading this quote, I realized I had fallen victim to the Curse of Knowledge. Seeing it written out in a book made me realize that not everyone thought this way. Many (most?) people tend to think of measurements as something very exact. That tree is 12 feet and 3 inches tall. I weigh 181.2 pounds.

In science, we really do think of it as reducing uncertainty. I'm comfortable with designing a measurement system that only reduces uncertainty. The level of uncertainty that I'm ok with depends on the magnitude of the decision.

Example:

How do I measure how engaged a student is in the lesson?

I remember reading a book on Japanese Lesson Study and it said that the teachers actually counted "shining eyes" as a way to tell if students were engaged and interested in the lesson. You might argue that counting shining eyes seems fuzzy or that it doesn't offer complete information. That might be true, but what is also true is that you now have more information than when you started. You have reduced your uncertainty.

If you combine this measurement with other ones you can start getting a pretty good idea of how engaged your students actually were.

Here are a few things I say might occur if  students are more engaged in a lesson:
1. Ask questions
2. Less bathroom requests
3. Asking for more time
4. Number of people participating
5. Less disruptions
6. Keep going even after they're technically completed
7. Asked for resources to continue at home

If I just measure a few of those, I can begin to get a very good, less uncertain, idea of the level of engagement in my class.

Obligatory XKCD link:

It's anticlimactic compared to the movie version but those scientists know a little bit more than they did before.