Why I Remain Committed to Educational Measurement

Gene V. Glass's name appears dozens of times in my EndNotes library. I bought his latest book the day it came out and will slow my skim to a read when I see his name, or that of his colleague Audrey Amrein Beardsley in my RSS feed. Like many whose day job involves the messy world of assessment and curriculum, I respect his thinking and value his take on events. His recent decision to shift his title and focus out of educational measurement was likely not an easy one, but it was likely the inevitable next step in his journey.  He, more so than most, know the boondoogle that is trying to quantify learning and his voice against large-scale testing used for accountability purposes will hopefully be listened to by those who set policy.

His points, though, haven't influenced my commitment to educational measurement. When I begin working with a new group of teachers, I often share a part of my educational philosophy: I believe we can talk about, describe, evaluate, measure, and learn about student learning without using numbers. I believe assessment is at its best when it is indistinguishable from learning. 

This, I believe, is the next level of educational measurement and why I remain committed to the field. 

My EndNote entries that included Glass were from a time when I was studying classic test design, cut score setting, and tests designed to evaluate the system from 10,000 feet. When I got back to the 1-foot perspective and shifted to focus on assessments that are more learner-centered, dynamic, and useful to teachers and students, my citations switched to researchers like Linda Darling-Hammond, Grant Wiggins, Giselle Martin-Kniep, and others. It is my firm belief that through the efforts like New Hampshire's teacher designed accountability measures, NY's Performance Consortiuum, California's exit portfolios, we're learning from the authentic assessment experiences of the 90's and creating a new approach to measurement in which the walls between curriculum, assessment, and instruction are blurred or non-existent; that all that remains is the learning. Our job as curriculum writers, assessment designers, sages, guides, teachers, and/or facilitators to create the conditions for learning and then get out of the way. 

The video below was created to talk about measuring learning in moocs but I nearly stood up and applauded at the two minute mark when Gardner Campbell makes the important - albeit simple sounding point: Why is measuring learning so hard? It depends on what you mean by measuring.



Is Opting Out *REALLY* the only option?

If I can be so bold, may I beg your indulgence for committing the sin of telling, not showing as I begin this post? Unfortunately, I neglected to take a picture of the student’s work and will do my best to convey its awesomeness.

A few years ago, I was working with a group of middle-level teachers designing assessments to meet NYS’s teacher evaluation system. They had a fair amount of leeway around the assessments’ design but had a few non-negotiables:
·         There would be no new testing. Any tasks had to be embedded in the curriculum and be seen by students to be a writing task, not a writing test.
·         The task had to offer novelty to the students by asking them a compelling, interesting authentic question. This required an interdisciplinary approach.
·         The task had to align to the Common Core Learning Standards as well as the district goal of perseverance. So while we were focusing on the craft of writing, the task needed be an opportunity for students to work on a task until they were satisfied with it and see their growth.

The task itself was complex in its simplicity. Students were presented with the essential question, “Does setting influence character?” and two writing prompts. [During the initial discussion of the question, a Social Studies teacher pushed back, raising a concern that the question didn’t cross disciplines. That it was an ELA question and didn’t really fit into SS or into life outside of school. One teacher turned to him and with a lovely raised eyebrow asked, “Any concerns about your daughter going on Spring Break to Mexico with her friends?” The ensuing conversation really highlighted how powerful essential questions can be.]  In the beginning of the year, students were prompted to select any character from any text they wanted and place them in NYC on September 10, 2001 and then a second scene dated September 12, 2001. Students’ work was evaluated on their ability to construct a narrative (W.8.3) through dialogue (W.8.3b) and show a change in their character development based on the change in setting.

I’ll date this anecdote by sharing that in the first round of the task, we read a lot of scenes involving Edward, Bella, Jacob, and Ironman in NYC. Although the adults struggled with cognitive dissonance, the students created scenes that were complex, powerful, awkward, and lovely. They struggled with the idea of character beyond just the moveable widgets in their stories but as it was a key component of the curriculum, it was something that could be addressed during the year. After revising the student handouts, we worked to anchor the rubric, revised it as needed, and gave the students feedback using a developmental writing rubric. The teachers took the work and feedback back to their classrooms and used them as a part of regular instruction, giving the students the option of using it for a portfolio piece. [And by the way, we used the work to set SLO baseline scores and targets for teacher evaluation but that’s another post.]

Time passed and it was time to try out the post-assessment. By then, students had been studying American History for about six months. So, this time, they had to create two scenes, placing a character in Philadelphia on July 3, 1776 and July 5, 1776.  During the “try out” period in which the teacher-designers gave the task to one or two groups of students and brought their work back to the group for rubric anchoring, one of the participating teachers floated into the meeting room with a grin that nearly split her face.  “Wait. Until. You. See. These.” She laid out her student work and not unlike a group at a baby shower, we ooed and awed over the responses. One particular students’ work elicited the same response from every reader: Well… how about that. It captured the complexity of character, nuance around American independence, and that eagerly sought trait of voice. The student’s first scene began: I am not a patient man but I have no choice. The middle part of the scene involved the unnamed character doing a variety of chores and physical labor on a farm on the outside of the city and ended with the line: Only one thousand, nine hundred, and ninety-nine days until I can earn my freedom and be with my family. The second scene was virtually the same, there were a few small changes in details but the character’s routine was similar. The ending, though. Oh the ending. Only one thousand, nine hundred, and ninety-eight days until I earn my freedom and be with my family. What this student understood was that for the vast majority of people living in the country born between July 3 and July 5, 1776, little had changed. 

Our first task was to use that students’ scene to anchor the top level of the rubric (“The student’s narrative is unexpected, surprising, or otherwise reflects an approach to the prompt that does not follow a common structure.”) The teachers followed the same routine as before; the tasks and feedback went back to the students who revised them and considered them for their portfolios. In addition, this time, they compared their baseline writing to their newer pieces and reflected on their growth as a storyteller. About a month later, the students took the 2013 state assessments. It’s my understanding that the teachers are still doing these kinds of tasks and their students are taking the state assessments. Their “Opt Out” numbers didn’t make the news.

In response to a flurry of recent editorials about the Opt Out movement, Leonie Haimson and Jeanette Deutermann make the claim that “opting out is the only option.” I recognize that my own confirmation bias flared up when I read their first sentence using two of my trigger words: exam and failed. Setting that aside, I’m struggling with their claim. Should the students from the anecdote above have opted out? Did their parents do the wrong thing by not forcing them to opt out? I’m especially struggling with their last line: The Opt Outs will continue until real teaching and learning return to our classrooms. 

 There’s a part of me that takes umbrage on behalf of teachers who do incredible work to ensure that state tests are only a pause in that real learning. That don’t devote weeks to test prep and support students through challenging conversations. I also want to speak up on behalf of teachers who do devote weeks to test prep because the pressure they feel is so great, they feel they have no other option.

My counterclaim: Opting out is not the only option. What about the option of ensuring that the other 177 days of the year are full of amazing, incredible, powerful learning moments? What about the option of having open conversations about what test prep can be instead of buying yet another test prep book? What about using the science of our profession – as troubled as it may be – to push back against what are perceived as poor quality items? 

The thought that keeps rolling around in my head: Opt Out won. The mouse roared and the elephant blinked. Now what?