The Big Picture of High School Graduation Criteria

In American public high schools, students generally need three things to be considered done with a free public education.

1. Passing scores on exit exams
2. Sufficient course credits
3. Be at least 17 years old

Not all states use the same criteria for exit exams, credit hours, and date for aging out of compulsory education. I'm in NY so I use that as a reference point. Your mileage and experiences may vary. What also varies are the slow, small changes some districts and communities have been taking over time to shift away from these particular criteria, which was all the rage when the United States made the decision to educate all of its younger citizens, not just those who could pay tuition.

Exit Exams
For the majority of students in NY, a passing grade on Regents exams is needed in order to demonstrate they've learned what's expected. This idea of "what's expected" lies at the heart of the standards conversation - which is too big an issue for this particular post. Currently, students need to pass 4 or 5 exams, depending on which pathway they are on. NYSED is in the process of expanding the pathways to include an arts degree and revamping the Global Studies Exam with has kept many a student from graduating on time.

For a minority of students in NY, mastery of the standards is demonstrated via research papers, portfolios, and projects. The criteria for success on their work is determinded through a consortium that operates with the full consent of NYSED and its members are regular ol' public education schools. Anne Cook, the director, reports they have fewer students drop out than Regents-giving high schools and their students report being better prepared for college. The consortium is not new. It's been around since the late 90's and as public interest in alternatives to high stakes exams grows, media outlets are covering more and more schools across the country that are quietly looking to document this criteria in a non-exam based way.

Course Credits
For the majority of students in public education across the country, they have to earn a sufficient number of course credits to graduate. These Carnegie units [Yes, it got its name exactly how you think it got its name] are strictly time-based. A common joke when discussing the issue of course credits is to point out the part of the learner they measure; students get credits based on how long their butt was in the seat, not necessarily how well or much is learned. Typically, students aren't given credit for having sat in a particular seat unless they get a grade that reflects they did what the teacher expected while sitting in that seat AKA pass the course. The challenge of how we describe "passing" is at the heart of the anti-grade movement and likewise a separate issue from this post.

For a minority of students in public education, measurement shifts from time to mastery. As it is with all things in education, this approach has many names: Competency-based education (CBE), performance-based, mastery-based, etc. [For what it's worth, I'm fairly confident that it took a while for Carnegie to shake out as the name for time-based education and then it became the only name. So it goes in education nomenclature. This does a good job trying to define mastery-based learning] This approach is not new nor does it look the same everywhere. It was not invented by Gates, Pearson, or Rocketship. It's based on the basic philosophy held by any adult who has met more than one child: not all children develop in the same way or at the same pace. What is new is the attention it's getting - especially when a publisher or vendor relies on teaching machines (H/t Audrey Watters) to make learning personal. It's important to note - and the purpose for this post - this approach is no more representative of CBE than a pit bull is representative of the subspecies canis. 

Aging out of Compulsory Education
For the majority of students in the world, passage through public education is based on how many times they've gone around the sun. To paraphrase Sir Ken Robinson - they're organized based on their date of manufacturing. Parents debate enrolling a "young 5" or waiting until they're a "young 6." To put it more bluntly, the "staircase" many point to as a problem with the Common Core was built long before Common Core came along. It's merely a runner on those cement steps.

For a minority of students here in the states, some districts are shifting how they think about the concept of time and age. The Adams 50 School District in Colorado is one that moved away from a traditional concept of grades. Ira Socol writes a great deal about his district's approach to grouping. For others, the shift away from course credits forced a reconsideration of how students are grouped and graduate. New Hampshire has passed policies that allows local districts to determine if demonstrating competencies allows students to graduate "early."

States, districts, and schools have choices about how they handle the three components of exiting a free public education. To that end, we can make claims about each of them in turn. Some possible ones might include [and note, I'm not married to any of these, just taking my claim writing skills out for a walk]:

• Claim: Exit exams are the cheapest, most cost-effective way to ensure students have mastered the expected content.
• Counterclaim: Portfolio-based and performance-based exit tasks, though more costly, are worth it as they allow us to expand what it measured and how students demonstrate their learning.

• Claim: Traditional course credits are the most effective way of ensuring students get the full developmentally-appropriate liberal arts experience including group work, discussion, and review of previously learned content.
• Counterclaim: Competency-based learned shifts the focus from time-based measurement to actual student ability and allows for more varied, personalized engagement with the content. 

• Claim: A society needs to keep children in school until the age of 17 or 18 to develop their social and emotional skills, regardless of how much learning they're experiencing in school. 
• Counterclaim: By allowing students to exit out of school once they have mastered the outcome expectations for public education, students are free to pursue their own areas of interest. 

Looking forward to hearing your claim.

"Thanks for the feedback!" NYSED to NYS Educators

Edited on November 23 to add the NYSAPE Common Core Survey. 

For years, likely since the first day the website went up, there has been a "Teacher Participation Opportunities" link on the New York State Education Department's Office of State Assessment (OSA) website.

Following the link leads to a series of options available to NY teachers to participate in a variety of test design and assessment writing activities. These activities typically require sub coverage and travel to Albany, a 9-hour round trip and an overnight stay for those in the Southern Tier. Some are "once and done" work in which the teachers go to Albany, engage in a particular task, get a nice thank you letter, and not know what will become of their work until the test is published or the scores released. Some are extended projects in which teachers return multiple times to Albany or continue the work back at home. The biggest challenges of this approach to getting teacher feedback: 

•  teachers have to volunteer or be nominated,
• SED can filter who they bring to do the work, and
• the proceedings aren't public. 

This novel idea of involving NYS teachers in the design of the NYS tests and exams isn't new. Teachers in 1891 were asked their opinion on the exams.

At least as early as 1891, blanks for suggestions and criticisms "relative to the character and scope of the examinations" were shipped with each set of examination papers. These comments are tabulated and studied carefully.

So basically, teachers have been involved in the writing of the NYS tests and exams since pretty much the beginning. Opinions about if it's the *right* kind of feedback, if the *right* teachers are giving feedback, and what that feedback looks like in the modern area vary.

The feedback process around standards isn't nearly as long. The formal presence of standards didn't start until the 90's. Any NYS teacher of a certain age remembers the booklets with the 1996 standards, printed on really thin paper with different colored covers. Inside the front cover of each book was a list of the teachers who participated in their construction and anchoring. This is from the LOTE standards, the only ones that haven't been updated since 1996. 

When the time came to update the standards following the change in NYS law in 2007, Albany came to the field. In April 2008, I was at the Western NY forum and used this new thing called Twitter to share out what was happening. It's interesting to note that many of the things I tweeted, the things the teachers in the room were asking for, are a part of the Common Core design. But I digress. 

Shortly after the forums concluded, the committee wrote up their findings and began working on what are now called the "lost" standards by some advocates. I prefer the moniker the "paused" standards as NY stopped that work in order to be a part of a new initiative to create multi-state standards. "Common" standards, as it were. NYSED provides a timeline of those decisions here. Opinions about why New York made that decision, if the "paused" standards are better or worse than the CCSS, and what it means to have 50 states with 50 sets of standards area vary.

Which brings us to 2015 and NYS is again seeking out teacher feedback.

1. Want to comment on each specific Common Core Learning Standard? Commissioner Elia wants to know if the standard is acceptable, if it should be moved, changed, or re-worded.
2. Want to comment on the CCLS, tests, or APPR in general? Governor Cuomo and his task force are all ears. (It remains to be seen, though, how discrepancies between Elia's survey and Cuomo's task force will be resolved.)
3. Want to comment on the latest draft of the Science standards? The Science department at NYSED will open a survey on December 2. Draft standards are available now.
4. Want to comment on the proposed changes to the NYS Social Studies Regents? The look, design, and structure of the exams are open for feedback.
5. Want to be a part of writing NYS tests, assessments, and exams? The offer from them still stands. (Be sure to check dates though, some have closed for now.)
6. NYS Allies for Public Education (NYSAPE) created their own survey which touches upon testing, APPR, and the CCLS standards. It's unclear how these data will be used. 

In addition, updates from SED frequently appear on the agenda for events like Middle-Level Liaisons, DATAG,  Social Studies conferences, etc. It's a safe assumption that those SED personnel are talking to the teacher- and administrator-leaders of those organizations. So let it not be said NYSED in 2015 doesn't want your opinion. 

But as we know, that's only step 1. 

Lie back and think of England

There is a certain cognitive twitch that occurs only when one is writing a multiple choice question for poetry. It's a brain hiccup caused by the tension of doing something that shouldn't be done but has to be done.

Everyone in the room, myself included, knew the non-negotiables:

• goal was to write a common assessment
• it needed to include poetry
• it had to generate quantitative data (ergo, scanning MC questions versus hand scoring written responses)

It simply wasn't a viable option at that moment to switch gears completely to curriculum-embedded, performance tasks like some districts had done. Things had been negotiated. Compromises had been reached. So, there we were. Trying to find a poem that was equal parts complexity and simplicity.

As most teachers do, this group found a way through and used student choice, several different poems, and a focus on the CCSS Language standards to make it an assessment that would generate useful information without causing too many brain cramps. 

On my travels to my next adventure, I kept re-living the day. Why didn't we push back harder? Why didn't I advocate more vocally for a better assessment when asked to support this group of teachers? Did we capitulate because as a room full of white women, we were socialized to follow the rules? We did what was asked. We met the mandate while doing our very best to ensure quality assessment. No one left feeling like we would be imposing something unethical or unfair on the students but at the same time... a multiple choice question about poetry. Did we do the right thing?

In a twitter exchange on the theme of the thinking behind certain mandates, Peter Greene tweeted to me, "So just lie back and think of England?" Which, first, no. And second... no. (No time to click the link? The phrase is wrongly attributed as advice from Queen Victoria to her daughter-in-law about producing an heir but has come to represent a trope that women need to suffer through sex for the greater good. Here is where every real nerd will repeat, "The Greater Good.")

It would seem there are three ways to deal with policy mandates with which we disagree.

1. Refuse it. The Opt Out approach appears to be about changing policy by refusing to participate. It's not necessarily about finding a way through, it's about finding a way around. 
2. Be excused from it. New Hampshire's approach to annual testing is asking permission to come at it from a totally different direction. 
3. Find a way through it. Leveraging mandates to make the best of what's been asked.

I compulsively read everything I can on cognitive biases and how our brains are lazy by design. So I spend a lot of time while traveling trying to figure out the holes in my logic model. See - I'm okay with #3. I'm okay with schools looking at policy and saying, "Welp. This is silly. But, it's policy. How can we attend to this in a way that honors what we value and protects our students?" And then moving on. I don't see it as capitulation, but I suspect that's because I'm treating it as a narrow issue of assessment/curriculum design. 

Some authors like to compare the Opt Out movement to activities through history, especially during the Civil Rights era and each time I read one of those blog posts, I struggle against my instinct to reject them as hyperbole. In some cases, I've no problems connecting parts of a system (impact of cultural appropriation on the well-being of Native Americans) but here... (Opting Out of a state test as a gesture towards more equitable schools), I struggle. 

There is, I suspect, a great deal to be said about what it means to leverage mandates. It's a close cousin to "asking for forgiveness instead of permission" and lives in the narrow space between doing what is required and what is right. Is it a matter of changing of what we can? Or do I have a giant blind spot around the Opt Out movement? 

What exactly is "standardization" in assessment design?

I'm going to do my best to keep this really short and concise and write according to The Notorious RBG: 'Get it right, keep it tight.'"

Background:

Peter Greene made a claim that the correct number of standardized tests is zero.

I presented a counterclaim that standardization isn't the problem. 

Greene expanded on his claim to clarify his intentions around the tests. 

While reading Peter's updated claim, I realized that at no point was the phrase "standardized" actually defined. We both gave our opinions on what it means:

From Greene: 

"Standardized" when applied to a test can mean any or all (well, most) of the following: mass-produced, mass-administered, simultaneously mass-administered, objective, created by a third party, scored by a third party, reported to a third party, formative, summative, norm-referenced or criterion referenced.

From Me:

Welp, first, minus ten to me because I didn't state a definition, I asked questions that implied one. So to restate the intention of my questions. Standardized means doing the same thing for a group of students. The "thing" can be the nature of the task, the amount of time, the scoring criteria, or the directions to the students.

This is the quote from Peter that made me consult my bookcase and/or Google.

This broad palate of definitions means that conversations about standardized testing often run at cross-purposes. When Binis talks about the new performance assessment task piloting in NH, she thinks she's making a case for standardization, and I'm think that performance based assessment is pretty much the opposite of standardized testing.

I wasn't making a case for standardization, I was identifying an example in which a standardized process is used to develop a performance-based assessment. This may be switch-tracking (from The Hidden Brain Podcast - check it out. It's really cool!) by both of us but it remains that when we use the word or phrase, we've a different meaning in mind. So... to the Googles!

From Standards for Educational and Psychological Testing AKA "The Testing Standards" (This is basically the sourcebook for writing a quality measure of student learning), written by the AERA, APA, NCME, 2014

A test is a device or procedure in which a sample of an examinee's behavior in a specific domain is obtained and subsequently evaluated and scored by a standardized process. Tests differ on a number of dimensions... but in all cases, however, tests standardize the process by which test takers' responses to test materials are evaluated and scored. 

 According to the alpha and omega, a test by its very nature is standardized. Which makes the phrase "standardized test" redundant, it seems.

From the Code of Fair Testing Practices, which is a supplementary document for the Testing Standards.

The Code applies broadly to testing in education regardless of the mode of presentation, so it is relevant to conventional paper-and-pencil tests, computer-based tests, and performance tests.... Although the Code is not intended to cover tests prepared by teachers for use in their own classrooms, teachers are encouraged to use the guidelines to help improve their testing practices.

From Stanford's primer on performance-based assessments:

[Describing performance-based assessments] Teachers can get information and provide feedback to students as needed, something that traditional standardized tests cannot do.

.... in the early years of performance assessment in the United States, Vermont introduced a portfolio system in writing and mathematics that contained unique choices from each teacher’s class as well as some common pieces. Because of this variation, researchers found that teachers could not score the portfolios consistently enough to accurately compare schools. The key problem was the lack of standardization of the portfolios.

Here, the authors use standardized in two ways: first to refer to the multiple choice test we tend to picture when we hear "standardized test" and then to refer to the process of creating a uniform approach to scoring student writing samples.

From Handbook of Test Development, edited by Downing & Haladyna:

The test administration conditions - standard time limits, proctoring to ensure no irregularities, environmental conditions conducive to test taking, and so all - all seek to control extraneous variables in the experiment and make conditions uniform and identical for all examinees. Without adequate control of all relevant variables affecting test performance, it would be difficult to interpret examinee test scores uniformly and meaningfully. This is the essence of the validity issue for test administration.

Now, for the kicker. Why does any of this matter? Because of this - assessment literacy.  If you follow no other link from this post, please follow that one. Peter and I are reading the same book but we're not on the same page, as it were. He's a teacher, I'm out of the classroom, working with teachers around assessment design. This isn't an issue of "He's right and I'm wrong" or "I'm the expert, trust me." It's more compelling, instead, to consider the implications - and there are many of how we talk about testing and assessment. From teacher preparation, to academic writing, to communicating with parents and the public. I suspect, that until the profession agrees on a common glossary, we're going to keep nibbling at the edges.

Unintended Consequences of Making Standardized Tests the Enemy

President Obama's "2%" video has generated a number of claims, counterclaims, rants and praise. Larry Ferlazzo rounded up many of them if you're looking for texts that support your opinion or challenge it. There's a lot to be said about it and there's a text written basically for every possible counterclaim or supporting claim. My general take away vacillates between "meh" as unless Congress changes ESEA, state-wide annual tests will persist and yeah! let's talk about what healthy assessment systems look like! The challenge remains, though, around how to deal with mandates - to leverage them to support student learning, to ignore them, to advocate parents opt their children out to send a message, or door #4. Regardless, it's a deeply personal decision every educator, school, and district must make.

That, however, is not the issue at hand or why I dusted off my semi-irregular blog. I get ranty about semantics and I own it. It comes from a place of absolute adoration for the teaching profession. Since my first ed course in college, I've loved the hubris that comes with labeling, describing, and attempting to capture the unseeable. It's the place that made me comfortable as a young teacher to speak up when a teacher's aide referred to my students with special needs as "TMR." In her day, "Trainable Mentally Retarded" was an acceptable moniker for a certain type of student. I had the concept of people first language to fall back on and the rules of the framework to help me find the words to speak up and find a way through the awkwardness. 

It's the love for the language of teaching and learning that rears up occasionally and results in me offering unsolicited opinions. Before I offer up the claim I disagree with for semantical reasons, I'd like to lay out the evidence for my counter-claim.

Ever give the same test or task to a group of students?

Ever use an answer key to score students' test?

Ever ask students to hand in their work after a certain period of time?

Ever assign a grade on a scale of 0-100 with a pass/fail cutpoint (i.e. 65)?

If your answer to any of these questions is yes, then you've given a standardized test. 

Ever talk with other teachers to reach consensus around what quality work looks like?

Ever set aside examples of student work to refer to later as an example of a "good" paper?

Ever review a task to make sure it's fair, accessible to students with disabilities, free of bias?

If your answer is yes, then you've used standardization to help you do your job.

Peter Greene wrote a post in response to Obama's comment titled The Correct Number of Standardized Tests. His claim is that the correct number is zero. He says, "Students need standardized tests like a fish needs a bicycle." If that's the case, then it raises a whole slew of questions about grades, final exams, and what it means to fairly assess students. Meanwhile, the NYS Performance Consortium uses standardized processes. These teachers in NH are using standardized tests (which are actually performance tasks) to assess their students. And more to the point, when you ask Kiana Hernandez about standardized tests, she talks about those created by the state, her district, and her teachers.

I recognize that his point isn't about standardized tests per se but the large-scale once a year tests. However, he ends his post with "the number of necessary standardized tests is zero." I'm all about performance tasks, portfolios, and authentic assessment. I'm all about the maker movement and kids doing things in school that have meaning to them outside of school. I'm also all about the profession of teaching and having a robust public education system. I want there to be a standardized approach to how we collect large-scale evidence of learning in order to inform systemic decisions and policy. Right now, we're using multiple choice because they're easy, familiar, and faster than the alternative. Hopefully, we'll move to a system like what NH is cooking up where the tasks are embedded within the curriculum and assessment is a part of a learning, not an interruption. But even when we get there, there will still be a need for some degree of standardization. To suggest that standardization itself is a problem ... well, that's a whole nother case of worms about the purpose of public education and society.

The aide in my school way back when didn't mean my students were "trainable." She didn't think less of them, she was used to "TMR" and rattled it off as a placeholder for "the students with mild to moderate disabilities in the 15:1 math class but in a general education class for Science and Social Studies." I suspect Greene is using "standardized tests" as a placeholder for "the tests given once a year that cause an incredible amount of stress and do little to inform what happens in my classroom." I go back to the questions I asked in my semantics post: What's gained or lost by using or not using precise language? What does the profession gain or lose when using shorthand to refer to a given concept or idea? 

My CCLS Changes and Recommendations

Earlier today, NYSED released the link to the tool they're using to collect recommended changes to the Common Core Learning Standards. The CCLS are slightly different than the original CCSS in a few key ways: NYS added #11 to reading about literature, a handful of other standards around creativity, culture, and choice and one or two to the math standards.

The survey is clear about what it is and what it isn't. It's not a place to share general opinions about the giant ball of sticky wax referred to as "Common Core." It's a place to comment on individual standards. Each. Individual. Standard. Which, according to my Excel file, is 1115 literacy standards. I will be sharing feedback with SED based on my experiences around curriculum and assessment over the last four years. Some of my feedback will include:

* Creativity was added during the adoption process - in some grades, though, it appears under Reading Informational Texts and in others, it's in Speaking and Listening, some it's both. I'd advocate for putting them all under Speaking and Listening like it is in 12th grade.

* Cultural connections in Kindergarten and First Grade (RL.9a) is worded oddly. I suspect it's about inviting students to see connections between their own lives and the experiences of those in a book they're reading but it should be cleaned up and clarified to ensure alignment in curriculum design.

* The "seek to understand" standard has always been one of my favorites but like cultural connections, the wording seems a bit hastily. I've drafted a proposed re-write based on work from anti-racism/cultural competency educators.

* The study of dialects and accents appears only in 5th grade. Feels like a waste of an opportunity to invite students to engage with the English language and all its odd quirks. I have ideas for how to expand that into other grades.

* In some of the original CCSS, there's a sense of writing by committee that becomes clear when you've started at the standards many, many, many times during design sessions. Small things like a word being used in 3rd, dropped in 4th, but re-appears in 5th. Because that's the kind of person I am, I have a running list of those odd quirks and will be passing them along.

Those who advocate opting out of state tests have reported that, in order to opt back in, they want the standards to be fixed so they are developmentally and age-appropriate. Let's hope they pass along their feedback around what changes need to be made to ensure that happens.

Why I Remain Committed to Educational Measurement

Gene V. Glass's name appears dozens of times in my EndNotes library. I bought his latest book the day it came out and will slow my skim to a read when I see his name, or that of his colleague Audrey Amrein Beardsley in my RSS feed. Like many whose day job involves the messy world of assessment and curriculum, I respect his thinking and value his take on events. His recent decision to shift his title and focus out of educational measurement was likely not an easy one, but it was likely the inevitable next step in his journey.  He, more so than most, know the boondoogle that is trying to quantify learning and his voice against large-scale testing used for accountability purposes will hopefully be listened to by those who set policy.

His points, though, haven't influenced my commitment to educational measurement. When I begin working with a new group of teachers, I often share a part of my educational philosophy: I believe we can talk about, describe, evaluate, measure, and learn about student learning without using numbers. I believe assessment is at its best when it is indistinguishable from learning. 

This, I believe, is the next level of educational measurement and why I remain committed to the field. 

My EndNote entries that included Glass were from a time when I was studying classic test design, cut score setting, and tests designed to evaluate the system from 10,000 feet. When I got back to the 1-foot perspective and shifted to focus on assessments that are more learner-centered, dynamic, and useful to teachers and students, my citations switched to researchers like Linda Darling-Hammond, Grant Wiggins, Giselle Martin-Kniep, and others. It is my firm belief that through the efforts like New Hampshire's teacher designed accountability measures, NY's Performance Consortiuum, California's exit portfolios, we're learning from the authentic assessment experiences of the 90's and creating a new approach to measurement in which the walls between curriculum, assessment, and instruction are blurred or non-existent; that all that remains is the learning. Our job as curriculum writers, assessment designers, sages, guides, teachers, and/or facilitators to create the conditions for learning and then get out of the way. 

The video below was created to talk about measuring learning in moocs but I nearly stood up and applauded at the two minute mark when Gardner Campbell makes the important - albeit simple sounding point: Why is measuring learning so hard? It depends on what you mean by measuring.

Is Opting Out *REALLY* the only option?

If I can be so bold, may I beg your indulgence for committing the sin of telling, not showing as I begin this post? Unfortunately, I neglected to take a picture of the student’s work and will do my best to convey its awesomeness.

A few years ago, I was working with a group of middle-level teachers designing assessments to meet NYS’s teacher evaluation system. They had a fair amount of leeway around the assessments’ design but had a few non-negotiables:

·         There would be no new testing. Any tasks had to be embedded in the curriculum and be seen by students to be a writing task, not a writing test.

·         The task had to offer novelty to the students by asking them a compelling, interesting authentic question. This required an interdisciplinary approach.

·         The task had to align to the Common Core Learning Standards as well as the district goal of perseverance. So while we were focusing on the craft of writing, the task needed be an opportunity for students to work on a task until they were satisfied with it and see their growth.

The task itself was complex in its simplicity. Students were presented with the essential question, “Does setting influence character?” and two writing prompts. [During the initial discussion of the question, a Social Studies teacher pushed back, raising a concern that the question didn’t cross disciplines. That it was an ELA question and didn’t really fit into SS or into life outside of school. One teacher turned to him and with a lovely raised eyebrow asked, “Any concerns about your daughter going on Spring Break to Mexico with her friends?” The ensuing conversation really highlighted how powerful essential questions can be.]  In the beginning of the year, students were prompted to select any character from any text they wanted and place them in NYC on September 10, 2001 and then a second scene dated September 12, 2001. Students’ work was evaluated on their ability to construct a narrative (W.8.3) through dialogue (W.8.3b) and show a change in their character development based on the change in setting.

I’ll date this anecdote by sharing that in the first round of the task, we read a lot of scenes involving Edward, Bella, Jacob, and Ironman in NYC. Although the adults struggled with cognitive dissonance, the students created scenes that were complex, powerful, awkward, and lovely. They struggled with the idea of character beyond just the moveable widgets in their stories but as it was a key component of the curriculum, it was something that could be addressed during the year. After revising the student handouts, we worked to anchor the rubric, revised it as needed, and gave the students feedback using a developmental writing rubric. The teachers took the work and feedback back to their classrooms and used them as a part of regular instruction, giving the students the option of using it for a portfolio piece. [And by the way, we used the work to set SLO baseline scores and targets for teacher evaluation but that’s another post.]

Time passed and it was time to try out the post-assessment. By then, students had been studying American History for about six months. So, this time, they had to create two scenes, placing a character in Philadelphia on July 3, 1776 and July 5, 1776.  During the “try out” period in which the teacher-designers gave the task to one or two groups of students and brought their work back to the group for rubric anchoring, one of the participating teachers floated into the meeting room with a grin that nearly split her face.  “Wait. Until. You. See. These.” She laid out her student work and not unlike a group at a baby shower, we ooed and awed over the responses. One particular students’ work elicited the same response from every reader: Well… how about that. It captured the complexity of character, nuance around American independence, and that eagerly sought trait of voice. The student’s first scene began: I am not a patient man but I have no choice. The middle part of the scene involved the unnamed character doing a variety of chores and physical labor on a farm on the outside of the city and ended with the line: Only one thousand, nine hundred, and ninety-nine days until I can earn my freedom and be with my family. The second scene was virtually the same, there were a few small changes in details but the character’s routine was similar. The ending, though. Oh the ending. Only one thousand, nine hundred, and ninety-eight days until I earn my freedom and be with my family. What this student understood was that for the vast majority of people living in the country born between July 3 and July 5, 1776, little had changed. 

Our first task was to use that students’ scene to anchor the top level of the rubric (“The student’s narrative is unexpected, surprising, or otherwise reflects an approach to the prompt that does not follow a common structure.”) The teachers followed the same routine as before; the tasks and feedback went back to the students who revised them and considered them for their portfolios. In addition, this time, they compared their baseline writing to their newer pieces and reflected on their growth as a storyteller. About a month later, the students took the 2013 state assessments. It’s my understanding that the teachers are still doing these kinds of tasks and their students are taking the state assessments. Their “Opt Out” numbers didn’t make the news.

In response to a flurry of recent editorials about the Opt Out movement, Leonie Haimson and Jeanette Deutermann make the claim that “opting out is the only option.” I recognize that my own confirmation bias flared up when I read their first sentence using two of my trigger words: exam and failed. Setting that aside, I’m struggling with their claim. Should the students from the anecdote above have opted out? Did their parents do the wrong thing by not forcing them to opt out? I’m especially struggling with their last line: The Opt Outs will continue until real teaching and learning return to our classrooms. 

 There’s a part of me that takes umbrage on behalf of teachers who do incredible work to ensure that state tests are only a pause in that real learning. That don’t devote weeks to test prep and support students through challenging conversations. I also want to speak up on behalf of teachers who do devote weeks to test prep because the pressure they feel is so great, they feel they have no other option.

My counterclaim: Opting out is not the only option. What about the option of ensuring that the other 177 days of the year are full of amazing, incredible, powerful learning moments? What about the option of having open conversations about what test prep can be instead of buying yet another test prep book? What about using the science of our profession – as troubled as it may be – to push back against what are perceived as poor quality items? 

The thought that keeps rolling around in my head: Opt Out won. The mouse roared and the elephant blinked. Now what? 

Gee willikers, Mister! I never thought of that!

It's a common theme among those who are not fans of Common Core to use sarcasm and derision when talking about teachers who attribute a change in their practice to those same standards. (I weighed the pros and cons of linking to particular writers who practice this habit and decided no. It's about patterns, not personalities.) The refrain is usually:

Educator: As a result of Common Core, I started doing this.
Response: Scoff. You needed Common Core to do *that*? Why weren't you doing it before?

A recent article in the NYTimes about changes in the English classroom has received the same response. Only this time, the subject of the refrain is the reporter. Several teacher-bloggers took to their pages to share their disdain at the reporter's lack of understanding around English classrooms pre-CCSS and their opinion on her seemingly uninformed understanding of the travesty that the CCSS are.

It's a provocative writing device as it attempts to establish the author of the response as a voice of authority. A sort of a: *I* knew about this before so my opinion on the matter is more righter than yours. The drawback is the attempt to shame the educator or reporter who spoke about the change in practice. And shame is the really the best adjective as many of these posts imply that the only right thing that teacher can do is hide her head, apologize for not knowing about *that* and rescind all support for the CCSS.

When I read posts with that tone, I'm reminded of Ignaz Semmelweis. Well, not him personally, I had to Google his name, but of his work. Prior to Mr. Semmelweis, midwives and doctors would rotate between deliveries without washing their hands. Mothers were dying at high rates and it was seen as just one of the consequences of giving birth. So these medical professionals were doing the best they could with the information they had and along comes Ignaz and they discovered they needed to change something in their practice. They weren't bad before, they weren't uninformed. In fact, they likely thought they were doing everything in their power to keep the mothers alive.

There are two connections I see to the education profession. 

First, why the disdain? Why the condemnation of teachers who come forward to share how they've reflected and evolved? Going after individuals who share stories of how their practice has changed is a bit like claiming you've been washing your hands all along. It's possible some doctors were. It's possible some had Clean Room Level 4 birthing suites. At the same time, when the profession realized a change was needed, the doctors who didn't know about hand washing weren't bad doctors the day before the "wash your hands because ew..." staff meeting and then good doctors once they started washing their hands. One would think we'd want to elevate the voices of teachers who share their thinking, not shame them back into silence. More to point, it's possible some doctors never lost a mother or a child. To their thinking, they didn't need to wash their hands as their practices were just fine, thank you very much. To those teacher bloggers, good for you but why not dial up the pride in reflective practitioners and dial back the disdain?

The second, more important, issue is race and social justice in education. Mad props to #educolor for their work on the matter and for offering a place for educators to listen and learn. The conversations about race and social justice aren't always smooth. They're not easy and people are going to get it wrong. 80%-ish of educators are white women, many of whom were good students praised for being well-behaved and doing the "right" thing in school and speaking up takes an incredible amount of courage. To reveal a past mistake in pubic? Not easy for anyone - especially teachers that are in predominately white schools where students are most often introduced to Black Americans in the month of February or via the Social Studies curriculum and slavery. When teachers are getting scoffed and derided for sharing their thinking around pedagogy, what's the benefit to sharing their thinking about the really hard and important stuff?

Teachers are increasingly asking. "Here's where I'm struggling. Where am I stuck?" or sharing, "Here's what I'm trying now. It was hard to get here but I'm glad now that I'm here." It baffles me that anyone would respond in any way other than, "Thanks for sharing. That must have been hard. Can you tell me more about that?"This isn't about protecting delicate white lady feelings or otherwise suggesting teachers can't handle critiques. (I'll refrain from commenting on the patterns when some bloggers are offered critiques via Twitter.) It's about saying, "Dude. Don't be such a jackass." Or fine. Be a jackass but take a moment and consider what happens when that teacher Googles her name.

On Negotiating Nuance

The difference between the right word and the almost-right word is the difference between lightning and a lightning bug. Mark Twain

I have awesome cats. Two of whom like to hang out in my office when I’m working from home and provide company while I work AKA walk across my keyboard when I’m not honoring them sufficiently. More than once, they’ve been my audience as I’ve scrolled Twitter and mumbled a strongly worded opinion about something I’ve read. And I swear, the cats rolled their eyes at me and I hear my mother’s voice telling me, "there are starving children in China!" when I refused to eat fish sticks as a kid. The message is sort of the same – this thing that is so important to you, Jenn, isn’t that big of a deal in the grand scheme of things and I totally get it. I rationalize my obsession with nuance and semantics around assessment design as my commitment to the science of the craft of teaching. My nuance nudges, in no particular order.

1.       Assessment literacy requires that we consider the system and recognize that different assessments serve different purposes. Diane Ravitch recently said that the state tests were invalid because they served no “diagnostic purpose.” My nuance nudge: Well neither do the final exams that many students take in High School. So while the state tests provide limited diagnostic data for a particular child, they provide useful information for the system. Case in point? Next year, 2015-2016, will be the first year that the students taking the CCLS tests will have only known curriculum aligned to CCLS. The state assessments are the [likely] most objective way of documenting, or one might even say, diagnosing the consistency of alignment to the standards across the state.

2.       “Exam” is not the same as “test”. Don’t believe me? Check the cover pages.

3. Why does it have to be about “sides”? That is, it’s possible to do all of the following - at the same time.

• support authentic, curriculum-embedded assessment and portfolio design,
• struggle with the intent and purposes of the opt out movement,
• be in favor of annual testing as a large-scale measure of the system,
• think the Common Core Learning Standards are better than what we had before and not really care about where they came from,
• be okay with providing students with direct instruction on how to take a test (AKA test prep done right), and
• be against VAM as it currently being used in teacher evaluation.

So I guess this is less a statement on struggling to find a place for nuance and more a "here's where I am right now." I think Steve! the cat agrees or at least is pretending to. At the very least, he makes me feel less guilty about not carrying the laundry upstairs on a Sunday morning (Kevin is our tabby, Steve! is behind him. Baby, our oldest, hangs out elsewhere.)

You say Tomato, I say Tomahto ... it's [almost] 2015, why are we still talking about this?

We all like to think we're open-minded; that we arrived at a reasonable, logical and right conclusion after careful consideration of the facts, perspectives, and various opinions. Sometimes we get the benefit of cardiac assessment ("it just feels right") or a gut response to an issue. We weigh the evidence, reach a conclusion, and can rest comfortably in our superiority over those who haven't reached the clearly obvious conclusions we have. Ok... so that last bit may or may not be true, depending on your approach to critical thinking and awareness of cognitive biases.

Here's an interesting experiment. Below are two Kindergarten standards. One is from the Common Core Learning Standards (NYS's version of the CCSS) and one is from the so-called "Lost Standards" (the version NYS was working on when Common Core came along).

Standards A	Standards B
Ask and answer questions in order to: • seek help, • get information, • or clarify something that is not understood.	• Ask and answer questions about classroom activities • Request help when needed • Know when and how to ask permission

My bias is that standards are the least important part of ensuring a quality public education for all students. I'd co-sign this post by Kathleen Porter-Magee on standards and curriculum if it were a petition. So when I make the claim: The text on both sides of the table are basically saying the same thing, it's informed by my bias and hunch that difference between any two set of standards isn't really all that big. In my opinion, the biggest change from the old NYS ELA standards and the CCLS (besides the six shifts) was the introduction of coding, shared language between grade levels, and the explicit inclusion of culture and choice in the language of the standards.

Someone with a different bias, perhaps that the Common Core are "developmentally inappropriate" will likely see the text in the two columns as different. The would likely make the counterclaim: one is more "appropriate" than the other. 

All of that said, here's my question: Does it matter? It's 2015. How will the problems created by the Common Core be fixed by dropping them and going back to standards that NYS walked away from in 2009?

This isn't about being right or being wrong about Common Core or which is better. This post is about humbleness and hubris. I've been writing this for a while now. While walking through airports, driving home from programs, falling asleep at night, in-between designing programs, and reading assessment research and I'm still struggling to find the right words. Usually, not always, I found myself mentally composing this post after scrolling through Twitter and watching the absolute confidence that a large number of educators speak about a particular issue. Mostly white male educators. Mostly about Common Core. I thought perhaps it was a function of 140 but the language of their blogs is often the same. Just for fun, I've "pushed back" (which I've been told is "trollish" and a "bad habit") and in most instances, I get a response that one might classify as doubling-down. I've reflected on why I feel compelled to comment and poke. It's partially because I'm fascinated by how we engage via Social Media. It's partially because I advocate for process assessments (asking students about HOW they think) and logical discourse. It's partially because I'm annoyed. I'm annoyed by hubris. I'm annoyed by the number of white male educators who write and post their thoughts on issues, rather than boosting the voices of women and men of color who have been writing about the given issue for months, even years. Maybe I'm a little jealous of the sheer hubris that some exhibit as they write and post about an issue, wrapped in a toasty blanket of confidence that they are absolutely, incontrovertibly right.

Mostly though, mostly it's because I'm angry. I'm angry that at the end of 2014, following months in which Black Americans had to say - aloud - "my life matters", people are still having conversations that feel like they should have been resolved in 2010. I'm angry at the data below.

I'm angry that someone claimed (with a seemingly straight face) that replacing one set of standards that are basically the same as another set of standards will reduce misbehavior among Black preschoolers and therefore, reduce the suspension rate. I'm angry that a number larger than 1 of middle-aged, white men wrote long-form essays on the impact of Common Core - without citing or even referring to the lived experiences of classroom teachers or current college students. I'm angry that many of those who are anti-test (seriously folks, take Jose Vilson's advice and advocate for the "Whole Child") aren't offering alternatives to annual testing other than "Opt Out." It's 2014, not 2009. How about we move on from the standards and onto pedagogy, quality curriculum, equity, and cultural literacy?

How powerful would it be if instead of continuing the same conversations in 2015 that have been going on since 2009, we start or join new ones? The ones about race and culture and whose voices we trust and the role of public education and the tension between the learner and schooling? How about instead of tweeting "this is the truth", we ask "what makes you think that's the truth?" What if we asked more than told? Questioned more than pontificated? Reflected more than bumper sticker-ed? But eh... whadda I know? I'm just a troll.

Edit: I wrote this is December 2014. I was going to remove the above few paragraphs as they're repeating what has been said much more clearly and concisely by other authors such as #educolor members and equity researchers. I elected to keep them in an effort to be an honest author. Now that Cuomo has announced his panel, my frustration has flared up again. Instead of talking about the problems and flaws of APPR, we're going to be playing a game of wordsmithing to create "NYS Standards" that basically say the same thing as the CCSS but are juuuust different enough to require new curriculum and assessment design. I for one, am fairly confident, that's not the most effective use of our time, not the thing that should be the top of our "to do" list.

Broadening Our Concept of the Whole Educator

Attending to the social and emotional needs of individual educators is an important component of a healthy classroom, school, and public education system. At the same time, ensuring a healthy system and a safe space for all children requires that all adults in the system go through the difficult work of uncovering and confronting assumptions, biases, and mental models. This work is confounded by the shared human trait of egocentrism. It is impossible for any human – regardless of age, culture, or experience – to truly “get” where another human is coming from. We are all limited in that regard. Yet, it is this very shared trait that allows us to create dynamic, diverse communities in which each person is seen and treated as a whole and rich being with their own perspective, passions, interests, and experiences. The creation of those communities, though, requires that we acknowledge our shared failings and work to openly address them. In the absence of these conversations, we may end up unintentionally causing harm to students. For example:

Last Year: A high school teacher created an assessment to assess a complex Common Core standard and selected a series of texts for students to use to support or refute a claim. When all was said and done, 13% of the teacher’s students wrote papers making the claim, supported by evidence the teacher provided, that the Holocaust didn’t happen.

It’s not uncommon for school communities to say, in the wake of similar instances, “the teacher was unaware of any [Jewish] students in his/her class.” In other communities, parents of color have shared that their children only see texts by authors of color during Black History Month or the role of women is mostly addressed during the month of March. Questions for discussion that events like this might provoke include: What are the implications when a teacher or a school’s approach to what they put in front of children is informed by the absence of a demographic group within the school walls or by particular events on the calendar? What do teachers need to know, be able to do, and value in order to to select the kinds of texts that will lead to a more accurate, fair-minded understanding of historical events and trends?

Six Months Ago: A transgendered student was told that the student would have use the restroom the school principal and teachers thought was appropriate for the student, not the restroom the student would have preferred to use.

In assuming a seemingly neutral stance, the school ended up denying an aspect of the student’s personal identity or may have creating shame or guilt in a young person struggling to protect her own emotional well-being. A similar defense around school policies (“It’s part of the dress code”) was used when a Navajo student was told to cut his hair (kept long for religious reasons) before returning to school. Questions for discussion that events like this might provoke include: What are the implications when our inherent inability to see through another’s eyes informs the policies and practices of school? Is empathy enough?

This Month: A teacher was suspended when, after the topic of Michael Brown’s death came up in class, a lesson involved having students act out Michael’s death. The re-enactment included researching the number of times the young man was shot and having students play the role of the officer who shot him.

In a powerful piece called “Facing Race Issues in the Classroom: How to Connect with Students”, an educator reminds the reader, “We may not be able to prevent everything, but we can control how we react to things.” Questions for discussion that events like this might provoke include: What are the implications when, having the best of intentions in reacting to student questions about a real and important issue, educators design tasks that in hindsight are clearly poorly informed? What steps can and should be taken before putting such activities in front of students?

In each instance, the educators involved were doing what they thought was best. Each of these teachers is also likely a family member, with friends and hobbies and pets. Based on the demographics of the American public education teaching force, the probability is high that these educators were white and female. It is also probable, that if asked, each woman involved would deny her actions were racist or biased[1].

Attending to the whole educator requires that we revisit who we think we are and how that fits into the larger social narrative and structures we exist, teach, and learn within. Learning more about how to confront our own biases and “blind spots” is the responsibility of all adults but given the impact that educators have, it’s crucial work for members of the profession. It’s also equally as important that as we do the work of broadening our conception of who we are as whole educators, we don’t infringe upon others’ emotional safety. It seems like a natural step to reach out to faculty members of color to discuss race or to Lesbian, Gay, Bisexual, Trans* or Queer (LGBTQ) colleagues around issues they may experience, but that small move is identified as “othering” by social justice advocates. Othering is where we use our frame of reference to define someone else’s identity. Instead of Ms. Jones being defined as a grade 5 Science teacher or however she self-identifies, she’s approached as Ms. Jones, a woman of color, based on how she’s seen by her white colleagues. Although intentions – seeking to understand – may be noble, constantly being “othered” can take its toll. [2] 

The students involved in the Student Six Tips in the “Facing Race” article shared that as a result of their teachers’ intentional work, they felt safer and found greater success. “The teachers treat us like peers and we respect that.” In a recent Twitter chat around LGBT issues, a teacher reported that a subtle shift in her language – from “husband and wife” to the more neutral “spouse” allowed one of her LGBT students to feel safe enough to confide in her the emotional and social challenges he was facing as a gay youth.

The work and heavy lifting of expanding the boundaries of the whole educator to include social justice and equity considerations has to be done by each individual. This work is not optional and should not happen in response to events like those listed. It needs to happen now, without hesitation, and without fear of saying the wrong thing, a common, shared fear. The hardest part may very well be admitting that for the majority of educators, discussions about race, gender, or sexuality are often event-based rather than a part of professional, reflective conversations. In order for us to answer the essential question, “How do we become a more just society?”  we have to start exploring the boundaries of our identities while seeking to understand others’ – even if, and especially if, the classrooms, faculty rooms, and media we see on a daily basis reflect mostly faces and experiences that look like our own. This work is important. The work must start now.

In addition to the resources linked in this column or referenced in footnotes, educators may find the following resources useful to inform their reflective practice.

	Quick Reads	Medium Length Reads	Long Read
Race in education and the classroom	5 Ways to teach about Ferguson  #educolor	“We cannot be color-blind” Race, Antiracism, and the Subversion of Dominant Thinking by George J. Sefa Dei	This is not a Test by Jose Vilson
LGBT issues in education and the classroom	#LBGTeach	GLSEN school resource guide	Raising My Rainbow by Lori Duron
Learning more about fairness in assessment design	“Identify and Eliminate  Assessment Bias” (video) by James Popham	Regents Exams Item Criteria Checklist	Gender bias and fairness by Ruth Axman Childs

---

[1] This article by Ta-Nehisi Coates, The Good, Racist People  or the TedX Talk by Jay Smooth are two great resources on the important distinction between racist actions and “being a racist”.)

[2] The blog space maintained  by Jose Vilson speaks to his experiences as an educator, parent, and man of color, and frequently makes connections to larger social issues.

Originally published in the NY ASCD newsletter (September 2014) 

Pick an item, any item - the reveal

Yesterday, I wrote a quick run through of item p-value in which I ignored a bunch of stuff about item analysis in order to focus on the big idea of predicting item difficulty. 20 anonymous people - of unknown age, teaching experience and background, answered a quick poll I put up about two items from the released NYS items. 20 adults (I assume -and based on their responses to the optional question, it's a reasonable assumption) picked which item they thought was harder. And 17 adults - about the size of an elementary school PLC - or 85% of them picked Item 1. Reasons for picking 1 included:

• To answer question 1 you need to go off and look up the contents of the 4 referenced paragraphs and then keep them in mind while evaluating the most plausible answer. This would put more strain on the working memory. The content needed to answer question 2 is right there in the question, easily visible.
• Question 1 requires students to go back into the reading selection; that is not required for Question 2.
• "Predicts the action" is awkward phrasing, question requires students to flip back into the story to reread. Also, question number 2 is a more familiar format

All reasonable. All made by (again, I assume) well-educated adults using the evidence in front of them to draw a conclusion. And yet, the reason we need student item data...

49% of NYS 4th graders who took the test got this question correct. 

25% of NYS 4th graders who took the test got this question correct. 

Does this mean that the 20 adults don't understand teaching, students, or education? Not by a mile. It's a reminder that when it comes to assessment - especially multiple choice item design - when adults read items we see difficulty different than the test taker. Think one of the released items is especially hard? Check the p-values by using the released charts. See the small blue number in the top left? That's the item's code. Look for that code on the p-value chart. If you're in a school in NY wondering how to make connections to your students, look at your students' p-values in the reports released by the RIC and start to have conversations about the implications. SED has released guidance on how to analyze the data and educators across the state are writing (written by my co-blogger, Theresa) about how to state assessment data to inform conversations.

State ed testing takes 300 minutes, 1%, 2 weeks - however you chose to present the numbers - of a child's year. It is the LEAST important assessment children will engage in over the course of a year. What's to be gained - or lost - by framing the LEAST important thing students do as a way to advance our agendas? How does it help alleviate students' (and parents') stress when we give the LEAST important thing in the education landscape the most attention? 

Disclaimer: And as you read these posts, please know, gentle reader, that I am an advocate of performance-based, portfolio, and authentic assessment. I love roses but have committed to the science of the teaching profession which means working to ensure we're describing the daisies correctly. So the usual disclaimer - I am not defending NCLB, VAM, APPR. I'm not even defending the NYS assessments. It's my hunch that we're making it harder to fix the big picture when we neglect to accurately define the parts of the whole.

Pick an item, any item

So we're going to play a little game in this post. But first, let me set the stage.

While lurking on a Twitter exchange about race, education, and schools, I saw a great reminder from Bill Fitzgerald scroll by. In effect [and apologies, Bill, if I've summarized incorrectly], it's worth engaging around important topics even it's clear the discourse isn't going anywhere because you never know who might be listening, seeking to learn. To revisit an earlier post, I decided not to worry about the manager of the nursery, and consider instead the walkers just out for a Sunday stroll who may overhear the discussion about the daisies.

I am going to make one claim here in this post and one claim only: when adults look at multiple choice items, we see them differently than students do. Experience, background knowledge, expertise, confirmation bias, 20 years of living - a wide variety of things influence how we read an item. Any teacher who's seen students ace a "hard" item or tank on an "easy" one will know that it's not until students actually take the items that we get a real sense of the item's difficulty.

Item design is a science - and an art. Objectivity plays a large role. BUT:

One cannot determine which item is more difficult simply by reading the questions. One can recognize the name in the second question more readily than that in the first. But saying that the first question is more difficult than the second, simply because the name in the second question is easily recognized, would be to compute the difficulty of the item using an intrinsic characteristic. This method determines the difficulty of the item in a much more subjective manner than that of a p value. Basic Concepts in Item Design

This is why there's field testing. Or why we should field test classroom tests and why states have to field test items from their large scale tests. The test designer (teachers or Pearson writers) do their level best but we need certain statistics (available only after students have actually taken and responded to an item) to reach conclusions about how high quality an item is. The most common statistic we can use is what's known as a p-value. This value is the percent correct - the higher the p-value, the more students who got an item correct, the easier the item was for the group of students who took the test. There are guidelines around p-values but generally speaking, "p-values should range between 0.30 and 0.90." There's a lot more to unpack around item difficulty but we'll just leave this here.

In the absence of these p-values, our observations about the difficulty of an item are just that - observations. Hambleton and Jirka (2006), two psychometricians/unicorns reviewed the literature around estimating item difficulty and found studies where highly qualified, well-experienced teachers were inconsistent when it came to accurately estimating how students would do on an item. "No one in any studies we have read has been successful at getting judges to estimate item difficulty well." Pretty compelling evidence that we need to temper our opinions with supporting evidence from students who, you know, actually took the assessments.

So now onto the game. Let's pick an. any item. How about Item 132040149_1 from the released Grade 4 Assessment items?

Now, in order for this game to work, you have to play along. Click the link above to read the Pecos Bill story and do your best to answer the question. You may look at it and conclude it requires skills "required skills out of the reach for many young children" or that the number of steps to answer this question are too many and too complicated for 4th graders. Now consider the question below also from the fourth grade test:

Which one would you expect to be harder for the students? The top one or the bottom one? What's informing your decision making? What evidence are you using? What percent of students do you think got the top item correct? How about the bottom one?

Hit me up here or on Twitter and share your thinking. I'll follow up with the answer in an upcoming blog, provided I get through my rose-tending to do list.

The reveal is here.