There are 2 bills in front of Ohio’s elected officials that seek to provide partial reforms to its current accountability model. I liken it to chefs trying to create a very good soup and they’re arguing over the ingredients that make a particular soup a very good one. The problem in this particular case is that those taking part in the cooking are forgetting that at the end of the day, the soup needs to taste good in order to be great. In addition, some aren’t even qualified chefs and shouldn’t be in the kitchen in the first place, but alas. Sure, a person may begrudgingly eat the soup that’s presented, but if you want to create something memorable, sometimes you need to start over.

In this case the debate is over using a 5-tiered grading system (A – F) versus a 6 level system using stars. As it relates to systems of measurement (much like a grading system) you can expand such a system as much as you want. Do you want to use an “A-F” system or introduce pluses and minuses? Heck you could go B- or C++ or C+++, A—. but doing so does not make your system of measurement more precise or more valid. In fact, it stretches the measurement scale to the point where those you’re reporting to have no idea what you’re trying to communicate. So…both are bad because they don’t do what an accountability system should do…clearly communicate and build trust with the audience that you’re communicating to. Yes measurement should occur. We do need assessment results to help us understand where we’ve been, but imagine driving to a location and all you’re doing is looking through your rear-view mirror.

In the words of John Tanner, one of the challenges of a school reporting system is that a group of people who are inside the organization (those that have all of the technical knowledge) are trying to report on a great deal of information to those outside of the organization so that they can see the benefit of forming a relationship with that organization. In this case…the public school district with its stakeholders. The rub is, those outside of the organization have very little technical knowledge. Tanner’s True Accountability model enables “an understanding regarding the technical work within an organization for a non-technical audience.” https://www.brave-ed.com/blog When the information presented enables others to understand the benefit, trust is built. When it is unclear, trust is broken.

It is clear that in Ohio we have some significant trust issues because we continue to mess around with the same types of metrics (e.g. the same soup ingredients) rather than trying to do something brave…something just…something transformative…something great. So from my perspective, use 5 grades, 8 grades, or 10 grades. Or, introduce 5-stars, 7 stars or 9 stars. The model is still bad, the benefit of public education isn’t easily understood and the trust doesn’t increase. In fact, the system that has been in place for the past 20 years has created a sense of distrust with public schools and Ohio’s elected officials have used it as leverage to continually expand the concept of school choice. Doing the same thing over and over while expecting different results is Einstein’s definition of insanity.

I’ll end with a few citations from people who have shaped my understanding of the topic of organizational measurement.

Tanner’s most recent blog that resonated with me:
Effective organizations all over the world begin and end their accountability conversations with the notion of benefit and when schools can do the same on a regular basis the amount of information and understanding our stakeholders will have will far surpass anything that exists today. And if our stakeholders can understand what is happening in our schools, so can our policy makers. -John Tanner

Finally, to those who are part of Ohio’s accountability conversation from the Fordham Institute and Ohio Excels who claim to be representing businesses, I’ll remind you of what Jim Collins wrote–1) “Business thinking isn’t the answer and 2) Greatness is not a function of circumstance. Greatness, it turns out, is largely a matter of conscious choice and discipline, 3) It doesn’t matter whether you can quantify your results. What matters most is that you rigorously assemble evidence–quantitative or qualitative–to track your progress” -from Good to Great and the Social Sectors

People will read about Ohio’s Report Card/Accountability debate and you will hear things like, “this isn’t perfect but no reporting system is.” When you hear comments like I’d encourage you to put it into this context. Think about a major purchase that you’d make. Something that will impact your daily financial life and cause you to put a significant amount of thought and care into such a purchase. Will you say things like–
“Well this car isn’t safe, but no car is….this house isn’t well built, but no house really is.” Would you accept that? If not, why would you for your children?

Kids have but one journey through their K-12 experience. If you’re not willing to accept an answer like that for something you’re spending a great deal of money on, why are you willing to accept it from our policy makers and elected officials. Expect better. Understand the benefit of public education and demand that they do it right. Require them to “do it right or do it over!”

~Jim Lloyd, Ed.D.

The email below as sent to Ohio State Board of Education members and other interested parties on April 14, 2021. It is in response to a quote from a communication that originated from a document called Focus Education which is put out by the Hannah News Service, Inc. This news service provides information to subscribers about proposed Ohio Laws and policies. Those receiving this email are listed below. NOTE: I owe a debt of gratitude to John Tanner for his continual inspiration on what true accountability is. He is both wicked smart and truly inspiring to read and listen to.

Laura.Kohler@education.ohio.gov,
Charlotte.McGuire@education.ohio.gov,
Christina.Collins@education.ohio.gov,
Kirsten.Hill@education.ohio.gov,
Meryl.Johnson@education.ohio.gov,
Paulo DeMaria <paolo.demaria@education.ohio.gov>,
John Tanner <johnt@testsense.com>,
Diana.Fessler@education.ohio.gov,
Antoinette.Miranda@education.ohio.gov,
Tim.Miller@education.ohio.gov,
John.Hagan@education.ohio.gov,
Michelle.Newman@education.ohio.gov,
Brendan.Shea@education.ohio.gov,
Eric.Poklar@education.ohio.gov,
Mark.Lamoncha@education.ohio.gov,
Martha.Manchester@education.ohio.gov,
Stephen.Dackin@education.ohio.gov,
Paul.Larue@education.ohio.gov,
OFCS BOE <ofcsboe@ofcs.net>

Dear Mr. Toal,
I saw in a recent communication that comes from the ODE a quote by you that said, “you’re never going to have equity until you measure, it’s impossible.”  While I don’t completely disagree with your quote, I thought I’d offer some perspective to you from the field. I’ve been an educator for 27 years and a superintendent for 8. In a previous life I was a school psychologist and student measurement was my forte. I also wrote my dissertation on Ohio’s Value Added Assessment model. I share this with you because I have a good understanding of how to measure student learning and school quality from a traditional sense, and also from a more contemporary view. 

Both personally and as a school superintendent, I am not in favor of either bill that has been proposed to measure district or school accountability, and based on the models that are being proposed (both past and future), my district will likely benefit from the model based on the zip code of where my district is located. We are suburban and have a socio-economic situation that has traditionally benefited us with regards to school ratings. My district has the good fortune to be mostly immune from the consequences of not performing well on Ohio’s accountability model–both in the past and likely in the future. It would be in my best interest and the interest of my district to simply shut my mouth and wait for either model to move forward, however that would be wrong for me to do. 

Simply put, we will never “measure our way to equity”…not with the previous accountability models nor with the proposed ones. Measurement doesn’t improve school quality. Changing adult behavior improves school quality and we have over 2 decades of evidence to demonstrate that measurement does not improve equity of education. What has occurred during my professional career (and what is occurring now), is an exercise in “tinkering” with reform rather than a desire to be transformative. While I do believe people mean well and are trying to do the right thing, sadly I don’t think they fully understand the transformative work that needs to be done in Ohio and at the National level. Both of Ohio’s proposed models will do nothing more than put a new coat of paint on a failing and broken down car. It may look good for a few years, but it will run poorly.  You see, because of the zip code where I both reside and have the good fortune to be superintendent, I have the opportunity to work on many of the curricular initiatives that my educational colleagues do not because I don’t have to worry about accountability to the Ohio School Board or Ohio Legislature. While I would put our instructional program and how we conduct the business of school up against any of the top performing districts in the state, our focus doesn’t have to be centered on Ohio’s accountability model. In fact, my board has locally rejected the state’s model and have come out against any accountability model that relies on Ohio’s achievement tests to rate the “goodness” of what we provide to our kids. These same circumstances and conditions do not exist for my colleagues in East Cleveland, Cleveland City, Akron, Warrensville and other school districts that struggle with the reality of poverty. They do not have the good fortune of starting the accountability game with runners standing on second and third base like districts that have median family incomes of $90,000+. 

I would argue that Ohio’s current and proposed accountability models do not work for any school district–including the very wealthy school districts that do not have to worry about poverty. The students that come through the doors there come through with generally stable homes with both parents who are financially secure. Those students have benefitted from an enriched background that has a very well documented and proven direct and correlated impact on their student achievement from the very beginning of their formal schooling experience as measured through traditional standardized achievement tests and those assessments that we’re required to deliver to students in the spring.

I would argue that Ohio’s assessments do not do a reliable and valid job of measuring the impact of schooling because they’re skewed by the opportunity (or lack thereof) that home lives provide. I would argue that is the reason why Ohio’s accountability model does not work for students who come from a more privileged life AND those that do not. It is why I’ve tried to advocate for a more holistic accountability system and why I’ve spent the past few years creating one at the local level. It is why I will spend the last 9 years of my career advocating for something better.

While the State Board certainly deserves to understand how the system of public schools are performing, the models that have been used for the past 2 decades and the ones that will likely come in the future, will not provide you with any new information. You already know what the data will look like prior to implementing the proposed systems. The wealthy suburban’s will continue to do well and the poor urbans will underperform…and remember this is coming from a superintendent who works in one of the “wealthy suburban’s.”    

I’m sure that you and your colleagues care about the low performing students and want to do what you’re able to do to assist districts, however past (and likely future) accountability models in our state have done nothing more than rearrange the chairs on the deck of what appears to be a ship stuck dead at sea. I would encourage you and your fellow board members to learn more about true accountability and look to be transformative rather than tinkering with reform. Transforming the state’s accountability model will not be easy and it will be difficult, but it is very necessary if you are truly dedicated to seeking equity for Ohio’s public school students. By sticking with the past and future models you are not ensuring equity…you are sentencing districts to more of the same that will allow the suburban schools to focus on all of the good stuff and leave the urbans with test prep to improve their scores in order to get out from under the state’s proverbial thumb. 

Please consider what true accountability can offer Ohio’s students.  I’ve spent the past 4 years learning more and exploring something more progressive in order to hold myself and our school district accountable to our local community. It’s still new and only in its 2nd generation…the pandemic didn’t do us any favors. I’ve attached a few things here for you to investigate. I believe the State Board of Education in Ohio wants to make a difference in the lives of students, but I’m concerned that those pushing the buttons and pulling the levers are listening to politically charged groups who are trying to keep us in the 20th Century because they’re either afraid or they don’t know any better. Like many, I was in the accountability cave watching the shadows on the wall thinking they were something that they are not. It wasn’t until I left that cave and saw that there was a great big world of other ideas to consider that I became a real advocate for accountability.

Now is the time to be brave and to be transformative. You don’t achieve equity through measurement. You achieve it by transforming the behavior of the adults in the system…those that create the system and those that operate within it. Our kids need us to do what is right, not what is easy.  

Sources for a more holistic model of accountability:
https://www.brave-ed.com/
https://www.mciea.org/

Attachments (to the reader–>email me directly if you would like these…Google will find them for you too):

What Does a True Accountability System Look Like in Schools? by John Tanner

Building a Better Measure of School Quality by Jack Schneider, Rebecca Jacobsen, Rachel White and Hunter Hehlbach

2018-2019 OFCS Balanced Scorecard 2nd Edition

Thanks for your time and consideration.
Sincerely,
Jim

Dr. Jim Lloyd
Superintendent for the Olmsted Falls City School District
26937 Bagley Rd. Olmsted Township, OH 44138
Phone: 440-427-6000
Twitter: @OFCSSuper
Web: www.ofcs.net

Our VISION is to inspire and empower all students to achieve their full potential and become meaningful contributors in a global society.