Each year, when Ohio’s School Report Cards are released, the reporter from Cleveland.com reaches out to local superintendents for comment. Traditionally, those in my position — myself included — have emphasized that students are more than test scores. Our local reporter knows my feelings on this topic, and over the years my responses have ranged from a simple “our students are more than test scores” to more pointed remarks.
Recently, what I intended as a sarcastic, tongue-in-cheek comment was misunderstood, and I want to clarify. In the Cleveland.com piece, I said:
“Our district needs to attract more wealthy parents and residents… so that we can score higher.”
This was meant to be a critique of Ohio’s accountability system, which often rewards wealth rather than reflecting the true quality of a school district. When sarcasm doesn’t land, it can hurt. I’ve since learned that some in our community — especially those who don’t consider themselves “wealthy” — felt slighted by my words. That was never my intention. Those who know me personally understand this, but I want to be clear with everyone.
I grew up in Sheffield Lake, Ohio, with my two brothers and our mom. We lived in a very small, 900-square-foot house on Howell Street that no longer exists — and probably should have been condemned even back then. My mom worked as a clerk for the Lorain Court System, earning about $20,000 in 1980, and did the best she could. We lived on government assistance: free school lunches, food stamps, and sometimes free kerosene to heat the house in winter. Our home had no foundation, no garage, no paved driveway — just gravel. In the coldest months, my mom would turn on the oven in the mornings to warm the kitchen. When our pipes froze (often), my uncle, a welder, would come with a blowtorch to thaw them. Once, a family of skunks moved in under our house. I can still remember going back to college with the faint smell on my clothes. The same happened to my brothers. We never revealed the source of the smell to our classmates.
I don’t share this for pity. I share it because it’s my story. It shaped who I am and the people I advocate for. If anyone felt hurt or looked down upon because of my words, I sincerely apologize. As the grandson of a Pennsylvania coal miner and an autoworker — neither of whom made it past middle school but who were remarkable, hard-working people — I deeply understand the pride in doing the best you can with what you have.
My comment about attracting wealth was not about diminishing anyone’s worth. It was meant to highlight the unfairness of Ohio’s current accountability system: one that often equates financial means with educational success. Districts in wealthier areas “outperform” because of socioeconomic advantages, not necessarily because they serve students better. Cleveland.com reporter Rich Exner has illustrated this issue well in this article.
For more than a decade, I’ve fought for a fairer, more accurate way to measure school quality. That passion can sometimes sound edgy or even frustrated, but it comes from wanting kids and communities to get honest, meaningful information. Here in our district, we’ve been working hard to build a better accountability model. If you live in our community, I encourage you to read Our Story to understand what we’re trying to do — and to help us create a more truthful and equitable way to talk about the great things happening in all public schools. In Olmsted Falls we embrace truthful accountability and believe that there are far better ways to demonstrate what we’re doing and how we’re doing it as a way to build trust. Please take a look at what we’ve built (here) and let us know what you think.
Last week, the State of Ohio released its “report card” for each school district and the schools within them. Ohio’s report card is the system the state uses to weigh in on the quality of what districts are doing in six categories: Achievement, Progress, Gap Closing, High School Graduation, Early Literacy, and College, Career, Workforce & Military Readiness.
Because the Ohio Revised Code required it—and also because they couldn’t help themselves—a group of people sat around a table and assigned arbitrary weightings to each of these six categories to develop an Overall Rating. The overall rating is meant to boil everything down into something that looks good to hang on a wall or drop into a newspaper headline. But doing so creates a system that reflects the socioeconomics of a community more than the actual quality of its schools. Also, remember that the assessments aren’t the bad guys. It is how the assessments are being used that makes this model a bad way to rate the quality of schools. These assessments were never designed to rate and rank.
What determines how many stars end up in their accountability soup? Good question.
Achievement is measured through assessments given to students beginning in 3rd grade through high school. In grades 3–8, they test reading and math. Science is tested in grades 5 and 8, and in high school, end-of-course exams are given in Algebra I, Geometry, U.S. Government, U.S. History, Biology, and English II. Scores are standardized, and then a group of people assigns descriptive titles to student performance.
Progress is measured through Ohio’s Educational Value-Added Assessment System (EVAAS). This system is based on the idea that a student’s prior test performance serves as the baseline for future test performance, attempting to calculate whether a student made expected, less-than-expected, or more-than-expected progress. It was a good model in theory (I even wrote my dissertation on its implementation years ago and once had dinner with its creator, Bill Sanders). However, Ohio ruined it years ago when it stopped measuring individual progress and instead standardized growth by putting it on the normal curve. That change automatically creates winners and losers.
Here’s why: basic statistics. Once student growth is put on a bell curve, the lowest score (regardless of what a student actually knows) becomes a “1,” and the highest score becomes a “99.” The normal curve doesn’t measure knowledge—it just ranks scores. Statistically, there must always be a “1” and a “99.” So the notion that all students or groups can make progress is a lie.
Bill Sanders’ original EVAAS model wasn’t intended to pit schools and districts against each other. It was designed to identify which students were making progress at the individual and group level, then study what teachers in high-growth classrooms were doing so their practices could be shared. What exists now is, frankly, harmful.
Gap Closing is next—and it’s ugly. I’m a former school psychologist, which means I grew up as a psychometrician and consider myself pretty adept at student assessment. In short: I’m a test dork. And I’ll tell you this—if you’re a measurement person and you study how Ohio’s Gap Closing metric was built, it will make you want to vomit. Honestly, it feels like something cobbled together at midnight on the back of a napkin at an OSBA conference bar. It combines subgroup test scores, graduation rates, gifted student performance, English language proficiency improvement, and chronic absenteeism. It’s Ohio’s equivalent of a Harry Buffalo party: a messy mix that produces more confusion than clarity.
I’ll be blunt: as a superintendent, former assistant superintendent of curriculum, instruction, and assessment, and a former school psychologist who ran RTI teams, I can’t imagine a more useless component. Even with my background in statistics and assessment, I couldn’t coherently explain it to you.
Early Literacy and Workforce Readiness are a hodgepodge. Yes, literacy and readiness matter, but Ohio’s model for literacy progress is broken. It relies on standardized tests, which by design will always place some students at the top and some at the bottom. That structure undermines Ohio’s professed dedication to the “science of reading.” Readiness at least shows some promise because it’s criterion-referenced. Finally, graduation rates simply reflect what this accountability model really measures: community income levels. High-income communities have high graduation rates; low-income communities do not.
Reporter: “So, how do you feel about your district report card? Do you have any comments?” Superintendent: :::thinks and engages in self talk::: “There’s what I want to say and what I have to say.”
Here’s the truth: the more money your community makes, the higher your district rating will be. That’s it.
Districts like mine have the luxury of ignoring the results and focusing on deep learning, engagement, and preparing students for life. But districts under the thumb of the state—those without wealth—have to double down on test prep, curriculum narrowing, data dives, and constant compliance. The deck is stacked against them. Those of us not as impacted need to speak up for those who cannot.
So how did we do? As a district, our per capita income is $47,511. Guess how we scored?
Should success within a state’s accountability model be something obtainable for all districts in a given year?
Answer: It should be. But Ohio’s isn’t.
The current system is built on standardized testing and rests on a bell-shaped curve. For the top 10% to exist, there must also be a bottom 10%. On Ohio’s tests, if 60–70% of students get an item right, that item is thrown out as “too easy.” A “good” test item is one where half the students get it right and half get it wrong. That means success for all is impossible—the tests are designed to sort first, not measure mastery.
If Ohio cared about academic competency, it would use criterion-referenced tests that measure what a student knows, not how they rank against everyone else. Think of it like parachute packing or performing surgery. Either the parachute deploys or it doesn’t. Either the surgery works or it doesn’t. Those are clear, obtainable standards. We could do the same for reading, math, and every other subject. But that’s not what we do.
Fine… Stop Complaining & Offer an Alternative
I’ve spent years chasing better ways to determine school quality. Two scholars I follow closely are John Tanner (Brav-Ed) and Jack Schneider (MCEIA). When building an accountability model, we should ask:
Should it be correlated with variables that predict long-term student success?
Should it provide stakeholders with timely, actionable evidence?
Should it be relevant and understandable to the community?
Should success be obtainable for all districts in a given year?
Should it be highly correlated to income levels?
Should it be truthful and honest?
Should it help build trust between districts and communities?
Now apply Ohio’s model to those questions. It fails the test. It prevents Olmsted Falls from realizing its locally created vision. Earlier in my career I worshiped at the altar of the bell curve. I’ve since recovered.
Why This Matters
Test-based accountability is institutionalized at the federal and state levels. Both political parties own this. It’s in law, and changing it requires changing law. States can petition to try alternatives, but Ohio hasn’t. So districts that want to innovate must do so while still trapped in this broken model that has never improved student outcomes.
At the very least, decades of poor results should have led to change. But I’m from Cleveland, so I get it.
As a local practitioner, I’m frustrated but also motivated. I feel a moral obligation to speak out against a model that doesn’t help students. More importantly, I want to be part of the resistance to create something new—something that reflects our vision to inspire and empower students.
We’ve begun asking our stakeholders what their hopes and dreams are for students, then creating evidence-based ways to report progress back to them. And here’s the thing: when you ask families, employers, and communities what they want for kids, they don’t talk about test scores. They want the basics, yes. But more than that, they want well-rounded, prepared, engaged young people.
Students spend 1,080 hours a year in school. Accountability models reduce all of that to a few 70-minute tests. That may be convenient for realtors and politicians, but it’s a disservice to families and kids. It erodes trust in schools. We can do better—and we must.
The triumverant at Ohio’s new DEW (DeWine, Husted and Dackin) have been sharing their newest and best idea ever discovered–all school districts must implement a cell phone policy. In my opinion it is a group of guys who are out of touch and want to spend time making laws and policy where there are already local policies in place. Some of these policies were created at a time when the current graduating Class of 2024 wasn’t even born. I will admit, we need to update ours and remove a few things– beepers, pagers and the BlackBerry (miss this guy). Historical photos are below. Our district policy, created in 2006 sits below the photos. ~Enjoy
This blog is a bit longer than I’d hope, but please hang in there.
Publicly funded school vouchers are an abomination. They are unconstitutional and, in my opinion, illegal.
If you live in the Ohio counties of Hocking, Athens, Clinton and Lawrence, let this sink in. A family of 4 with an annual income of $135,000 living in Cuyahoga County can receive $8,000 per child to attend a private, Catholic school in the Cleveland area and your tax dollars help pay for it. While that will not cover the full tuition for the child, (in 2022 that was listed at $18,360) it will subsidize it. What’s more is that the early data shows us that the majority of families taking voucher money never attended the public school in their community and many live in places where the public school districts are highly rated on traditional measures of school quality (examples–Rocky River, Westlake, Bay Village, Olentangy, Dublin and so on).
I have no problem with a family opting out of the public school system and seeking to send their children to a private school, or even choosing to homeschool them. While I didn’t do that, some parents choose to send their children to private school because of religious reasons. That’s fine. People can choose how to raise their children and how to educate them. The problem that I have is with the majority of Ohio Legislators using tax dollars to subsidize private schools. Listed below are a few examples to consider that are meant to illustrate my point regarding public services versus private choice. While it seems as though many have an issue with “government,” and the services that it provides, our American way of life was built on creating things for the purpose of contributing to the public good. Politically some would argue that the government should offer more to citizens, and some would say that it should be less. Regardless of your philosophical beliefs, I think that most would agree that a constitutional document should serve as the True North…that’s my opinion anyway. If a group does not like or agree with what is written in a constitutional document there is a process to change it.
The Public Park System: Example 1
At some point in time (and I’m not an expert on this subject), the Ohio Government decided that it was in the public’s best interest to create a system of parks for recreation. They allocated resources to create a public park system , and provided funding (only about $4.4 million) to ensure the parks were taken care of so that the public could enjoy them. If the local government chose to improve upon the park system that falls within its respective jurisdiction, they could go to their local voters and ask for additional resources to build upon the original system that was subsidized by the resources that the larger government body provided. The Cleveland Metroparks system does this.
There is no provision for citizens to say, “we would like our income arising from taxation to be used for a private park system (example–Cedar Point) because we want to choose our own.” A real life argument would be, “I don’t bike, play softball, use the walking paths and so on, so give me my money by way of a coupon to spend it elsewhere.” If private choice existed on this matter, a family who wanted to exercise choice could ask for a voucher to have their tax dollars go towards a private park system of their choosing. They could opt for a Cedar Point membership and fast pass, rather than the public system. As much as I like Cedar Point, I think I should have to pay for my admission to go there.
Fire and Safety: Example 2
At some point in time, the “government” thought it was in the best interest of the public to have a fire and safety system put into place so that the citizens that live within a particular geographic boundary have services available to protect them from things like fires, crime and so on. The larger government body (federal and state) provided a baseline of resources to provide this service to the public. At the local level, the government had the freedom to choose to build upon, or augment those services. If they chose, the government could go to the local voters and ask for additional resources.
There is no provision that allows Ohio citizens to say, “We don’t like our safety services. We would like our State tax money to be used to fund a private fire and safety force because we want to choose our own.” While an odd argument, if a family wanted to make a personal choice and use their own money to hire an armed security guard, or have a person on standby to put out any fire or transport any family member to the hospital, they could do so. If they chose this, it would be at their own expense. As nice as it might be to have my own police officer to keep my family safe, I think I should pay for it with my own money. That would be my choice.
Education: Example 3
At some point in time, the Federal government decided that it was in the best interest of our Nation to ensure that its citizenry were an educated group. There are a number of reasons one could imagine why this was a good idea. For instance, the more “educated” a group of citizens are, the more likely they are to create things that would benefit society as a whole. While the Federal government didn’t want to fully fund this, they did provide resources, and then indicated in the Constitution that it would be up to the states to determine what they should do if they wanted to augment it. In short, in the US Constitution Amendment 10 tells us, “the powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.”
According to my research, in 1961, a group of elected officials, on behalf of the people of Ohio, wrote how education should be addressed and they codified it in the Ohio Constitution. Specifically, in Article VI, Section 6 of the Ohio Constitution they said–
The General Assembly shall make such provisions, by taxation, or otherwise, as, with the income arising from the school trust fund, will secure a thorough and efficient system of common schools throughout the State; but, no religious or other sect, or sects, shall ever have any exclusive right to, or control of, any part of the school funds of this state. (OH Const art VI § 2)
Common schools means schools that could be accessible by the public (i.e. public schools). The thorough and efficient system makes a “quality” argument that has already been debated and ruled upon by Ohio’s Judicial System. The “no religious or other sect, or sects, shall ever have any exclusive right to, or control of, any part of the school funds” seems pretty straightforward to me. The school trust fund should NOT go towards a “religious or other sect, or sects” nor “shall they even have an exclusive right to… (see what’s cited above).”
Where Am I Going With This?
A citizen (i.e. a parent) has the right to privately choose many things. He or she can choose where they shop for clothes and groceries, where he or she may go on vacation, the doctor one might choose, and so on. When one makes a private choice, it is understood that said private choice is paid for by the individual. There are certain things that society (and in this case, it is codified in the Ohio Constitution) has indicated ought to be provided as a matter of law. As it relates to schooling, it is clear–Ohio should provide for a thorough and efficient system of common schools.
When it comes to a public park system, emergency and safety services, and a thorough and efficient set of common schools, these are things that are supposed to be available to Ohio’s citizens because of the system that has been set up to provide for them. In short, our tax dollars pay for public education. They pay for a safety force and they pay for public parks. They do not pay for private security guards, fire-fighter people, and they don’t pay for my Cedar Point pass. Public services are meant to serve the good of the public in an organized society, and the government has a responsibility to use our tax dollars to provide them. According to how our system is set up, people aren’t permitted to say “no thank you” and take their tax dollars and funnel them to private entities under the banner of “choice.”
If the Ohio Constitution says that the State Government is required to provide a “thorough and efficient system of common schools” and that “no religious or other sect, or sects, shall ever have any exclusive right to, or control of, any part of the school funds of this state,” why is appropriate (one could even ask, why is it even legal) for the government to earmark funds to pay for private school tuition–especially when the majority of those schools are religious?!?
A Final Example
Here’s a final example of why this shouldn’t be permitted. I know a senior citizen who lives in my school district. He has a pretty nice house and pays his taxes. His property value is high because he keeps his house in fine condition and the public school district where his house is located is pretty good as well (I’m biased, but it is). Good schools increase property values. While he has no students in the school system, under Ohio’s current Voucher Program for all, his tax dollars are paying for someone else’s choice to attend a private, parochial school in Cleveland. He’s told me that he isn’t happy about this, and for the life of me I can’t explain to him why it is appropriate that the Ohio Legislators have consciously chosen to disregard the Ohio Constitution. He benefits from a strong public school system. He does not benefit from an out of control voucher system.
If you want a choice, know that you’ve always had one. Like your grocery store that you choose to shop at, or the doctor that you choose to go to–families should use their private money to pay for private schooling. Respectfully, you shouldn’t expect those who pay into a system that benefits the public good to subsidize your individual choice. There is one pot of tax resources to fulfill the Ohio Constitutional obligation of a system of common schools and it is being split up to pay for private choice because a political group has a stranglehold on what is permitted to occur and what is not. I think that’s wrong and, I would argue, it is unconstitutional.
The point of this article is to illuminate how poorly designed Ohio’s accountability system is by using a very personal example.
The legislators from Ohio continue to tinker with the educational accountability system and in 2023 those of us that are forced to operate under this compliance driven model will be subjected to a new and improved reporting style that looks much like a “Yelp Review.” They will tell you that superintendents and other educators were involved in the creation of this “new” system, however the reality is the success of the project was doomed from the beginning because 1) innovation was shackled, 2) the right people weren’t at the table and more importantly, 3) the right questions weren’t being asked. Rather than being disruptive and starting over by reimagining and transforming, the group simply put a new shade of lipstick on the same old pig, and when you do that you’re left with what you started with (my apologies to all pigs).
Based on the tone of this writing you’d probably expect my district to be one that did poorly under this revised system, however that is generally not the case. Olmsted Falls Schools received a 4.5 out of 5 star review from Ohio. Receiving a favorable rating is advantageous for me because it provides us with a reasonable amount of cover to criticize the model for what it is–at best a complete waste of taxpayer resources, and at worst, a lie telling, spirit crushing machine. Generally we have done well under Ohio’s accountability model. A model that many believe is riddled with institutionalized racism and classism. In general the deck is stacked in favor of a district like mine so I’m going to load up and let go over the next several weeks because I think people need to know how badly it’s flawed.
Our school district is unique. We are relatively small (about 3,600 kids) and have 5 school buildings. Our schools are arranged in the following manner: OF Early Childhood Center has grades PreK-K; Falls Lenox Primary has grades 1-3; OF Intermediate School has grades 4-5; OF Middle School has grades 6-8 and OF High School has grades 9-12. We like how our grade levels are arranged. It allows our teachers to really focus instructionally on the developmental levels of the students specific to the school, and each building has a great sense of community.
Each of our 5 buildings will be subjected to Ohio’s Compliance-Based Accountability Model and each will have a school report card. I could devote a dissertation as to why Ohio’s accountability model lacks truth, precision and the robustness that it should have in order to provide communities with a real understanding of what is going on in their schools, but I will not. I’m going to devote this space to demonstrate why one size does not fit all AND the harm that Ohio’s model causes my colleagues that work within my school district and the families that continue to move into the Olmsted Communities.
What’s in Ohio’s Rating System?
Ohio’s reporting system provides an Overall Rating that is composed of 6 Components: Achievement; Progress; Gap Closing; Graduate Rate; Early Literacy; and College, Career, Workforce & Military Readiness. Four out of six use student standardized test scores in the component. Some of the components are grade level specific (e.g. Graduation Rate is specific to a high school not an elementary school and Early Literacy is specific to elementary not high school). Below is a screen shot from Ohio’s reporting system that shows how many stars the district will receive and from which components. Remember that this rating is “rear-view.” It is based on data obtained from last school year.
Most primary schools in our state are arranged with more than 1 grade level. Generally they contain grades K-2, K-3, K-4 or K-5. The Olmsted Falls Early Childhood Center (ECC) has 1 grade level (kindergarten) and a wonderful preschool program that serves students from ages 3-5 with special needs and peer models. While Ohio’s preschool programs have a compliance-based accountability system, that system is separate from the K-12 system. Due to the unique grade level configuration at the ECC (it contains 100 preschool students and approximately 215 kindergarteners), it will be rated and judged inaccurately and unfairly because the system has many missing components.
The ECC is a school that has only preschool and kindergarten students so it does not have a State math or reading test which eliminates Ohio’s report card components of achievement, growth, and early literacy. When you think about that it is pretty absurd to imagine a state system that’s supposed to report on school quality being incapable of accounting for and reporting on the achievement, growth and literacy of its youngest learners when research would suggest that the earliest years are the most critical years when it comes to student development, but I digress. Shown below is what Ohio will tell our community. They’ll say that our Early Childhood Center is not good. It will receive a rating of 1 out of 5 stars. Not only is this inaccurate and damaging as it relates to the quality of this particular school, it’s a damn lie which makes it even worse.
The rating for this school will be based on only 2 factors–whether the school serves 5 year old’s identified as gifted (which doing so is just plain dumb and this is coming from a former school psych who has evaluated quite a few kids) and what Ohio calls, “chronic absenteeism.” Chronic absenteeism happens when kids miss too much school regardless if those absences are excused or unexcused. So…if you have a young child who has an underdeveloped immune system, is prone to ear infections, colds, COVID, whatever, and the child misses “too much school,” the state takes it out on your school district. Imagine doing this post COVID-19 when we’ve told families to keep sick kids at home so they’re not coughing all over each other.
The picture below shows how the school chronic absenteeism indicator is generated. I’ll be honest. I have a doctorate, am a former psychometrician and have been involved with school accountability for a long time. I can certainly follow along, but the picture below is “Educational NC-17.” If our school building received a 1 star as a result of this indicator I think it would follow that Ohio should get a “0” as a state. Does it surprise you that the absenteeism rate has increased post-COVID?
The point of this article isn’t that school attendance doesn’t matter or gifted education isn’t important because both are. The issue is releasing a state endorsed school quality measure that is so poorly designed that it cannot account for the quality of a school. The repercussions of a system that uses gifted service delivery and who kept their kid home the most in order to publicly pronounce the quality of a school is a system designed by lazy people who do not understand the complexity of determining school quality. Ohio kids and parents deserve honesty, precision and so much more. Be better!
I’ll end with this…in Olmsted Falls City Schools we’ve been engaged in creating a more innovative and transformative approach to communicating what we’re focusing on and how we’re doing. We’ve engaged our families and students and asked them what their hopes and dreams are. Our new system for accounting for what we’re doing will be based on those hopes and dreams. We will be launching this new approach in November and will communicate to our primary stakeholders more specifically about what we’re trying to do in a language that is easily understood and free of the educational jargon that educators love and parents do not.
Parents with preschoolers and kindergarteners at the ECC have hopes and dreams for their kids that include early literacy development, the ability to think mathematically, begin communicating through print and all kinds of other things. We will account for those things, show that evidence to our community and celebrate our youngest learners. Everything that Ohio’s accountability model cannot do, we will do. In the meantime, I hope that those that created, endorsed and voted for this new yelp model stub their toe tonight because your model has done harm to students, families and staff. Listed below is the kind of trust building, truth telling machine we are looking to create in order to tell a more complete story.
We believe that it is our job to inspire and empower students to make a contribution to the world. We believe that engaged students are ones that can think critically, are creative, collaborative, communicative, self-confident and self-directed citizens. In Olmsted Falls, “different on purpose” means that our emphasis is on providing our kids with a comprehensive experience through academics, the arts and athletics. When we asked our families what matters most to them, they described the benefits they seek from the school district. Those benefits fall into 7 broad categories and include: Student Learning; Student Readiness for the Future; Engaged and Well-Rounded Students; Safety & Well-Being and Connections to the Community; Effective Systems and Effective Adults. It’s not that test score performance doesn’t matter. It’s simply not enough and we believe that our community deserves a more holistic report on the value that their school district provides to our most precious resource–our kids.
I think the question is really why should we chase them? The entire foundation of basing school quality on test scores is a lie.
Test scores were never designed to rank schools and weigh in on school quality.
Test scores are influenced by too many outside factors and primarily driven by parent education and family income.
Test scores predict other test scores and do not predict how a student will be successful later in life. Shouldn’t time be spent on developing something that would predict this?
It is possible to “beat the test” and falsely raise test scores. We used to do this by teaching to the test and what it did was shrink the curriculum. Rather than teaching critical thinking and problem solving skills, we taught students to solve test-like problems–there’s a big difference there.
Chasing test scores does not inspire and empower students or teachers. It lowers school climate and promotes practicing problems to chase a number and leads to score inflation.
As a school district we’ve made a conscious choice to stop chasing scores and seeking to beat the test. District rankings do not provide us with any tangible reward. You don’t “win” anything for a ranking and quite honestly we believe that chasing test scores creates demoralized educators and students that don’t feel a sense of purpose for what they’re doing.
Standardized Test Scores Fall Short When Measuring School Quality
Standardized test scores don’t really tell us what skills students know or what they can do; rather they give us a score that serves as a proxy (a representation) for what the test is trying to measure (e.g. where does the student rank compared to other students who took the same test in reading, math, etc.). Sure, tests can be useful when planning intervention for individual students, however elected officials use them for purposes they were not designed for (i.e. ranking school districts), and most elected officials do not understand the harm that they are doing by using these assessments for a purpose they weren’t designed for. While a school and individual teacher have an impact on student achievement, extensive research has demonstrated that factors that are outside of the school’s control have a more significant influence on test scores. Median household income and parental educational levels are two such influences that have a profound impact on how students score.
For instance, consider Ohio’s top 20 highest family median income communities. They had an average achievement test pass rate of 86.04%. The highest median family income in this group was $234,423 and the lowest was $122,078. Conversely, Ohio’s lowest 20 family median income communities had an average achievement pass rate of 37.49%. The highest median family income in this group was $42,257 and the lowest was $35,191. The passage rates essentially transform into Ohio giving a rating to which school district is better than the other. Are they measuring school quality or something else? The communities that have higher wealth score high. Those that don’t, score low. Those districts that are relying on their high test scores to tell them how good they are may not be as great as they think they are because the score influencing factors external to the school may be weighing heavily on student test performance–more so than the educators inside their school buildings. As a result, a school district that relies exclusively on these kinds of scores to boast about school quality may be overlooking key components of school quality. In short, high test scores may not be attributable to the quality of the school or the instruction within it. They may merely be a function of how much money parents make and the opportunities that those parents provide to their children. In addition, schools that have had lower test scores when compared to those districts with wealth and opportunity may be deemed “bad schools,” when in fact they are making significant impacts that aren’t detected by standardized test scores. They in turn could be abandoning promising practices because those practices may not be immediately impacting the standardized test scores they’re looking at.
Criterion-Referenced/Competency-Based Tests Serve The Purpose
There are assessments that are designed to assess student skill along a continuum. They’re called criterion referenced or competency-based assessments. State tests are not criterion referenced or competency-based. What are these kinds of assessments? They are assessments that evaluate students against a pre-specified criteria without referencing how the student compares to others. Does the student know all 26 letters of the alphabet? Can the student identify all shapes and colors? Does the student know how to blend sounds? Assessments like these tell us what kids know, not how they compare to others.
If you want to know whether or not kids are literate, use a skills-based assessment that makes this determination, not one that simply ranks kids on how high or low they scored.
Assessments developed by teachers are also criterion referenced or competency-based. Can the student perform 2 by 2 subtraction problems? Can the student explain how a cell divides? Teachers want their students to be successful because it shows that the teaching worked and the students learned. The hope with any classroom assessment is that all of the students in class will demonstrate mastery. This is NOT how standardized achievement tests work. Standardized tests aren’t meant to determine mastery of a skill. They are meant to rank students and the government uses them to rate school districts. If you want to know whether or not kids are literate, use a skills-based assessment that makes this determination, not one that simply ranks kids on how high or low they scored!
You may have seen that Ohio has released its School District Report Cards recently and articles on the local news have appeared. The district’s position is in opposition to using this method to rate the quality of schools and school districts. Realtors and newspapers like this kind of system because it allows them to neatly talk about which districts are “good” and which are not. While we aren’t opposed to assessments that provide us with information on how students are performing in academic areas (yes…tests can be useful), using tests to rank schools and make judgments on their quality is inappropriate, unethical and counter-productive.
The Nation Wasn’t At Risk
The creation of the testing movement began after a report that was commissioned by the Federal Government was released that painted a very negative picture of public education. That report, called A Nation At Risk, was published in 1983 and, plainly, it lied. It had a profound impact on many important things that occurred in public education beginning in the 80s through present day. I recently had the opportunity to hear first hand from the person that facilitated the group of individuals that took part in the meetings (Jim Harvey) and wrote the final report. He relayed the discourse of the meetings to a group of superintendents and said that the information the commission used to write the report was gathered and used in such a way to support the group’s preconceived beliefs. In short, they wrote a predetermined outcome and then pulled together evidence to support it.
In order to answer the question of, why don’t we emphasize test score performances, one needs to think about why these types of assessments exist in the first place. While the state assessments do give individual student performance results to parents, one of the primary purposes of the assessments is to rank school districts and this kind of assessment approach was never designed to rank or weigh in on school quality. That’s a concept that elected officials made up. In addition, because the assessments are standardized, it means they are designed to distribute themselves to create a bell shaped curve. For every high achievement score, there must be a low achievement score and because the growth measure (a.k.a. value-added) is directly related to the achievement test, that method of measurement is also flawed because it too is standardized so It is statistically impossible for everyone to have a chance at success in this kind of model. What transpires is again a bell shaped curve that is used to say, “this school is a quality one and this one is not; these groups of students are literate and these are not.” The problem with this approach is that inaccurate conclusions are reached.
Several years ago the district pulled together a group of external and internal stakeholders to revise our Vision, Mission and strategic plan. It was during that time that we began to brand our “Triple A” moniker which demonstrated our long standing commitment to educating the whole child. While our core business is student learning and achievement, we recognized that student participation in extracurricular activities such as the Arts and Athletics are two very important components to the Olmsted Falls School District experience.
As we thought about how we would be in a better position to actively live out our District Vision of Inspiring and Empowering all students to achieve their full potential and become meaningful contributors in a global society, we recognized the primary method to communicate our district success to the community falls short using traditional methods. Ohio’s school quality reporting system uses standardized achievement tests to tell a school district’s story and we believe that by doing so, it uses a very small portion of the student experience to make a judgment on the quality of that experience; something that we believe is very inaccurate and lacking depth. While test scores will likely be with us for the foreseeable future and they do play a part in determining student achievement, we believe that we can and should provide our community with a more accurate and truthful representation as to what we’re working on and how we’re doing.
Over the past two years our District Leadership Team, which consists of administrators and teachers, has learned a new methodology to relay to our primary stakeholders what we’re working on and the progress that we’ve made. While we are the first district in Ohio to undergo this transformation, other school districts in Georgia, Texas, Kentucky and Pennsylvania have been engaging in this work.
Why use this approach?
As I previously mentioned, standardized achievement tests sample a very small portion of a student’s knowledge and do so in a single sitting. The information that is obtained is very technical. We needed to find a way to account for our efforts in a far different manner than what happens in our test-obsessed world. Moreover, when we’ve talked to parents, teachers, students, community members and businesses about the kinds of skills and dispositions they desire for students in K-12 education, their responses were very comprehensive. In Olmsted Falls we’ve led with this question–What are your hopes and dreams for your students? The responses included the following: “I need them to show up; A solid foundation in reading, math, history and science; I want my child challenged; They need to develop their thinking skills; Intellectually curious; Find their passion and purpose; Have friends, be happy and feel a sense of belonging; Learn to be communicative, collaborative, creative, and self-directed.” When you ask people what they want from their school district they speak in the language of “benefits.” You buy a car for a number of reasons, but one reason is the benefits that it will provide you. You do the same with a phone, the hospital you choose to use and so on. As we continued to explore the hopes and dreams question with stakeholders, we found the answers we were given focused on the comprehensive benefits people expect the school district to provide.
We believe that by creating a new accountability system that is grounded in the benefits that people want and the district provides, would lead to a more accurate and much more truthful accounting of what we do compared to the method that currently exists. The reporting system that we’re developing will provide progress in language that parents understand rather than “educator-speak.” We will provide a report in non-technical language to those that want the information, and for those that want to take a deeper dive into the evidence, data and how we’re gathering it, we can take them on that trip as well.
When are you starting this and what can we expect?
We plan to launch our benefits-based accountability system this fall and will provide our first progress report in November. We will provide additional updates in February and May. While we will be required to continue to live within the test-based model our state requires, our community can expect to see reporting on the benefits that each school will select from 7 main categories: Student Learning, Student Readiness; Student Engagement and Well-Roundedness, Student Well-Being, the Effectiveness of our Staff, Community Connections and an Effective System. In totality there are about 31 benefits that fall within each of the 7 categories.
Each of our five schools have engaged their primary stakeholders to determine the benefits that will be their focus and while data will not be collected on all 31, our accountability system will report on those things that matter most as identified by our staff and primary stakeholders (parents and students). Below is an example of the format that we will use at the district level. The category in the example provided is from the Student Learning area and as you can see there are 4 main benefits within the category. Families can expect each building to provide an update on the benefits that were chosen from 7 main categories. While the 31 benefits are common across the district, the benefits chosen by each building will be unique.
We look forward to continuing to talk about this work and having conversations with our community to let them know that we are focusing on and gathering evidence on the things that they care about the most.
OFCS Superintendent's Update 