The public will never know how Smoky Hill High School ninth-graders scored last spring on English tests that challenged them to do things like interpret ancient Greek poetry.
Nor will it know how many fifth-graders at Monterey Community School in Commerce City grasp concepts like identifying a story’s main idea. Or whether sixth-graders at Ortega Middle School in Alamosa can puzzle out the complexities of algebraic equations.
Those results from last spring’s PARCC tests were among roughly 4,000 data points shielded from public view — the result of a new, more restrictive state policy designed to protect individual students from being identified. More than 1 in 4 data points from the math and English tests are not available for public inspection because of the year-old policy.
The move to redact more data from the state’s publicly available standardized test results is a dramatic shift for a state known for rich and easily accessible educational statistics. Inspired in part by the State Board of Education’s zeal for student privacy, the change has sparked a new debate pitting data transparency advocates against student privacy supporters.
“It’s really problematic that we don’t know how thousands of kids at large high schools are doing,” said Lisa Berdie, policy director for A Plus Colorado, a school reform advocacy group. “(These results aren’t) just for punitive accountability decisions. It’s so communities and students and families have a sense of how their schools are serving them and whether they meet grade-level requirements.”
Officials at the Colorado Department of Education stress that districts and schools are receiving complete data sets, and that parents will be provided with comprehensive reports explaining how their students and schools are performing.
The new rules, state officials acknowledge, are among the most stringent in the nation.
Department of Education officials and State Board of Education members say that the state remains committed to using data for accountability, and that the rules are designed to make it impossible for a member of the public to pinpoint how a particular student performed on state tests.
“The intent and the purpose of the rules are important to protect individual privacy and prevent the identification of individual students through the manipulation of the data,” said Colorado Springs Republican Steve Durham, the state board’s chairman. “And as far as I’m concerned, it’s more important to protect those individual students than give the press something to write about.”
What do the new rules do?
Before 2015, when the new rules took effect, Colorado’s data rules were pretty simple. If fewer than 16 students at a school took any test — say, fourth-grade English — the state would not release the results.
The new rules say that if fewer than four students score at any one of the exam’s five proficiency levels, the state must redact results from that level and results from at least one other level. (If just the one were blacked out, doing simple math would allow someone to easily fill in the blanks).
Consider this example: Twenty fourth-graders at a school take a test and four place at Level 1, two place at Level 2, five place at Level 3, four place at Level 4 and five place at Level 5. The state would redact the results for Level 2 and one other level and report the rest.
In the last round of achievement results this month, the state didn’t release school-level results by individual proficiency level at all. Instead, it placed students in two broader categories — those who scored in levels 1, 2 and 3, and those who scored in 4 or 5, meaning they met or exceeded expectations.
Students at levels 4/5 were reported. If fewer than four students fell into that category, the scores were suppressed.
The state took an additional step that rubbed some schools the wrong way.
If a school’s results were withheld on any one test — and it is the only school in the district with redacted results on that test — the state took the additional step of also redacting the results on the same test from another school (the one with the fewest number of scores in the district).
The state believes this step is critical because it would be possible for someone to subtract the school’s student population from the district’s overall results to learn the school’s results.
That’s why Smoky Hill High School’s ninth-grade English scores were withheld this year. Because the results from the Cherry Creek School District’s alternative high school, Endeavor, were redacted, the state withheld the Smoky Hill scores because Smoky Hill had the fewest valid scores on the ninth-grade test.
Cherry Creek school officials are not happy about the new rules.
“A school that had more than 300 kids testing isn’t a school that should have any stars,” said Judy Skupa, the district’s assistant superintendent, referring to the typographical symbol the state uses when scores are redacted.
What’s the actual privacy threat?
According to federal law, the state must redact data that could allow any person using reasonable measures, such as basic subtraction, to figure out how a particular student performed on a test.
Joyce Zurkowski, the state’s chief assessment officer, explains it like this:
Imagine a neighborhood middle school had 100 sixth-graders take the state’s math test, and not a single one met the state’s expectations. Under the old rules, it would be very easy for folks on the block to know, at the least, the student next door did not pass the test.
That, Zurkowski said, would violate student privacy.
Here’s a slightly more complex example: Say a school had 17 boys and 10 girls take the third-grade math test. While the girls’ scores would be redacted under the old rules, their results could be determined through simple subtraction.
“You don’t get to know how your neighbor’s child performed,” she said. “We needed to do a better job of protecting that individual student’s data, that we hadn’t been doing historically.”
The Colorado Department of Education says it never received a complaint about privacy violations under the old, less restrictive system.
What’s the concern about transparency?
Advocates for more data are worried that the new rules will prohibit the public from knowing two things: which schools are doing poorly and which schools are doing exceptionally well, especially with traditionally underserved populations.
“That’s information we need to know,” said Luke Ragland, vice president of policy for Colorado Succeeds, a nonprofit group that advocates for school reform on behalf of the business community.
Advocates lay out another scenario: you could have a school where most students aced the exam, but that data could be withheld if too few students placed in the lower categories.
That’s what happened at West Ridge Academy in Greeley, as the Greeley Tribune reported.
“We’re not just hiding schools that are underperforming,” said Berdie, of A Plus Colorado. “We’re also hiding success stories.”
An even greater fear is that as the state breaks students into subgroups — students of color, students who qualify for free or reduced-priced lunches, students with special needs — the data will increasingly be redacted because not enough students are scoring in each category.
Starting in 2017, states will be required to break down student performance data into even more subgroups to include students of military families and those who are homeless.
Elena Diaz-Bilello, associate director of the Center for Assessment, Design, Research and Evaluation at the University of Colorado Boulder, said it will be increasingly difficult to draw any conclusions from state test score results if so much data is held back from the public.
“I don’t think the state thought through all the implications,” she said.
Where does the state go from here?
Many advocates in the education reform community are hoping the state softens the rules.
“My biggest fear would be that in the privacy environment we’re in right now, we’d swing so far in one direction and no longer have an opportunity to recalibrate,” said Dan Schaller, director of governmental affairs for the Colorado League of Charter Schools.
For the moment, the new rules only apply to state standardized tests given in grades three through nine, Zurkowski said. Other important measures that go into a school’s quality rating are not affected — including the SAT, graduation rates and growth data that show much students learn year to year.
The rules are not explicitly required by any law — not even Colorado’s landmark student privacy law passed earlier this year. Zurkowski said the policy is influenced by the federal Family Educational Rights and Privacy Act and guidance from the U.S. Department of Education.
That leaves room for the rules to shift, which Zurkowski said is a possibility.
“It’s an ongoing conversation,” Zurkowski said. “I”m not saying we hit this right. I know we haven’t hit this right. And these rules will continue to evolve. In the end, we got to balance transparency and privacy.”
Photo by Nicholas Garcia, Chalkbeat Colorado.
Originally posted on Chalkbeat by Nicholas Garcia on September 23, 2016. Chalkbeat is a nonprofit news site covering educational change in public schools.
