In a postmortem, leaders of Denver’s Imaginarium say test scores, bureaucracy are stifling innovation

The Denver school district’s intense focus on test scores makes it nearly impossible to spread teaching practices that would improve education.

That was the conclusion reached by district staffers who spent years in the Imaginarium, an office dedicated to fostering innovation in Denver schools. The district closed the office earlier this year, in part to pay for higher teacher salaries. Officials who worked in the Imaginarium put their findings in a report back in May, but the district didn’t release it.

Now, as Denver teachers return to work, the authors, who no longer work for the district, have published it on an independent website, where they hope it will stimulate hard conversations about the changes that need to be made if the district is to truly serve its students, in particular the black and brown students who make up the majority of Denver Public Schools.

That will require moving away from standardized tests as the best measure of student progress, they wrote.

“High-stakes standardized tests instill fear and anxiety, which leads to a narrowing of the curriculum, leaving little room for personalization,” the report says. “To change the measurement system, a district must have the kind of bold leadership that pushes the boundaries of state and federal mandates on testing and accountability.”

Amy Keltner, Denver Public Schools’ chief impact officer, said the report was not released because the district considers it a first draft, one that shone necessary light on challenges but didn’t have enough information about lessons learned or next steps.

“It didn’t include enough insights about what we’ve learned about what does work,” Keltner said. “It did a great job calling out barriers that we have faced.”

The district plans to release its own version of the report within a few weeks, she said.

Denver Public Schools is Colorado’s largest school district, and it has a national reputation for trying new ideas, some of which are controversial. The district has seen its test scores rise in the past decade, but the scores of white students and those from wealthier families have risen faster than the scores of students of color and those from poor families, resulting in big gaps.

Without calling out individual district leaders by name, the report is critical of the bureaucracy that exists in districts across the country, including Denver. School leaders and teachers don’t have the autonomy to make real changes, it says. But rather than criticism, the report’s authors see it as a call to action for the district to do better by its students.

“To me, the greatest success to come out of this report would be this conversation of, ‘What we’re doing right now is not serving all kids. And how do we have some bold changes?’” said Meg McCormick, a former teacher and principal, and one of three authors of the report.

The report was written by three former Imaginarium staff members, who described themselves as the Imaginarium’s leadership team, and didn’t include input from all of the dozen or so others who once worked for the department, some of whom still work for the district in other roles.

Denver Public Schools “is rightly known as one of the most innovative, forward-thinking school districts in the country,” the report says. “But our gains will always be incremental, and our excuse-making more prominent than our progress, unless we muster the collective courage to say: Enough!”

Enough top-down mandates, the report says. Enough excuses for institutional racism and cultural incompetence. Enough “turning meaningful ideas into meaningless buzzwords.”

The district created the Imaginarium in 2014 as a way to combine separate projects, most of which were funded by philanthropy, under one department. The department was tasked with providing schools and teachers a space to test new ideas, developing a consistent way to test whether the ideas were working — and if so, spreading them to more schools.

While other innovation labs existed throughout the country, few, if any, were district-run. That made the Imaginarium unusual. Also unusual was that in a district that measures school quality based on standardized test scores, the Imaginarium chose to focus on the “soft skills” its staffers theorized would help eliminate academic gaps between more and less privileged students.

“We concluded that the most effective innovation would be transforming classrooms into places where students have real agency,” the report says.

To do that, the Imaginarium helped teachers, the majority of whom are white, learn new ways to build relationships with their students, many of whom are black and Latino.

The report cites examples like the Writing the Road to Freedom writing center at Manual High School, where students used newly developed skills to advocate for their school, or an initiative at Montclair School of Academics and Enrichment in which students and staff mapped their relationships and held family meetings to build a closer community.

The Imaginarium also promoted “personalized learning,” a buzzword that often looks like students doing computer work at their own pace. The Imaginarium’s approach involved helping students become self-directed learners by having them co-teach lessons or set their own goals.

“The goal is to have students take ownership of their learning and learn how to learn,” said Susan Trickett, a researcher and another of the report’s authors.

But that can be slow to implement and tricky to measure. The Imaginarium’s detailed rubrics and interviews showed students making progress from setting a fantastical goal of being a professional baseball star at the beginning of the year to a more concrete goal of reading “Harry Potter,” but the evidence of success wasn’t enough for the district, the former staffers said.

“What we were ultimately being measured on is improvement in standardized test scores,” said Danna Ortiz, the third author of the report.

“We did have all these measurements,” McCormick said. “But DPS was like, ‘But reading scores didn’t go up.’”

“We say we value the whole child,” Trickett said, “and yet our one measure of success is this single data point.”

This report comes as the Denver school district has convened a working group to talk about changing how it measures school quality. The formula has been tweaked repeatedly over the years but remains heavily based on test scores. These measurements carry high stakes because low ratings can be used to close schools.

The Colorado Department of Education, meanwhile, has its own school performance measures. Denver has largely avoided state intervention in low-performing schools by aggressively managing its own school improvement processes, but if the district moves too far away from using test scores to measure school quality, that could change.

While it’s not clear what the next school rating system will look like, Keltner said district officials are taking to heart the criticism about relying on test scores.

“Broadly, success cannot be boiled down to CMAS scores,” she said. “There has to be a more nuanced picture.”

When it came time for Denver Superintendent Susana Cordova to cut more than 220 positions in the district’s central office, the Imaginarium ended up on the chopping block. The cuts freed up $17 million, some of which was funneled into the higher teacher salaries negotiated to end a three-day strike in February.

At the time, Cordova said the Imaginarium’s annual budget was about $2.4 million. (Ortiz estimated that since 2015, the Imaginarium received about $9.5 million in district funds and about $12 million in grants from philanthropies and private foundations.)

Keltner said the work of the Imaginarium was valuable and won’t be lost. Four former members of the Imaginarium are now part of a five-member Innovation and Improvement team within the Impact Office, with their work to be more closely connected with the district’s long-term strategies. Personalized learning efforts are now part of the office of curriculum and instruction, she said.

The methods that the Imaginarium developed to measure changes in student mindset — critical to how students learn — will be used going forward, Keltner said. Programs that were incubated in the Imaginarium, including one that aims to create culturally responsive classrooms with strong community connections and another that trains school leaders in a problem-solving approach known as improvement science, in which new ideas are tested in rapid succession to learn what works and what doesn’t, are now used in dozens of Denver schools.

The authors said they showed the report to district leadership before the Imaginarium closed on May 31. They said the feedback they got was that the report lacked nuance and wasn’t as positive as it could be. They responded that they were open to revisions, but said they didn’t hear anything back until a few days before the planned publication.

Keltner, for her part, says she was not concerned with the report being positive, but with it being as useful as possible. She called the publication “premature.”

The three former staffers decided to publish the report Monday to a website they built to host the Imaginarium’s teaching materials and other resources. Monday is when most district educators return to work for the new school year. Most students start Aug. 19.

“This was public money,” Trickett said of the Imaginarium’s funding. “It was an experiment. You could say it succeeded, you could say it failed, but our perspective is we’re always learning. Learning we keep to ourselves is not useful learning.

“We need to share it in order to move things forward.”

Read the full report here.

Originally posted on Chalkbeat by Melanie Asmar and Erica Meltzer on August 12, 2019. Chalkbeat is a nonprofit news site covering educational change in public schools.