Facebook Twitter

New study says Chicago’s teacher evaluation may penalize Black educators

A teacher erases writing on a chalkboard.

Emerging evidence from Boston, Michigan, and other places shows significant gaps between Black and white teachers when it comes to classroom observation scores.

Jamie Grill/Getty Images/Tetra images RF

Chicago’s Black educators who taught in high-poverty schools were more likely to receive lower scores on teacher evaluations, new research shows, pointing to bias in a widely used measure that determines promotion and tenure. 

Left unadjusted, the scores could lead to disproportionate and incorrect identification of Black teachers for remediation and dismissal — an outcome that would be counterproductive to efforts to diversify the teacher workforce. About half of Chicago teachers are white.

The study, which examined two years of classroom observation data from 2013 to 2015, published Thursday in American Educational Research Association’s peer-reviewed journal “Educational Evaluation and Policy Analysis.”  Classroom observation scores make up the bulk of a teacher’s evaluation under Chicago’s REACH system, which separate studies have linked to measurable improvements in student achievement. 

Now researchers have found that some adjustments may be needed. 

The study by Matthew P. Steinberg, an associate professor at George Mason University, and Lauren Sartain, formerly of the University of Chicago Consortium on School Research and currently at the University of North Carolina at Chapel Hill, found that the average Black teacher in Chicago ranked at the 37th percentile in classroom observation scores, compared to the 55th percentile for the typical white teacher.

But there was another critical consideration: Those Black educators also were more likely to teach in the city’s most economically disadvantaged schools.

Once researchers adjusted for certain factors — such as the percentage of a teacher’s students from low-income families, the students’ prior year test scores and behavior records, and even school-level resources — that gap statistically disappeared. 

“It’s not because Black teachers are less effective or have lower performance than white teachers in Chicago. It’s that Black teachers in Chicago teach systematically in very different school settings,” said Steinberg. “Black teachers are systematically in schools that are less resourced. It’s not just the classroom environment, it’s the package of goods and services at the school level that is explaining differences in observation scores.”

Researchers said Black teachers also tended to teach in schools with less consistent leadership and less support for ambitious instruction. The race of the evaluator did not appear to affect the score. 

Steinberg said the findings have national implications. Emerging evidence from Boston, Michigan, and other places shows significant gaps between Black and white teachers when it comes to classroom observation scores. 

“We think these findings are important for districts nationwide,” he said. 

The good news: Districts can make statistical adjustments to try to control for differences across schools and classrooms. 

“Teachers and principals generally like the observations and use the information from those observations to make improvements to their practice. But clearly as we show in this paper, there are issues as well,” said Sartain, who has previously studied the REACH system. “Policymakers can now make decisions about how to strengthen the evaluation system with this information in hand.”

James Gherardi, a spokesman for Chicago Public Schools, said it was a district priority to provide an evaluation system that is “fair, equitable and drives school improvement” and the district had commissioned a joint study with the Chicago Teachers Union to fine-tune the system and better prevent bias.

“The district remains committed to working with our research and labor partners to refine the tools used to provide teacher feedback and improve the quality of instruction our students receive,” Gherardi said.

The study examined scores for more than 5,500 teachers from 411 CPS elementary schools. The teachers taught in grades kindergarten through fifth grade.

A spate of recent research links teachers of color to better outcomes for students of color. But across the country, and in Chicago, retention of Black educators has been a point of concern. 

According to a Chalkbeat analysis, by 2019, Chicago had lost a quarter of its Black teaching force over a six-year period. At the same time, more than 9 in 10 white teachers remained in the district, and the number of Latino teachers grew.

Union officials previously have pointed to layoffs of large numbers of Black teachers and filed litigation against the district over the issue. 

In more recent years, Chicago officials have made investments to better support, recruit and retain staff, particularly in high-need schools.

Chicago’s adoption of the REACH system made the city a test case for how to revamp teacher evaluations. The system, piloted in 2012 and tweaked over time through a joint committee staffed by district leadership and the teachers union, was widely considered more comprehensive and effective than a previous checklist that the district had used since the 1970s. 

In March 2020, researchers at the University of Chicago Consortium published a five-year look back at the system based largely on educator surveys. Most teachers and administrators reported that the evaluation system improved teacher practice and student learning. 

The Latest
Paul Vallas and Brandon Johnson disagree about how to give under-enrolled campuses a boost
Supporters of the former Chicago schools chief Paul Vallas interrupted a press conference held by Brandon Johnson backers aimed at denouncing Vallas’ schools record ahead of the heated mayoral runoff election.
As the Chicago Teachers Union found its political footing, Johnson worked on the union’s front line. Now, his years of knocking on doors might be his secret weapon to win over voters.
The number of children under 18 who killed someone with a firearm jumped from 836 in 2019 to 1,150 in 2020.
Chalkbeat Chicago spoke to three Illinois teachers about how they are teaching Black history to their students.
A new report has found that Illinois’ K-12 evidence-based funding formula is working to increase funding for districts that serve students from low-income families, students of color, and students around the state since being implemented in 2017.