Chicago education officials are reconsidering how the district uses standardized tests in light of a critical report from the top schools investigator that points to unusually long testing times and other timing irregularities.
Chicago has already taken action in response to the report about the untimed test known as NWEA MAP, from outgoing Inspector General Nicholas Schuler.
The report has stirred up lingering concerns about the high-stakes nature of this particular test and standardized testing in general.
Here’s what to know about the investigation, the potential outcomes, why it matters, and why it has struck a nerve.
1. The report flagged a big problem.
Inspector General Nicholas Schuler said he found a dramatic difference between the amount of time Chicago students took to complete the untimed test known as NWEA MAP during one testing cycle and the average time nationwide. The discrepancies could “be an indicator of cheating or of attempts to game the test,” according to a report provided to the Chicago Board of Education.
Schuler’s office reviewed nearly 321,000 tests taken by third through eighth graders in spring 2018. It found that 36% of tests took at least twice as long, and up to five times as long, as the national average. The report noted that Chicago was not timing the test the same way among schools, and that some students took three or four times as long to complete their tests, and paused them more often, as the average national rate.
And, as time went on, the tests took longer. The average time Chicago students spent on the test got progressively longer from 2016 to 2018.
2. Still, Chicago Public Schools points to a record of academic growth independent of the disputed test.
A 2017 report by Stanford University Professor Sean Reardon found that from 2009 to 2014, Chicago’s students grew academically at a rate that was faster than 96% of other urban districts.
That report did not review NWEA results. Schuler’s findings do not question the study, which Chicago has frequently cited as evidence of districtwide improvement.
More recently, results of the 2019 NAEP TUDA, a national test covering about two dozen big city school districts, show Chicago posted among the largest gains in scores since 2009.
Reardon and his researchers used low-stakes tests, in part, to minimize the chances that pressure would push teachers and students to cheat.
He said that Schuler’s report now makes it more difficult to use the NWEA test to compare Chicago with other districts. It is also “a troubling finding, and reasonable people might worry about whether it reflects a larger pattern of testing irregularities in CPS.”
3. There is disagreement over whether the frequent pauses and long test times contributed to better scores.
Schuler said he found a dramatic difference between the amount of time Chicago students took to complete the NWEA during one testing cycle and the average time taken nationwide, discrepancies which could “be an indicator of cheating or of attempts to game the test,” according to a report provided to the Chicago Board of Education. The report found long tests had a “strong relationship” with unusual growth in achievement.
But the district has questioned those results, saying they didn’t show a correlation between duration and academic growth.
At a board meeting this week, several board members appeared visibly frustrated by the term “ cheating.”
“It’s a serious accusation to our teachers, students and school communities, and I don’t see the evidence here,” board member Elizabeth Todd-Breland said.
Experts said it’s difficult to draw the line between cheating, gaming the test, or just trying to do your best quickly on a test.
Daniel Koretz, an expert on educational assessment and testing policy at Harvard University, said he views any type of test prep that can only produce misleading gains, rather than real growth, as cheating.
4. The NWEA is just one of several tests Chicago students take.
Chicago Public Schools has used the NWEA since 2012 to assess how well students are learning what are called the Common Core standards. Other tests also measure proficiency.
Illinois requires districts to give the Illinois Assessment for Readiness, or IAR, starting in third grade.
Chicago district officials have said they want their own test, in part because it takes longer to get the results from the IAR, making it more difficult for teachers to evaluate how well students are learning during the year.
There are a battery of other assessments, too. Fifth and eighth graders statewide take a science assessment, and students in early grades often take reading and math assessments. Every two years, some Chicago fourth and eighth graders also take the National Assessment of Educational Progress, or NAEP, which is called the “nation’s report card.” High school juniors take the college-entrance exam SAT.
Parent groups often complain about the number of tests and the time devoted to them.
5. Some teachers are critical of the NWEA.
Brentano Math and Science Academy teacher Aaron Bingea said schools take a long time on the NWEA test because they know the scores carry big consequences. For his seventh grade students, the NWEA is a significant portion of the score that helps decide which high school they will be accepted to.
Getting a good score on the test involves extra work from both teachers and students, he said. Teachers have to teach harder material because the adaptive online test adjusts the difficulty based on correct answers. Students will go slowly on the test, Bingea said, and will research content above their grade level.
Bingea, who wants the district to unlink the test from high school entrance, said that otherwise, “we are going to continue to put a lot of stress on kids, it will take a lot of time, and you are going to get a lot of bogus data.”
6. Current board members and others also question using the NWEA to rate schools.
Chicago is revising its School Quality Rating Policy, which assigns schools a rating on a scale from 1-plus (the highest) to 3 (the lowest) based on factors including attendance and NWEA score growth.
The NWEA can make up nearly 50% of a school’s score. Board Vice President Sendhil Revuluri suggested the district also use school visits to evaluate schools.
At a recent public meeting, a parent also said she would like to see the city draft a ratings policy that doesn’t rely so much on standardized test scores in reading and math, but that also considers the availability of arts programs, technology, and social emotional investments for students.
7. District officials defended their use of the test, but promised change.
At Wednesday’s board meeting, Chicago’s Chief Education Officer LaTanya McDade pointed out that the NWEA is useful because it delivers results relatively quickly and allows the district to more fairly assess individual student growth and school-level growth.
Administrators said they would adopt the majority of the report’s recommendations including defining a time limit for the test, more closely reviewing how it is administered, hiring a testing security firm and prohibiting teachers from being the sole person overseeing a test on which they are graded.
8. Chicago has an opportunity to contract with a new testing company.
The $2.2 million contract with Northwest Evaluation Association expires in June. Schuler said on Wednesday, “At some point, CPS might want to consider whether this test is the right test for its multiple high-stakes needs.”
9. The report has raised questions about the veracity and utility of all high-stakes testing.
The district said this is an opportunity to reevaluate all the district’s standardized testing.
Board member Todd-Breland said Schuler’s report was spurring top-level thinking about whether the district should continue to use the NWEA as a high-stakes test.
“I am very critical, frankly, about the way the NWEA is used, and I think we need to have that discussion.”