A report released Friday afternoon finds unusual patterns in test score growth at some Chicago public schools and, while it doesn’t prove cheating, it contends that proper test procedures were repeatedly violated that could have given students an edge.
The irregularities include students taking a longer-than-normal amounts of time to complete the tests as well as excessive pauses or breaks while testing is underway.
The data analysis looks at the district-mandated standardized math and reading elementary school tests called the NWEA in 2018. These tests are used to evaluate teachers, give schools an all-important rating and are one key factor used for determining admission to the city’s elite test-in high schools. The review looked at annual tests in third through eighth grade.
In an unusual move, the school district issued the report, which was produced by the Office of the Inspector General. It comes about a week before Inspector General Nick Schuler’s last day in office. Mayor Lori Lightfoot announced his resignation after an investigation reportedly found that Schuler had created a “toxic work environment.”
Schuler said Lightfoot asked for his resignation. He did not offer comments about the report on Friday.
Board member Elizabeth Todd-Breland said that she accepts criticism around test security and procedures. But she strongly rejected the idea that the patterns identified by Schuler’s office are indicative of cheating.
“It feels irresponsible and frankly disrespectful to our teachers and students to accuse them of cheating without showing the [evidence] for it,” she said.
In addition, the school district in a statement stressed that the report “does not call into question the accomplishments of our students and school communities.”
However, the school district is accepting all recommendations made by the inspector general, including hiring a testing security expert, strengthening testing policies, improving training and more test monitoring.
The report notes the school district appears to have ignored test security and best practices recommendations made by its audit office two years ago.
Testing expert Bob Schaeffer said it is harder to cheat on tests like the NWEA, which is done on a computer and is adaptive, which means students are given individual questions that get progressively harder. But he said teachers get creative when the stakes are so high.
“The more consequences that are attached to test scores, the higher the pressure to boost those scores by any means possible,” said Schaeffer, who is the interim director of FairTest: The National Center for Fair and Open Testing. “It is human nature when you ratchet up the pressure like that some people will crack and do the wrong thing.”
Schaeffer points out that school districts don’t have an incentive to root out problems, because they also tout test scores.
The data analysis focused on the connection between high test score growth and the length of time students took to complete the test. The NWEA, produced by the Northwest Evaluation Association, is not timed.
The report questions the practice of using an untimed test for such high stakes accountability.
The report also focused on how often students paused the test. Once a test is paused, a student gets a new question. Pauses are allowed, but the test maker said they should be reserved for things like bathroom breaks.
The analysis showed one in four Chicago Public School students took double the average amount of time on the test, and those students who took longer tended to do better. Some tests were paused up to 60 times.
According to the report, the NWEA testing company told the OIG, “The assessment’s validity may be compromised if a student intentionally allows a question to time out because he/she does not know the answer and would like a new question.”
School district officials also stress the inspector general’s report doesn’t change that Chicago Public Schools students have made academic gains.
“Our students — with the support of dedicated educators and principals — have made real and sustained academic progress over the past decade on a variety of assessments and metrics,” the school district said in a statement. “The review did not find any correlation between test duration and high academic growth and it does not call into question the accomplishments of our students and school communities.”
They point to studies, now several years old, that show Chicago students made more improvement than other students across the nation. Those studies examine gains on a state-mandated test that is no longer used.
Chicago Public School students also take the new state-mandated test, but the school district doesn’t use its results for its accountability purposes.
Click here to read the full report.
Sarah Karp covers education for WBEZ. Follow her on Twitter at @WBEZeducation and @sskedreporter.