Your NPR news source

ISAT Do-Over Raises Questions

SHARE ISAT Do-Over Raises Questions
ISAT Do-Over Raises Questions

AP/Charles Rex Arbogast

Standardized tests are given throughout the country every year to see how much our students know and how well they think. Pretty straightforward. But there are different ways these tests can be scored. And depending on the approach, you can come up with different results. That's what's happening in Illinois this year. The state is getting a "do-over" on about 900,000 ISAT's, or Illinois Standards Achievement Tests. Officials say this year's results are too different from last year. But some say that's not playing fair.

Think of John Wick as a kind-of renegade.

Now, he's not a cop-turned-vigilante, like in the old TV show "Renegade." Wick's a renegade who wants to make sure standardized tests are scored equally. He's a psychomatrician.

WICK: Psychomatricians are people who spend their life, primarily applying statistics to testing.

Wick used to teach this kind of stuff at Northwestern University. He runs a psychometrics business now. The testing world needs people like Wick because tests change from year to year. And there has to be some way to compare all these different tests to each other. That way you can see if students are doing better or worse.

About two years ago, in 2006, the state changed the ISAT. They added some color printing and let students take longer to finish, among other things. The company that graded the test used psychometrics so it could compare this test to the one from a year before. And the scores jumped dramatically.

WICK: But since they got higher results that time instead of lower, they all kept it very quiet and didn't re-audit it.

RHEE: But you did.

WICK: But I did yeah. And they didn't like it.

RHEE: And what'd you find?

WICK: They had basically weighted the raw scores in a manner that, as far as I could see, they had done it in 2005 one way and 2006 the other way. And I could see that that was probably the reason for the phony gains that they reported between the two years.

RADNER: I think the state should've taken a closer look right then to say, 'What happened?'

Barbara Radner heads the Center for Urban Education at DePaul University. Radner says it is possible this year's scores are off. And if so, the state should adjust them. But she wonders, what if the earlier scores were off?

RANDER: This year's results actually might be correct. Then, you have to look back two years and say, 'What was going on then, did we actually get real data then? Or were we making this mistake systematically?' It's possible and it is worrisome. Because this means we don't know where we are.

So we might not know how much students really know.

Matt Vanover is a spokesperson with the Illinois State Board of Education. He says the scores from earlier years changed a normal amount. This year's scores are different, though. As he puts it, they're quote, "all over the place." Vanover says the board consulted with outside experts back in June.

VANOVER: What these experts recommended and what we're doing is following a new methodology that will provide more accurate scores.

Pearson is the company in charge of doing this. They scored the test originally. The company declines to comment, but in a statement, agrees they need a new methodology.

Now Wick, the renegade psychomatrician, says they need more than that. He says they need to start from scratch.

WICK: They can't change their mind in the middle of a testing stream and change scoring models. But if they do that, they absolutely have to go back to 2005 and re-score the 2005, 6, 7, and 8, all of them using the same model.

In the end, Wick says it's important to be able to compare students, schools and districts to each other. And to see how they're changing.

ISAT scores usually are just one factor districts use to determine school performance. But the clearer they are, the better decisions parents and educators will be able to make.

I'm Mike Rhee, Chicago Public Radio.

More From This Show