One of the thorniest issues in education is measuring how kids learn.

And what’s even thornier is using those measurements to determine how well teachers are teaching.

lIlinois, like 41 other states, now links teacher evaluations directly to student performance.

But in Chicago Public Schools, the shift is causing a lot of anxiety. New research out of the University of Chicago Consortium on Chicago School Research shows 79 percent of teachers feel more stress and anxiety about their performance reviews.

One of the underlying things feeding into that anxiety is a big mistrust of the tests being used to measure student learning. Take Sarah Chambers testimony from last week’s Board of Education meeting:

“What do you think is the reading level for the 5th grade REACH performance task? Do you have any guess? It is at a 12th grade reading level! This does not make sense,” Chambers said. “You are setting up our children to fail. (applause) You are setting them up to fail so you can label our schools as failing, close our public schools in black and brown communities, fire our experienced teachers, privatize our public schools!”

Chambers teaches special education at Saucedo Academy. When pressed by board member Jesse Ruiz at the end of the meeting, CPS CEO Barbara Byrd-Bennett said she thought maybe Chambers was worried specifically about her own students, who need special services.

But Chambers is not alone in her concerns about the tests.The University of Chicago study also found that half of all teachers believe the tests used to measure student learning are not fair or accurate.

Most of a teacher’s job performance is still based on classroom observations, and so far there haven’t been dramatic changes in how many teachers are rated in the top or bottom categories. In accordance with the Chicago Teachers Union contract, this will be the first year all teachers get a rating under the new system.

And this year, 30 percent of that rating will be based on results of two kinds of tests kids take—those common fill-in-the-bubble tests and then the ones Chambers is concerned about, called REACH Performance Tasks.

Last month, another teacher who asked to remain anonymous sent me some copies of the REACH Performance Tasks.with the same concern about the reading levels. So I called in an expert.

Reading levels too hard?

Barbara Radner is the head of DePaul University’s Center for Urban Education. She’s worked with CPS schools, developing and implementing literacy curriculum, for years.
“This is all very interesting,” Barbara Radner told me after I sent her the tasks and asked her to help me analyze their difficulty.

Radner quickly ran all of the reading passages through six different readability indexes, including Flesch Kincaid and Gunning Fog.  She helped me calculate the lexile levels and together we compared them to the new recommended reading levels for each grade under the new, more rigorous Common Core State Standards.

No matter how we sliced it, the 4th, 5th and 8th grade levels were registering at least three grades above level. By some calculations, the passages were registering at a college reading level.

“At this time of year, I do not want to scare the children, Radner said. “I want to find out, can they figure out the main idea and supporting details.”

She said when she writes literacy tests, she’ll choose short passages, usually only one page long, that are either at grade level or one below, so that you’re actually testing a student’s skills.

At this point, CPS officials aren’t too concerned about the tasks’ reading levels being too high.

John Barker, the district’s chief of accountability, said there are a lot of things that affect readability levels, like “the number of commas in a sentence.”

Radner echoed that point. The example she gave was the sports section of the newspaper.

“I thought (in) the LeBron James passage kids would say, ‘Oh it’s about a basketball player let me try,’” Radner said, referring to the 4th grade passage. “One thing people don’t know is that the sports pages of the newspaper are written at a higher level of complexity than the rest of the newspaper because they tend to use more descriptive terms and more unusual terms. And what is ironic is that kids who cannot read the rest of the newspaper can read the sports pages.”

Radner said there are a lot of ways to look at reading levels, so it could be that students are fine reading some of these passages. But if not, the results may not tell you much.

“You really cannot make decisions about the kid or the teacher if the text is inaccessible,” Radner told me. “That is the peril.”

‘By teachers, for teachers’

CPS’s Barker repeatedly said the REACH Performance Tasks “are written by teachers, for teachers.”

And that’s true. It was one of the big compromises when the new evaluation law and the subsequent teachers’ union contract were being negotiated over two years ago.

For years, teachers have been critical of the district’s use of “value-added” scores in sorting and ranking schools. The value-added scores CPS uses are calculated by running student scores on the Illinois Standards Achievement Test, or ISAT, through a complicated formula to figure out how much they are learning over the course of a year.

CPS initially proposed using only value-added measures, but backed off. The final agreement with the union included REACH Performance Tasks, developed by teams of teachers and district officials.

CTU’s Carol Caref said the teacher-developed tasks are better than other standardized tests, but that the union still doesn’t think a teacher’s job performance should be tied to student scores.

And they don’t think teachers should take the fall for problems with the performance tasks.

“There’s so many things that teachers don’t have a say-so in and so then there’s this very narrow little spot where, ‘OK. We’re going to let you have a little bit of say-so in these tests’ and then if there’s a mistake on them, blame the teachers. I just think that’s so wrong,” Caref said.

It’s unclear if students struggled with the most recent set of literacy tests— scores won’t be out until next fall.

Caref said if you want reliable results the district will have to doublecheck the end of the year exams.

“Since this is already out and we gave this test that’s too hard at the beginning of the year, we also have to give a test that’s too hard at the end of the year, because otherwise, then, you’re not measuring growth,” Caref said.

CPS spokesman Bill McCaffrey says the district is looking closer at the issue, and the CEO will be the one determining if any changes need to be made.

That wait-and-see situation isn’t likely to quell any anxiety among teachers.

Becky Vevea is a producer and reporter for WBEZ. Follow her @WBEZeducation.