Good school, bad school: How should we measure?
Editor's note: Reporter Becky Vevea has finished answering this question, and her final report is here: http://wbez.is/1IB3XTW. We're currently closing discussion on this earlier post, which proved useful in helping her and the Curious City team focus Becky's reporting. Thanks to everyone who contributed, both here and on social media!
Recently, a listener submitted a question that WBEZ’s education reporters hear pretty often.
Here’s a version that Julie from North Center sent Curious City:
There is reporting about how Chicago Public Schools are slowly getting better. Was there ever a time when they were good?
The short answer is: It depends.
Not only do families have different values and ideas of “good” or “successful,” but society as a whole has placed value on different things over time. At one point, vocational education was essential to train the next generation of residents, workers, and citizens. Today, value’s placed in college placement and readiness.
This post is just step one in answering Julie’s question. We’re going to lay out several ways that “good” has been defined and ask you to do a little homework. Take a look at what we’ve compiled below and let us know on Twitter, Facebook, or in the comment section which of these, if any, we should use to answer Julie’s question. What do you think matters the most for determining a school’s success?
Standardized tests scores
The most popular tool for measuring a school is to measure how its students perform academically, specifically how they perform on standardized tests.
Illinois students started taking a new test called the Partnership for Assessment of Readiness in College and Careers, or PARCC. Some high school students take the PARCC and many also take the ACT. Up until last year, all juniors in Illinois took the ACT.
Those tests have changed dramatically over time and the scores needed to be considered good or bad are moving targets. The University of Chicago Consortium on Chicago School Research analyzed Chicago Public Schools performance from 1988 to 2009, running sophisticated statistical analysis on the standardized tests used throughout that time.
“The publicly reported statistics used to hold schools and districts accountable for making academic progress are not accurate measures of progress,” the report concluded. “The indicators have changed frequently — due to policies at the local, state, and federal levels; changes made by test makers; and changes in the types and numbers of students included in the statistics.”
The report looked at Chicago Public Schools reading and math performance across two separate tests, the Iowa Test of Basic Skills, or ITBS, and the Illinois Standards Achievement Test, or ISAT. It found math performance inched up, while reading performance stayed nearly the same for almost two decades.
The average ACT score of a high school is often used in conjunction with graduation rates to determine if it is a “good” school or not. The average ACT score in CPS has remained relatively flat over the past decade and a WBEZ analysis showed many of the best high schools in the city are enrolling freshman who already perform well on standardized tests.
Standardized test scores can reflect the performance of a school, but academic reputation is usually defined by a combination of things.
Anne Kremer, Associate Director of Undergraduate Admission at DePaul University, said colleges first look at a student’s transcript.
“We’re looking at the rigor of your curriculum, that you are taking not just the bare minimum,” Kremer said, though she added that colleges evaluate students based on what’s available at their school. For example, if Advance Placement classes aren’t offered, that won’t be held against an applicant.
There are a host of schools in Chicago that tout rigorous college prep curriculums, including selective enrollment schools that require test scores for admission. Many charter schools, like those in the Noble Street Network, offer AP classes and tout a long list of alumni at prestigious public universities, and even some Ivy league schools. Mayor Rahm Emanuel continues to expand rigorous International Baccalaureate programs within CPS high schools, as well.
Beyond what schools advertise about curriculum and notable alumni, there’s word of mouth reputation, as well as the following data:
Academic awards, such as the “Blue Ribbon School” award, U.S. News & World Report rankings, etc.
When Chicago Public Schools decided to shut down 50 schools in 2013, it became clear that some parents don’t choose schools based only on academics; they often choose a school based on how safe it is — or appears to be.
Illinois and Chicago school officials started collecting data on safety in recent years, in order to identify schools where violence is an issue and figure out how to shield schools from it. There are two ways to talk about the safety: what happens inside the school and what happens outside the school.
Inside, schools can struggle with fighting, poor classroom management, and bad behavior. As an outsider, the best way to figure out how unruly a school might be on the inside would be by looking at the following data points:
Total number of infractions
Level or severity of those infractions
Suspensions and expulsions
What happens outside a school is another matter. Even the best, most orderly school can struggle to be considered “good” because of the neighborhood it sits in. Data points we could look at to determine the safety of the school’s surroundings:
Vacant lots and buildings
In the same way parents don’t want to send their children to unsafe schools, many don’t want to travel long distances to send their children to school. In Chicago, many do in order to attend magnet schools, charter schools and, frankly, other neighborhood schools that are safer than their own. But many parents say they want a good school within walking distance of their home.
Ask anyone what they remember most about high school and they probably won’t say the standardized tests or fights in the hall. Most of what people remember about their school-aged years is the overall experience — playing sports, performing in music and drama clubs, or participating in student government. So one way to define “good” relates to the reputation the school has in areas unrelated to academics.
Recent research out of the University of Chicago shows “soft skills” are the key to long-term success in life. Tim Kautz, a researcher at the Center for the Economics of Human Development, recently wrote a book about non-cognitive skills in high school graduates.
“Cognitive skills, that is what’s measured by test scores, becomes relatively fixed by age 10,” Kautz said. “So, that sort of completely changes what a good education system should do.”
The state’s best schools don’t just focus on academics; they also provide robust extracurricular experiences. Whitney Young, while known for its academics, is a basketball powerhouse in Chicago’s Public League, and the school’s chess team won the state championship. Another school, Simeon High School, also draws students because of its basketball program, despite low scores on standardized tests.
ChiArts, modeled after the high school in the musical Fame, attracts students interested in pursuing careers in fine arts such as drama, music, and dance. Comprehensive suburban high schools in Evanston and Oak Park River Forest draw families who want their children in diverse schools.
Kautz said most of that is not measured by districts and states in part because it’s really hard to capture.
“What you learned on the soccer team probably isn’t going to help you all that much with your ACT scores, but may actually help you with your job or getting through college,” he said.
It may be hard to gather empirical data to correlate the amount of time spent in soccer with success in careers, but there are ways to gauge whether students have opportunities to develop “soft-skills”. They are:
Racial and ethnic diversity in enrollment data
Non-core classes, sports, and clubs offered
Budgets and grants for extra-curricular programs
One common phrase used by everyone from President Obama to your child’s teacher in recent years is “College and Career Readiness.”
No one argues with it, but hard to nail down exactly what someone means when they use the phrase.
Our question asker, Julie, touched on this issue when we first asked for her to explain what “good” meant.
“I know an older gentleman who moved to Chicago as a teenager to escape WWII in the 1940s,” she wrote. “He arrived here knowing no English. He graduated from a North Side public school, got a job at a department store, and went on to have a successful career as an entrepreneur.”
Julie pointed to Senn High School in Rogers Park as a school that’s trying hard to change its reputation.
“Just a few years ago, it was one of those schools that one would say ‘No way is my child going to go there.’” She went on: “Was Senn once a ‘good’ school decades ago? Did it do its job of helping a mostly immigrant population to learn English and a few vocational skills to assimilate into American culture? Were expectations lower?”
The latest push of education reform, in part, comes because the American economy is dramatically different than it was decades ago: factory jobs are gone; technology and information jobs are abundant; and policymakers worry our school system doesn’t align with the skills needed to get a job in today’s world.
There’s not as much data to measure success in this and the data one would need would require patience. Tying a student’s success in life to a school and the education it offers requires tracking beyond graduation, something some public schools have just started collecting on a systematic basis.
There are two measurements that Chicago uses to measure success in college. They are:
There are no official measurements for tracking career success district wide. But many schools with vocational programs can typically say how many students earn a certification in the trades available at their school.
Becky Vevea is an education reporter for WBEZ. Follow her @WBEZeducation.