Why Are Chicago's Kids Getting Smarter? Here's My Guess.

If you just read the headlines, you might suspect that Chicago Public Schools is in a terrible tailspin. Part of this is the noise of big city politics. Part of this stems from cloud of violence that hangs over the city. But here's the truth: Chicago has improved on academic test scores more than most other cities in the country. Rather than being one of the country's worst districts, Chicago is one of the best. A  recent report by Sean Reardon and Rebecca Hinze-Pifer found that between 2009 and 2014:
This [student achievement] growth rate [of Chicago] is higher than 96 percent of all districts in the U.S. Among the 100 largest districts in the country, the average growth rate from third to eighth grade is 0.95 grade equivalents per year; Chicago has the highest growth rate between third and eighth grade of any large district in the United States.
The authors admit that they don’t know why this occurred. I can’t prove why Chicago kids are getting smarter, but I have a hypothesis.

What’s Been Going on in Chicago Public Schools? 

One way to try to solve the mystery of why Chicago children are getting smarter is to look at the district’s previous major initiatives. As detailed in a report out of the University of Chicago's Consortium on Chicago School Research, between roughly 1990 and 2010 there were three overarching eras of reform in Chicago: the decentralization era, the accountability era and the do-a-lot-of-things era. The authors are very careful not to attribute a causal relationship between reform eras and outcomes. The reforms were messy and not rolled out in an experimental manner—which is fair enough. But in this post I’ll try to make my best guess on what was causal and what was not.

The Decentralization Era

The decentralization era was best known for the creation of local school councils. This reform gave local councils real control over decisions about how schools were run. The councils were made up of school leadership, parents and community members. The councils always seemed like a terrible idea to me. It’s basically taking all we know about charter schools (good central offices, scalable instructional programs, governance matters) and doing the exact opposite! Unsurprisingly, research on the reforms found that the councils had some positive effects on advantaged communities, but were least likely to improve schools in low-income communities. Communities with low social capital didn’t gain a lot from ad hoc and poorly constructed local boards. I’m very skeptical that the decentralization era and school councils were the root cause of later gains.

The Accountability Era

In 1995, Mayor Richard M. Daley installed Paul Vallas as superintendent (full disclosure: I later worked with Paul when he was the superintendent of the Recovery School District in Louisiana). Vallas, who did not have deep instructional expertise, used test-driven accountability to try to make things better. New tests, promotional standards and interventions for failing schools were all put in place. The reforms had better impacts for low-performing schools. The researchers noted:
This was the only era to show large improvements in the lowest-achieving schools. However, the patterns in test scores in the lowest-performing schools suggest that some of the improvements resulted from instruction that was aligned specifically to the high-stakes tests.
This matches other research on accountability reforms: You tend to see gains in the lowest-preforming schools, but high stakes can cause the curriculum to narrow.

The Do-a-Lot-of-Things Era

Arne Duncan replaced Vallas, and he instituted a lot of reforms. Arne launched 100 new schools, implemented internal district instructional and curricular reforms, overhauled school leadership pipelines and focused especially on on-time high school progression. Perhaps the biggest initiative of this era was the  Renaissance 2010 project, which launched about 100 new district, charter and contract schools between 2005 and 2010. Unfortunately, no one has conducted a full evaluation of the program. Someone should do this! Two interim research reports came out around 2010. One study, which only included a few years of data from the early Renaissance cohorts, found that the new schools performed about the same as the existing district schools. The other study was inconclusive. The research community wasn't much help. A lot of work was also done on school leadership. The Chicago Public Education Fund, in partnership with the district, invested heavily in school-leader development, placing bets on both district-based and nonprofit providers. The  latest research I could find on these programs found that “results indicate that one-year learning gains in elementary and high schools led by Fund-supported principals were not different than those in other similar schools.” Another major reform, another mediocre result. All told, researchers found that this era produced more gains in high school than elementary schools, but wrote, “While the effects of the dominant policies of Eras 1 and 2 are largely understood, much research remains to be done to understand both the positive and problematic effects of the policies in Era 3.” Not super helpful, especially since this is the era that preceded the large gains in test scores that occurred after 2009.

What About the Charter Sector? 

CREDO published a report on Chicago charters that covered test scores from 2010 to 2012, which is right in the middle of the period where Chicago saw a lot of gains. The study found +.01 effects in reading and +.03 effects in math. These effects amount to about a month or so of extra learning per year, maybe a bit less. Given Chicago’s relatively small charter-school market share, and the modest size of these positive effects, it’s unlikely that charters themselves accounted for the 2009-2014 gains. A more recent study, which just looked at charter high school performance from 2010-2013, found much larger effects: +.2 effects on ACT-related tests and much higher college-enrollment rates. These are large effects, but they are for high school only. The study lauding Chicago’s gains only covered grades three through eight.

So WTF Happened in Chicago to Make Kids Smarter?

To summarize: Chicago improved its test scores more than any other big city in the country, and researchers really don’t know why. So why are Chicago kids getting smarter? Here’s my guess: Competition and accountability lifted all boats. When you put accountability in place (the Vallas era) and then launch 100 new schools (the Duncan era) you get a city where school leaders know there are consequences for failure and the best of the new schools begin to raise the bar for what’s possible. This theory helps explain why the Renaissance schools and charter effects were a bit muted. In the studies on these reforms, researchers compared the new schools to existing schools. So if the existing schools were improving due to increased competition, you would not see large relative effects for the new schools. I can’t prove that accountability and competition caused the results, but in many sectors accountability and competition make everyone better. It also fits stories we’ve seen elsewhere. In place like Denver and Washington, D.C., increased competition led to all boats rising in the public school system. If you have a better theory, let me know.

What Should Chicago Do Now?

Here’s another tough question: If accountability and competition caused Chicago’s gains, how should this impact Chicago’s future strategy? Since 2002 (while the district was getting much better!)  Chicago enrollment plummeted from 440,000 students to 370,000 students. This means that there are lot of under-enrolled schools in the district and the city might have to go through another round of painful closures. This also it means it’s harder to push the very reform (opening new schools) that might have driven Chicago’s previous gains in achievement. So what should the city do? Reasonable people can surely disagree, but I would continue to create new schools, albeit in a different fashion. First, I’d open new schools in the areas where population is increasing. Chicago is made up of a lot of neighborhoods, and not all neighborhoods are losing children. Second, I would do some replacement work. Instead of closing all the under-enrolled schools, I’d try and select some neighborhoods where there’s enough child density that you could imagine families coming back to the public schools if there were better options. I’d launch replacement schools in these neighborhoods. There are clear drawbacks to this strategy. Politically, it’s hard to justify opening schools when you’re in the midst of closures. Programmatically, it’s a hard sell to get the operators of new schools to open up in neighborhoods with shrinking enrollment. But I think it’s the best thing for children. Lastly, I might also try and launch some diverse-by-design schools. In a city as diverse as Chicago, it’s sad that its  schools are so segregated.

One Last Thing

Chicago’s Chief Education Officer, Janice Jackson, recently  gave her take on why things are better. Her list: pre-K, better professional development, better curriculum, competition from private and charter schools and clear accountability standards. In her own words:
I believe the level of transparency we have provided around what a quality school is has been transformational in this district.
An original version of this post appeared on the Relinquishment as Does anyone know why Chicago children are getting smarter?.
Neerav Kingsland is the senior fellow at the John and Laura Arnold Foundation and CEO of the Hastings Fund. He has been an advocate for education in the New Orleans area since he moved there at age 18 to attend Tulane University. He was part of the founding team, and eventually became CEO, of New Schools for New Orleans. In this role, he helped build the nation’s first school system where over 90 ...

Join the Movement