Facebook Twitter

As Chicago mulls learning recovery plan, study offers new clues

A University of Chicago study found major gains from an intensive high school tutoring program.

A teacher is reading books with an elementary age girl in the library.

A University of Chicago study released this week found remarkable gains from an intensive, or “high-dosage,” tutoring program tested in 20 Chicago high schools.

FatCamera / Getty Images

As school districts across the country weigh how to address academic fallout from the pandemic, some experts have touted large-scale tutoring programs as one promising solution. Chicago could become ground zero for those making the case. 

A University of Chicago study released this week found remarkable gains from an intensive, or “high-dosage,” tutoring program tested in 20 Chicago high schools, in which math tutors met during each school day with one or two students at a time.

The authors found the program, provided by a Boston-based nonprofit called Saga Education, doubled or tripled how much math students learned, and improved student grades in both math and other subjects. 

Those findings are especially noteworthy because it goes against the common assumption that it’s too hard to intervene with students who arrive in high school significantly behind grade level.  

The study took place more than five years ago, but an updated, final version of the findings comes out as billions of dollars in federal stimulus funding head to school districts to power their recovery from the pandemic. 

Chicago leaders, including Mayor Lori Lightfoot and district education chief LaTanya McDade, have signaled that an expansion of tutoring services might factor into what officials have said is a comprehensive plan in the works to deal with learning gaps caused by the pandemic’s disruption. 

Questions remain about the district’s ability to scale the tutoring model across schools, grades and possibly subjects. 

Monica Bhatt, senior research director at the university’s Education Lab, and Jonathan Guryan, the lab’s co-director and a professor of education and social policy at Northwestern University, spoke with Chalkbeat Chicago about their findings.

How did you come to study high-dosage tutoring, and did the onset of the pandemic affect your work on this research?

Bhatt: This study of Saga Education’s in-person, high-dosage tutoring program took place in the 2013-14 and 2014-15 school years. Earlier versions of this report have been released in working paper form. But we’re able in this updated version to shed some light on why we think high-dosage tutoring has such big effects.

Since this study took place, we have continued to partner with Saga Education and Chicago Public Schools to study iterations where we tweak the model slightly. So instead of two kids to one tutor, maybe we have four kids to one tutor where they spend half the time online. 

Of course, in the times of the pandemic, all of the tutoring has gone online. We’re interested in whether you can take the insights from the in-person study and apply them to our current context. 

You mentioned efforts to tweak the model to see how effective different versions can be. What are the essential elements? 

Bhatt: This is a study of high-dosage tutoring, which is embedded during the school day, not before or after school. It takes place every day as a credit-bearing class, called Math Lab, in addition to a student’s regular math class. Students in the study attended 45 minutes of Math Lab a day where they worked with a professional tutor, in the sense that the tutor was hired and worked with the students on a long-term basis. 

That model is very different from tutoring akin to homework help, where a volunteer works with kids maybe once a week for a short period of time assisting them with course assignments. 

Saga also has a curriculum that is aligned with the Chicago Public Schools’ ninth grade algebra curriculum. 

Guryan: The personalization part is very important. Because the tutor is working with two kids at a time and not with 30, the tutor can personalize the math topics and problems to the level that each kid is ready to learn at. If there’s a kid who is a couple of grade levels behind in math, that kid can get math problems that are at the level that the kid is ready to learn. 

The tutors are also there all year. For the most part, kids work with the same tutor every day for the whole year as they develop a relationship with the tutor. 

How much do we know about how applicable the results of the study would be to this current moment, when learning is massively disrupted and, for many students, could likely continue virtually for the foreseeable future? 

Bhatt: We are interested in whether we’re able to preserve effects when students are working online versus in person. We did a study across Chicago and New York looking at a blended model where instead of two kids to one tutor, there were four kids working with one tutor, and the kids were spending half the time working on an online platform.

It’s very preliminary, but early indications suggest that it is possible to preserve effects while reducing the cost and making the program a little bit more scalable. 

Guryan: If we’re right that one of the key reasons why high-dosage tutoring is effective is its ability to personalize instruction, then that has become only more crucial, given the disruptions that kids have experienced because of the pandemic, and how those disruptions have not been borne equally by all kids. 

How challenging would it be to scale an intensive tutoring program across a large urban district such as Chicago? 

Guryan: A focus of our current research is trying to understand how well a program like Saga’s high-dosage tutoring could scale if it were to expand. 

One barrier is whether the cost is something that a large school district would be willing and able to spend. That is why we are trying to test different versions that provide the individualized support but at a lower cost. 

The other barrier is the ability of Saga or some other tutoring provider to attract and recruit tutors who would be just as effective as the tutors in the program we studied. 

Our preliminary results suggest that a cyber program allows lots of people to be able to come in and be effective tutors. But it’s still an open question of how much they could scale.

You touched on the question of cost. The per-student cost for the model you studied is not excessive. But when you scale the program in a large district such as Chicago, the price tag is still significant. Could a small group instruction program or some other kind of intervention be as effective at a lower cost?

Guryan: It might seem obvious that if you provide every kid or every pair of kids with a personalized tutor, that’s an effective way to help them learn math. What Saga was able to figure out was how to deliver that at a cost that a school district would consider spending.

They’ve done that by simplifying instruction in a way that you don’t need an experienced classroom teacher to deliver the tutoring. That’s how they were even able to get it to $3,000 or $3,500 a kid. 

In the intervening years, we have been working on different versions to try to reduce costs. Part of that is about integrating technology into the model. 

They’ve also been able to reduce the costs to school districts by getting funding through things like AmeriCorps. The cost to the school district of some of the newer models is more in the range of like $1,300 to $1,800 a kid. 

Bhatt: If we think that this is one promising way to accelerate student learning, then the question really becomes: How can we do it in a way that is cost-effective for school districts? What we see in our study is a very high benefit-cost ratio, comparable to and in some cases even higher than some early childhood programs.

Are there any reasons why it might be more challenging to adapt this model to other subjects such as language arts or science?

Bhatt: We know from prior research that math skills are more malleable and susceptible to in-school interventions than reading skills. But we did actually do a small pilot study in partnership with Saga Education on an early childhood literacy program for first graders to explore just that question. 

The results of that study are coming out soon. But we find comparable effects, which is to say that we think that the core technology of having personalized support is not specific to math and can actually translate across subjects and grades.

The Latest
The district said it will begin to mail out checks of up to $500 this week. Parents can pick up checks from their child’s school.
Officials said about 40% of kindergarten through second grade students were at or above grade level by May, up from 9% in September. They declined to share school-level data or any information about how students fared on early math assessments
Officials are considering opening more so-called specialty schools meant to help students with more challenging disabilities transition into the real world.
People interested in running for a seat at their local public school can apply between Oct. 16 and Feb. 8, 2024. Elections will take place on April 10, 2024 for elementary schools and April 11, 2024 for high schools.
Preliminary data analyzed by Chalkbeat shows just over 322,000 students were enrolled as of the 20th day of school, when the district takes an official count. The stable number comes after a decade of dramatic annual declines.
School-level data from the 2023 Illinois Assessment for Readiness shows many schools have not returned to pre-pandemic levels of students meeting standards in reading and math.