Positive Results from Mathematica Study of KIPP
Some great news from a Mathematica study of KIPP – here’s KIPP CEO Richard Barth:
Dear KIPP Team and Family --
I want to take a moment to tell you about an important new independent report on KIPP.
Today, Mathematica released its second report on KIPP middle schools—the most comprehensive and rigorous research on KIPP to date. You can read the full report here, and the executive summary here. This report represents not only a new phase in Mathematica’s multi-year study of KIPP, but also a huge new opportunity for us to see where we are as a network.
Mathematica’s independent study of KIPP supports our commitment to transparency and research. Through third-party research like this—along with our own annual Report Card and our Healthy Schools and Regions Initiative—we identify areas where we are fulfilling our mission and where we can improve. Our network-wide commitment to research and evaluation is also crucial to our goal of serving as leaders in the education community at large and proving what is possible in public education for our students.
Mathematica’s 2013 report expands on the analysis in the original 2010 report in several ways. First, it examines nearly twice as many middle schools and twice as many academic subjects. Mathematica also looks as additional measures of impact—including a national norm-referenced test and self-reported surveys on student and parent behavior and attitudes. Most notably, Mathematica compared academic outcomes for students who got into KIPP via lottery with those who entered the lottery and didn’t get in. This kind of analysis is considered the “gold standard” of educational research.
There are four key findings in this report around student achievement:
1. KIPP middle schools have positive and statistically significant impacts on student achievement across all years and all subject areas examined. According to a matched comparison design study, KIPP students showed gains in math, reading, science, and social studies on state assessments. This finding confirms that we have been able to maintain the quality of our middle schools as we have expanded our network.
2. The magnitude of KIPP’s achievement impacts is substantial. Across all grade levels and subjects studied, KIPP’s achievement impacts are large enough to be educationally significant.
3. The matched comparison design produces estimates of KIPP’s achievement impacts similar to estimates of the same impacts based on an experimental, lottery-based design. Researchers found that KIPP’s achievement gains are similar for the matched comparison design and the experimental lottery analysis--demonstrating that parental motivation cannot explain our student’s achievement gains.
4. In the lottery sample, average KIPP impacts on a nationally normed test that includes items assessing higher-order thinking skills were similar to impacts on high-stakes state tests. For students in the lottery sample, gains on the national norm referenced test mimicked those on state tests.
In addition to the academic findings, Mathematica found little evidence that we are attracting the most able students. KIPP’s middle school students look very much like their neighborhood peers: non-white, low-achieving, and from low-income backgrounds. The typical KIPP student’s test scores in math and reading are lower than the average in neighborhood feeder schools. Mathematica also found that our students are less likely than their district peers to be designated Limited English Proficient or be receiving special education services. We remain committed as a network to increasing the number of Limited English Proficient and Special Education students we serve.
Mathematica also looked at self-reported surveys to see how KIPP is influencing attitudes and behaviors among both students and parents. In the survey results, Mathematica found that KIPP students complete almost an hour more homework daily than they would at a non-KIPP school. Mathematica also reported that KIPP students were more likely to self-report undesirable behavior like arguing with parents or giving teachers a hard time. The report presents two potential reasons for this: students might engage in these behaviors more often, or KIPP’s culture might make them more likely to acknowledge these behaviors.
Finally, Mathematica looked at what makes individual KIPP schools successful. Although Mathematica found no one factor that determines a successful KIPP school, they did find two characteristics that were correlated with positive achievement gains. First, KIPP schools with a strong school culture (as measured by comprehensive behavior systems) tend to be more successful. Second, KIPP schools that devoted relatively more time to core academic subjects had larger achievement gains, but given an already-extended school day, Mathematica did not find a positive relationship between longer schools days and improved academic results.
I wanted to end by giving a huge shout out to our teachers, staff and school leaders across the network. It is your hard work, passion, and dedication that has gotten us to this point. Mathematica’s findings both affirm the validity of our approach and identify areas for us to work on as network. Now, with these new findings, we can move forward in our quest to prove the possible and change the life trajectory of our students.
Onward,
Richard Barth
CEO|KIPP Foundation
Work hard. Be nice.
Here are links to some of the news coverage of this study:
posted by Whitney Tilson at 11:30 PM
<< Home