Whitney Tilson's School Reform Blog: More responses to WMU hatchet job on KIPP

Here's an email from Ryan Hill, the incomparable founder and leader of the KIPP schools in Newark, using his local data to rebut the WMU report:

Attached (and below my note) is an email that KIPP CEO Richard Barth sent a group of supporters in response to the grossly misleading report that a professor at Western Michigan University recently released. The study claims that KIPP schools' success can largely be explained by greater funding, serving a more privileged population, and the loss of African American males through transfers. These findings are false, and based on misinterpretation of a very selective data set.

KIPP is a massive threat to the status quo, because our 99 schools and 28,000 students across America prove what for so long was alleged to be impossible: that low-income kids can do just as well as rich kids, regardless of their circumstances. Thus, those who benefit from the status quo attack us whenever they can.

On the positive side, we at KIPP, and specifically at TEAM, have been trying to get the mass media and the general public to focus on factors beyond test scores for a long time. As a national network, KIPP is leading the way on this front by holding schools accountable for not only scores, but also for student attrition rates, for serving a representative sampling of our respective cities' student population. And the independent studies done by credible research firms have confirmed this point: KIPP schools work for low-income kids, and the gains that our kids make cannot be explained away by demographics or attrition.

Another important point to keep in mind is that KIPP is a very decentralized organization, and as such, each region has a lot of autonomy in determining how to serve its students – thus, the numbers vary from region to region and school to school. Attached is the update I sent in December that details TEAM's latest results, and describes how we think about how our schools are doing. TEAM Schools is the New Jersey region of KIPP schools, so below are our TEAM-specific responses to the main points that are raised in the report.

----------------------

------------------------

Letter from Ryan Hill, 4/1/11

Dear Friends,

1) Student Mobility and Attrition: TEAM Schools has VERY low student attrition, with the majority of students who leave us leaving because they're moving out of town. Attrition and mobility are not the same as dropping out, because most kids who leave us enroll in other schools. In fact, the percentage of kids who leave us in high school after 10^th-12^th grades (the grades where kids who drop out usually do so) is roughly 1%.

a. Mobility: In New Jersey, mobility is calculated as the percentage of a school's population that leaves or enters between Oct. 15^th (the first enrollment count) and the end of the year. Last year, TEAM Schools had a 2.5% mobility rate. By contrast, Newark Public Schools had a 20.4% mobility rate. For a more apples-to-apples comparison, divide the NPS number in half, because they admit students mid-year, and we usually don't. So the correct comparison is 2.5% vs. 10.2%. We are actually lowering the mobility rate of kids in Newark (it helps that we have buses and serve kids from all over town). Also note in the attached document that the schools that are nearest to ours have even higher mobility rates.

b. Attrition: Attrition is the percentage of kids who leave our school in an entire year, including summer (October to October of the following year). TEAM's attrition rate last year was 7.7%, which is not only lower than the district mobility rate, but is exceedingly low for a charter school. Some more notes about our attrition:

i. Attrition does not have an impact on our test scores. Our data team analyzed the scores of our students who leave vs. those who stay, and found that there is no meaningful difference. The accusation that our test scores can be explained by us losing low-performing students is thus patently false.

ii. TEAM loses, on average, 9% of our boys every year (this is the attrition statistic). Our mobility rate for boys is under 4%. Again, this is well below the mobility rate of Newark Public Schools (over 10%), and even further below the mobility rates in NPS schools that serve mostly African-American students (see attachment for other schools; our schools are 95% African American) so the implied accusation that KIPP does not serve its African-American boys well is false.

c. Dropout Rate: We calculate our dropout rate as the percentage of students who drop out of school altogether divided by the number of students who are with us in 8^th grade. In other words, if a student is with us until 11^th grade, transfers to another high school, and then drops out, we count this student as a dropout in our numbers. We don't have to do this – we could just count the kids who drop out directly from our schools, but we do it this way because we believe that once a kid is with us through 8^th grade, we should support them all the way through college graduation.

i. TEAM's dropout rate is under 1%. Of the more than 400 students who have completed 8^th grade with us, three have dropped out of school.

ii. Calculated the way the state calculates dropout rates, TEAM's dropout rate is 0%. If we only counted the kids who dropped out while attending our high school, our dropout rate would be 0%, as no student has ever done so.

2) Per-Pupil Funding: We not only DO NOT have a "financial advantage over public schools" as yesterday's NYT headline claims, we actually are at a massive disadvantage – to the tune of around $8,600 per pupil in underfunding compared to the district.

Though we don't raise so much money that we outspend the Newark Public Schools (not even close!), we do raise private funds, in large part to offset two major funding inequities:

a. Unequal funding of charters: We receive ~$15,000/student in state funding, compared to the roughly $22,000 NPS receives for each student. Moreover, for every charter school student served, NPS keeps 10% of the funding for that student, yielding over $30 million in revenue in that Newark has to spend for students they don't serve.

b. No facilities funding: Unlike district schools, charters are not given facilities. So we spend approximately $1,600/student on facilities (mostly rent). We have to fundraise to offset these costs that are imposed upon us, but not the district.

Despite these disadvantages, OUR SCHOOLS BREAK EVEN ON PUBLIC FUNDING AT FULL SIZE. Though startup costs prevent new schools from breaking even, we expect all of our schools (and our full-size schools are doing this already) to break even on the state and federal funding we receive for their students.

3) Private Fundraising: Considering our breakeven model, there are a few reasons for why we have to fundraise. Last year, we raised approximately $3 million for over 1,000 students, or approximately $3,000/student, which is far less than the report claims.

a. We don't receive funding for facilities. The $1,600/student mentioned above is low for NJ charter schools, because our real estate and finance teams have achieved incredible efficiencies through their negotiating skills and their ability to put together aggressive and creative financing packages, usually including New Markets Tax Credits or other financing vehicles. Meanwhile, the Newark Public Schools district has a number of buildings sitting empty or half-empty, which could be used for charter schools (thankfully, Commissioner Cerf and Mayor Booker understand this).

b. We are a growing organization. Unlike the Newark district, which is shrinking, or most districts, which are static or growing slowly, TEAM Schools is growing at about 30% per year. This rapid growth means that we have to hire ahead of our needs, we have to staff up our central office in places that districts don't have to spend money (e.g. real estate staff, fundraising personnel, recruiters for the huge amount of teachers we hire every year, etc.).

c. Start-up funding for new schools is not provided in New Jersey. New schools are too small to have the efficiencies that will enable them to break even at full size. So we fundraise to help us open more schools and cover startup costs.

As always, thank you for your generous support of our schools. This will undoubtedly not be the last attack we face, but with friends, parents, and supporters like you, we will continue to give our kids the education they deserve, regardless of their circumstances, without making excuses. Please don't hesitate to reach out to me with any questions or comments.

Thank you,

Ryan Hill

----------------

Letter from Ryan Hill, 12/14/10

Dear Friends,

What follows is an unusually long and rather comprehensive look at our results for last year. We think that analyzing the health of a school is a complicated task, so you'll be getting more data than you may expect, but our aim is to show our performance in a variety of domains that we think are important.

If you would like a different kind of look at our schools than this data-intensive approach: Many of you have already received – or soon will receive – our annual report (a copy is attached). This year's report shows not only our results, but the schools through the eyes of a number of our teachers across all four of our schools. We hope you enjoy it, and feedback is welcome. If you'd like to receive a hard copy and have not done so yet, please reach out to Jenn Shetsen at jshetsen@teamschools.org.

At a very high level, the news on our results is this: Almost across the board, TEAM Schools had its best test scores ever last year. In areas where in previous years we were not performing up to our expectations, we saw very large performance gains (continuing a multi-year upward trend), and in areas in which we were already doing well, our scores stayed high. In grades that we added for the first time last year (8^th grade at Rise, Kindergarten at SPARK, 11^th grade at NCA) our scores were extremely high, and in grades we already served, our scores improved.

In the context of these good scores, a note on how seriously (or not) we believe you should take standardized test scores, and what we consider to be the most important factors in determining a school's success:

Test scores are like vital signs – they can't by themselves tell you that a school is healthy, but they can tell you it's sick. Just like you can have a good blood pressure, a strong pulse, and a normal temperature but still be terminally ill, schools can get high scores on state tests without really adding much value to their students or the world. By serving an already high-performing population (often called "cream-skimming"), by pushing out or expelling low-performing students, by test-prepping all year, or by severely narrowing the scope of their curriculum, schools can artificially inflate their test scores. These things do happen in schools of all kinds, so it's important to be aware of the limitations of data that can't differentiate between schools that do these things and schools that don't. We believe that the improvements we've made are legitimate and that our scores are reflecting deep improvements in our schools' practices and systems, rather than superficial score-boosters, but in the absence of further data, it is always hard to tell from afar.

We thus think it is critical to remind ourselves to do three things:

1) Take the scores for what they are: a snapshot in time of our kids' performance on one test. They don't tell us about social studies, sports, art, our students' character and resilience, or any other important elements of a strong school. The proficiency tests (NJASK, for example) don't even tell us whether a student's achievement has improved or whether it was already high when the student joined our school. Test scores are highly susceptible to testing conditions and many other confounding factors. They are helpful in that they allow us to compare schools on a standardized metric, but they don't paint a very complete picture.

2) Look at other objective measures of success, the most important of which is college matriculation. We believe that student demographics, student attrition, and teacher satisfaction and attrition (teacher stability is a key driver of the long-term stability of a school) are all important indicators of school health.

3) Visit the schools. Even the most robust array of numbers can't tell us the complete story of a school's performance.

With that in mind, here is our performance on a few key metrics from 2009-10.

The most important number: College Matriculation: (our first class went to college this year) Please note that we report the percentage of our 8^th graders (as opposed to 12^th graders) who made it to college, since most kids who drop out do so in grades 9-12. In four years, our most important metric will become college completion.

Summary: 90% of our first class of 8^th graders entered some sort of college this year.

Total college: 90%

4-year college: 78%

2-year college: 12%

Student Demographics: These are the latest numbers, for the current year (2010-11). Newark Public Schools and New Jersey averages are provided as well, where possible.

Summary: We have a student body with very similar demographics to those of Newark Public Schools.

% African American: TEAM: 95% (NPS: 56%)

% Latino: TEAM: 5% (NPS: 36%)

% Other: TEAM: 0% (NPS: 8%)

Qualify for Free lunch: TEAM: 69% (NPS: 70%)

Qualify for Reduced lunch: TEAM: 16% (NPS: 12%)

Total Free/Reduced: TEAM: 85% (NPS: 82%, NJ: 32%)

Percentage of Student Body with Special Needs:

TEAM: 14%

NPS: 14%

NJ: 15%

Student Attrition and Mobility: (Attrition = students who left at any time of year, including summer; Mobility = students who left or came into the school between Oct. 15 and the end of the school year). As KIPP CEO Richard Barth has said, "we track and report our student mobility. We put it right in our Report Card every year. Bottom line… a school with strong state test scores and high student attrition is not a healthy school."

Summary: We have very low attrition rates compared to most other district and charter schools.

Attrition:

TEAM Academy: 11.45%

Rise Academy: 5.54%

Newark Collegiate Academy: 8.37%

SPARK Academy: 2.00%

TEAM Schools Average: 7.7%

(NJ and NPS do not report this number)

Mobility: (TEAM tends not to bring in many kids mid-year, unlike NJ and NPS, so their numbers should be approximately halved for a more apples/apples comparison)

TEAM Schools: 2.5%

Newark Public Schools: 20.4%

New Jersey: 10.7%

Teacher Attrition: An important indicator of long-term stability and success.

Summary: Our teacher attrition was slightly higher than in the previous year, but not to a degree that alarms us. Our percentage is low compared to similar charter schools.

TEAM Schools:

- Voluntary Attrition: 7%

- Involuntary Attrition: 11%

- Total Attrition: 18%

National Average: 13%

NJ Average: 10%