Little Progress for City Schools on National Test

It would be hard to find a better example of lousy (either biased or unintelligent) journalism than this highly negative cover story in today's NYT, which is headlined: "Little Progress for City Schools on National Test". That's the only conclusion I can come to after carefully going through this article and comparing it to the actual data, which can be found in a 47-page Powerpoint presentation posted at: http://schools.nyc.gov/daa/reports/2007_NAEP_TUDA_Results.pdf (the DOE's press release is at: http://schools.nyc.gov/Offices/mediarelations/NewsandSpeeches/2007-2008/20071115_naep.htm)

Let's analyze the just-released NAEP scores, which cover four areas: 4th grade math, 4th grade reading, 8th grade math and 8th grade reading. Here's how I'd score it (note that I exclude the gains from 2002-2003, since critics claim that Klein only became Chancellor in July 2002 and doesn’t deserve the credit for these first-year gains):

- 4th grade math: Students at or above basic rose from 67% in 2003 to 73% in 2005 to 79% in 2007, far bigger gains than for other large cities and the nation as a whole. Notably, New York City’s biggest gains were by black and Hispanic students, who are now exceeding both the national and big-city averages. There was an even bigger gain for students at or above proficient, from 21% in 2003 to 26% in 2005 to 34% in 2007 – a 62% increase (from 21% to 34%) in four years. These are exceptionally strong results.

- 4th grade reading: the key to making fair comparisons here is to adjust for the fact that nearly double the number of English Language Learners (ELL) took the test in 2007 vs. 2005 (8% of all test takers to 15%; see slide 19). Without adjusting, the scores are flat; with the adjustment (looking only at English proficient students), the students at or above basic rose from 55% in 2003 to 60% in 2005 to 63% in 2007, a strong positive trend (see slide 26). And for English proficient students at or above proficient, the increase has also been strong: 23% in 2003, 24% in 2005 and 28% in 2007 (slide 27). Again, black and Hispanic students showed the largest gains. Overall, this is good performance.

- 8th grade math: Students at or above basic rose from 54% in 2003 and 2005 to 57% in 2007 -- OK performance.

- 8th grade reading: At or above basic declined from 62% in 2003 to 61% in 2005 to 59% in 2007. These numbers are weak, but keep in mind that they are also impacted by the greater number of ELL students, though not as much as the 4th graders.

In summary, there is progress in three of four areas, with strong progress for 4^th graders. If I were to give letter grades, they would be one A, a B+, a B- and a D. Though there is certainly plenty of room for improvement, this is a respectable report card and, when considered in the context of other evidence, reinforces my belief that Bloomberg and Klein’s reforms are showing positive results, which I believe will accelerate over time as the mutually reinforcing reforms kick in.

And it certainly doesn't warrant this headline: "Little Progress for City Schools on National Test". How about: "Mixed Results for City Schools on National Test" or, if you wanted to be biased (ignoring the 4th grade results) but at least accurate: "Little Progress for City's 8th Graders on National Test".

This stuff really matters because most people will only read the headline and the first few paragraphs and such a falsely negative article erodes support for the bold reforms Bloomberg and Klein have undertaken and gives ammunition to those trying to protect the status quo (for example, here's Randi's predictable quote: "Randi Weingarten, president of the United Federation of Teachers, described the scores as ‘bad news’.”). Shame, shame!

As for the differences between the state and NAEP scores, the article at least gets this partly right: it's a combination of what students are prepped for and lower NY state standards, which should indeed be strengthened (but this is a state legislature issue, not a Bloomberg/Klein one). I'm not buying what Randi's selling here -- that the state tests are bogus and/or there's widespread cheating:

“When scores become so high-stakes, then you have to really think about and ensure the reliability of these testing systems,” she said, adding that the federal scores “call into question the reliability of the New York State testing system.

“Is it right?” she continued. “Is it wrong? I’m not a psychometrician. But what I do know is when everything is so high-stakes, you have to be doubly, triply, quadruply sure of the accuracy of the data.”

I think the difference is mainly that the bar for passing is set much lower at the state level (which is true for the vast majority of states, sadly).


-------------------------
Little Progress for City Schools on National Test

By JENNIFER MEDINA
Published: November 16, 2007
www.nytimes.com/2007/11/16/education/16scores.html

New York City’s eighth graders have made no significant progress in reading and math since Mayor Michael R. Bloomberg took control of the city schools, according to federal test scores released yesterday, in contrast with the largely steady gains that have been recorded on state tests.