Michelle Rhee DC Cheating Scandal

The seemingly never-ending saga of the DC cheating scandal is back in the news – see John Merrow’s article, Michelle Rhee’s Reign of Error, here and Greg Toppo’s, Memo warns of rampant cheating in D.C. public schools, here.

I’ve done some digging into this and have a few thoughts:

A) This is a HUGE issue that goes far beyond DC, which we reformers need to be REALLY focused on. If we screw this up, it will severely undermine our movement. One of the key pillars of what we’re trying to achieve is differentiating between educators so that – as in any profession (as opposed to, say, the longshoreman’s union, which is the model followed by the teachers unions) – strong ones can be rewarded and promoted while weak ones can get the help they need to improve – and if they don’t, they can be removed.

We know with 100% certainty that there are A LOT of ineffective (and, in many cases, far worse) educators in our schools today who think (correctly until recently) that they have a job for life, regardless of performance, absenteeism (see below), etc. Thus, when real reform is introduced, by definition ineffective educators fear for their jobs and thus have a big incentive to cheat. Therefore, we need to do two things when we introduce reform: a) Most importantly, make it really hard to cheat; and b) Rigorously look for cheating after the test (erasure analysis, etc.), with rapid, thorough investigations and severe consequences when wrongdoing is uncovered. For best practices on this, see this memo by NYS Super John King, and this presentation to the NY Board of Regents, Ensuring the Integrity of the New York State Testing Program, and this follow up. 

B) While there was no doubt some cheating in DC, what happened here is NOT like Atlanta, where there’s evidence that there was a widespread conspiracy. By my count, there have been six investigations by four different entities, all done independently of Rhee and her successor, Kaya Henderson, none of which show any widespread cheating in DC or a cover-up by Rhee or Henderson. So the questions are: how much cheating was there and did Rhee take appropriate action once she learned of it?

C) Regarding the former, reasonable people, using different methodologies, can have differences of opinion on how much cheating there was. I honestly don’t know, but note that the Office of the State Superintendent of Education significantly narrowed its probe:

OSSE has winnowed from 128 to 35 the number of classrooms that it will ask an independent contractor to investigate for possible cheating on the 2011 DC CAS, the agency announced Thursday.

The 128 classrooms, spread across 54 public and public charter schools, represent less than three percent of classrooms citywide in which tests were administered last April. They were identified in a study last July by CTB/McGraw-Hill, publisher of the DC CAS, as having “inordinate numbers” of wrong-to-right erasures on answer sheets. The study, which wasn’t made public by OSSE until late December, said the data “may indicate inappropriate intervention on students’ answer documents by an educator.”

But in a statement, OSSE said only 35 classrooms met the expanded set of criteria the agency is now applying to determine whether a classroom’s test results should be fully investigated. These are: unusual test score gains by individual students from 2010 to 2011; wide variances or unusual patterns of scores within a classroom, and prior year’s test results that showed inordinate wrong-to-right erasures in that teacher’s classroom. OSSE said it consulted with an independent advisory committee of testing experts to develop the criteria.

“Erasures themselves are not an automatic flag,” said OSSE spokesman Marc Caposino.

The CTB/McGraw-Hill analysis of 2011 scores also cautioned against using elevated erasures as the sole criterion for investigation.

Continue Reading...

D) Regarding the accusation of Rhee cover-up, the main evidence critics point to is a supposedly new, smoking gun memo, but it turns out that it isn’t new – it was reviewed by the various investigators – and was very preliminary – it was written only two days after the consultant was hired, based solely on another report. Rhee doesn’t remember seeing the memo, nor does she remember the meeting that Merrow writes about (using an anonymous source) at which she supposedly discussed it. Here’s her statement on 4/11/13:

"As chancellor I received countless reports, memoranda and presentations. I don’t recall receiving a report by Sandy Sanford regarding erasure data from the DC CAS, but I'm pleased, as has been previously reported, that both inspectors general (DOE and DCPS) reviewed the memo and confirmed my belief that there was no wide spread cheating."

Below is a timeline of events:

November 21, 2008 - OSSE sends a letter to DCPS and 13 charter schools regarding two erasure analyses, attaching data sets. The letter directs LEA’s to take appropriate steps to investigate within 60 calendar days.

January 7, 2009 - DCPS updated OSSE on its investigation’s progress and, because of the level of seriousness and large number of identified potential irregularities, requested an extension to February 28, 2009 for a full response.

January 10, 2009 - The Deputy Superintendent of Education granted DCPS’s request for an extension, writing “we appreciate your acknowledgement that this is a serious matter and note your commitment to conduct a thorough investigation.”  

January 28, 2009 - DCPS reached out to consultant Dr. Sanford providing him the November 21, 2008 OSSE documentation. 

January 30, 2009 - Dr. Sanford responded with a “project brief” document outlining a proposed work plan and stating:  “I need more information from OSSE (CTB and AIR) in order to do more accurate analysis, come up with confident conclusions, and make cogent recommendations.  I’m attaching the nine questions I recommend we send to OSSE in our initial response to their November 21st letter.” His analysis of “191 teachers representing 70 schools” that were “possibly culpable at some level” was based entirely on data received from OSSE with the November 21, 2008 letter (i.e. not new information). 

February 16, 2009 - DCPS sent Dr. Sanford’s recommended questions to OSSE.  

February 25, 2009 - OSSE responded (but failed to address several fundamental questions, after which DCPS expressed concerns about OSSE’s failure to provide this vital information).

Feb 28, 2009 - DCPS updated OSSE on its investigation and requested that OSSE provide the methodology used in CTB’s analysis and also request an opportunity to review the original 2008 DC-CAS answer sheets.

March 7, 2009 - CTB McGraw-Hill -- the testing company that has conducted erasure analyses for other states -- advised OSSE “against concluding that cheating behavior may have occurred in these schools, based on these analyses.”  Specifically, in its memo, CTB wrote: “The statistic calculated for each school and classroom does not appear to be consistent with any known approach to analyzing erasure patterns. As a result, the approach for the September analyses does not support evaluation of a hypothesis that cheating has occurred.” 

March 23, 2009 - Dr. Sanford recommended, if DCPS determined a need to hire a consultant to troubleshoot potential cheating situations, to hire a company called Caveon known as the industry leader. 

April 1, 2009  - Responding to DCPS’s February 28, 2009 request for further guidance, OSSE reduced the number of classrooms/test groups recommended for further investigation to just 13.  

March-April 2009 - DCPS strengthens test integrity, providing on-site DC-CAS training to test administrators and coordinators, monitoring test administration, and creating a test security agreement to be signed by any person having access to the assessment. 

September 3, 2009 - The D.C. Deputy Mayor for Education issued a report noting that a new administration at OSSE reviewed the analyzed provided by OSSE to DCPS on November 21, 2008 and determined them to have been “ultimately inconclusive” and to have “created a confusing picture.” The report read: “The erasure analyses of the 2008 DC CAS results were ultimately inconclusive. The analyses ‘flagged’ anomalies in certain DCPS schools and thirteen public charter schools. Two different analyses led to divergent data – the number of classrooms and the number of schools identified as having potential concerns varied significantly. The new administration at OSSE, along with DCPS, reviewed the methodology used and the data presented, and determined that insufficient guidance around interpretation and calculation created a confusing picture.” 

November 20, 2009 - OSSE directs DCPS to conduct a review of the security and test administration procedures with regard to the 2009 DC-CAS. 

December 17, 2009 - DCPS executed a contract with Caveon to investigate 2009 DC-CAS tests. 

February 2010 - Caveon concluded its investigation, finding no evidence of widespread cheating. 

August 8, 2012 - The District of Columbia Office of the Inspector General concluded its investigation into possible cheating on the 2008 DC-CAS, stating: “the investigation did not reveal evidence of criminal activity or widespread cheating.” 

January 17, 2013 - The U.S. Department of Education Office of Inspector General concluded its investigation into the 2008 DC-CAS -- which included review of the January 30, 2009 Dr. Sanford memo and other documents from Sanford  -- by concluding that “No information was obtained or developed during the course of the investigation that substantiated the allegation of false claims made to the federal government or confirmed widespread cheating on standardized tests.”  

As for even more investigations, Rhee isn’t standing in the way. StudentsFirst spokesman Matt David said, “we welcome all investigations.”

I’m not buying Merrow’s final claim is that “Sadly, DC’s schools are worse by almost every conceivable measure.” He decries how many teachers and principals have been fired, but I think this was necessary given the catastrophic state of the system, which no doubt had plenty of people who weren’t doing their jobs, when Rhee took over. As for the test scores, based the latest (2011) NAEP results (in which no-one claims there was cheating), DC was one of only four jurisdictions in the country to show positive growth in both 4th and 8th grade math. Here’s an excerpt from a 7/28/12 Washington Post editorial:

Results of the 2012 D.C. Comprehensive Assessment System (DC CAS) announced on Thursday showed steady and solid growth in student achievement over the past five years. In 2007, when then-Mayor Adrian M. Fenty (D) took over the schools and Michelle A. Rhee became chancellor, 27.9 percent of students were proficient in math, and 34 percent were proficient in reading. The results for 2012 — while still woefully unacceptable — are substantially improved, with 46 percent of students proficient in math and 43.5 percent proficient in reading.

Continue Reading...

And the 2011 DC CAS results show solid progress from 2007-2011:

And this federal report that compares standardized test scores of students from 21 urban districts nationwide showed continued improvements for DC schools fourth- and eighth-grade students in math and fourth-grade students in reading since 2007:

Below is the most thoughtful article I’ve read on this:

More than five years after Michelle Rhee took over Washington, D.C., public schools, and nearly three years after she left her position as chancellor, critics are still looking for closure and demanding a more rigorous investigation of the seeming rise in student test scores under her reign.

In what's become an annual ritual of sorts, D.C. school administrators on Friday released an investigation into the district's 2012 standardized tests. The report concluded that 11 schools had "critical" test security violations -- and in some cases, teacher cheating -- on last year's DC Comprehensive Assessment System.

A&M investigated 41 testing groups -- students tested by the same administrator in the same environment -- after they were flagged for certain indicators, like having many answers erased and changed from wrong to right and low score variations.

At Kenilworth Elementary School, according to auditing firm Alvarez and Marsal, a student told investigators that a teacher would "point to the question I got wrong and say to pick another [answer]."

Still, only 18 of the testing groups investigated were found have critical violations, amounting to 0.6 percent of all those in the district. So far, no teachers have been dismissed because of the investigation, DCPS spokeswoman Melissa Salmanowitz said.

Continue Reading...