Skip to content

What DC Schools Won’t Report: Many NAEP Subgroup Gains Slowed under Rhee/Henderson

December 10, 2016

On December 05, 2016, retired DC teacher Erich Martel sent an investigation request to DC’s Attorney General Karl Racine. In short, Martel is asking Racine to investigate two issues: 1) the failure of DC Mayor Muriel Bowser to follow procedure in nominating Antwan Wilson as the next chancellor of schools (Kaya Henderson left effective October 01, 2016), and 2) inconsistencies in reporting DC’s NAEP improvement from 2007 to 2015 and the failure to report a more thorough picture of DC’s NAEP story by excluding info from 1998 to 2007.

Martel has painstakingly compiled his evidence and explanations in his December 05, 2016, letter to Racine, which can be accessed here: dc-ag-investigation-request-dc-chancellor-search-16-1205

(Martel also includes the following supplemental documents: dcps-naep-tuda-2007-to-2015-11-district-compdcps-weberp-discov-email-1352-54, and naep_2015_sped_ell_exclusion_district-1.)

In this post, I focus on issues related to DC’s NAEP results and the shaping and selectively reporting (and outright misrepresenting) of said results in order for former DC Chancellor Kaya Henderson to be credited with making DC “the fastest improving urban school district in the country.”

NAEP is primarily a state-level assessment; however, in 2002, NAEP began administering a “Trial Urban District Assessment” (TUDA). NAEP TUDA was administered to six districts in 2002 in reading and writing and has grown to include 21 districts in math and reading by 2015. DC has participated in NAEP TUDA since 2002. However, DC is both a “district” and a “state” as far as NAEP is concerned; therefore, DC has scores for reading that go back to 1992 (grade 4) and 1998 (grade 8) and scores for math that begin in 1992 (grade 4) and 1990 (grade 8). (Those scores can be accessed using this NAEP search engine.)

One major issue with the sale of DC as “the fastest improving school district in the country” is that the 2015 NAEP results reported on DC Public Schools (DCPS) website have been misreported in a manner that makes two of the four 2007 scores appear lower than they actually were. Thus, the 2015 scores falsely show “improvements” that are not real. The genius in this inaccurate reporting is that it alters the older, 2007 scores, which readers are less likely to take the effort to verify. Instead, most reader attention is on the recently-released, 2015 scores. Martel also adds that upon initial release, DCPS made it more difficult for readers to access the 2007 scores via its website.

From Martel’s letter:

Three of the four 2007 aggregate averages are fabricated: they differ from NAEP reports;

Ex-Chancellor Henderson posted three altered 2007 NAEP-TUDA aggregate scale score averages on the DCPS website as “proof” that DCPS is thefastest improving urban school district.” The TUDA website (select DCPS) shows the correct aggregate averages for the four 2007 and 2015 tests.

Table 1 shows the three altered and five accurate scale score averages, the resulting changes or “gains” by test, then as a total of the four tests (Altered shown in red Italics)

Year Reading, Gr4 Reading, Gr8 Math, Gr4 Math, Gr8 Totals
2007 198 237 214 244  
2015 214 245 232 258  
07 > 15 Gains +16 +8 +18 +14 + 56

 

Table 2 shows the three correct and five accurate scale score averages, the resulting changes or “gains” by test, then as a total of the four tests (Correct shown in blue) 

Year Reading, Gr4 Reading, Gr8 Math, Gr4 Math, Gr8 Totals
2007 197 241 214 248  
2015 214 245 232 258  
07 > 15 Gains +17 +4 +18 +10 Correct: +49

 

Martel further shows that when DC’s scores are compared to the 10 other districts participating in NAEP TUDA, that “fastest improving urban school district” story shows its limitations as more of a grade 4 story than a grade 8.

DC’s overall NAEP reading and math for grade 4 ranked 1st for most improved from 2007 to 2015.  However, grade 8 scores for math have DC tied for 2nd place overall– but the grade 8 math black student subgroup ranked 6th, and the grade 8 free/reduced lunch subgroup ranked 8th.

The above subgroup story is not stellar, but it is easy to hide these lagging subgroups by simply reporting an overall ranking.

Note that students who were not eligible for free/reduced lunch ranked 1st.

For grade 8 reading, even the overall ranking cannot conceal the “fastest improving” myth:

As for DC’s grade 8 reading score improvements from 2007 to 2015: DC ranked 6th overall– with the black and Hispanic student subgroups ranking 9th, and free/reduced lunch subgroups tying for 9th. What apparently “raised” DC’s grade 8 reading improvement to 6th overall were the students who were not eligible for free/reduced lunch; they ranked 1st.

When it comes to DC’s grade 8 math score improvement, the subgroup that ranked 1st was the one for the students who were not eligible for free/reduced lunch. The same was true for grade 8 reading.

From 2007 to 2015, the gap between NAEP scores for all students and black students is profound, but the DC public does not get to see the subgroup breakdown of scores in order to see this growing rift, as Martel notes:

The general public knows that urban school districts have high percentages of minority and/or economically disadvantaged students, many of whom have low test scores. Most will, therefore, assume that “fastest improving urban school district” was referring to them. Not only are they are the focus of TUDA, they were the rationale for mayoral control and the greater authority delegated to the chancellors. Not so. The 2007 to 2015 gains she posted, i.e. when  using correct averages (17+4+18+10 = 49), are aggregate gains.

…The gaps between All students and Black students more than doubled in grade 4 and tripled in grade 8 between 2007… and 2015….

Martel outlays the above in several detailed tables, which are available as part of his investigation request linked above.

One final point from Martel’s work, and it’s a big one:

DCPS’s reporting DC’s NAEP “improvement” only as far back as 2007 conceals the fact that DC was making greater gains on NAEP prior to Henderson’s chancellorship as well as that of her predecessor, the notorious and polarizing Michelle Rhee. As Martel reports:

Under Rhee and Henderson student improvement slowed dramatically.  Rhee and Henderson suppressed data showing that Black and Disadvantaged students’ averages rose twice as fast from 1998 to 2007 as from 2007 to 2015; White & Hispanic averages rose four times faster.

Table A:

NAEP-TUDA Scale Score Avgs & Comparative Gains: Grade 4 Reading: 1998-2007 vs 2007-2015

Year All Black Hisp White FRL-Elig Not Elig
1998 179 174 173 247 172 215
2007 197 192 206 258 188 216
2015 214 202 206 262 192 256
Gains

1998-2007

+18 +18 +33 +11 +16 +1
Gains

2007-2015

+17 +10 0 + 4 +10 +40

Table B:

NAEP-TUDA Scale Score Avgs & Comparative Gains: Grade 8 Reading: 1998 -2007 vs 2007 -2015

Year All Black Hisp White FRL-Elig Not Elig
1998 236 233 246 280 229 253
2007 241 238 249   301* 234 253
2015 245 236 244 299 233 281
Gains

1998-2007

+ 5 + 5 + 3 +21 + 5 0
Gains

2007-2015

+ 4 –  2 –  5 –  2 –  1 +28

(*No average for White students was reported for 2007; 301 is the average from 2005)

Table C:

NAEP-TUDA Scale Score Avgs & Comparative Gains: Grade 4 Math: 2000-2007 vs  2007-2015

Year All Black Hisp White FRL-Elig Not Elig
2000 192 188 190 254 186 219
2007 214 209 220 262 207 228
2015 232 220 233 275 219 266
Gains

2000-07

+22 +21 +30 + 8       +21      + 9
Gains

2007-15

+18 +11 +13   +13       +12 +38

Table D:

NAEP-TUDA Scale Score Avgs & Comparative Gains: Grade 8 Math: 2000-2007 vs  2007-2015

Year All Black Hisp White FRL-Elig Not Elig
2000 235 231 236 300 226 258
2007 248 245 251  317* 243 259
2015 258 248 263 314 247 300
Gains

2000-07

+13   +14 +15    +17      +17      + 1
Gains

2007-15

+10 + 3 +12     – 3      + 4 +41

(*No White average reported for 2007; 317 is the average from 2005)

As Tables A – D (renamed by me) show, under Rhee and Henderson, the subgroup that has made the greatest NAEP gains for DC is the group of students who are not eligible for free/reduced lunch.

Martel offers this summary:

Black, Hispanic and Disadvantaged students’ improvement rates were two to three times faster in the nine years that ended in 2007 than in the eight years from 2007 to 2015.

Rhee took the DCPS helm in June 2007; when she left in 2010, her deputy, Kaya Henderson, took over.

According to NAEP, they both failed. So has the mayoral control of schools responsible for both Rhee and Henderson. And what is particularly striking is that these “reformers” would rather lie to the public about their success by concealing information than confront their failure and change their corporate-reform-fed course.

I challenge DC Mayor Muriel Bowser to offer a public response to Martel’s NAEP story as publicized in this post, and I challenge DCPS to post the full spectrum of DC’s NAEP results, beginning with the 1998/2000 results; to make such posting easily accessible on the DCPS website, and to use accurate numbers.

sticky-mess-on-shoe  Time to scrape that mayoral control off of your shoe, DC.

____________________________________________________________

Released July 2016– Book Three:

School Choice: The End of Public Education? 

school choice cover  (Click image to enlarge)

Schneider is a southern Louisiana native, career teacher, trained researcher, and author of both A Chronicle of Echoes: Who’s Who In the Implosion of American Public Education and Common Core Dilemma: Who Owns Our Schools?.

both books

Don’t care to buy from Amazon? Purchase my books from Powell’s City of Books instead.

15 Comments
  1. Great

    Posting tomorrow

    Sent from my iPad

    >

  2. Harlan Underhill permalink

    WONDERFUL job of research again, Mercedes. What DC did reminds me of falsification in the early temperature records used to “prove” anthropogenic global warming.

  3. And thus we enter the age of a technocratic love affair with statistics: statistics being so easily manipulated in efforts to pull the wool over the eyes of those blithely offering up that Big Money funding….

    • Statistician key question: “What do you want me to prove?”

  4. Jennifer Pacten permalink

    DCPS is just telling the story that the gentrifiers want to hear. DC schools work for your kids. For Other People’s Children . . . not so much.

  5. sallyo57 permalink

    I’d love to see some follow-up from Martel or others that looks at what specific changes occurred between the 1998-2007 period and the 2007-2015 period. In particular, what might account for relatively flat progress for non-FRL students pre-Rhee and the very rapid improvement under Rhee and Henderson, and the great improvement by Hispanic students, especially 4th graders, between 1998-2007 and the zero or minimal improvement by 2015.

    Very interesting and important analysis here. Thank you!

  6. alainjehlen permalink

    The strongest predictors of test scores for large groups of students continue to be income and education of the parents. That’s what seems to prepare students best for taking the tests, although not necessarily for other kinds of achievement. So when city scores go up, the first place to look is gentrification. DC’s white population is probably richer than the white populations of most big cities and getting richer still.

  7. Thank you Mr. Martel. This investigation will be applauded and in my opinion should have happened when an approval for a “Chancellor” in this city was passed. My granddaughter has been failed tremendously with uncertified teachers and administrators. We need are EDUCATORS, not teachers. DC gets a “F”.

  8. Erich Martel permalink

    Mercedes and Diane,
    Thank you for giving this the wider coverage it needs!
    I want to acknowledge others who have written about the hypocritical claims of progress since the imposition of mayoral control: Guy Brandenburg, WashPost columnist Valerie Strauss and an insightful person who frequently comments on Wash Post education articles and goes by the handle efavorite.
    Erich

  9. Erich Martel permalink

    sallyo57, The biggest changes in DCPS since 2007 and mayoral control have been demographic. If you go to pp8 & 9 of my investigation request (linked above), you will see tables 9 through 12. Tables 9 & 10 show the percentages of Black, Hispanic and White students in grade 4 and in gr8 in 2007 and 2015. In gr4, Black student enrollment dropped from 84% to 64%, a 20 point decline, while White student enrollment rose from 6% to 16%. Black students enrolled in DC schools (DCPS and charters) are disproportionately poor. White students in DC schools are almost entirely from middle income and higher homes; by contrast, White students in most other TUDA cities include a percentage of poor white students.
    In grade 8, Black student enrollment declined 16 points, white student enrollment rose 6 points, an indication of the rolling impact of gentrification on the public schools. When I applied these NAEP-TUDA statistics to the raw enrollment numbers in the annual October audit (tables 11 & 12), Black student enrollment fell by 711 students in grade 4. In grade 8, there were 917 fewer Black students and 120 more white students.
    You can expect this trend to continue as gentrification consumes the entire city!
    In other words, Henderson and her supporters are really celebrating Black student removal, but are calling it “improvement.”
    The %s of disadvantaged (F/R Lunch eligible) students’ in 2015 is exaggerated: when a school’s enrollment of F/R Lunch eligible students surpasses 60% (approx.), NAEP simplifies it to 100%.

Trackbacks & Pingbacks

  1. Erich Martel and Mercedes Schneider show that Supposed Big Gains on NAEP in DC mostly evaporate when examined carefully | GFBrandenburg's Blog
  2. Mercedes Schneider: The Rigging of DC Scores to Claim “Improvement” That Didn’t Happen | Diane Ravitch's blog
  3. Pelos States | O Meu Quintal
  4. Ed News, Tuesday, December 13, 2016 Edition | tigersteach
  5. Trump Whisperer Kellyanne Conway Snubbed by DC Private Schools | deutsch29

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: