July 14, 2005

Should we not be jubilant about No Child Left Behind?

This is great news:
The math and reading test, known as the National Assessment of Educational Progress, Long Term Trends, has been given to a representative national sample of 9-, 13- and 17-year-old students every few years since the early 1970's, virtually without modification, and social scientists study it carefully. The results announced today were from a test given to 28,000 public and private school students in all 50 states during fall 2003 and spring 2004. The test had not been administered since 1999.

Nine year old students born in the mid-1990's, on average, earned the highest scores in three decades, in both subjects.

In the reading test, the average score of 9-year-old black students increased by 14 points on a 500-point scale, to 200 in 2004 from 186 in 1999. Reading scores of 9-year-old white students increased by 5 points, to 226 in 2004 from 221 in 1999. As a result, the black-white achievement gap for 9-year-old students narrowed to 26 points from 35 points over those five years. In 1971, the gap was 44 points.

Education Secretary Margaret Spellings attributed the gains among elementary students to President Bush's school reform law, No Child Left Behind. Sounding jubilant, she also credited the nation's teachers, principals and state and national policymakers, including Democrats who have supported the federal law.

Despite Spellings' efforts at sharing the credit, I expect to hear lots of people going out of their way to discredit No Child Left Behind.

26 comments:

Gerry said...

Whatever the cause, it is good news.

John Jenkins said...

Someone needs to remind Madame Secretary that correlation does not mean causation. It would be interesting to know what "virtually without change" means and whether other things were changed, like test-taking conditions (i.e. calculators). There simply aren't enough data to draw any conclusions other than scores are up. Of course, I'm a cynic so I'm guessing the numbers were juggled when they scaled the raw scores.

JamesTr said...

It would be nice to know how the scores had changed prior to 1999. My guess is that they've gone up every year since they started. My question is ... Did they go up at a higher pace this time, then previous tests?

I would like to think that future generations are always smarter than past generations. I'm sure our teachers and ways of teaching have a lot to do with that.

I'm not discrediting "No Child Left Behind" but I'm also not going to jump on the bandwagon and get up cheer over it.

Maybe the numbers would be even higher if they program had been funded as it was supposed to be funded. Just think where our students would be with an extract $200 billion [that was spent needlessly somewhere else].

Gerry said...

James, some but not all of the info you want is on this page.

Gerry said...

And having just looked at it, whatever is going on is mainly localized to the 9 year old cohort. The math trend took a slight uptick for age 13, but the math and reading both took fairly sizeable upswings for the 9 year olds-- more than in any five year period before. The 17 year olds were mainly stagnant.

If they cooked the books, they did so for only one age group.

Its very hard with dynamic systems such as education where the number of variables is impossibly large to nail down the cause of any particular improvement. But I fall into the camp, if you change something trying to elicit an effect, and you see the effect, it might not prove that it caused it, but it is an indicator towards the possibility.

Ann Althouse said...

James: the key point is that the GAP between black and white students is closing very significantly, and it's happening for the younger but not the older students. This suggests something new is happening.

John Jenkins said...

The gap for the 13 year olds narrowed even more to only 22 points.

These data show that this isn't really much of a departure and that the measure is average score. I don't see the median data anywhere, so this data REALLY doesn't tell us anything.

Al Maviva said...

Key factors in education:

1. The most important learning occurs by 4th grade. If a kid can't read well by then, it is very, very difficult to get him on track. Work on the young kids, get results. Work on the older kids... well, good luck.

2. Before NCLB, schools could just result aggregate scores. Typically, districts with large minority populations bused the students around to break up minority concentrations, and to try to defeat "poverty pocket" effects. Test results would be released and would show 80% of students at grade level -nifty, huh? NCLB forced disaggregation of the data, and drove the school districts to be accountable. What was discovered, is that most of the kids flunking in many districts were Black or Hispanic. Instead of having one majority-minority school with a high failure rate, the districts were breaking up the poor students between otherwise decent schools, to hide the scale of their failure. They can't do that any more, without generating outrage among all the parents.

3. The Administration has cracked down on "MinSped" (minorities in special education) problems, mainly the "overclassification" of minority students into special ed classes. Typically, a teacher evaluates a second or third grader, and determines whether they will make it or not. Black and Hispanic children in many districts are written off and pushed into slow moving special ed classes all to easily. By forcing schools to make special ed classifications by more rigorous, hard-to-meet standards, it encourages the schools to make extra efforts to ensure the children really are write-offs. There is also a self-fulfilling prophecy effect attendant to the greater teaching efforts. When teachers are quick to write off a kid, the kid does worse; when the teachers hope and expect a kid to do well, the kid tends to do better.

4. The Administration's efforts to develop race-neutral methods of college admissions have stressed "pipeline" development, a la California, with the public universities taking an active role in improve education for minority kids in "the pipeline" of elementary, middle and high schools. Enrichment materials have been developed, more AP courses are offered, and internships are more extensive. Additionally, with a greater push on academic achievment, expectations are higher - again taking advantage of the self-fulfilling prophecy effect.

Some of the improvement started in the late 90s, but the performance gap was stubborn and unmoving until 2002. It has started to shrink rapidly since then. The statistical materials examining this are available through the Department of Education, which tracks the performance of all schools on standardized tests.

Ann Althouse said...

And let's be clear about the math: it took 28 years to cut the gap by 20% (from 1971 to 1999 the gap is cut from 44 to 35) and but in the 5 years thereafter the gap was cut by an additional 26% (from 1999 to 2004 the gap is cut from 35 to 26). That's pretty impressive! And I would think that as a big gap narrows, it becomes harder, not easier to close it by new increments.

Gerry said...

James, you can find the percentile data elsewhere on that site. The median will be at the 50 percentile mark, by definition.

Mark said...

I think this is one of the payoffs of welfare reform.

Pastor_Jeff said...

Ann,

This is great news! I'm not sure that I agree with your comment about the difficulty of closing this kind of a gap, though. That's true in many areas (a .300 hitter can't improve 20% every year), but here the issue is bringing under-performing students up to a standard that has been shown to be acheivable. That kind of gap has lots of causes, but there is no inherent reason why minority students can't perform on a par with whites.

John Jenkins said...

The gap closed from 44 to 35 in 4 years, from 1971 to 1975. The long term trend is never lower than 29 and never higher than 35 over that period with no change greater than 6 points and most changes 3 points.

1971 44
1975 35 -9
1980 32 -3
1984 32 0
1988 29 -3
1990 35 +6
1992 33 -2
1994 33 0
1996 29 -4
1999 35 +6
2004 26 -9

From 35 to 26 is a shift of 9 and the last shift before that was a shift of +6. It's really a delta of -3 from the 1996 scores, which is wholly consistent with the trend. The aberration is likely the 1999 scores, not this year's scores. I just don't see the excitement

Ann Althouse said...

John: Very interesting. The 1971 to 75 change is amazing and must have made people think the problem was close to being solved. How diappointing to see the plateau after that. You've got a 35 point gap basically sitting there all that time. Hard to know what it means. I guess I am an optimist. I see an improvement and know something very different was done and want to believe we are on our way to a better future. But people in 1975 must have thought the same. You're right that we don't really know very much until we see the next set of scores.

Matt said...

The biggest problem I have is that most achievement tests (by their nature) don't test much more than rote memorization and "spit back." Months of time in many schools are now spent on "achievement test preparation" when they could be spent on teaching something better (and ultimately more useful in the outside world). These test scores don't test a whole lot more than how good a test-taker the students have become.

Ann Althouse said...

Matt: I don't understand how your comment relates to reading comprehension and math tests, the subject of this article.

Pastor_Jeff said...

Ann,

There is reason to be encouraged. There has been statistically significant improvement in both gap reduction and average black reading scores since 1975. At the same time the gap has closed, the baselines have increased. This is good news.

John Jenkins said...

Jeff, I am not sure we can draw any conclusions from these data. The data are ALL averages, not a median one. I couldn't find medians, but these data could be very skewed. For example, a large number of white kids from suburban schools (generally recognized as better than rural and inner city schools, and the one limited set of data for this variable bears this out) could skew the numbers for white students up against those of black and hispanic students. Things could be a lot closer than they appear.

Alternatively, lots of kids from rural areas could outnumber those affluent suburbanites and it could draw down the scores making things look better than they are. Averages just aren't a good measure here and my cynic's eye tells me that means the reporting of averages versus medians was calculated because it made things look better. If I can find those data, I'll come back with them here.

John Jenkins said...

The percentile data are not indexed by race, but the median score on the test has gone up (50th percentile is the median) since 1971, so that much is good, at least insofar as the exam is as difficult as it was then, that's something we can't know.

Incidentally, the gap between blacks and whites is statistically indistinguishable from the gap between whites and hispanics and those who are eligible for free school lunches and those who are not. Given that the poor outnumber blacks or hispanics, that might be considered a better indicator. Unfortunately, I have to get up early tomorrow and do some research on, of all things, equine law, so I am done playing with these data for now...

Bruce Hayden said...

Equine law? I knew things were getting bad, but this? Where will it end?

Gerry said...

John,

You said "I couldn't find medians".

They give the percentiles. The definition of the 50 percentile is that there are equal numbers of scores above and below that mark. That also is the definition of the median.

From the front page I linked to, you can find links to index pages for reading and for math. On each of these pages, immediately below the link to average scale scores (and here) are links to the trend results in terms of percentiles (and here).

Since you were having trouble finding it from my directions earlier, I just provided direct links. But to save you some of the trouble, here are the numbers for the 50th percentile, aka the median.

Age 9, Reading:
1971- 209
1975- 212
1980- 217
1984- 213
1988- 214
1990- 210
1992- 214
1994- 215
1996- 215
1999- 215
2004- 221

Age 9, Math:
1978- 220
1982- 220
1986- 223
1990- 231
1992- 231
1994- 233
1996- 232
1999- 233
2004- 243

At the links above, you can find other percentile scores as well, which gives a good indication of the pattern of distribution for the scores.

Regards,

John Jenkins said...

Gerry,

I did find the percentile scores, and you're right that the 50th percentile is the median score AND that the median score has risen. I noted that last night before you posed, in fact.

What I cannot find is median scores indexed by race for those years. All of the indexed scores are averages. They are averaged SCALED scores at that, so they've already been adjusted up and down for difficulty of each test form.

I'd like to see median data indexed by race before I draw conclusions and since I (1) don't have the raw data and (2) don't have SPSS anymore because lawyers don't use it, I can't get that unless they release it, which they haven't. Even percentiles by race would be nice, but they don't appear to have released that either.

AJ Lynch said...

I heard this great news in a press release from the NEA and the AFT. They have also determined their stark-raving moonbat criticism of NCLB was wrong and they apologize.

On a serious note, I agree with John there may be no or little causation. I believe a good strong economy works wonders in terms of reducing crime and improving the quality of almost everyone's life.

Matt said...

One of the "models" for these testing programs is the TAAS/TEAMS testing program in Texas. Even in my "advanced/honors" High School classes, we were forced to spend a week on "learning about the test." Everything revolved around the standardized test for WEEKS ON END. I think the intent is good, but it changes the focus of learning in the wrong direction from learning for its own sake (as an inherent good) to learning for THE TEST.

I'll also note that these sorts of tests frequently discourage creative thinking and use of outside information. For instance, I scored only "adequate" rather than "exemplary" on my "exit writing" test because I didn't feel the formulaic "essay template" suited what they asked us to do. (It was a "on the one hand, on the other hand" sort of thing that they wanted--only taking a position at the end, whereas I prefer to state my position up front.)

What this data tells us is "our children have gotten better at taking reading and math standardized tests." That may (but does not necessarily) mean that our children have gotten better at reading and math.

John Jenkins said...

Matt, never, *never*, take a law school essay exam ;-)

Matt said...

That's one of many reasons that my law school grades weren't as high as I would have liked, and why I fretted about the bar exam. I HATE writing to a formula of any sort. I prefer putting clarity and directness above formula. Sometimes a formula achieves that. Other times, not so much.