Lost in Un-chartered Waters: A Newspaper Drowning

It's only August, but Samuel G. Freedman wins the Economic Bozo of the Year Award. To win this award, you must make comments that would earn you an F in any introductory course in economics at any junior college in the country. His article is the culmination in a series of NYTimes articles that could only be described as a journalistic train wreck. So let's do this in chronological order. First, Diana Jean Schemo’s article reproduced the research fed to her by the American Federation of Teachers. Schemo and the AFT drew some interesting inferences from the data:

The first national comparison of test scores among children in charter schools and regular public schools shows charter school students often doing worse than comparable students in regular public schools.

But the results in the report do not compare charter school students to *comparable* students in other schools. The study shows that charter school students often under perform students in other public schools but most of the difference goes away when a control for race is included. The fact that results are not robust to the introduction of a single control variable (the usual econometric standard is that many family background variables should be included) invalidates the inference.

Further, you need at least two consecutive test scores to discern whether a school is doing anything for its students, to discern whether students are learning, whether they are improving. In other words, you need a reference point from which to judge whether students have made progress. The data here have only one score. Thus, the data do not and cannot speak to how well charter schools are performing.

What the analysis does show is that charter schools appear to be drawing disadvantaged students. This is an ecouraging sign. Charter schools are not cherry-picking, but serving needy students. Whether they will succeed in large numbers is anybody's guess at this point. (This is not a pro-charter schools post. It is a pro-sanity post.)

Eduwonk, Kausfiles and others have already written extensively about the flaws in the AFT inferences. The article was based on bad social science. Period. I do not blame the AFT. You don’t blame a dog for having 4 legs and a tail. The AFT is a special interest group with an agenda. If Exxon Corporation were to feed the New York Times a study saying that oil spills are good for the environment, would the Times run that study, unquestioned? Would the Times parrot Exxon's conclusions? The paper's reputation would plummet if it did. But this is exactly what the Times did here.

But, wait... It gets worse... Shortly after the Schemo article appeared, the Times editorialized thusly:

The Bush administration's education program received a devastating setback this week when long-awaited federal data showed that children in charter schools were performing worse on math and reading tests than their counterparts in regular public schools. Among other things, the data casts doubt on a central provision of the No Child Left Behind Act…

[Emphasis mine.] Devestating setback?? The data show absolutely nothing, and you don't have to be a Nobel Laureate to figure that out. This is either gross negligence on the part of the editorial staff or deliberate propaganda. Has the Times thrown reason, responsibility, and integrity to the wind in pursuit of, say, a Kerry presidency? So it seems.

But wait! It gets worse...

Under the headline "Report Offers No Clear Victory for Charter School Opponents," Samuel G. Freedman writes a follow-up article. The headline contradicts the Times editorial, does it not? Where is the "devestating setback" to Bush's education policies that the Times announced so boldly? Has the Times retreated from this statement? And if so, shouldn't the Times issue a clear retraction saying that the "devestating setback" editorial was dead wrong? Wouldn't journalistic integrity require that?

Stick with me… the Bozo award is coming up…

The usually thoughtful Eduwonk fawns over Freedman's column , but I cannot imagine why. Rather than coming across as an indictment of botched coverage, the Freedman column oozes smugness. What is the lede in the piece? Three paragraphs of gloating. I kid you not. What does he or his paper have to gloat about, you ask? Read for yourself:

Anyone who has tired of hearing more than 20 years of relentless attacks on public education, dating back to the Reagan administration report on "A Nation At Risk," could be forgiven the temptation to gloat last week, when the critics were hoisted with their own petard of standardized testing.

Logical flaws abound here:

1. Yes, those who have attacked public schools have used selective and often misleading statistics, as have organizations like the AFT who have sought to defend public schools and avoid possibly useful reforms. To here Freedman tell it, one side has been using bad statistics and bad methods while the other side, the blessedly pure and longsuffering AFT, after 30 years of virtue, has finally given the evil right-wingers a taste of their own medicine. What nonsense!

2. This is not even a public school versus private school debate! Hello? Anybody home? Charter schools ARE public schools. The gloating is irrelevant.

3 Few honest researchers are without concern for public schools. Freedman makes it sound as though harsh criticism of public schools is the purview of extremist know-nothings. The grounds for criticism of the public schools are real, disturbing, and in no way paranoid. Studies far more sophisticated than the self-serving AFT report--studies that try to compare apples to apples--have generated reasons for concern.

4. Those who argued for education standards (far from being “hoisted” on some hackneyed “petard”) wanted them for precisely this reason: so that successful approaches could be identified and failing approaches could be spotted and remedied.

Lost in some sort of delusional revenge fantasy (it's almost embrassing to read: you figure he should be doing this in the privacy of his home), Freedman continues with his gloating:

Sweeter still, charter school proponents, led by the secretary of education, Rod Paige, were left to sputter out all the rationales and excuses heard so often from apologists for dismal public schools - that they have a concentration of poor, troubled, and nonwhite pupils; that no single test can fairly measure performance; and that the data was analyzed in the wrong way.

What exactly is the point? Proponents of charter schools were left to sputter… the truth? Sputter? Is that how one usually speaks the truth? And why, exactly, is this sweet? Many charter school proponents have always respected sound statistical methodologies. There have been lots of valid reasons for criticizing public schools. So it is somehow “sweet” when someone, somewhere, uses a bad report to draw bad inferences about charter schools? (Which, of course, ARE public schools to begin with!) There is no logic here. None. Zero.

And why spend 3 paragraphs on… mock gloating? Put aside the lack of logic. The tone of this article is infantile. Fantasizing about humiliating the evil Bush administration, I suppose, is so important that reasoning or--heaven forbid--actually engaging in journalism, is a vanishing afterthought.

It goes on and on:

Now, however, is the time to let go of the guilty pleasures of payback.

Oh, Ok, so after 3 paragraphs of bizarre, delusional, and illogical fantasizing, Freedman tells us we should not gloat. This is like the neighbor who says, “If I were a gossip I’d tell you that Bill’s wife is having an affair but since I’m not a gossip I won’t tell you.” Sorry, but it’s way too late to pass yourself off as a grown-up.

So what happens when he stops gloating. He suggests, essentially, that charter schools should not be condemned because there are people other than Bushies who like them too. Ah, so if people with the proper ideological credentials like an idea, then the idea must be O.K? Is that really a good reason to be for or against something?

But wait. It gets worse...

Midway through the article, Freedman finally gets to some substance. After admitting, essentially, that the AFT’s take on the statistics is wrong or misleading and giving a few reasons why we should suspend judgment on charter schools, Freedman offers this:

At the same time, the most perceptive supporters of charter schools do recognize the A.F.T. report for the wake-up call that it is.

Nope. Wrong again. The “most perceptive” supporters of charter schools recognize no such thing. A veritable Who’s Who of prominent education economists, some of whom are charter school supporters, emphatically refute that the report says anything at all about charter school performance. I can’t imagine that any economist--regardless of party affiliation--would interpret this report as a wake-up call for charter schools. It just doesn’t follow.

Freedman gets to decide for all the rest of us who the “most perceptive” supporters are, it seems. What makes someone perceptive? you ask. Not his or her expertise as an education economist or an interpreter of statistics. Nope. Not that. What then? (Agreeing with Freedman, maybe?)

But wait, it gets worse. (Here comes the Bozo award...)

After pointing out that some charter schools have been bad (big surprise, that), and that “Most recently, the largest charter school company in California abruptly closed all 60 of its schools, stranding 10,000 pupils only weeks before classes were to begin,” Freedman concludes:

These disparate incidents should have brought reasonable people to a reasonable conclusion: the free market for innovators, visionaries, and idealists is the free market for incompetents, demagogues, and charlatans, too. Nobody wins if things stay that way.

[Emphasis mine]. Anyone who’s ever stood next to an introductory economics textbook and inhaled knows this is wrong. In the free market, the “incompetents,” forced to compete, either improve or are driven out.

THIS IS THE WHOLE POINT.

Markets do not require that bad firms somehow be prohibited from competing. Bad firms exit or improve. This is what a market is, for heaven’s sake. The idea that “nobody wins if things stay that way” is just ignorant. The market for a college education works "that way," and lots of people win. Hell, almost every market works that way. Incompetents and ideologues are allowed in, but get competed out.

(There is, in fact, intriguing evidence that market forces operate as expected in the charter school setting. "Parent decisions to exit a school appear to be much more sensitive to education quality in the charter sector than in regular public schools." Bad charter schools, then, would appear to be more likely to be "weeded out." See here for an academically respectable study.)

I don’t know whether market mechanisms will work well enough to be useful in charter schools. It's an open question. It will be interesting to find out, as time passes. But Freedman's claim is nonsense, any way you look at it. The thing that seals the deal for Freedman’s Bozo award, however, is the smugness. He claims that all “reasonable people” should agree with his embarassing misinterpratation of Econ 1A. How's that for unearned smugness?

A word of advice to Times writers: Before you use a term like “free market” and before you start telling everyone else what is and is not reasonable to believe, try to make sure you have at least a passing familiarity with what the term actually means.

In any event, I’m left to assume that by Freedman’s standards, and perhaps by the standards of the New York Times, I’m not a “reasonable person.”

Lucky me.

(If the folks responsible for this journalistic train wreck think you’re “reasonable” or “perceptive,” it just might be time to jump off a bridge …)

Philip S. Babcock

Comments

[Vinayak Nagaraj]

Wow...!

I hope you sent the Times this response. Sounds like they need a good wake up call. Responsible journalism just doesn't seem to exist anymore. Perhaps the man needs a kick in his ***, and a copy of "Economics" to set things right.

Uhm Mr. Freedman...

That the California schools closed down is a good sign that the market forces are indeed operating just the way they should be.

[P.S.Babcock]

Exactly, Vinayak. People in this debate who don't know what a market is always think the closing of some specific charter school (or 60 schools, in Freedman's quote) spells disaster for the very idea of charter schools. But the real problem with failing traditional public schools has been that **they don't close.** Ever.

This is the whole point.

Example: It took the better part of a decade for public schools in California to realize theat the "whole word" reading method wasn't working. Huge numbers of kids weren't learning how to read. It went on for years and years. If competitive pressures had forced administrators to acknowledge this problem more quickly, things would have turned out much better, even if that meant some schools had to close.

Alas, I have been told that the Times almost never publishes scathingly critical letters like mine, whether they have merit or not. But at least there's the blogosphere. I'm hoping that this post (and critiques elsewhere online) will get some play.

It's getting harder and harder for major newpapers to pass off pretentious posturing and plain old bullshit as "analysis" without being exposed.

Here's to the bloggers...

[Chris]

Phil, if you pen a good letter I'd be happy to sign my name to it. Probably wouldn't help you very much but this sort of idiocy drives me nuts.

[Zogg]

Go get 'em, boy. I used to read the NYT every day when I lived in NY, feeling kinda smug about the intelligent language I was reading while folks in Kansas City were stuck with a daily 7-word summary on Europe. Now that I have been on the Internet, I wouldn't ever buy the NYT if it weren't for their excellent crossword puzzle. They, perhaps of their "newspaper of record" status, epitomize the hypocrisy of believing oneself an objective reporter entrusted with delivering truth to the public. I don't know how to make reporting better, but I do think that editors should pay more attention to obvious bias figleafed by regurgitated spin [metaphors in a blender, sorry], as you pointed out so well.

Aloha.

[p.s.babcock]

Glad to hear it annoys all of you as much as it does me. I did look into writing a letter to the Times, but get this: They allow a maximum of 150 words. (My post above is about 2000 words.) Can't begin to tell this story in 150 words. Back to blogging...