September 25, 2002

More Sleazy New York Times-Bashing From Slate

*Slate* appears to have developed a *bad* quality control problem.

Another example of sleazy *New York Times* bashing from yesterday's *Slate*: Mickey Kaus's claim to report "some things the NYT didn't tell you about those new Census stats,"--including that during the recession of 2001 "the child poverty rate for [B]lacks actually continued to fall."

Did the Black child poverty rate continue to fall between 2000 and 2001. The right answer is that we cannot tell. The Black child poverty rate is not a very precisely estimated number. If I remember CPS sample sizes correctly, the Current Population Survey's point estimate of Black child poverty in 2001 of 30.2 percent comes with a 95-percent confidence bound of plus or minus 1.2 percent--that is, under classical statistical assumptions, there is a 95 percent chance that the range from 29.0 percent to 31.4 percent contains the true 2001 value. In 2000 the CPS's point estimate of Black child poverty was 31.2 percent, meaning that there is a 95 percent chance that the range between 30.0 percent and 32.4 percent contains the true 2000 value.

Notice one key thing: the confidence interval ranges overlap. More than 1/2 of the year-2000 range falls within the year-2001 range. More than 1/2 the year-2001 range falls within the year-2000 range. What does this mean? It means that we can say that the point estimate for 2001 is less than the point estimate of 2000. It means that we can say that the data make it look more likely than not that Black child poverty continued to fall. But we cannot say--at least not if we remember anything from our statistics courses--that we have *confidence* (at the 95 percent level) that it continued to fall.

I wish the *New York Times* would talk of confidence intervals and sampling variability, but its editors have made a judgment that such discussions would lose too many of their readers. It doesn't. But this failure to talk about the uncertainties of sample-based estimates leaves the door open for sleazy attacks like *Slate's*.

Is it fair to speak with an air of certainty ("the child poverty rate for [B]lacks actually continued to fall") when the confidence intervals have a substantial overlap? No. Something more along the lines of "the data suggest..." coupled with a "the difference, however, is not statistically significant" is called for.

Is it fair to bash *New York Times* reporters for not parroting the party line of Mickey Kaus's overstatement--an overstatement for which the CPS does not provide a confident statistical base? Of course not.

Black Child Poverty Decreases in a Recession? - Some things the NYT didn't tell you about those new Census stats. By Mickey Kaus: ...The economy went into a recession, poverty went up. That usually happens. But at least two things happened in this recession that don't usually happen. (1) Welfare caseloads continued to fall... child poverty was unchanged -- and the child poverty rate for blacks actually continued to fall...

You can read the Census Bureau's Poverty in the United States: 2001 report. It will tell you about many of the confidence intervals for the 2000 and 2001 estimates and for the estimated yearly change (see table 1 on page 3). However, the table that reports point estimates of Black child poverty (table A-2, in the appendixes, page 28) does not report the associated confidence intervals: a presentational mistake, I think.

Posted by DeLong at September 25, 2002 10:16 PM | TrackbackEmail this entry

Oh come on. If Slate has a bad quality control problem for not mentioning confidence intervals and statistical significance in a few lines of a blog post, then what kind of problem does the NY Times have, when they decide not to mention them in their full-length articles?

Posted by: rncarpio on September 26, 2002 12:59 AMEr...you miss the point. The New York Times didn't say the 'black poverty level rose', it didn't mention it. Slate is picking it up for not saying that it fell. Therefore the onus is on Slate to get its facts right.

Posted by: Robert Ponter on September 26, 2002 02:20 AMKaus' argument has plenty of other holes. The data is for the whole of 2001, which means that his claim that we have seen a full cycle is wrong. And at least in 2001, the recession basically hit male manufacturing workers. The impact on services, where most women are employed, didn't really take place until this year.

Posted by: John Quiggin on September 26, 2002 04:34 AMOf course, Kaus is hardly the only person to fall into this trap. Everybody likes to pretend that statistics are absolute numbers, and only rarely do pundits and journalists bother to tell you that the three-percentage-point rise or fall in the President's approval rating is actually within the error bars (and the error bars on political polls always seem awfully optimistic to me...).

Bob Park of the APS does a very entertaining talk on the use of bad statistics to sell bad science, particularly in medicine. Politics is even worse.

The problem is not so much Slate's NYT poor quality conrtol, but Kaus' continual slide. I don't recall him being nearly so conservative and single-minded about the Times before he got the Slate blog. I also don't recall him being so petty. Maybe he needs more editing. [Ed. -- By someone other than himself.]

Posted by: Ethan on September 26, 2002 06:55 AMTwo points:

1) The best estimate we have is that 3 of 10 black kids are in poverty -- and Slate is congratulating itself for finding out that that number didn't go up? Right on the nits, perhaps, but way wrong on the substance. Three of ten is still a horrendous problem--the criminals of tomorrow. That's the real story, and Slate misses it compeletly

2) The CPS one which the report is based is taken as of March of 2001 -- 18 months ago. Anyone care to bet that the poverty rate (for all segments) hasn't risen since then?

Posted by: Don Freeman on September 26, 2002 07:45 AMKaus had become a parody of journalism; now he's a parody of a parody.

Posted by: nick sweeney on September 26, 2002 09:09 AMSlate has become a vehicle for distortion. That is why the New York Times is such a problem for them. Was it 7 Pulitizer Prizes the Times won last year, and 0 for Slate? Duh....

Posted by: on September 26, 2002 09:31 AM"the Current Population Survey's point estimate of Black child poverty in 2001 of 30.2 percent comes with a 95-percent confidence bound of plus or minus 1.2 percent--that is, under classical statistical assumptions, there is a 95 percent chance that the range from 29.0 percent to 31.4 percent contains the true 2001 value."

No! No! No! If you want a probability distribution for the true value of a point estimate, then please use Bayesian methods and do not mention 95% confidence bounds. What confidence bounds tell you is that in 95% of similar surveys, the estimate of the value will be no more than 1.2 percentage points from the (unknown) true value. You simply cannot construct a probability distribution of the true value based on your estimate and confidence bounds. (If this survey is one of the outliers, and the estimate is, say, 3 percentage points away from the true value, then there is a 0% probability "that the range from 29.0 percent to 31.4 percent contains the true 2001 value" -- the true value lies where it lies, after all.)

This is not to say that confidence bounds are not useful, or do not inspire confidence. They just do not produce a probability distribution for the true value, and it is misleading to say they do.

(My thesis advisor was Professor of Math and Stat (Emeritus) at UCB, Lester Dubins. This rant is adapted from one he gave often.)

Posted by: David Margolies on September 26, 2002 10:12 AMBrad

The point of attacking the Times has nothing to do with any particular issue, rather it is to call to question all issues that touch on social equity.

Slate editors have no concern with truth, merely with keeping the world safe for their style of conservatives.

Posted by: on September 26, 2002 10:26 AMThe poverty rate for African-American children is a disgrace, a disgrace. We must do all we can to reduce it. Imagine, 30% of African-American children live in poverty and some hack reporter finds comfort.

We must be about lessening the poverty rate for African-American children, all children.

Posted by: on September 26, 2002 11:04 AMInteresting how the sadness that 30% of African-American children live in poverty is not the real focus.

Anne

Posted by: on September 26, 2002 12:04 PMUhh, I don't think I buy this criticism of Kaus. Even the factoid that "poverty went up"--the title statistic for the article--relies on similarly shaky confidence intervals: From table 1 of the report (linked by both Kaus and DeLong) we have the poverty rates of 11.3% in 2000 (±0.2%, with a *90%* confidence interval) and 11.7% in 2001 (again ±0.2% with 90% CI). The 95% confidence intervals which DeLong wants to use (I haven't read the report closely enough to know whether the values he quotes are correct) are not specified, but are probably about twice these, i.e., about ±0.4%. So the overlap in the 95% confidence intervals for the overall rise in the poverty rate is nearly as large as the one for the black under-18-years figure.

So if you're going to criticize Kaus for not using confidence intervals correctly, you'd better criticize the NYT too--especially since in the NYT's case the relevant table actually lists the appropriate intervals. The NYT is using the point estimates you disdain in its article; it seems reasonable that Kaus, in his reply, should do the same.

Posted by: Dave on September 26, 2002 01:15 PMThe confidence interval issue is a red herring.

As noted by David Margolies (above), you can't use them in the manner Brad and some others here want to.

If you want to delve into the significance - or lack thereof - between the 2,000 and 2001 point estimates you would have to obtain the raw data and perform an analysis of variance (ANOVA) comparing the two years.

Then - and only then - would you be able to meaningfully discuss the probability that the two years data suggest a change in black poverty levels.

Posted by: E. Avedisian on September 26, 2002 01:46 PMPS you could use a T-test for just comparing two years.

ANOVA would allow you to assess the significance of differences between more than two years data.

Posted by: E. Avedisian on September 26, 2002 02:11 PMAn interesting post and (mildly interesting) responses. There's one central point, however. People like Kaus (and his middlebrow ilk like Sully) are Bayesians. (That is, to the extent they have any real statistical training at all. The closest Sully has come to an estimator is mating with his statistician boyfriend.) They have very strong priors (welfare rolls will fall once we get rid of welfare as we know it, Bush is right, etc). Therefore, it takes a ton of evidence to the contrary to move them off their priors.

Posted by: Tim on September 26, 2002 02:55 PMAlthough the NYT did not explicitly talk about confidence intervals and such, they did add asterisks to some of the figures (in the tables on the main page of the article) to point out evolutions that were "statistically insignificant". A very welcome addition.

Posted by: kimon on September 26, 2002 02:58 PMThe NYT continually bungles statistics when it sees fit. In a NYT Magazine piece this summer, they (deliberately) mixed statewide demographics on race with ntional (oversampled) racial demographics to try to prove their preconceived point about Florida's racial demographics. When it was pointed out to them that their conclusion could not possibly be correct, they chose not to correct. As so many others have pointed out, they are attacked because they like to play fast and loose with whatever "truth" suits their political spin.

Posted by: Josef on September 26, 2002 06:48 PMI agree with Brad that the Times should mention more about statistical methods. It might stop such distrust much of the public has with statistics.

Posted by: Sean Hackbarth on September 26, 2002 11:05 PMTo Seth Hackbarth -- it's not distrust of statistics, its distrust of the Times.

Posted by: Josef on September 27, 2002 07:00 AMFair play requires that we ding Prof. DeLong for misquoting Kaus, as:

"the child poverty rate for [B]lacks actually continued to fall."

Here's what Kaus actually wrote:

" 2) even though there were fewer people on welfare, child poverty was unchanged -- and the child poverty rate for blacks actually continued to fall a bit."

Leaving out: "a bit" seems to change the meaning, a lot.

Posted by: Patrick R. Sullivan on September 27, 2002 08:10 AMPatrick Sullivan wrote:

---

`Here's what Kaus actually wrote:

" 2) even though there were fewer people on welfare, child poverty was unchanged -- and the child poverty rate for blacks actually continued to fall a bit."

Leaving out: "a bit" seems to change the meaning, a lot.'

---

Why? If it fell by any amount, `continued to fall' is true, and Kaus's point is that it is still falling. Brad's point is that while it may be falling, and that not rising is more likely than rising, the available statistics do not allow anything like a firm conclusion. Therefore, complaining that someone does not state a firm conclusion is disingenuous, and stating as undisputed fact that the rate fell (as Kaus does) is wrong.

As to leaving out `a bit' beside not affecting the point, one could argue that Brad was just correcting Kaus's lousy writing. Kaus should have said `continued to fall' or `fell a bit' or `continued to fall, if only a bit'. But saying `continued to fall a bit' is bad English.

Disgraceful - Where is the least concern shown about the poverty rate for African-American families and children? The Times printed the article just as it should have been printed.

The criticism of the Times is completely beyond merit and heartless in that it shows not the slightest concern for people who are in need.

Many many African-American children are hurting - get it.

Posted by: on September 27, 2002 08:39 AMQuoting Dave (and if this quote is not literally correct, blame cut and paste!):

"From table 1 of the report (linked by both Kaus and DeLong) we have the poverty rates of 11.3% in 2000 (±0.2%, with a *90%* confidence interval) and 11.7% in 2001 (again ±0.2% with 90% CI). The 95% confidence intervals which DeLong wants to use (I haven't read the report closely enough to know whether the values he quotes are correct) are not specified, but are probably about twice these, i.e., about ±0.4%."

95% confidence intervals are large than 90% confidence intervals by a factor of 1.2, not a factor of 2.

Oops, forgot to attach my name to the post above, starting "Quoting Dave..."

Your description of the 95% confidence interval is wrong. After the interval is constructed there is no more uncertainty, said interval either contains the parameter or it does not. The probabilties involved are 0 and 1, or 0% or 100%.

Posted by: Steve on September 30, 2002 01:01 PMPost a comment