August 14, 2003

David Brooks Gets Burned by Trusting Charles Murray

David Brooks gets burned by trusting the American Enterprise Institute's Charles Murray:

The Atlantic | September 2003 | People Like Us | Brooks: My favorite illustration of this latter pattern comes from the first, noncontroversial chapter of The Bell Curve. Think of your twelve closest friends, Richard J. Herrnstein and Charles Murray write. If you had chosen them randomly from the American population, the odds that half of your twelve closest friends would be college graduates would be six in a thousand. The odds that half of the twelve would have advanced degrees would be less than one in a million...

Ummm... No. Definitely not. Back when The Bell Curve was published, 22.2% of Americans over 25 had bachelor's degrees (an additional 7% had associate's degrees) and 7.5% of Americans over 25 had advanced degrees. Draw 12 people at random from this set, and if my hasty back-of-the-envelope calculation is correct* the odds that half of them will have college degrees is 2.5% (7.2% if we are counting associate's degrees)--not "six in a thousand." The odds that half of 12 people drawn at random from this set will have advanced degrees is 0.1%--not "less than one in a million." I can't for the life of me figure out what calculations Murray was trying to make that would produce his numbers. But whatever calculations he made, he is off by a factor of 4 (or 12, if we are counting associate's degrees) for the college-educated and off by a factor of 100 for those with advanced degrees.

"Does being off by a factor of a hundred (or four) really matter?" you ask. "2.5% or 0.6%, 0.1% or 0.001%, the odds are still low--and the point that American society is not well-mixed is still true. " But Murray's (and Brooks's) point is not that American society is not well mixed. Their point is that American society is totally stratified--and that is surely false.

And there is another point. Brooks's reference to the "first, noncontroversial chapter of The Bell Curve" is hard to read as anything other than a partial attempt to try to rehabilitate the reputation that Charles Murray shattered by writing The Bell Curve. It is worth noting that nothing Charles Murray writes can be trusted without being independently verified, and that even the first chapter of The Bell Curve is "controversial"--that is, flat-out wrong.

*Suppose we draw twelve people at random. The chance that all of the first six we draw will have college degrees is 0.222^6. The chance that all of the last six we draw will not have college degrees is 0.778^6. The chance that both of these things will happen together is the product of those two numbers--0.0000265. But we don't care about the order: we would be perfectly happy if numbers 2, 4, 7,8,9, and 12 had college degrees. So we need to multiply 0.0000265 by the number of possible ways in which six college and six non-college graduates can be ordered. There are (12!)/((6!)(6!)) such ways--924 such ways. Multiplying 0.0000265 by 924 gives us 0.025--our 2.5% number.

Posted by DeLong at August 14, 2003 06:56 AM | TrackBack

I'm too lazy to check the math today, but it looks somewhat worse even. Most people reading the sentence would interpret "the odds that half" as "the odds that EVEN half."

Posted by: dfinberg on August 14, 2003 07:14 AM

The difference between permutations and combinations becomes clearer if you spend a winter with the Eskimos: the permutations keep you entertained, but the combinations keep you warm.

Posted by: James Wimberley on August 14, 2003 07:36 AM

Also you really care about the event that AT LEAST half of your 12 friends are college educaed, not that EXACTLY 6 are college educated

Posted by: artfish on August 14, 2003 07:56 AM

Also you really care about the event that AT LEAST half of your 12 friends are college educaed, not that EXACTLY 6 are college educated

Posted by: artfish on August 14, 2003 07:59 AM

Most people reading the sentence would interpret "the odds that half" as "the odds that EVEN half."

Right, and if you do that the number rises to around 3.2%. It's a bizarre error - whatever dialectical value the thought experiment had would hardly change were we to use 0.6% or 3.2%. That's not to say it has any value at all, of course.

Posted by: Brian Weatherson on August 14, 2003 08:16 AM

I might be misinterpreting the previous post, but aren't the odds 3.175% and 11.315%, respectively? I would expect that you take the binomial results for anything n=6 or greater to say what are the chances of AT LEAST half having their degree (which seems to be more in line with the qualitative argument being made).

Just wondering.

Posted by: Mike Rand on August 14, 2003 08:41 AM

Solving the binomial distribution probability "backwards" to get the p (=prob of being a college grad) if the probability of 6 successes in 12 trials is one in a thousand, I get around 13%. Any possibility that they were using some such-like number in their calculations? Looks like it might be close to the percentage of college grads in the whole population, rather than in the 25+ yr old population Brad's using.

Posted by: maciej on August 14, 2003 08:53 AM

Solving the binomial distribution probability "backwards" to get the p (=prob of being a college grad) if the probability of 6 successes in 12 trials is one in a thousand, I get around 13%. Any possibility that they were using some such-like number in their calculations? Looks like it might be close to the percentage of college grads in the whole population, rather than in the 25+ yr old population Brad's using.

Posted by: maciej on August 14, 2003 08:55 AM

The figure also assumes that there is no age bias amongst your friends, younger cohorts having greater participation in higher education.

If he had chosen ten as the number of friends, things would have been less unlikely still.

Posted by: Jack on August 14, 2003 09:01 AM

Brooks writes:
If you had chosen them randomly from the American population, the odds that half of your twelve closest friends would be college graduates would be six in a thousand.

......

This is the kind of malarkey that should permanently disqualify him from writing about statistics, ever. People read this and completely forget about the 'random' part.

The ordinary person, on reading this tripe, looks around him, finds his twelve friends and checks if they have college degrees. Now, if I look around at my friends, every one of them does. But that is because, I DO NOT PICK MY FRIENDS AT RANDOM FROM THE AMERICAN POPULATION!

sorry for the rant, but it is this sort of drivel that gives statistics a bad name.

The binomial math is correct, of course.

Posted by: Suresh krishnamoorthy on August 14, 2003 12:05 PM

But that is because, I DO NOT PICK MY FRIENDS AT RANDOM FROM THE AMERICAN POPULATION!

That is what Brooks is saying. His larger point is that while we celebrate diversity, in fact, we prefer the company of those most similar to us.

Posted by: Tripp Ritter on August 14, 2003 01:42 PM

Last time I posted here, IE splattered my posts across the board; apologies in advance if it happens again. :/

I Maple-ized the numbers you put up, Prof DeLong, under the assumption that "half" meant "at least half" and not "exactly half". In that case:

* For ordinary college degrees, assuming p = 0.222, the probability is 0.03175, or 3.175%, as Mike Rand said above.
* For ordinary college degrees, assuming p = 0.292, the probability is 0.1058, or 10.48%.
* For advanced college degrees, assuming p = 0.075, the probability is 0.0001105, or 0.011%.

Conversely, trying to derive the results stated...

* For ordinary college degrees, a probability of 0.006 gives us p = 0.1577 = 15.77%.
* For advanced college degrees, a probability of 0.000001 gives us p = 0.03297 = 3.297%.

[For reference, if we're assuming "exactly half", then your forward analysis is correct; but contrary to what maciej said above, when backsolving I get p = 0.1632 = 16.32% for the ordinary degree, and p = 0.03314 = 3.314% for the advanced college degree.]

Posted by: Anarch on August 14, 2003 02:12 PM

Sorry, there's a typo on the second number. It should have read:

* For ordinary college degrees, assuming p = 0.292, the probability is 0.10558, or 10.558%.

Posted by: Anarch on August 14, 2003 03:16 PM

Is it possible that the number arises from not restricting the statistic to Americans over 25--in other words, perhaps he also included children in his statistic. Perhaps dishonest, but technically true (just like our president).

Posted by: cynic on August 15, 2003 06:19 AM

I suppose this is an example of the difference between peer reviewed science, and opinion dressed up as science. If Murrey et al included a footnote on the source of their figures, or even better a footnote with the derivation of the value, we would know if they were miscalculating or skewing the calculation by using unrealistic assumptions.

Ps - I'll eat my bits if there actually is such a footnote.

rbb

Posted by: Mobius Klein on August 15, 2003 10:45 AM

Forget college grads. What are the chances of two *siblings* being in your circle of friends assuming that your friends are drawn randomly from the entire population? What are the chances of two people in a randomly drawn circle of friends being *married* to each other?

The thought experiment is meaningless and deceptive.

Posted by: Simon on August 15, 2003 12:37 PM

Forget college grads. What are the chances of two *siblings* being in your circle of friends assuming that your friends are drawn randomly from the entire population? What are the chances of two people in a randomly drawn circle of friends being *married* to each other?

The thought experiment is meaningless and deceptive.

Posted by: Simon on August 15, 2003 12:39 PM

Forget college grads. What are the chances of two *siblings* being in your circle of friends assuming that your friends are drawn randomly from the entire population? What are the chances of two people in a randomly drawn circle of friends being *married* to each other?

The thought experiment is meaningless and deceptive.

Posted by: Simon on August 15, 2003 12:42 PM

If Murray means exactly 6 people out of 12, then the binomial parameter must be p =0.1632. If Murray means at least half, then p= .1577. On page 47 of “The Bell Curve” Murray gives endnote number 34 as a reference for his statement. According to this endnote, Murray used proportions from the National Longitudinal Survey of Youth. So by inference Murray is claiming that NLSY says about 16% of the adult population has attained a bachelor’s degree. In any case, I don’t see that the difference between 16% and 22% amounts to much.

Here are the Mathematica commands you can use to make the calculations:

FindRoot[PDF[BinomialDistribution[12,p],6] = 006,{p,.1,0.,1.}]

FindRoot[Sum[PDF[BinomialDistribution[12,p],k],{k,6,12}] = 006,{p,.1,0.,1.}]

For get the binomial probabilities for advanced degrees, replace 0.006 by .000001, giving 0.0348 and 0.0349 respectively. This is about half of Brad’s figure of 7.5%. Again I don’t see much of a problem. Either figure seems like a reasonable approximation.

Note if you read Murray’s statement of page 47 carefully, he actually says “more than a million to one.” Not “less than one in a million.” I think the book has a misprint.

What makes the whole first chapter of “The Bell Curve” “flat out wrong?”--certainly not the difference between 16% and 22% or 3.5% and 7.5%.

Posted by: A. Zarkov on August 16, 2003 12:06 AM

Didn't Brooks also get burned by trusting Richard Herrnstein? Or does he get some sort of free pass in disseminating this nonsense? Why isn't Herrnstein's reputation also - and deservedly - in tatters?

Posted by: stacy stein on August 20, 2003 05:12 PM

The first problem that I have is the notion of your 12 closest friends. I don't have 12 closest friends. I have a couple. I have a lot of friends. I have a lot of friendly acquaintances. But the question of whether or not 50% or more are college graduates has nothing to do with statistics. Statistics have no relevance here. How I acquired these friends depends upon the unique circumstances of my own personal history, which varies from person to person. The statistical analysis that would predict a pattern throughout a random sample is absolutely meaningless. There is no intellegence of any use that could be derived from this. And this is obviously to start the proposition off by making some kind of point. But there IS no point to make. Sometimes statistics are valid. Such as 100% of people who drive there car off a bridge and are submerged and cannot escape drown. There is useful information. But what can I do with the odds that half or more of my friends will be college educated? There is a whole useless mind set at work here that is entirely apriori.

Posted by: hologlyph on September 13, 2003 12:43 PM

I wouldn't be surprised if Murray's original numbers factored in the sizable sector of the population that is not old enough to have earned an academic degree.

Posted by: neil on September 13, 2003 01:20 PM

OK, you are in the habit of riding elevators with exactly twelve people in them of all ages. In the US, with only people who live in the US. OTOH, you might be riding in elevators that are X rated, and only open to adults above the age of 25.

josh halpern

Posted by: Joshua Halpern on September 13, 2003 03:32 PM

"Didn't Brooks also get burned by trusting Richard Herrnstein? Or does he get some sort of free pass in disseminating this nonsense? Why isn't Herrnstein's reputation also - and deservedly - in tatters?"

Richard Herrnstein is dead. There's nothing that could be accomplished by rehabilitating his reputation that couldn't also be accomplished by rehabilitating Murray's rep. And Murray has the added advantage of still being around to disseminate his pseudoscience.

Posted by: Chris D. on September 13, 2003 04:54 PM

It looks like only Tripp Ritter has so far pointed out, however, that Brooks, while his calculations may have been foolhardy, is actually making the same point that many of those criticizing him here seem to be echoing: We don't choose our friends randomly. We choose them quite strategically: people like ourselves. And this most often results in folks of similar race, education, and class being friends with each other. Which, oddly enough, proves Brooks' point and argues against what DeLong seems to be implying here- that we're a big happy mixed family.

Posted by: Alex on September 14, 2003 06:58 AM

It looks like only Tripp Ritter has so far pointed out, however, that Brooks, while his calculations may have been foolhardy, is actually making the same point that many of those criticizing him here seem to be echoing: We don't choose our friends randomly. We choose them quite strategically: people like ourselves. And this most often results in folks of similar race, education, and class being friends with each other. Which, oddly enough, proves Brooks' point and argues against what DeLong seems to be implying here- that we're a big happy mixed family.

Posted by: Alex on September 14, 2003 07:03 AM

singal mothers

Posted by: abbas on September 23, 2003 07:05 AM

charles murray is a facist!

Posted by: katie price on November 26, 2003 04:08 AM