In 1973, UC Berkeley was sued for gender bias, Simpson's Paradox and Statistical Urban Legends: Gender Bias at Berkeley

In 1973, UC Berkeley was sued for gender bias

https://www.refsmmat.com/posts/2016-05-08-simpsons-paradox-berkeley.html

Simpson's Paradox and Statistical Urban Legends: Gender Bias at Berkeley

Simpson's Paradox and Statistical Urban Legends: Gender Bias at Berkeley

Alex Reinhart – Posted May 8, 2016the refsmmat report · a blog at refsmmat.com

Introductory statistics textbooks usually point out Simpson’s paradox, an interesting phenomenon that’s usually illustrated with a story from the University of California, Berkeley. The story goes something like this:

In 1973, UC Berkeley was sued for gender bias, because their graduate school admission figures showed obvious bias against women:¹

	Applicants	Admitted
Men	8442	44%
Women	4321	35%

Men were much more successful in admissions than women, leading Berkeley to be “one of the first universities to be sued for sexual discrimination”. (The difference is statistically significant with p ≈ 10^-26!) The lawsuit failed, however, when statisticians examined each department separately. Graduate departments have independent admissions systems, so it makes sense to check them separately—and when you do, there appears to be a bias in favor of women.

How does this happen? The simple explanation is that women tended to apply to the departments that are the hardest to get into, and men tended to apply to departments that were easier to get into. (Humanities departments tended to have less research funding to support graduate students, while science and engineer departments were awash with money.) So women were rejected more than men. Presumably, the bias wasn’t at Berkeley but earlier in women’s education, when other biases led them to different fields of study than men.

Now, this example has been analyzed to death in many places: on Wikipedia, in various blogs, in many textbooks (including my own book), and pretty much everywhere else. I’m not going to present a new analysis of the data or of Simpson’s paradox.

I just want to point out something simpler: There never was a lawsuit!

The real Berkeley story

A Wall Street Journal interview with Peter Bickel, one of the statisticians involved in the original study, makes clear that Berkeley was never sued—it was merely afraid of being sued:

Simpson’s Paradox has fooled many. In the fall of 1973, for instance, the University of California, Berkeley’s graduate division admitted about 44% of male applicants and 35% of female applicants. That raised eyebrows among school officials, who feared bias and asked Peter Bickel, now a professor emeritus of statistics at Berkeley, to analyze the data.

“The associate dean of the graduate school thought that the university might be sued,” Mr. Bickel says.

When Mr. Bickel and his colleagues scrutinized the data, they found little evidence of gender bias. Instead, they discovered that more women had applied to departments that admitted a small percentage of applicants, like English, than to departments that admitted a large percentage of applicants, like mechanical engineering.

The core paradox matches the usual story, but no lawsuit was involved. I’ve done some digging and I haven’t been able to find the original source of the mythical lawsuit—perhaps an early textbook or journal article author misheard the original story, wrote about a lawsuit, and authors ever since have copied the story unchanged.

Academic urban legends

A scan through Google Books reveals this urban legend has infected many recent books, and undoubtedly many older ones:

• Paradoxes in Scientific Inference (2012), by Mark Chang
• Learning Statistics with R, by Daniel Navarro
• R Graphics Cookbook (2013), by Winston Chang
• Math on Trial: How Numbers get Used and Abused in the Courtroom (2013), by Leila Schneps and Corali Colmez. Surprisingly, this book makes the claim in an entire chapter dedicated to Simpson’s paradox, after carefully explaining a separate lawsuit about gender bias in Berkeley’s faculty hiring process. After explaining the suit in detail, the mythical graduate admissions lawsuit is thrown in as an aside.
• Einstein’s Riddle: 50 Riddles, Puzzles, and Conundrums to Stretch Your Mind (2009), by Jeremy Stangroom
• Impossible? Surprising Solutions to Counterintuitive Conundrums (2011), by Julian Havil

It’s also present in the scientific literature:

• Harris Cooper and Erika A. Patall. “The relative benefits of meta-analysis conducted with individual participant data versus aggregated data,” Psychological Methods (2009), vol 14(2), pp. 165-176. 10.1037/a0015565. “Perhaps the best known example of Simpson’s paradox involves a lawsuit brought against the University of California at Berkeley alleging bias against women in the selection of graduate school applicants”
• Stanley A. Taylor and Amy E. Mickel. “Simpson’s Paradox: A Data Set and Discrimination Case Study Exercise,” Journal of Statistics Education (2014), vol 22(1). “An example of this phenomenon is when the University of California, Berkeley was sued for bias against women who had applied for admission to graduate schools in 1973.”
• Matteo G A Paris. “Two quantum Simpson’s paradoxes,” Journal of Physics A (2012), vol 45(13). 10.1088/1751-8113/45/13/132001. “One of the best known real life examples of Simpson’s paradox occurred when the University of California, Berkeley was sued for bias against women who had applied for admission to graduate schools there.”

Now, this isn’t the first time scientists have unwittingly propagated a myth through decades of the literature. The best example is possibly the century-old myth that spinach is a rich dietary source of iron: even the myth is a myth, as it turns out the common explanation (that German chemists measuring the iron content of spinach had misplaced a decimal point) is also a myth, propagated through the literature by scientists copying references without checking up on their provenance.²

It seems Simpson’s paradox has experienced a similar problem. Someone, perhaps back in the 1970s or 1980s, before the story was easily Googleable, wrote about a lawsuit to spice up their example, and the story has been repeated ever since.³ I only detected the problem because I am the type of nerd who wonders “Is the court opinion in this case available online? I’d love to read what the judge thought of the statistics”, and so I started hunting for a nonexistent court case.

Perhaps when we use stories to illustrate common statistical errors, we should make sure our stories are not in error as well.

---

1. Bickel, P. J., Hammel, E. A., & O’Connell, J. W. (1975). Sex bias in graduate admissions: Data from Berkeley. Science, 187(4175), 398–404. http://doi.org/10.1126/science.187.4175.398↩

2. Rekdal, O. B. (2014). Academic urban legends. Social Studies of Science, 44(4), 638–654. http://doi.org/10.1177/0306312714535679↩

3. Kudos to the anonymous Wikipedian who noticed that none of the Simpson’s Paradox article’s sources could confirm a lawsuit and removed the mention.↩