It has become très chic to criticize and dismiss the concept of statistical significance.
The new Reference Manual on Scientific Evidence contains a sly reference and endorsement to a book by the two would-be statistics experts who submitted an amicus brief to the Supreme Court in Mattrix Initiatives v. Siracusano:
“For a hypercritical assessment of statistical significance testing that nevertheless identifies much inappropriate overreliance on it, see Stephen T. Ziliak & Deidre N. McCloskey, The Cult of Statistical Significance (2008).”
Michael D. Green, D. Michael Freedman, and Leon Gordis, ” Reference Guide on Epidemiology” 549, 579, in Federal Judicial Center and National Research Council, Reference Manual on Scientific Evidence (3d ed. 2011).
The Reference Manual authors are, in fact, hypo-critical of the rhetoric of Ziliak and McCloskey. I have previously written at some length of these authors’, and brief writers’, submission to the Supreme Court, and their subsequent harrumph in Significance. See The Matrixx Oversold; Matrixx Unloaded; The Matrixx – A Comedy of Errors; Matrixx Galvanized – More Errors, More Comedy About Statistics; and Ziliak Gives Legal Advice — Puts His Posterior On the Line (June 2, 2011).
To date, I have not addressed Ziliak and McCloskey’s book-length treatment of statistical significance, and their demonization of Sir Ronald Fisher, their beatification of William Gossett, and the need for a measure of “ummph: The Cult of Statistical Significance. Thankfully, Professor Deborah Mayo, a professor of statistics and philosophy, has delivered the coup de grâce to Ziliac & McCloskey, in her interesting and timely blog, Error Statistics Philosophy. See, e.g., Part 2 Prionvac: The Will to Understand Power (October 3, 2011); and Part 3: Prionvac: How the Reformers Should Have done Their Job (October 4, 2011).
Mayo’s “will to understand power” is a nice play on Nietzsche, and a rebuke of Ziliak and McCloskey’s strident call for a measure of “ummph.” Most of their argument is beside the point for the current practice of epidemiology, which insists upon reporting a measure of “effect” size, as well as statistical precision in a confidence interval. The Reference Manual‘s citation to Ziliak & McCloskey’s book thus badly misses the point and the errors of the book’s criticisms of significance.
Mayo’s posts should remove any sense of need or desire to obtain and read Ziliac & McCloskey’s book. You can safely wait until Cult shows up in on the discount rack, or in the recycling pile.
