A couple of years ago, Sander Greenland and I had an interesting exchange on Deborah Mayo’s website. I tweaked Sander for his practice of calling out defense expert witnesses for statistical errors, while ignoring whoopers made by plaintiffs’ expert witnesses. See “Significance Levels Made a Whipping Boy on Climate-Change Evidence: Is p < 0.05 Too Strict?” Error Statistics (Jan. 6, 2015).1 Sander acknowledged that he received a biased sample of expert reports through his service as a plaintiffs’ expert witness, but protested that defense counsel avoided him like the plague. In an effort to be helpful, I directed Sander to an example of bad statistical analysis that had been proffered by Dr Bennett Omalu, in a Dursban case, Pritchard v. Dow Agro Sciences, 705 F. Supp. 2d 471 (W.D. Pa. 2010), aff’d, 430 F. App’x 102, 104 (3d Cir. 2011).2
Sander was unimpressed with my example of Dr. Omalu; he found the example “a bit disappointing though because [Omalu] was merely a county medical examiner, and his junk analysis was duly struck. The expert I quoted in my citations was a full professor of biostatistics at a major public university, a Fellow of the American Statistical Association, a holder of large NIH grants, and his analysis (more subtle in its transgressions) was admitted” (emphasis added). Sander expressed an interest in finding “examples involving similarly well-credentialed, professionally accomplished plaintiff experts whose testimony was likewise admitted… .”
Although it was heartening to read Sander’s concurrence in the assessment of Omalu’s analysis as “junk,” Sander’s rejection of Dr. Omalu as merely a low-value target was disappointing, given that Omalu also has a master’s degree in public health, from the University of Pittsburgh, where he claims he studied with Professor Lew Kuller. Omalu has also gained some fame and notoriety for his claim to have identified the problem of chronic traumatic encephalopathy (CTE) among professional football players. After all, even Sander Greenland has not been the subject of a feature-length movie (Concussion), as has Omalu.
I lost track of our exchange in 2015, until recently I was reminded of it when reading an expert report by Professor Martin Wells. Unlike Omalu, Wells meets all the Greenland criteria for high-value targets. He is not only a full, chaired professor but also the statistics department chairman at an ivy-league school, Cornell University. Wells is a fellow of both the American Statistical Association and the Royal Statistical Society, but most important, Wells is a frequent plaintiffs’ expert witness, who is well known to Sander Greenland. Both Wells and Greenland served, side by side, as plaintiffs’ expert witnesses in the pain pump litigation.
So here is the passage in the Wells’ report that is worthy of Greenland’s attention:
“If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the same population.”
In re Testosterone Replacement Therapy Prods. Liab. Litig., Declaration of Martin T. Wells, Ph.D., at 2-3 (N.D. Ill., Oct. 30, 2016). Unlike the Dursban litigation involving Bennett Omalu, where the “junk analysis” was excluded, in the litigation against AbbVie for its manufacture and selling of prescription testosterone supplementation, Wells’ opinions were not excluded or limited. In re Testosterone Replacement Therapy Prods. Liab. Litig., No. 14 C 1748, MDL No. 2545, 2017 WL 1833173 (N.D. Ill. May 8, 2017) (denying Rule 702 motions).
Now this statement by Wells surely offends the guidance provided by Greenland and colleagues.3 And it was exactly the sort of misrepresentation that led to a confabulation of the American Statistical Association, and that Association’s consensus statement on statistical significance.4
And here is another example, which occurs not in a distorting litigation forum, but on the pages of an occupational health journal, where the editor in chief, Anthony L. Kiorpes, ranted about the need for better statistical editing and writing in his own journal. See Anthony L Kiorpes, “Lies, damned lies, and statistics,” 33 Toxicol. & Indus. Health 885 (2017). Kiorpes decried he misuse of statistics:
“I am not implying that it is the intent of the scientists who publish in these pages to mislead readers by their use of statistics, but I submit that the misuse of statistics, whether intentional or otherwise, creates confusion and error.”
Id. at 885. Kiorpes then proceeded to hold himself up as Exhibit A to his screed:
“Remember that p values are estimates of the probability that the null hypothesis (no difference) is true.”
Id. Uggh; we seem to be back sliding after the American Statistical Association’s consensus statement.
“Almost all scientists have stated (or have been tempted to state) something like ‘the mean of Group A was greater than that of Group B, but the difference was not statistically significant’. With very few exceptions (which I will mention below), this statement is nonsense.”
* * * * *
“What the statistics are indicating when the p-value is greater than 0.05 is that there is ‘no difference’ between group A and group B.”
Id. at 886.
Let’s hope that this gets Sander Greenland away from his biased sampling of expert witnesses, off the backs of defense expert witnesses, and on to some of the real culprits out there, in the new year.
1 See also “Sander Greenland on ‘The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics’” (Feb. 8, 2015).
2 See also “Pritchard v. Dow Agro – Gatekeeping Exemplified” (Aug. 25, 2014); “Omalu and Science — A Bad Weld” (Oct. 22, 2016); Brian v. Association of Independent Oil Distributors, No. 2011-3413, Westmoreland Cty. Ct. Common Pleas, Order of July 18, 2016 (excluding Dr. Omalu’s testimony on welding and solvents and Parkinson’s disease).
3 See, e.g., Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman, “Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations,” 31 Eur. J. Epidem. 337 (2016).
4 Ronald L. Wasserstein & Nicole A. Lazar, “American Statistical Association Statement on statistical significance and p values,” 70 Am. Statistician 129 (2016)
