TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

From Here to CERT-ainty

June 28th, 2018

An enterprising journalist, Michael Waters, recently published an important exposé on the Council for Education and Research on Toxics (CERT). Michael Waters, “The Secretive Non-Profit Gaming California’s Health Laws: The Council for Education and Research on Toxics has won million-dollar settlements using a controversial public health law,” The Outline (June 18, 2018). Digging deep into the shadowy organization, Mr. Waters reported that:

“CERT doesn’t have a website, a social media account, or any notable public presence, despite having won million-dollar judgments by suing corporations. However, files from the California Secretary of State show that in May 30, 2001, four people co-founded the non-profit: C. Sterling Wolfe, a former environmental lawyer; Brad Lunn; Carl Cranor, a toxicology professor at University of California Riverside; and Martyn T. Smith, a toxicology professor at Berkeley.”

Id.

Mr. Water’s investigation puts important new facts on the table about the conduct of the CERT corporation. The involvement of Christopher Sterling Wolfe, a Torrance, California, plaintiffs’ lawyer, is not terribly surprising. The involvement in CERT of frequent plaintiffs’ expert witnesses, Carl F. Cranor and Martyn T. Smith, however, raises serious ethical questions. Both Cranor and Smith were expert witnesses for plaintiffs in the infamous Milward case,1 and after the trial court excluded their testimony and granted summary judgment, CERT filed an amicus brief in the Court of Appeals.2

The rules governing amicus briefs in federal appellate courts require disclosure of the amicus’s interest in the proceedings. By the time that CERT filed its amicus brief in Milward, Cranor and Smith may not have been officers of the corporation, but given CERT’s funding of Smith’s research, these “Founding Fathers” certainly had a continuing close relationship with the corporation.3Coffee with Cream, Sugar & a Dash of Acrylamide” (June 9, 2018). Given CERT’s name, which suggests a public interest mission, the corporation’s litigation activities on behalf of its founders, Cranor and Smith, exhibit a certain lack of candor with the court.

======================

My discussions with Mr. Waters, and his insightful piece in The Outline, led to a call from Madeleine Brand, who wanted to discuss CERT’s litigation against Starbucks, under California’s Proposition 65 laws, over acrylamide content in coffee. David Roe, a self-styled environmental activist and drafter of California’s bounty hunting law, was interviewed directly after me.4

As every California now no doubt knows, acrylamide is present in many foods. The substance is created when the amino acid asparagine is heated in the presence of sugars. Of course, I expected to hear Roe defend his creation, Proposition 65, generally, and the application of Proposition 65 to the low levels of acrylamide in coffee, perhaps on contrary-to-fact precautionary principle grounds. What surprised me were Roe’s blaming the victim, Starbucks for not settling, and his strident assertions that it was a long-established fact that acrylamide causes cancer.

Contrary to Roe’s asseverations, the National Cancer Institute has evaluated the acrylamide issues quite differently. On its website, the NCI has addressed “Acrylamide and Cancer Risk,” and mostly found none. Roe had outrageously suggested that there were no human data, because of the ethics of feeding acrylamide to humans, and so regulators had to rely upon rodent studies. The NCI, however, had looked at occupational studies in which workers were exposed to acrylamide in manufacturing processes at levels much higher than any dietary intake. The NCI observed “studies of occupational exposure have not suggested increased risks of cancer.” As for rodents, the NCI noted that “toxicology studies have shown that humans and rodents not only absorb acrylamide at different rates, they metabolize it differently as well.”

The NCI’s fact sheet is a relatively short précis, but the issue of acrylamide has been addressed in many studies, collected and summarized in meta-analyses.5 Since the NCI’s summary of the animal toxicology and human epidemiology, several important research groups have reported careful human studies that consistently have found no association between dietary acrylamide and cancer risk.6


1 Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir. 2011), cert. denied sub nom., U.S. Steel Corp. v. Milward, 132 S. Ct. 1002 (2012).

2 See “The Council for Education and Research on Toxics” (July 9, 2013).

3 A Guidestar Report show that in 2007, the corporate officer were Nancy L. Quam-Wickham and Nancy Perley, in addition to Lunn and Wolfe.

4 Not to be confused with David Roe, the famous snooker player.

5 Claudio Pelucchi, Carlo La Vecchia, Bosetti C, P. Boyle & Paolo Boffetta, “Exposure to acrylamide and human cancer–a review and meta-analysis of epidemiologic studies,” 22 Ann. Oncology 1487 (2011); Claudio Pelucchi, Cristina Bosetti, Carlotta Galeone & Carlo La Vecchia, “Dietary acrylamide and cancer risk: An updated meta-analysis,” 136 Internat’l J. Cancer 2912 (2015).

6 C. Pelucchi, V. Rosato, P. M. Bracci, D. Li, R. E. Neale, E. Lucenteforte, D. Serraino, K. E. Anderson, E. Fontham, E. A. Holly, M. M. Hassan, J. Polesel, C. Bosetti, L. Strayer, J. Su, P. Boffetta, E. J. Duell & C. La Vecchia, “Dietary acrylamide and the risk of pancreatic cancer in the International Pancreatic Cancer Case–Control Consortium (PanC4),” 28 Ann. Oncology 408 (2017) (reporting that the PanC4 pooled-analysis found no association between dietary acrylamide and pancreatic cancer); Rebecca E. Graff, Eunyoung Cho, Mark A. Preston, Alejandro Sanchez, Lorelei A. Mucci & Kathryn M. Wilson, “Dietary acrylamide intake and risk of renal cell carcinoma in two large prospective cohorts,” 27 Cancer Epidemiol., Biomarkers & Prevention (2018) (in press at doi: 10.1158/1055-9965.EPI-18-0320) (failing to find an association between dietary acrylamide and renal cell carcinoma); Andy Perloy, Leo J. Schouten, Piet A. van den Brandt, Roger Godschalk, Frederik-Jan van Schooten & Janneke G. F. Hogervorst, “The Role of Genetic Variants in the Association between Dietary Acrylamide and Advanced Prostate Cancer in the Netherlands Cohort Study on Diet and Cancer,” 70 Nutrition & Cancer 620 (2018) (finding “no clear evidence was found for interaction between acrylamide intake and selected genetic variants for advanced prostate cancer”).

Coffee with Cream, Sugar & a Dash of Acrylamide

June 9th, 2018

Causal statements are made all the time without much thought of their epistemic warrant. On a day that the stock market indices fall, would-be economic pundits point to some putative cause, such as concern about wage inflation. When the stock market rises on the following day, the explanation is that investors were buoyed by corporate tax cuts, even though those tax cuts were supposedly designed to help companies increase wages. As philosophy professor Harry Frankfurt has explained:

Bullshit is unavoidable whenever circumstances require someone to talk without knowing what he is talking about. Thus the production of bullshit is stimulated whenever a person’s obligations or opportunities to speak about some topic exceed his knowledge of the facts that are relevant to that topic.”1

Of course, Frankfurt’s dictum aptly describes the situation with much of expert witness testimony in health effects litigation.

Nothing seems to stimulate speculative causal claiming as much as the potential rewards of rent-seeking litigation under Proposition 65. By popular referendum, the State of California has taken upon itself to make pronouncements about the causal effects of various foods, drugs, and exposures. The referendum became a California statute with the Safe Drinking Water and Toxic Enforcement Act of 1986. Proposition 65 and the subsequent Enforcement Act require the State of California to publish a list of chemicals it “knows” cause cancers or birth defects. California knows a lot. The list, updated annually, now includes about 800 chemicals.

When California knows that a chemical or an exposure causes cancer, the state does not necessarily know that the chemical or exposure causes cancer in human beings; nor does it necessarily know that the chemical or exposure causes cancer at the exposure level experienced by the citizens of the state. Furthermore, many exposures occur in the context of complex mixtures in which a hypothetical effect of one chemical might be offset or antagonized by another chemical in the mixture. But nonetheless, what California “knows” can hurt you if you are on the wrong side of a Prop 65 enforcement action.

What has California gotten for all its “knowledge”? Clearly, the Proposition 65 statute has created huge incentives for private citizens to sue for violations by creating private rights of action against businesses that supposedly violate the law by failing to warn about what California knows. The proof standards for “known to cause cancer” are so removed from scientific discourse that forcing monetary settlements out of California businesses has become at once a big business itself, and a twisted process that distorts the truth of health hazards and benefits. There have been occasional outcries about the abusive system created from what once was perhaps a well-intentioned reform,2 but for the most part, Prop 65 has become the abnormal normal in California.

Mostly California has gotten lawsuits and a glut of warnings with no difference in cancer or birth defect rates than those observed in states less knowledgeable on such matters.3 Some of California’s cancer rates may be a bit lower than the national rates but this outcome is largely the result of lower state rates for smoking and obesity. Some birth defect rates (neural tube defects) are actually higher in California than in the country as a whole.4

Last year, 681 Prop 65 settlements worth $25.6 million were reported to the California attorney general’s office. Attorneys’ and expert witness fees and other litigation costs made up more than 75% of the total.5 The rate of return has been steady over the years. In 2011, 74 percent of Proposition 65 awards went to attorneys’ fees and costs.6

Council for Education and Research on Toxics (CERT)

For all the hoopla over CERT’s lawsuit against Starbucks,7 there has been little coverage of the actual testimony from the trial. One journalist did report that Peter Infante, a frequent testifier for the lawsuit industry, testified on epidemiology for CERT’s lawyer, Raphael Metzger.8 Apparently, Infante described some studies as showing statistically significant correlations between coffee drinking and some kinds of cancer. Infante demurred on whether coffee caused these kinds of cancer, and admitted that one “would need a clinical trial to resolve the issue.” David Kessler, the former FDA commissioner who helped create the breast implant litigation fiasco and who now testifies frequently for the Lawsuit Industry, testified for Starbucks. Despite his substantial fear-mongering credentials, Dr Kessler emphasized that coffee is a “staple of the American diet,” and that drinking coffee has known health benefits. As everyone now knows, Starbucks failed to persuade the California trial judge that coffee, acrylamide and all, should come under the statute’s safe harbor provisions.

Almost five years ago, I first blogged about the CERT, in connection with the Milward case.9 When I first wrote back in 2013, and until the present, CERT, has not had a website, which is odd for an organization that professes to have an educational mission. In 2013, my research on CERT showed it to be a California corporation, EIN: 42-1571530, founded in 2003, with a business address at 401 E. Ocean Blvd., Ste. 800, Long Beach, California 90802-4967, and a telephone number:  1-877-TOX-TORT. CERT’s reported mission statement was furthering scientific understanding of toxins. Plaintiffs’ lawyer Ralph Metzger, a denizen of the Prop 65 world, was noted as the contact person for CERT, and indeed, the telephone number for CERT was the same as that for Metzger’s lawfirm, the Metzger Law Group.

As I started to watch the activities of CERT, I detected some curious patterns. I saw CERT file amicus briefs in legal cases, which is not the typical activity of a scientific research organization.10 Even more curious, and somewhat dubious, in two cases in which Ralphael Metzger of the Metzger Law Group represented the plaintiffs, another firm, Richard Alexander of the Alexander Law Group, represented CERT as an amicus in the same cases.11

Given the publicity created by CERT’s victory in its Proposition 65 citizen’s action against Starbucks, I recently revisited this research. See Alexander Nazaryan, “Will coffee in California come with a cancer warning?Los Angeles Times (Feb. 18, 2018). One group, “Deniers for Hire,” which describes itself as committed to “debunk anti-science propaganda and expose the activists who produce it,” identified CERT as:

a sham environmentalist nonprofit that sues food companies and collects settlements to fund additional lawsuits against other food companies. Founded in 2002 by toxicologist Martyn T. Smith, with backing from the shameless trial lawyers at Metzger Law Group, CERT uses junk science to target California businesses that can be sued under the state’s ill-conceived Proposition 65.”

The connection with Martyn T. Smith, was news to me, and interesting given how frequently Smith testifies for plaintiffs in cases involving even minimal benzene exposure. If correct, this website’s connecting Martyn Smith with CERT raises additional conflict-of-interest issues.

Funding of Research

Does CERT actually support research? Perhaps, after a fashion, but the money trail is as sketchy as is the ownership issue. Searching in Google Scholar turns up several publications that openly acknowledge funding from CERT. Perhaps only the young and naïve will be surprised that CERT money went to Martyn Smith, alleged founder of CERT and testifier for plaintiffs’ counsel, and to Smith’s students.12 In one instance, CERT support has been acknowledged by Martyn Smith and co-authors for the production of a meta-analysis, which can then be relied upon by Smith and other plaintiffs’ expert witnesses in benzene litigation. Although this meta-analysis credits funding from CERT, most readers of a professional journal will have little idea of the funding’s litigation provenance.13 The corresponding author of the CERT-funded meta-analysis was an official in the California state Office of Environmental Health Hazard Assessment, an office which is in a position to make decisions that help CERT in its California litigation goals.14

The funding of Martyn Smith and his students would certainly be questionable if Smith was a founder of or a participant in CERT. When Smith’s litigation opinions were challenged in one high-profile case, CERT rallied to his rescue with an amicus brief, which did not disclose any relationship between CERT and Martyn Smith, or CERT’s funding of Smith’s research. Milward v. Acuity Specialty Prods. Group, Inc., 639 F.3d 11 (1st Cir. 2011).

A current online listing at Guidestar gives Nancy Quam-Wickham as the “principal officer,” with the same EIN for CERT, as I saw five years ago. Quam-Wickham is a professor of history at California State University, in Long Beach. She seems an unlikely person to head up an organization given to research and education on “toxics.” The phone number for CERT is now 6101824891, but the mailing address is still Ralphael Metzger’s law office.

The Charity Navigator website does not rate CERT because its annual revenue is below $1 million. The website describes CERT as a 501(c)(3), with the same current address as Metzger’s lawfirm. According to Charity Navigator, CERT’s IRS 990 return listed assets of $21,880, and income of $137,354, for 2017.

So what are CERT’s educational activities? The sketchiness of CERT’s appearance as an “amicus” in Ralphael Metzger’s own lawsuits seems matched by the sketchiness of the organization’s professed educational mission. A deeper dive discovered that CERT has garnered some acknowledgements on the websites of other organizations. For instance, the Green Science Policy Institute, founded in 2008, for instance, acknowledges CERT for its “generous support” of the Institute’s work.

Some of CERT’s “educational” efforts have not fared particularly well. In the Chemtura Corporation bankruptcy, CERT attempted to intervene to assert a $9 billion claim to compensate “the public” for alleged injuries from the bankrupt’s allegedly toxic chemicals. In re Chemtura Corp., No. 09-11233, U.S. Bankruptcy Court (S.D.N.Y. 2010). Bankruptcy Judge Robert Gerber was not impressed with CERT’s educational efforts, and dismissed the entity as lacking the necessary standing to make a claim.15


1 Harry Frankfurt, On Bullshit 63 (2005).

2 See, e.g., Lisa L. Halko, “California’s Attorney General Acknowledges Prop 65 Abuse,” 22 no. 29 Wash. Leg. Fdtn. Legal Backgrounder (July 27, 2007); Pamela A. MacLean, “California Judge Blasts Firm in Toxic-Warnings Case,” Nat’l L.J. (April 13, 2006); Consumer Defense Group v. Rental Housing Industry Members40 Cal. Rptr. 3d 832 (Cal. Ct. App. 4th 2006) (“As the Attorney General pointed out in oral argument, it does not serve the public interest to have the almost the entirety of the state of California ‘swamped in a sea [of] generic warning signs’.”).

7 Council for Education and Research on Toxics v. Starbucks Corp., BC435759, California Superior Court, Los Angeles County.

8 Edvard Pettersson, “Toxic Java? California Law Carries Big Fines, Little Evidence,” Bloomberg (Oct. 25, 2017).

10 See, e.g., Parker v. Mobil Oil Corp., 7 N.Y.3d 434, 857 N.E.2d 1114, 824 N.Y.S.2d 584 (2006).

11 In Uriarte v. Scott Sales Co., 226 Cal. App. 4th 1396, 172 Cal. Rptr. 3d 886 (2014); Ramos v. Brenntag Specialties, Inc., 63 Cal.4th 500, 203 Cal. Rptr. 3d 273, 372 P.3d 200 (2016). In both of these cases, CERT was joined by a band of scientists proclaiming neutrality and failing to disclose their significant litigation activities and income: Dr. Jerrold Abraham, Dr. Richard W. Clapp, Dr. Ronald Crystal, Dr. David A. Eastmond, Dr. Arthur L. Frank, Dr. Robert J. Harrison, Dr. Ronald Melnick, Dr. Lee Newman, Dr. Stephen M. Rappaport, Dr. David Joseph Ross and Dr. Janet Weiss. SeeSand in My Shoe – CERTainly” (June 17, 2014). Of course, California appellate courts require that amici disclose financial interests. A motion for leave to file an amicus brief must include, among other things, the names of all persons or entities that contributed financially to the brief, and acknowledgments about whether any party of party’s lawyer helped fund the preparation or filing of the brief. Cal. Rules of Court, Rule 8.200(c)(3)(A)(ii), (B), 8.882(d)(3)(A)(ii) and (B).

12 See, e.g., Jimmy Phuong, Simon Kim, Reuben Thomas & Luoping Zhang, “Predicted Toxicity of the Biofuel Candidate 2,5-Dimethylfuran in Envt’l & Biological Systems,” 53 Envt’l & Molecular Mutagenesis 478 (2012); Michele Fromowitz, Joe Shuga, AntonioYip Wlassowsky, Zhiying Ji, Matthew North, Chris D. Vulpe, Martyn T. Smith, and Luoping Zhang, “Bone Marrow Genotoxicity of 2,5-Dimethylfuran, a Green Biofuel Candidate,” 53 Envt’l & Molecular Mutagenesis 488 (2012); Reuben Thomas, Jimmy Phuong, Cliona M. McHale and Luoping Zhang, “Using Bioinformatic Approaches to Identify Pathways Targeted by Human Leukemogens,” 9 Internat’l J. Envt’l. Research & Public Health 2479 (2012).

13 Frolayne M. Carlos-Wallace, Luoping Zhang, Martyn T. Smith, Gabriella Rader & Craig Steinmaus, “Parental, In Utero, and Early-Life Exposure to Benzene and the Risk of Childhood Leukemia: A Meta-Analysis,” 183 Am. J. Epidem. 1 (2016).

14 Dr. Craig Steinmaus, Office of Environmental Health Hazard Assessment, 1515 Clay Street, 16th Floor, Oakland, CA 94612.

15 Caroline Humer, “Judge rules against big Chemtura bankruptcy claim,” Reuters (April 8, 2010); John Parry, “Chemtura hits back at $9 billion claim over toxins,” Reuters (Mar. 24, 2010).

Infante-lizing the IARC

May 13th, 2018

Peter Infante, a frequently partisan, paid expert witness for the Lawsuit Industry, recently published a “commentary” in the red journal, the American Journal of Industrial Medicine, about the evils of scientists with economic interests commenting upon the cancer causation pronouncements of the International Agency for Research on Cancer (IARC). Peter F. Infante, Ronald Melnick, James Huff & Harri Vainio, “Commentary: IARC Monographs Program and public health under siege by corporate interests,” 61 Am. J. Indus. Med. 277 (2018). Infante’s rant goes beyond calling out scientists with economic interests on IARC working groups; Infante would silence all criticism of IARC pronouncements by anyone, including scientists, who has an economic interest in the outcome of a scientific debate. Infante points to manufacturing industry’s efforts to “discredit” recent IARC pronouncements on glyphosate and red meat, by which he means that there were scientists who had the temerity to question IARC’s processes and conclusions.

Apparently, Infante did not think his bias was showing or would be detected. He and his co-authors invoke militaristic metaphors to claim that the IARC’s monograph program, and indeed all of public health, is “under siege by corporate interests.” A farcical commentary at that, coming from such stalwarts of the Lawsuit Industry. Infante lists his contact information as “Peter F. Infante Consulting, LLC, Falls Church, Virginia,” and co-author Ronald Melnick can be found at “Ronald Melnick Consulting, LLC, North Logan, Utah.” A search on Peter Infante in Westlaw yields 141 hits, all on the plaintiffs’ side of health effects disputes; he is clearly no stranger to the world of litigation. Melnick is, to be sure, harder to find, but he does show up as a signatory on Raphael Metzger’s supposed amicus briefs, filed by Metzger’s litigation front organization, Council for Education and Research on Toxics.1

Of the commentary’s authors, only James Huff, of “James Huff Consulting, Durham, North Carolina,” disclosed a connection with litigation, as a consultant to plaintiffs on animal toxicology of glyphosate. Huff’s apparent transparency clouded up when it came to disclosing how much he has been compensated for his consulting activities for claimants in glyphosate litigation. In the very next breath, in unison, the authors announce unabashedly that “[a]ll authors report no conflicts of interest.” Infante at 280.

Of course, reporting “no conflicts of interest” does not mean that the authors have no conflicts of interest, financial, positional, and idealogical. Their statement simply means that they have not reported any conflicts, through inadvertence, willfulness, or blindness. The authors, and the journal, are obviously content to mislead their readership by not-so-clever dodges.

The clumsiness of the authors’ inability to appreciate their very own conflicts infects their substantive claims in this commentary. These “consultants” tell us solemnly that IARC “[m]eetings are openly transparent and members are vetted for conflicts of interest.” Infante at 277. Working group members, however, are vetted but only for manufacturing industry conflicts, not for litigation industry conflicts or for motivational conflicts, such as advancing their own research agendas. Not many scientists have a research agenda to show that chemicals do not cause cancer.

At the end of this charade, the journal provides additional disclosures [sic]. As for “Ethics and Approval Consent,” we are met with a bold “Not Applicable.” Indeed; ethics need not apply. Perhaps, the American Journal of Industrial Medicine is beyond good and evil. The journal’s “Editor of Record,” Steven B. Markowitz “declares that he has no conflict of interest in the review and publication decision regarding this article.” This is, of course, the same Markowitz who testifies frequently for the Lawsuit Industry, without typically disclosing this conflict on his own publications.

This commentary is yet another brushback pitch, which tries to chill manufacturing industry and scientists from criticizing the work of agencies, such as IARC, captured by lawsuit industry consultants. No one should be fooled other than Mother Jones.


1See, e.g., Ramos v. Brenntag Specialties, Inc., 372 P.3d 200 (Calif. 2016) (where plaintiff was represented by Metzger, and where CERT filed an amicus brief by the usual suspects, plaintiffs’ expert witnesses, including Melnick).

P-Values: Pernicious or Perspicacious?

May 12th, 2018

Professor Kingsley R. Browne, of the Wayne State University Law School, recently published a paper that criticized the use of p-values and significance testing in discrimination litigation. Kingsley R. Browne, “Pernicious P-Values: Statistical Proof of Not Very Much,” 42 Univ. Dayton L. Rev. 113 (2017) (cited below as Browne). Browne amply documents the obvious and undeniable, that judges, lawyers, and even some ill-trained expert witnesses, are congenitally unable to describe and interpret p-values properly. Most of Browne’s examples are from the world of anti-discrimination law, but he also cites a few from health effects litigation as well. Browne also cites from many of the criticisms of p-values in the psychology and other social science literature.

Browne’s efforts to correct judicial innumeracy are welcomed, but they take a peculiar turn in this law review article. From the well-known state of affairs of widespread judicial refusal or inability to discuss statistical concepts accurately, Browne argues for what seem to be two incongruous, inconsistent responses. Rejecting the glib suggestion of former Judge Posner that evidence law is not “fussy” about evidence, Browne argues that federal evidence law requires courts to be “fussy” about evidence, and that Rule 702 requires courts to exclude expert witnesses, whose opinions fail to “employ[] in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Browne at 143 (quoting from Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152 (1999). Browne tells us, with apparently appropriate intellectual rigor, that “[i]f a disparity that does not provide a p-value of less than 0.05 would not be accepted as meaningful in the expert’s discipline, it is not clear that the expert should be allowed to testify – on the basis of his expertise in that discipline – that the disparity is, in fact, meaningful.” Id.

In a volte face, Browne then argues that p-values do “not tell us much,” basically because they are dependent upon sample size. Browne suggests that the quantitative disparity between expected value and observed proportion or average can be assessed without the use of p-values, and that measuring a p-value “adds virtually nothing and just muddies the water.” Id. at 152. The prevalent confusion among judges and lawyers seems sufficient in Browne’s view to justify his proposal, as well as his further suggestion that Rule 403 should be invoked to exclude p-values:

The ease with which reported p-values cause a trier of fact to slip into the transposition fallacy and the difficulty of avoiding that lapse of logic, coupled with the relatively sparse information actually provided by the p-value, make p-values prime candidates for exclusion under Federal Rule of Evidence 403. *** If judges, not to mention the statistical experts they rely on, cannot use the information without falling into fallacious reasoning, the likelihood that the jury will misunderstand the evidence is very high. Since the p-value actually provides little useful relevant information, the high risk of misleading the jury greatly exceeds its scant probative value, so it simply should not be presented to the jury.”

Id. at 152-53.

And yet, elsewhere in the same article, Browne ridicules one court and several expert witnesses who have argued in favor of conclusions that were based upon p-values up to 50%.1 The concept of p-values cannot be so flexible as to straddle the extremes of having no probative value, and yet capable of rendering an expert witness’s opinions ludicrous. P-values quantify an estimate of random error, even if that error rate varies with sample size. To be sure, the measure of random error depends upon the specified model and assumption of a null hypothesis, but the crucial point is that the estimate (whether mean, proportion, risk ratio, risk difference, etc.) is rather meaningless without some further estimate of random variability of the estimate. Of course, random error is not the only type of error, but the existence of other potential systematic errors is hardly a reason to ignore random error.

In the science of health effects, many applications of p-values have given way to the use of confidence intervals, which arguably provide more direct assessments of both sample estimates, along with ranges of potential outcomes that are reasonably compatible with the sample estimates. Remarkably, Browne never substantively discusses confidence intervals in his article.

Under the heading of other problems with p-values and significance testing, Browne advances four additional putative problems with p-values. First, Browne asserts with little to no support that “[t]he null hypothesis is unlikely a priori.” Id. at 155. He fails to tell us why the null hypothesis of no disparity is not a reasonable starting place in the absence of objective evidence of a prior estimate. Furthermore, a null hypothesis of no difference will have legal significance in claims of health effects, or of unlawful discrimination.

Second, Browne argues that significance testing will lead to “[c]onflation of statistical and practical (or legal) significance” in the minds of judges and jurors. Id. at 156-58. This charge is difficult to sustain. The actors in legal cases can probably best appreciate practical significance and its separation from statistical significance, most readily. If a large class action showed that the expected value of a minority’s proportion was 15%, and the observed proportion was 14.8%, p < 0.05, most innumerate judges and jurors would sense that this disparity was unimportant and that no employer would fine tune its discriminatory activities so closely as to achieve such a meaningless difference.

Third, Browne reminds us that the validity and the interpretation of a p-value turns on the assumption that the statistical model is perfectly specified. Id. at 158-59. His reminder is correct, but again, this aspect of p-values (or confidence intervals) is relatively easy to explain, as well as to defend or challenge. To be sure, there may be legitimate disputes about whether an appropriate model was used (say binomial versus hypergeometric), but such disputes are hardly the most arcane issues that judges and jurors will face.

Fourth, Browne claims that “the alternative hypothesis is seldom properly specified.” Id. at 159-62. Unless analysts are focused on measuring pre-test power or type II error, however, they need not advance an alternative hypothesis. Furthermore, it is hardly a flaw with significance testing that it does not account for systematic bias or confounding.

Browne does not offer an affirmative response such as urging courts to adopt a Bayesian program. A Bayesian response to prevalent blunders in interpreting statistical significance would introduce perhaps even more arcane and hard-to-discern blunders in court proceedings. Browne also leaves courts without a meaningful approach to evaluate random error other than to engage in crude comparisons between two means or proportions. The recommendations in this law review article appear to be a giant step, backwards, into an epistemic void.


1See Browne at 146, citing In re Photochromic Lens Antitrust Litig., 2014 WL 1338605 (M.D. Fla. April 3, 2014) (reversing magistrate judge’s exclusion of an expert witness who had advanced claims based upon p-value of 0.50); id. at 147 n. 116, citing In re High-Tech Employee Antitrust Litig., 2014 WL 1351040 (N.D. Cal. 2014).

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.