Offense to that man by whom the WOE cometh!
A few weeks ago, the Wake Forest Journal of Law & Policy published six articles from its 2012 Spring Symposium, on “Toxic Tort Litigation After Milward v. Acuity Products.” Not a single paper is critical of Milward, which is no surprise given that the Symposium was a joint production of The Center for Progressive Reform and the Wake Forest University School of Law.
In previous posts, I addressed concerns about papers from Professors Green and Sanders. One of the partisan expert witnesses from the Milward case, Carl Cranor, presented at the symposium, and published in the Journal. See Carl F. Cranor, “Milward v. Acuity Specialty Products: Advances in General Causation Testimony in Toxic Tort Litigation,” PDF 3 Wake Forest J. L. & Policy 105 (2013) [cited herein as Cranor].
The partisan nature of the Wake Forest/CPR symposium is obvious, and perhaps disclosures of conflicts of interest, so real and palpable, are unnecessary. Cranor acknowledges that he testified for plaintiffs in Milward, but his disclosure does not address how deep his conflict of interest was. Cranor at 105. In addition to his consulting, report writing, and testifying, Cranor has written briefs for plaintiffs in this and in other litigations. Unlike the potential for conflict of interest supposedly raised by payments, Cranor’s conflict of interest is actual. He has been a long-time advocate for radical precautionary principle regulation, legislation, and adjudication. Cranor’s conflicts are revealed by his writings, his associations, and his activities.
There is nothing wrong with advocacy per se, and Cranor’s ideas, such as they are, deserve to be judged on their merits. Cranor will no doubt complain that I am addressing one idea at a time, in a corpuscularian fashion, and that his ideas can only be appreciated as a complete gestalt. If the individual ideas and claims, however, are incorrect, incomplete, inconsistent, and incoherent, we may rightfully reject the entirety of his claims.
Cranor asserts that he is representing how scientists go about their business in reaching judgments of causality. He has, however, inaccurately described serious attempts to judge causality in order to distort causal assessments into precautionary practice. Cranor presents a reductionist, abridged notion of scientific assessment of causality in order to legitimate the CPR’s radical agenda in both regulation and adjudication.
VAPID AND VACUOUS
Cranor’s principal claim is that “weight of the evidence” (WOE) is a complete, sufficient description of how scientists do, and should, engage in judging causality. This claim, however, fails because WOE is not a methodology for attributing causation.
In his symposium article, Cranor introduces WOE by telling us that:
“‘Weight of the evidence argument’ is just another name for nondeductive reasoning.”
Cranor at 113 (citing Larry Wright, Practical 46-49 (Fogelin, ed. 1989)
So WOE is equivalent to induction, abduction, analogy, and every other form and manner of non-deductive reasoning. So, let’s see. A rat dropped from the top of my building falls to the ground, accelerating at 32 ft/sec/sec. A mouse dropped accelerates at the same rate. Rats and mice are both murine mammal species. This could be a great analogy. Rats fed large quantities of saccharin develop bladder tumors. So by analogy, mice develop bladder tumors from saccharin. It’s an analogy, but it goes very badly wrong. But Cranor’s explication of WOE fails to explain why this analogy fails to explain or predict the outcome in mice. See Kenneth Rothman, Sander Greenland, and Timothy Lash, Modern Epidemiology 30 (3d ed. 2008) (“Whatever insight might be derived from analogy is handicapped by the inventive imagination of scientists who can find analogies everywhere”).
Cranor proceeds to introduce a qualitative criterion, “best support,” in non-deductive settings:
“[N]o one conclusion is ‘guaranteed’ by the premises. Consequently, the evaluative task in assessing such inferences is to judge which conclusion the evidence best supports (or, to put it another way, which explanation best accounts for the evidence in the premises) and how well it does so.”
Cranor at 114-15. Although the requirement of a superlative qualitative assessment seems promising, as we will see, Cranor ensures that the assessment is empty. Cranor applauds the First Circuit for adopting his identification of WOE with non-deductive reasoning to the best explanation:
“[Nondeductive reasoning or reasoning] to the best explanation can be thought of as involving six general steps, some of which may be implicit. The scientist must
(1) identify an association between an exposure and a disease,
(2) consider a range of plausible explanations for the association,
(3) rank the rival explanations according to their plausibility,
(4) seek additional evidence to separate the more plausible from the less plausible explanations,
(5) consider all of the relevant available evidence, and
(6) integrate the evidence using professional judgment to come to a conclusion about the best explanation.”
Cranor at 115 (citing Milward v. Acuity Specialty Prods. Group, Inc., 639 F.3d 11, 17−18 (1st Cir. 2011)). Cranor’s and the Circuit’s embrace of this description is seriously flawed. They move from WOE to “reasoning to the best explanation,” but they do provide any guidance on the key elements:
(1) what constitutes an association?
(2) what renders an explanation plausible, and when is “unknown on the current evidence” an appropriate explanation to offer?
(3) what are the criteria for ranking plausible explanations?
(4) what evidence will discriminate between and among rival explanations?
(5) how do we consider all relevant evidence without using qualitative and quantitative weights? If we use weights, how?
(6) how do we integrate disparate lines of evidence, and which profession will provide the critical assessment of validity for the integration, and the determinant of what is the “best explanation”?
Cranor harrumphs with the First Circuit:
“[n]o serious argument can be made that the weight of the evidence approach is inherently unreliable.”
Cranor at 115 (quoting Milward, at 18−19). The double negative is revealing, and so is the utter lack of content to the so-called methodology. Even if WOE were a method, the Circuit’s statement is meaningless, much like saying that no one could say that physics is inherently unreliable. Such a statement certainly would not help us judge the bona fides of cold fusion advocates.
WOE IN COURT
Perhaps in an attempt to induce judges and lawyers to lower their intellectual guard, Cranor tells us that WOE is nothing other than what happens in jury trials:
“Jurors, or judges conducting bench trials, use such inferences to find the most plausible account of whether a person is guilty of a crime or has committed a tort. To convict a person of a crime, jurors must find that the total body of relevant evidence supports the conclusion of a nondeductive argument beyond a reasonable doubt; to hold a person accountable for a tort, jurors must find that the total body of relevant evidence supports the conclusion of a nondeductive argument by a preponderance of the evidence, a lower standard of proof.”
Cranor 116. To be sure, “weight of the evidence” does have a legal usage. Any superficial appeal of this analogy between scientific assessment of causation and litigation of facts quickly dissipates when we realize the relevant evidence has been filtered for the jury by recognized rules of evidence. Evidence deemed too weak, too speculative, too prejudicial will have been excluded in a jury trial. In addition, there are social norms and contexts that operate in a jury trial that may be inimical to the truth-finding process. My favorite Philadelphia trial anecdote from a former Assistant District Attorney is about a jury that convicted a man for murder. Although there were racial issues that made the case difficult and the outcome uncertain, after the trial, the forewoman explained that reaching a verdict was easy because the defendant’s mother, who was identified as living near the courthouse, never showed up for the trial. More “scientific” studies document the role of race, ethnic, and socio-economic prejudice and bias. Furthermore, in civil and criminal trials, the evidence is generally unweighted except by lawyers’ argument and rhetoric. A lawyer unhappy with a study’s result may argue that one author had a conflict of interest, even though the study was well designed and conducted, and provided the “weightiest” evidence on the issue to be decided. Perhaps Cranor advances the trial example because almost “anything goes” in lawyers’ argument and juries’ assessments of scientific issues.
WOE is vacuous as described by Cranor. Statements that all types of relevant research should be considered do not tell us anything. Stating that “scientific judgment” is necessary says everything, and thus nothing, because it leaves out any description of the methodology to inform and apply the judgment. Expert witnesses should not be allowed to invoke WOE as a way to avoid methodological scrutiny.
The WOE Cranor would inflict upon the judicial process has been described as a “black box,” which fails to provide any operative method of specifying relevancy or weight for differing kinds of evidence, or method for synthesizing the disparate studies. We should not be surprised by the lack of endorsement from the scientific community itself for WOE-ful methods. The phrase is vague and ambiguous; its use, inconsistent.
See, e.g., V. H. Dale, G.R. Biddinger, M.C. Newman, J.T. Oris, G.W. Suter II, T. Thompson, et al., “Enhancing the ecological risk assessment process,” 4 Integrated Envt’l Assess. Management 306 (2008)(“An approach to interpreting lines of evidence and weight of evidence is critically needed for complex assessments, and it would be useful to develop case studies and/or standards of practice for interpreting lines of evidence.”);
Igor Linkov, Drew Loney, Susan M. Cormier, F. Kyle Satterstrom, and Todd Bridges, “Weight-of-evidence evaluation in environmental assessment: review of qualitative and quantitative approaches,” 407 Science of Total Env’t 5199–205 (2009) (reviewing the use of WOE methods and concluding that the approach is not particularly rigorous, and that the approach “does not lend itself to transparency or repeatability except in simple cases”);
Douglas Weed, “Weight of Evidence: A Review of Concept and Methods,” 25 Risk Analysis 1545 (2005) (noting the vague, ambiguous, indefinite nature of the concept of “weight of evidence” review);
R.G. Stahl, Jr., “Issues addressed and unaddressed in EPA’s ecological risk guidelines,” 17 Risk Policy Report 35 (1998); (noting that U.S. Environmental Protection Agency’s guidelines for ecological weight-of-evidence approaches to risk assessment fail to provide guidance);
Glenn Suter II & Susan Cormier, “Why and how to combine evidence in environmental assessments: Weighing evidence and building cases,” 409 Science of the Total Environment 1406, 1406 (2011)(noting arbitrariness and subjectivity of WOE “methodology”);
Charles Menzie, Miranda Hope Henning, Jerome Curac, et al. “A weight-of-evidence approach for evaluating ecological risks; report of the Massachusetts Weight-of-Evidence Work Group,” 2 Human Ecological Risk Assessment 277 (1996) (“although the term ‘weight of evidence’ is used frequently in ecological risk assessment, there is no consensus on its definition or how it should be applied”);
Sheldon Krimsky, “The Weight of Scientific Evidence in Policy and Law,” 95 Supp.(1) Am. J. Pub. Health S129, S131 (2005) (“However, the term [WOE] is applied quite liberally in the regulatory literature, the methodology behind it is rarely explicated.”)
Describing WOE, Krimsky notes that “WOE seems to be coming out of a ‘black box’ of scientific judgment.” Krimsky at S131. Revealingly, Krimsky references a report from the Agency for Toxic Substances and Disease Registry (ATSDR) of the Department of Health and Human Services, which describes WOE as an alternative to causal determinations when trying to set policy, when “causality is out of reach.” Id. (citing citing ATSDR, “The Assessment Process: An Interactive Learning Program,” available at http://www.atsdr.cdc.gov/training/public-health-assessment-overview/html/module2/sv18.html (last visited May 8, 2013).
Krimsky thus acknowledges what Cranor tries so hard to obscure: WOE is a precautionary approach to be applied when the scientific answer is “I don’t know.”
