Carl Cranor pays me the dubious honor of quoting my assessment of weight of the evidence (WOE) pseudo-methodology as used by lawsuit industry expert witnesses, in one of his recent publications:
“Take all the evidence, throw it into the hopper, close your eyes, open your heart, and guess the weight. You could be a lucky winner! The weight of the evidence suggests that the weight-of-the-evidence (WOE) method is little more than subjective opinion, but why care if it helps you to get to a verdict!”[1]
Cranor’s intent was to deride my comments, but they hold up fairly well. I have always maintained that if were wrong, I would eat my words, but that they will be quite digestible. Nothing to eat here, though.
In his essay in the Public Affairs Quarterly, Cranor attempts to explain and support his advocacy of WOE in the notorious case, Milward, in which Cranor, along with his friend and business partner, Martyn Smith, served as partisan, paid expert witnesss.[2] Not disclosed in this article is that after the trial court excluded the opinions of Cranor and Smith under Federal Rule of Evidence 702, and plaintiff appealed, the lawsuit industry, acting through The Council for Education and Research on Toxics (CERT) filed an amicus brief to persuade the Court of Appeals to reverse the exclusion. The plaintiffs’ counsel, Cranor and Smith, and CERT failed to disclose that CERT was founded by the two witnesses, Cranor and Smith, whose exclusion was at issue.[3] Many of the lawsuit industry’s regular testifiers were signatories, and none raised any ethical qualms about the obvious conflict of interest, or the conspiracy to pervert the course of justice.[4]
Cranor equates WOE to “inference to the best explanation,” which reductively strips science of its predictive and reproducible nature. Readers may get the sense he is operating in the realm of narrative, not science, and they would be correct. Cranor goes on to conflate WOE methodology with “diagnostic induction,” and “differential diagnosis.”[5] The latter term is well understood in both medicine and in law to involve the assessment of an individual patient’s condition, based upon what is already known upon good and sufficient bases. The term has no accepted or justifiable meaning for assessing general causation. Cranor’s approach would pretermit the determination of general causation by making the disputed cause a differential.
Cranor offers several considerations in support of his WOE-ful methodology. First, he notes that the arguments for causal claims are not deductive. True, but indifferent as to his advocacy for WOE and inference to the best explanation.
Second, Cranor describes a search for relevant evidence once the scientific issue (hypothesis?) is formulated. Again, there is nothing unique about this described step, but Cranor intentionally leaves out considerations of validity, as in extrapolations between high and low dose, or between species. Similarly, he leaves out considerations of validity of study designs (such as whether any weight would be given to case studies, cross-sectional, or ecological studies) or of validity of individual studies.
Cranor’s third step is the formulation of a “sufficiently complete range of reasonable and plausible explanations to account for the evidence.” Again, nothing unique here about WOE, except that Cranor’s WOE abridges the process by ignoring the very real possibility that we do not have the correct plausible explanation available.
Fourth, according to Cranor, scientists rank, explicitly or implicitly, the putative “explanations” by plausibility and persuasiveness, based upon the evidence at hand, in view of general toxicological and background knowledge.[6] Note the absence of consideration of the predictive abilities of the competing explanations, or any felt need to assess the quality of evidence or the validity of study design.
For Cranor, the fifth consideration is to use the initial plausibility assessments, made on incomplete understanding of the phenomena, and on incomplete evidence, to direct “additionally relevant /available evidence to separate founded explanations from less well-founded ones.” Obviously missing from Cranor’s scheme is the idea of trying to challenge or test hypotheses severely to see whether withstand such challenges.
Sixth, Cranor suggests that “all scientifically relevant information” should be considered in moving to the “best supported” explanation. Because “best” is determined based upon what is available, regardless of the quality of the data, or the validity of the inference, Cranor rigs his WOE-ful methodology in favor of eliminating “indeterminate” as a possible conclusion.
In a seventh step, Cranor points to the need to “integrate, synthesize, and assess or evaluate,” all lines of “available relevant evidence.” There is nothing truly remarkable about this step, which clearly requires judgment. Cranor notes that there can be convergence of disparate lines of evidence, or divergence, and that some selection of “lines” of evidence may be endorsed as supporting the “more persuasive conclusion” of causality.[7] In other words, a grand gemish.
Cranor’s WOE-ful approach leaves out any consideration of random error, or systematic bias, or data quality, or study design. The words “bias” and “confounding” do not appear in Cranor’s essay, and he erroneously discusses “error” and “error rates,” only to disparage them as the machinations of defense lawyers in litigation. Similarly, Cranor omits any serious mention of reproducibility, or of the need to formulate predictions that have the ability to falsify tentative conclusions.
Quite stridently, Cranor insists that there is no room for any actual weighting of study types or designs. In apparent earnest, Cranor writes that:
“this conclusion is in accordance with a National Cancer Institute (NCI) recommendation that ‘there should be no hierarchy [among different types of scientific methods to determine cancer causation]. Epidemiology, animal, tissue culture and molecular pathology should be seen as integrating evidences in the determination of human carcinogenicity.”[8]
There is much whining and special pleading about the difficulty, expense, and lack of statistical power of epidemiologic studies, even though the last point is a curious backdoor endorsement of statistical significance. The first two points ignore the availability of large administrative databases from which large cohorts can be identified and studied, with tremendous statistical power. Case-control studies can in some instances be assembled quickly as studies nested in existing cohorts.
As I have noted elsewhere,[9] Cranor’s attempt to level all types of evidence starkly misrepresents the cited “NCI” source, which is not at all an NCI recommendation, but rather a “meeting report” of a workshop of non-epidemiologists.[10] The cited source is not an official pronouncement of the NCI, the authors were not NCI scientists, and the NCI did not sponsored the meeting. The meeting report appeared in the journal Cancer Research as a paid advertisement, not in the NCI’s Journal of the National Cancer Institute as a scholarly article:
“The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.”[11]
Tellingly, Cranor’s deception was relied upon and cited by the First Circuit, in its Milward, decision.[12] The scholarly fraud hit its mark. As a result of Cranor’s own dubious actions, the Milward decision has both both ethical and scholarship black clouds hovering over it. The First Circuit should withdraw the decision as improvidently decided.
The article ends with Cranor’s triumphant view of Milward,[13] which he published previously, along with the plaintiffs’ lawyer who hired him.[14] What Cranor leaves out is that the First Circuit’s holding is now suspect because of the court’s uncritical acceptance of Cranor’s own misrepresentations and CERT’s omissions of conflict-of-interest disclosures, as well as the subsequent procedural history of the case. After the Circuit reversed the Rule 702 exclusions, and the Supreme Court denied the petition for a writ of certiorari, the case returned to the federal district court, where the defense lodged a Rule 702 challenge to expert witness opinion that attributed plaintiff’s acute promyelocytic leukemia to benzene exposure. This specific causation issue was not previously addressed in the earlier proceedings. The trial court sustained the challenge, which left the plaintiff unable to show specific causation. The result was summary judgment for the defense, which the First Circuit affirmed on appeal.[15] The upshot of the subsequent proceedings, with their dispositive ruling in favor of the defense on specific causation, is that the earlier ruling on general causation is no longer necessary to the final judgment, and not the holding of the case when all the proceedings are considered.
In the end, Cranor’s WOE leaves us with a misdirected search for an “explanation of causation,” rather than a testable, tested, reproducible, and valid “inference of causation.” Cranor’s attempt to invoke the liberalization of the Federal Rules of Evidence ignores the true meaning of “liberal” in being free from dogma and authority. Evidence does not equal eminence, and expert witnesses in court must show their data and defend their inferences, whatever their explanations may be.
——————————————————————————————————–
[1] Carl F. Cranor, “How Courts’ Reviews of Science in Toxic Tort Cases Have Changed and Why That’s a Good Thing,” 31 Public Affairs Q. 280 (2017), quoting from Schachtman, “WOE-fully Inadequate Methodology – An Ipse Dixit by Another Name” (May 1, 2012).
[2] Milward v. Acuity Specialty Products Group, Inc., 639 F. 3d 11 (1st Cir. 2011), cert. denied, 132 S.Ct. 1002 (2012).
[3] See “The Council for Education and Research on Toxics” (July 9, 2013).
[4] Among the signatures were Nachman Brautbar, David C. Christiani, Richard W. Clapp, James Dahlgren, Arthur L. Frank, Peter F. Infante, Philip J. Landrigan, Barry S. Levy, David Ozonoff, David Rosner, Allan H. Smith, and Daniel Thau Teitelbaum.
[5] Cranor at 286-87.
[6] Cranor at 287.
[7] Cranor at 287-88.
[8] Cranor at 290.
[9] “Cranor’s Defense of Milward at the CPR’s Celebration” (May 12, 2013).
[10] Michelle Carbone, Jack Gruber, and May Wong, “Modern criteria to establish human cancer etiology,” 14 Semin. Cancer Biol. 397 (2004).
[11] Michele Carbone, George Klein, Jack Gruber and May Wong, “Modern Criteria to Establish Human Cancer Etiology,” 64 Cancer Research 5518 (2004).
[12] Milward v. Acuity Specialty Products Group, Inc., 639 F. 3d 11, 17 (1st Cir. 2011) (“when a group from the National Cancer Institute was asked to rank the different types of evidence, it concluded that ‘[t]here should be no such hierarchy’.”), cert. denied, 132 S.Ct. 1002 (2012).
[13] Cranor at 292.
[14] See “Wake Forest Publishes the Litigation Industry’s Views on Milward” (April 20, 2013).
[15] Milward v. Acuity Specialty Products Group, Inc., 969 F. Supp. 2d 101 (D. Mass. 2013), aff’d sub nom. Milward v. Rust-Oleum Corp., 820 F.3d 469 (1st Cir. 2016).
