TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Zoloft MDL Excludes Proffered Testimony of Anick Bérard, Ph.D.

June 27th, 2014

Anick Bérard is a Canadian perinatal epidemiologist in the Université de Montréal.  Bérard was named by plaintiffs’ counsel in the Zoloft MDL to offer an opinion that selective serotonin reuptake inhibitor (SSRI) antidepressants as a class, and Zoloft (sertraline) specifically, cause a wide range of birth defects. Bérard previously testified against GSK about her claim that paroxetine, another SSRI antidepressant is a teratogen.

Pfizer challenged Bérard’s proffered testimony under Federal Rules of Evidence 104(a), 702, 703, and 403.  Today, the Zoloft MDL transferee court handed down its decision to exclude Dr. Bérard’s testimony at the time of trial.  In re Zoloft (Sertraline Hydrochloride) Prods. Liab. Litig., MDL 2342, Document 979 (June 27, 2014).  The MDL court acknowledged the need to consider the selectivity (“cherry picking”) of studies upon which Dr. Bérard relied, as well as her failure to consider multiple comparisons, ascertainment bias, confounding by indication, and lack of replication of specific findings across the different SSRI medications, and across studies. Interestingly, the MDL court recognized that Dr. Bérard’s critique of studies as “underpowered” was undone by her failure to consider available meta-analyses or to conduct one of her own. The MDL court seemed especially impressed by Dr. Bérard’s having published several papers that rejected a class effect of teratogenicity for all SSRIs, as recently as 2012, while failing to identify anything that was published subsequently that could explain her dramatic change in opinion for litigation.

Stanford Conference on Mathematics in Court

June 26th, 2014

Last month, The Stanford Center for Legal Informatics hosted a conference, “Trial With and Without Mathematics: Legal, Philosophical, and Computational Perspectives.” The conference explored the what if any role mathematics plays in the law, and in the training and education of lawyers.

The program was organized by Marcello Di Bello (Stanford Univ., Department of Philosophy), and Bart Verheij (Stanford Univ., CodeX Center for Legal Informatics, and Univ. of Groningen, Institute of Artificial Intelligence). DiBello teaches an undergraduate course, Probability and the Law, at Stanford.

The program featured presentations by:

Sandy L. Zabell (Northwestern Univ.) on “A Tribe of Skeptics: Probability and the 19th Century Law of Evidence,” (Slides; Video), with commentary by Andrea Roth (Univ. California, Berkeley School of Law);

Susan Haack (Univ. of Miami School of Law), on “Legal Probabilism: An Epistemological Dissent,” (Slides; Video), with commentary by Charles H. Brenner (Univ. California, Berkeley School of Law) (Slides);

William C. Thompson (Univ. California, Irvine Dep’t Criminology, Law & Society), on “How Should Forensic Scientists Explain Their Evidence to Juries: Match Probabilities, Likelihood Ratios, or ‘Verbal Equivalents’? (Slides; Video), with commentary by Paul Brest (Stanford Law School);

Henry Prakken (Univ. Groningen), on Models of Legal Proof and Their Cognitive Plausibility,” (Slides; Video), with commentary by Sarah B. Lawsky (Univ. California, Irvine, School of Law) (Slides);

Vern Walker (Hofstra Univ. School of Law), on “Computational Representation of Legal Reasoning at the Law-Fact Interface,” (Slides; Video), with commentary by Bart Verheij (Slides); and

Ronald J. Allen (Northwestern Univ. School of Law) presented onWhat Are We Doing? Reconsidering Juridical Proof Rules,” (Slides; Video), with commentary by Marcello Di Bello.

An interesting collection of presentations and commentary, which I have not yet reviewed carefully.  Professor Haack’s presentation seems to cover much the same ground covered at a conference on Standards of Proof and Scientific Evidence, held at the University of Girona, in Spain.  Her previous lecture can be viewed on-line, and a manuscript of Haack’s paper is available , as well.  Susan Haack, “Legal Probabilism:  An Epistemological Dissent” (2011)(cited here as “Haack”).  SeeHaack Attack on Legal Probabilism” (2012).

Professor Haack’s papers and presentations on law, legal evidence, and probability are slated for republication in book form, this August. Susan Haack, Evidence Matters: Science, Proof, and Truth in the Law (Cambridge 2014). The contents look familiar:

1. Epistemology and the law of evidence: problems and projects

2. Epistemology legalized: or, truth, justice, and the American way

3. Legal probabilism: an epistemological dissent

4. Irreconcilable differences? The troubled marriage of science and law

5. Trial and error: two confusions in Daubert

6. Federal philosophy of science: a deconstruction – and a reconstruction

7. Peer review and publication: lessons for lawyers

8. What’s wrong with litigation-driven science?

9. Proving causation: the weight of combined evidence

10. Correlation and causation: the ‘Bradford Hill Criteria’ in epidemiological, legal, and epistemological perspective

11. Risky business: statistical proof of specific causation

12. Nothing fancy: some simple truths about truth in the law

 

 

 

Differential Etiology and Other Courtroom Magic

June 23rd, 2014

ITERATIVE DISJUNCTIVE SYLLOGISM

Basic propositional logic teaches that the disjunctive syllogism (modus tollendo ponens) is a valid argument, in which one of its premises is a disjunction (P v Q), and the other premise is the negation of one of the disjuncts:

P v Q

~P­­­_____

∴ Q

See Irving Copi & Carl Cohen Introduction to Logic at 362 (2005). If we expand the disjunctive premise to more than one disjunction, we can repeat the inference (iteratively), eliminating one disjunct at a time, until we arrive at a conclusion that is a simple, affirmative proposition, without any disjunctions in it.

P v Q v R

~P­­­_____

∴ Q v R

~Q­­­_____

∴ R

Hence, the term, “iterative disjunctive syllogism.” Fans of Sir Arthur Conan Doyle will recognize that iterative disjunctive syllogism is nothing other than the process of elimination, as explained by Doyle’s fictional detective, Sherlock Holmes. See, e.g., Doyle, The Blanched Soldier (“…when you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.”); Doyle, The Beryl Coronet (“It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth.”); Doyle, The Hound of the Baskervilles (1902) (“We balance probabilities and choose the most likely. It is the scientific use of the imagination.”); Doyle, The Sign of the Four, ch 6 (1890)(“‘You will not apply my precept’, he said, shaking his head. ‘How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth? We know that he did not come through the door, the window, or the chimney. We also know that he could not have been concealed in the room, as there is no concealment possible. When, then, did he come?”)

The process of elimination sometimes surfaces in court cases in which expert witnesses attempt to attribute a health outcome in a specific person to that person’s prior environmental, occupational, or lifestyle exposures.  A few general conclusions can be advanced about this mode of reasoning:

1. Differential Etiology NOT Differential Diagnosis

Although courts and expert witnesses sometimes refer to this process of ruling out as “differential diagnosis,” their terminology is a misnomer.  Their usage is not an innocent diction error because diagnosis is almost never involved, and the usage attempts to suggest that the causal attribution is part of a process typically conducted by a treating physician, when in fact, the treating physician rarely determines the actual cause in the person. Etiology is usually not needed to determine the nature of the disease or the proper course of treatment. Biomarkers, other than diagnostic criteria, rarely point to a specific cause(s) in a given case. The “differential diagnosis” misnomer tends to obscure clear reasoning about physician witnesses, who are often not experts in epidemiology or other sciences needed to assess general causation, not familiar with systematic reviews, not published on the scientific issue of general causation.  The specific causal attribution is analogous to differential diagnosis, in its process of ruling in, and then ruling out, and therefore is sometimes called differential etiology. See, e.g., Michael D. Green, D. Michal Freedman, and Leon Gordis, Reference Guide on Epidemiology 549, 617 & n.211, in Reference Manual on Scientific Evidence (3ed ed. 2011)[RMSE].

2. Differential Etiology Assumes, and Cannot Establish, General Causation

The differential etiology process assumes that each disjunct – each putative specific cause – has itself been established as a known cause of the disease in general. Id. at 618 (“Although differential etiologies are a sound methodology in principle, this approach is only valid if general causation exists … .”). In the case of a novel putative cause, the case may give rise to a hypothesis that the putative cause can cause the outcome, in general, and did so in the specific case.  That hypothesis must, of course, then be tested and supported by appropriate analytical methods before it can be accepted for general causation and as a putative specific cause in a particular individual.

3.  Differential etiology typically fails when a substantial percentage of cases are idiopathic in origin

When one of the disjuncts is “no known cause,” then it will be virtually impossible to negate and remove from the disjunction. If very few cases have idiopathic causes, the error rate may be low, and tolerable. Take for example, asbestosis, a diffuse interstitial lung disease caused by chronic, excessive inhalation of asbestos.  Clinically asbestosis will look similar to idiopathic pulmonary fibrosis (IPF), a lung disease of unknown origin.  IPF may remain a differential diagnosis in every case because it cannot be ruled out, clinically.  The likelihood of IPF, however, will be relatively low in a cohort of asbestos miners, and thus not a serious source of error.  In a study of household exposure cases, in which the exposure resulted from a family member’s bringing home dust from work, IPF may be a much likelier alternative, and the failure to rule it out may invalidate conclusions about the asbestosis diagnosis in every case in the cohort.

With respect to differential etiology, the same principle applies: the iterative disjunctive syllogism requires ruling out “unknown,” or at least minimizing the number of cases in the unknown disjunct that are not ruled out.  See RMSE at 618 (“Although differential etiologies are a sound methodology in principle, this approach is only valid if … a substantial proportion of competing causes are known. Thus, for diseases for which the causes are largely unknown, such as most birth defects, a differential etiology is of little benefit.”)(internal citations omitted). Accordingly, many cases reject proffered expert witness testimony on differential etiology, when the witnesses fail to rule out idiopathic causes in the case at issue. What is a substantial proportion?  Unfortunately, the RMSE does not attempt to quantify or define “substantial.” The inability to rule out unknown etiologies remains the fatal flaw in much expert witness opinion testimony on specific causation.

More Nonsense on Differential Diagnosis

The Supreme Court recently addressed differential etiology in Matrixx Initiatives, in stunningly irrelevant and errant dicta:

“We note that courts frequently permit expert testimony on causation based on evidence other than statistical significance. See, e.g., Best v. Lowe’s Home Centers, Inc., 563 F. 3d 171, 178 (6th Cir 2009); Westberry v. Gislaved Gummi AB, 178 F. 3d 257, 263–264 (4th Cir. 1999) (citing cases); Wells v. Ortho Pharmaceutical Corp., 788 F. 2d 741, 744–745 (11th Cir. 1986). We need not consider whether the expert testimony was properly admitted in those cases, and we do not attempt to define here what constitutes reliable evidence of causation.”

Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, 1319 (2011).  The citation to Wells was clearly wrong in that the plaintiffs in that case had, in fact, relied upon studies that were nominally statistically significant, and so the Wells court could not have held that statistical significance was unnecessary.[1]

The two other cases cited by the Supreme Court, however, were both about “differential diagnosis,” and had nothing to do with statistical significance.  Both cases assumed that general causation was established, and inquired into whether expert witnesses could reasonably attribute the health outcome in the case to the exposures that were established causes of such outcomes.  The Court’s selection of these cases, quite irrelevant to its discussion, appears to have come from the Solicitor General’s amicus brief in Matrixx.[2]

Although cited for an irrelevant proposition, the Supreme Court’s selection of the Best’s case was puzzling because the Sixth Circuit’s discussion of the issue is particularly muddled. Here is the relevant language from Best:

“[A] doctor’s differential diagnosis is reliable and admissible where the doctor

(1) objectively ascertains, to the extent possible, the nature of the patient’s injury…,

(2) ‘rules in’ one or more causes of the injury using a valid methodology,

and

(3) engages in ‘standard diagnostic techniques by which doctors normally rule out alternative causes” to reach a conclusion as to which cause is most likely’.”

Best v. Lowe’s Home Centers, Inc., 563 F.3d 171, 179, 183-84 (6th Cir. 2009).

Of course, a physicians rarely use this iterative process to arrive at causes of diseases in an individual; they use it to identify the disease or disease process that is responsible for the patient’s signs and symptoms. See generally Harold C. Sox, Michael C. Higgins, and Douglas K. Owens, Medical Decision Making (2d ed. 2014).  The Best court’s description does not make sense in that it characterizes the process as ruling in “one or more” causes, and then ruling out alternative causes.  If an expert had ruled in only one cause, then there would be no need or opportunity to rule out an alternative cause.  If the one ruled-in cause was ruled out for other reasons, then the expert witness would be left with a case of idiopathic disease.[3]

We can take some solace in the Supreme Court’s disclaimer that it was not attempting reliable evidence of causation. Differential etiology, however, is irrelevant to general causation, which is the context in which statistical significance arises.  The issue of statistical significance was not addressed; nor could it have been addressed in either Best or Westberry.

What follows is an incomplete selection of cases on differential etiology, good and bad.


Differential Etiology for Specific Causation

FIRST CIRCUIT

Baker v. Dalkon Shield Claimaints Trust, 156 F.3d 248, 252-53 (1st Cir. 1998) (stating that “ ‘differential diagnosis’ is a standard medical technique”)

District Courts within 1st Circuit

Whiting v. Boston Edison Co., 891 F. Supp. 12, 21 n.41 (D. Mass. 1995) (noting that differential diagnosis cannot be used to support conclusion of specific causation when 90% disease cases are idiopathic)

Polaino v. Bayer Corp., 122 F. Supp. 2d 63, 70 & n.7 (D. Mass. 2000) (“differential diagnosis is a useful means of distinguishing one disease from another with similar symptoms, it is not a technique typically used to investigate the cause of an illness”)

Plourde v. Gladstone, 190 F. Supp. 2d 708, 722-723 (D. Vt. 2002) (excluding testimony where expert failed to rule out causes of plaintiff’s illness other than exposure to herbicides)

Allen v. Martin Surfacing, 263 F.R.D. 47, 56 (D. Mass. 2008) (admitting general and specific causation testimony of ALS, to be tested by adversary process, rather than excluded altogether, despite paucity of epidemiologic evidence)

Milward v. Acuity Specialty Products Group, Inc., Civil Action No. 07–11944–DPW, 2013 WL 4812425 (D. Mass. Sept. 6, 2013)


SECOND CIRCUIT

McCullock v. H.B. Fuller Co., 61 F.3d 1038, 1043–44 (2d Cir.1995) (defining differential etiology as an analysis “which requires listing possible causes, then eliminating all causes but one”) (affirming admission of a treating doctor’s testimony despite his inability to “point to a single piece of medical literature that says glue fumes cause throat polyps”) (upholding admission of treating physician who relied upon his “care and treatment of McCullock; her medical history (as she related it to him and as derived from a review of her medical and surgical reports); pathological studies; review of [Defendant] Fuller’s [Material Safety Data Sheet], his training and experience, use of a scientific analysis known as differential etiology (which requires listing possible causes, then eliminating all causes but one); and reference to various scientific and medical treatises”)

United States v. Zuchowitz, 140 F.3d 381, 385-87 (2d Cir. 1998) (“[d]isputes as to . . . faults in [the] use of differential etiology as a methodology, or lack of textual authority for [an] opinion, go to the weight, not the admissibility of [the] testimony”)

Wills v. Amerada Hess Corp., 379 F. 3d 32, 45-46 (2d Cir. 2004)(noting that expert witness failed to account for other possible causes), cert. denied, 126 S.Ct. 355 (2005)

Ruggiero v. Warner-Lambert Co., 424 F.3d 249, 254 (2d Cir. 2005) (“Where an expert employs differential diagnosis to ‘rule out other potential causes’ for the injury at issue, he must also ‘rule in the suspected cause’ and do so using ‘scientifically valid methodology’.”) (quoting Cavallo v. Star Enter., 892 F. Supp. 756, 771 (E.D. Va. 1995), aff’d on this ground, rev’d on other grounds, 100 F.3d 1150 (4th Cir. 1996))

District Courts within 2d Circuit

Becker v. National Health Products, 896 F.Supp. 100 (N.D.N.Y. 1995).

Mancuso v. Consolidated Edison Co. of New York, Inc., 967 F. Supp. 1437, 1450 (S.D.N.Y. 1997)(“it is improper for an expert to presume that the plaintiff ‘must have somehow been exposed to a high enough dose to exceed the threshold [necessary to cause the illness], thereby justifying his initial diagnosis.’ This is circular reasoning.”)

Zwillinger v. Garfield Slope Hous. Corp., 1998 WL 623589, at *20 (E.D.N.Y. Aug. 17, 1998) (excluding testimony and granting summary judgment where expert failed to rule out alternative causes of plaintiff’s immunotoxicity syndrome)

Prohaska v. Sofamor, S.N.C., 138 F. Supp. 2d 422, 439 (W.D.N.Y. 2001) (excluding expert’s opinion and granting summary judgment where expert “was unable to rule out, to a reasonable degree of medical certainty, [plaintiff’s] pre-existing condition, scoliosis, as a current cause of her pain”)

Martin v. Shell Oil Co., 180 F. Supp. 2d 313, 320 (D. Conn. 2002)

Figueroa v. Boston Scientific Corp., 254 F.Supp. 2d 361, 368 (S.D.N.Y. 2003)(“failure to rule out alternative causes is not determinative of admissibility of evidence but goes to weight, which is for a jury to decide”)

Perkins v. Origin Medsystems, Inc., 299 F. Supp. 2d 45, 57-61 (D. Conn. 2004)

In re Rezulin Prods. Liab. Litig., No. MDL 1348, 00 Civ. 2843(LAK), 2004 WL 2884327, at *3-4 (S.D.N.Y. Dec. 10, 2004) (holding that differential etiology may not be used to prove general causation) (“differential diagnosis does not ‘speak to the issue of general causation. [It] assumes that general causation has been proven for the list of possible causes’ that it rules in and out in coming to a conclusion.”)

In re Ephedra Prods. Liab. Litig., 393 F. Supp. 2d 181, 187 (S.D.N.Y. 2005) (Rakoff, J.)


THIRD CIRCUIT

In re Paoli R.R. Yard PCB Litig., 916 F.2d 829, 862 (3d Cir.1990)

In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 758 (3d Cir. 1994) (“[D]ifferential diagnosis generally is a technique that has widespread acceptance in the medical community, has been subject to peer review, and does not frequently lead to incorrect results …. )

Wade-Greaux v. Whitehall Labs., Inc., 874 F. Supp. 1441 (D.V. I.), aff’d, 46 F.3d 1120 (3d Cir. 1994) (excluding testimony of expert who failed to rule out alternative causes of plaintiff’s birth defects)

Kannankeril v. Terminex Int’l, Inc., 128 F.3d 802, 807 (3d Cir. 1997)

Heller v. Shaw Indus., Inc., 167 F.3d 146, 154 (3d Cir. 1999) (a medical expert need not “always cite published studies on general causation in order to reliably conclude that a particular object caused a particular illness” so long as there are good grounds, such as differential diagnosis, for the conclusion)

District Courts within 3d Circuit

Wade-Greaux v. Whitehall Labs., Inc., 874 F. Supp. 1441 (D.V. I.), aff’d, 46 F.3d 1120 (3d Cir. 1994) (excluding testimony of expert who failed to rule out alternative causes of plaintiff’s birth defects)

Diaz v. Matthey, Inc., 893 F. Supp. 358, 376-377 (D.N.J. 1995) (excluding testimony and granting summary judgment where expert failed to rule out alternative causes for plaintiff’s asthma)

Rutigliano v. Valley Bus. Forms, 929 F. Supp. 779, 787 (D.N.J. 1996) (excluding expert’s testimony and granting summary judgment where the “record is replete with evidence, including [the expert’s] own admissions, that [plaintiff’s] symptoms could be attributable to medical conditions other than formaldehyde sensitization”)

Reiff v. Convergent Technologies, 957 F. Supp. 573, 582-83 (D.N.J. 1997) (excluding expert’s testimony and granting summary judgment where expert failed to rule out alternative causes of plaintiff’s carpal tunnel syndrome)

O’Brien v. Sofamor, 1999 WL 239414, at *5 (E.D. Pa. Mar. 30, 1999) (excluding expert’s testimony and granting summary judgment where plaintiff “offer[ed] no evidence that [plaintiff’s experts] performed a differential diagnosis, or even considered other potential causes” of plaintiff’s back condition)

Kent v. Howell Elec. Motors, 1999 WL 517106, at * 5 (E.D. Pa. July 20, 1999) (excluding expert testimony and granting summary judgment because expert could “not rule out reasonable alternative theories of what caused the retaining ring to fail”)

Schmerling v. Danek Med., Inc., 1999 WL 712591, at *9 (E.D. Pa. Sept. 10, 1999) (excluding expert’s testimony and granting summary judgment on the grounds that expert’s failure to rule out alternative causes “alone warrants a determination that the expert’s methodology is unreliable”)

Turbe v. Lynch Trucking Inc., 1999 WL 1087026, at *6 (D.V.I. Oct. 7, 1999) (excluding expert’s testimony where expert “expressed awareness of obvious alternative causes” yet “did not investigate any other possible causes”)

In re Paoli R.R. Yard PCB Litig., 2000 WL 274262, at *5 (E.D. Pa. March 1, 2000) (expert’s opinion should be excluded “because she failed to rule out alternative causes” of plaintiff’s injuries)

Magistrini v. One Hour Martinizing Dry Cleaning, 180 F. Supp. 2d 584, 608-610 (D.N.J. 2002) (excluding testimony of expert who sought to testify that dry cleaning fluid caused leukemia, but failed to rule out smoking as an alternative cause) (holding expert witness’s differential methodology unreliable when objection to the opinion points to a plausible alternative cause, and the expert witness offers no explanation for his conclusion that the exposure was a substantial factor in causing plaintiff’s injury)

Yarchak v. Trek Bicycle Corp., 208 F. Supp. 2d 470, 498 (D.N.J. 2002)

Soldo v. Sandoz Pharms. Corp., 244 F.Supp. 2d 434, 554-56, 567 (W.D. Pa. 2003) (excluding experts’ specific causation testimony based on a differential diagnosis because the witnesses “did not demonstrate any valid diagnostic methodology–any ‘sufficient diagnostic technique’–for excluding” other plausible causes as the sole cause of the plaintiff’s injury) (holding that the differential “diagnostic” process is not reliable, and not admissible, unless it reliably rules out reasonable alternative causes or idiopathic causes of the alleged harm); see id. at 524 (differential diagnosis cannot establish general causation)

Perry v. Novartis, 564 F. Supp. 2d 452, 469 (E.D. Penn. 2008)(Dalzell, J.) (“Standing alone, the presence of a known risk factor is not a sufficient basis for ruling out idiopathic origin in a particular case, particularly where most of the cases of the disease have no known cause.”)


FOURTH CIRCUIT

Benedi v. McNeil-P.P.C. Inc., 66 F.3d 1378, 1384 (4th Cir. 1995) (upholding admission of differential diagnosis , reasoning circularly that diagnosing physicians use it)

Cavallo v. Star Enter., 892 F. Supp. 756, 771, (E.D. Va. 1995) (noting that it is not sufficient for an expert to rule out other possible causes if he has no sound evidence that allows him to “rule in” the purported cause), aff’d in relevant part, rev’d in part on other grounds, 100 F.3d 1150 (4th Cir. 1996)

Oglesby v. General Motors Corp., 190 F.3d 244, 250 (4th Cir. 1999) (affirming exclusion of testimony where “as a matter of logic, [the expert] could not eliminate other equally plausible causes” of cracked plastic inlet)

Westberry v. Gislaved Gummi AB, 178 F.3d 257, 262-263 (4th Cir. 1999) (“Differential diagnosis, or differential etiology, is a standard scientific technique of identifying the cause of a medical problem by eliminating the likely causes until the most probable one is isolated”)

Cooper v. Smith & Nephew, Inc., 259 F.3d 194, 202 (4th Cir.2001) (holding that an expert’s opinion based on a differential diagnosis is generally admissible but that there must be adequate evidence that the differential is a cause of the disease)

District Courts within 4th Circuit

Higgins v. Diversey Corp., 998 F. Supp. 598, 603 (D. Md. 1997), aff’d, 135 F.2d 769 (4th Cir. 1998) (excluding expert’s testimony that the accidental inhalation of a bleach caused plaintiff’s injuries, where expert “admit[ted] that he [could] not rule out several other possible causes”)

Driggers v. Sofamor, S.N.C., 44 F. Supp. 2d 760, 765 (M.D.N.C. 1998) (excluding expert’s testimony and granting summary judgment where “expert failed to rule out other possible causes of [plaintiff’s back] pain”)

Aldridge v. Goodyear Tire & Rubber Co., 34 F. Supp. 2d 1010, 1024 (D. Md. 1999), vacated on other grounds, 223 F.3d 263 (4th Cir. 2000) (excluding testimony of plaintiffs’ experts where they “failed to adequately address possible alternative causes of plaintiffs’ illnesses”)

Fitzerald v. Smith & Nephew Richards, Inc., 1999 WL 1489199 (D. Md. Dec. 30, 1999) (excluding expert’s testimony and granting summary judgment where expert “failed to rule out what could have been another cause of [plaintiff’s] condition”)

Shreve v. Sears, Robuck & Co., 166 F. Supp. 2d 378, 397-98 (D. Md. 2001) (excluding testimony where expert failed to rule out other causes of plaintiff’s injury other than an alleged defect in snow thrower)

Smith v. Wyeth-Ayerst Laboratories Co., 278 F.Supp. 2d 684, 692 (W.D.N.C. 2003)(inexplicably rejecting argument that idiopathic causes prevent the use of “differential etiology” method to ascertain specific causation)

Roche v. Lincoln Property Co., 278 F.Supp. 2d 744 (E.D. Va. 2003) (excluding in part expert witness’s testimony that mold caused the plaintiffs’ allergy-like symptoms because he failed “to rule out the Roches’ significant allergies to cats, dust mites, grasses, weeds, and trees as potential causes for the Roches’ symptoms,” which pre-existed moving to the defendant’s apartment)

Doe v. Ortho-Clinical Diagnostics, Inc., 440 F.Supp. 2d 465, 476-78 (M.D.N.C. 2006) (excluding improperly conducted differential diagnosis in thimerosal vaccine autism case)

Hines v. Wyeth, Inc., 2011 WL 2792436, at *3 (S.D.W.V. July 14, 2011) (excluding expert witness who failed properly to rule out alternative causes of breast cancer in hormone therapy case)


FIFTH CIRCUIT

Moore v. Ashland Chem. Inc., 151 F.3d 269, 278-79 (5th Cir. 1998)(en banc), cert. denied, 526 U.S. 1064 (1999)(holding that trial court has discretion to conclude that an expert’s differential diagnosis was insufficiently reliable to be submitted to the jury)

Curtis v. M&S Petroleum, Inc., 174 F.3d 661, 670 (5th Cir. 1999)

Michaels v. Avitech, Inc., 202 F.3d 746, 753 (5th Cir. 2000) (excluding testimony when “plaintiff’s experts wholly fail[ed] to address and rule out the numerous other potential causes” of an aircraft disaster)

Black v Food Lion, Inc, 171 F3d 308 (5th Cir 1999) (expert witness, purporting to use a differential diagnosis, testified that plaintiff’s slip in the supermarket caused fibromyalgia, which is largely idiopathic) (“This analysis amounts to saying that because [the physician] thought she had eliminated other possible causes of fibromyalgia, even though she does not know the real ‘cause,’ it had to be the fall at Food Lion. This is not an exercise in scientific logic but in the fallacy of post-hoc propter-hoc reasoning, which is as unacceptable in science as in law.”)

Johnson v. Arkema, Inc., 685 F.3d 452, 467–68 (5th Cir. 2012) (suggesting that a proper differential diagnosis may be admissible)

District Courts within 5th Circuit

Bennett v. PRC Public Sector, 931 F. Supp. 484, 492 (S.D. Tex. 1996) (excluding testimony of expert who failed to consider and rule out alternative causes of plaintiff’s repetitive motion disorders)

Conger v. Danek Med., Inc., 1998 WL 1041331, at *5-6 (N.D. Tex. Dec. 14, 1998) (excluding expert’s testimony and granting summary judgment when expert “had not attempted to rule out [other potential sources] as causes for [plaintiff’s back] pain”);

Nobles v. Sofamor, 1999 WL 1129661 (S.D. Tex June 30, 1999) (Rosenthal, J.)

Leigh v. Danek Med., Inc., 1998 WL 1041329, at *4-5 (N.D. Tex. Dec. 14, 1998) (excluding expert’s testimony and granting summary judgment where expert failed to rule out alternative causes of plaintiff’s back pain)

In re Propulsid Products Liability Litigation, 261 F. Supp. 2d 603, 618 (E.D. La. 2003)(“They also cannot rule out other explanations for the measurements that form the predicate of the QTc, the heart rate, or heart rate variability)

Cano v. Everest Minerals Corp., 362 F. Supp. 2d 814, 844-46 (W.D. Tex. 2005) (addressing specific causation in context of known carcinogen (radiation), and holding that expert witness’s methodology of concluding that any cause that could have been a cause was in fact a cause and a substantial factor was invalid)

Ridgeway v. Pfizer Inc., No. 2:09-cv-02794, 2010 WL 1729187, *4 (E.D.La. April 27, 2010) (using “differential diagnosis,” or res ipsa loquitur, the proponent bears the burden of “excluding  reasonable explanations for the accident other than defendant’s negligence”)


SIXTH CIRCUIT

Glaser v. Thompson Med. Co., 32 F.3d 969, 978 (6th Cir. 1994) (differential diagnosis defined as “standard diagnostic tool used by medical professionals to diagnose the most likely cause or causes of illness, injury and disease”)

Hardyman v. Norfolk & W. Ry. Co., 243 F.3d 255, 260 (6th Cir.2001) (“Differential diagnosis … is a standard scientific technique of identifying the cause of a medical problem”)

Downs v. Perstorp Components, Inc., 26 F. Appx. 472, 476–77 (6th Cir. 2002) (holding that exclusion of expert’s opinion was appropriate when arrived at by a “methodology primarily [that] involved reasoning backwards from Downs’ condition and, through a process of elimination, concluding that [defendant’s product] must have caused it”)

Best v. Lowe’s Home Centers, Inc., 563 F. 3d 171, 178-80 (6th Cir. 2009)

Gass v. Marriott Hotel Servs., 558 F.3d 419, 426 (6th Cir. 2009) (“the ability to diagnose medical conditions is not remotely the same as the ability to deduce … in a scientifically reliable manner the causes of those medical conditions”)(internal citations omitted)

Tamraz v. BOC Group Inc., No. 1:04-CV-18948, 2008 WL 2796726 (N.D. Ohio July 18, 2008) (denying Rule 702 challenge to treating physician’s causation opinion), rev’d sub nom., Tamraz v. Lincoln Elec. Co., 620 F.3d 665, 673 (6th Cir. 2010) (carefully reviewing record of trial testimony of plaintiffs’ treating physician; reversing judgment for plaintiff based in substantial part upon treating physician’s speculative causal assessment created by plaintiffs’ counsel; “Getting the diagnosis right matters greatly to a treating physician, as a bungled diagnosis can lead to unnecessary procedures at best and death at worst. But with etiology, the same physician may often follow a precautionary principle: If a particular factor might cause a disease, and the factor is readily avoidable, why not advise the patient to avoid it? Such advice—telling a welder, say, to use a respirator—can do little harm, and might do a lot of good. This low threshold for making a decision serves well in the clinic but not in the courtroom, where decision requires not just an educated hunch but at least a preponderance of the evidence.”) (internal citations omitted), cert. denied, ___ U.S. ___ , 131 S. Ct. 2454, 2011 WL 863879 (2011)

Thomas v. Novartis Pharm. Corp., 443 Fed. App’x 58, 61-62 (6th Cir. 2011) (excluding expert witnesses in cases involving osteonecrosis of the jaw, allegedly caused by bisphosphonate medication, for failing to conduct proper differential analysis; emphasizing “the importance of correctly determining the cause of the osteonecrosis … does nothing to establish that [the doctor] can in fact, reliably determine the cause of a patient’s [osteonecrosis]”)

District Courts within 6th Circuit

Nelson v. Tennessee Gas Pipeline Co., 1998 WL 1297690, at *6 (W.D. Tenn. Aug. 1, 1998) (excluding testimony of expert who “failed to engage in adequate techniques to rule out alternative causes and offers no good explanation as to why his opinion is nevertheless reliable in light of other potential causes of the alleged injuries”)

Downs v. Perstorp Components, 126 F. Supp. 2d 1090, 1127 (E.D. Tenn. 1999) (excluding expert testimony as to whether exposure to chemicals caused plaintiff’s injuries where expert failed to rule out alternative causes)

Huffman v. SmithKline Beecham Clinical Lab., Inc., 111 F. Supp. 2d 921, 930 (N.D. Ohio 2000)

Asad v. Continental Airlines, Inc., 314 F. Supp. 2d 726 (N.D. Ohio 2004)


SEVENTH CIRCUIT

O’Connor v. Commonwealth Edison, 13 F.3d 1090, 1106 (7th Cir. 1994) (holding that physician’s testimony that  cataracts were caused by radiation exposure based upon visual examination of the plaintiff’s was not reliably supported by clinical examination), cert. denied, 114 S.Ct. 2711 (1994).

Ervin v. Johnson & Johnson, Inc., 492 F.3d 901, 904 (7th Cir.2007)(noting that “[a] differential diagnosis satisfies a Daubert analysis if the expert uses reliable methods”) (excluding differential etiological testimony that was based upon ruling a particular potential specific cause based on temporal proximity)

District Courts within 7th Circuit

Schmaltz v. Norfolk & Western Ry., 878 F.Supp. 1122 (N.D. Ill. 1995)

Lennon v. Norfolk & Western Ry., 123 F.Supp.2d 1143, 1153 (N.D.Ind. 2000) (excluding neurologist’s unreliable causal attribution of multiple sclerosis to fall)

Eve v. Sandoz Pharm. Corp., No. IP 98-1429, 2001 U.S. Dist. LEXIS 4531 (S.D. Ind. 2001)

Caraker v. Sandoz Pharms., 188 F. Supp. 2d 1026, 1030 (S.D. Ill. 2001) (when a differential diagnosis is employed “in the practice of science (as opposed to its use by treating physicians in the practice of medicine out of necessity) it must reliably ‘rule in’ a potential cause”)

Bickel v. Pfizer, Inc., 431 F.Supp. 2d 918, 923 (N.D. Ind. 2006) (“the Plaintiff cannot rely on [differential] diagnosis to establish general causation”)


EIGHTH CIRCUIT

National Bank of Commerce v. Assoc. Milk Producers, 22 F. Supp. 2d 942, 963 (E.D. Ark. 1998), aff’d, 191 F.3d 858 (8th Cir.1999) (excluding testimony and granting summary judgment where expert did “not successfully rule out other possible alternative causes” for cancer)

Turner v. Iowa Fire Equip. Co., 229 F.3d 1202, 1208-09 (8th Cir. 2000) (“[A] medical opinion about causation, based upon a proper differential diagnosis, is sufficiently reliable to satisfy Daubert.”)(“If a properly qualified medical expert performs a reliable differential diagnosis through which, to a reasonable degree of medical certainty, all other possible causes of the victims’ condition can be eliminated, leaving only the toxic substance as the cause, a causation opinion based on that differential diagnosis should be admitted.”)

Bonner v. ISP Technologies, Inc., 259 F.3d 924, 1208 (8th Cir. 2001)

Glastetter v. Novartis Pharms. Corp., 252 F.3d 986, 989 (8th Cir. 2001) (per curiam) (“[T]he district court excluded the differential diagnoses performed by Glastetter’s expert physicians because they lacked a proper basis for ‘ruling in’ Parlodel as a potential cause of [an intracerebral hemorrhage] in the first place. . . . We agree with the district court’s conclusion.”)

Jazairi v. Royal Oaks Apts., 217 Fed. Appx. 895 (8th Cir. 2007) (excluding differential etiological testimony that was based upon ruling a particular potential specific cause based on temporal proximity)

Bland v. Verizon Wireless, L.L.C., 538 F.3d 893, 897 (8th Cir. 2008) (affirming exclusion of treating physician’s differential diagnosis)

District Courts within 8th Circuit

Stover v. Eagle Products, 1996 WL 172972, at *11 (D. Kan. Mar. 19, 1996) (excluding testimony of expert who “[did] not explain in any meaningful detail how he [was] able to exclude the numerous multiple alternative causes” of injury to plaintiff’s dogs) (excluding expert testimony for failing to rule out alternative causes)

Bruzer v. Danek Med., Inc., 1999 WL 613329, at *8 (D. Minn. Mar. 8, 1999) (excluding expert’s testimony and granting summary judgment where expert did “not attempt to rule out any alternative potential causes for [plaintiff’s] continuing and increasing [back] pain”) (excluding expert testimony for failing to rule out alternative causes)

Thurman v. Missouri Gas Energy, 107 F. Supp. 2d 1046, 1058 (W.D. Mo. 2000) (expert’s opinion “that the pipeline failed because of corrosion” was excluded and summary judgment granted where expert reached the conclusion “without eliminating other causes”) (excluding expert testimony for failing to rule out alternative causes)

Jisa Farms, Inc. v. Farmland Indus., No. 4:99CV3294, 2001 U.S. Dist. LEXIS 26084 (D. Neb. 2001) (excluding expert testimony for failing to rule out alternative causes)

In re Viagra Prod. Liab. Litig., 658 F. Supp. 2d 950, 957 (D. Minn. 2009)


NINTH CIRCUIT

Kennedy v. Collagen Corp., 161 F.3d 1226, 1228-30 (9th Cir. 1998)

Clausen v. M/V NEW CARISSA, 339 F.3d 1049, 1057 (9th Cir. 2003)

Messick v. Novartis Pharms., ___ F.3d. ___, 2014 WL 1328182 (9th Cir. 2014)

District Courts within 9th Circuit

Hall v. Baxter Healthcare Corp., 947 F.Supp. 1387, 1413 (D.Ore. 1996) (explaining that differential diagnosis assumes general causation has been established) (“differential diagnosis does not by itself prove the cause, even for the particular patient. Nor can the technique speak to the issue of general causation.”)


TENTH CIRCUIT

Hollander v. Sandoz Pharms. Corp., 289 F.3d 1193, 1211 (10th Cir. 2002) (stating that “experts would need to present reliable evidence that the drug can cause strokes” before differential diagnosis could be admissible)

Goebel v. Denver & Rio Grande W. RR., 346 F.3d 987, 999 (10th Cir. 2003)

Tingey v. Radionics, 193 Fed. Appx. 747, 763 (10th Cir. 2006)

District Courts within 10th Circuit

Stover v. Eagle Products, 1996 WL 172972, at *11 (D. Kan. Mar. 19, 1996) (excluding testimony of expert who “[did] not explain in any meaningful detail how he [was] able to exclude the numerous multiple alternative causes” of injury to plaintiff’s dogs)

In re Breast Implant Lit., 11 F. Supp. 2d 1217, 1230, 1234 (D. Colo. 1998) (excluding expert testimony where expert failed to “explain what alternative causes he considered, or how he ruled out other possible causes” of plaintiffs’ auto- immune disease) (“Differential diagnosis may be utilized by a clinician to determine what recognized disease or symptom the patient has, but it is incapable of determining whether exposure to a substance caused disease in the legal sense.”)


ELEVENTH CIRCUIT

McClain v. Metabolife Int’l, Inc., 401 F.3d 1233, 1252-53 (11th Cir.2005) (detailing a reliable differential diagnostic process)(“A valid differential diagnosis, however, only satisfies a Daubert analysis if the expert can show the general toxicity of the drug by reliable methods.”)

Rink v. Cheminova, Inc., 400 F.3d 1286, 1295 (11th Cir. 2005) (holding that a differential diagnosis alone does not support a finding of causation where no expert testimony from a treating physician or toxicologist is presented, or any toxicological evidence produced; specifically rejecting the Westberry)  (“[I]n the context of summary judgment . . . differential diagnosis evidence by itself does not suffice for proof of causation.”)

Guinn v. AstraZeneca Pharms. LP, 602 F.3d 1245 (11th Cir. 2010), aff’g 598 F. Supp. 2d 1239, 1243 (M.D. Fla. 2009) (excluding expert witness’s specific causation opinion for failing “to articulate any scientific methodology for assessing whether, and to what extent, Seroquel contributed to Guinn’s weight gain and diabetes”)

Hendrix v. Evenflo Co., 609 F.3d 1183, 1194-95 (11th Cir. 2010), aff’g, 255 F.R.D. 568, 596 (N.D. Florida, 2009)(differential etiology not diagnosis)

Kilpatrick v. Breg, Inc., 613 F.3d 1329, 1342 (11th Cir. 2010) (noting that differential diagnosis “assumes the existence of general causation”)

District Courts within 11th Circuit

Coleman v. Danek Med., Inc., 43 F. Supp. 2d 637, 650 n. 23 (S.D. Miss. 1999) (stating that “in reaching his conclusion that these plaintiffs were injured by Danek’s product, Dr. Aldreti did not rule out other causes of their alleged injuries. Thus, his conclusion that their injuries were caused by Danek’s product is based on pure speculation – and is not a valid differential diagnosis.”)

Siharath v. Sandoz Pharms. Corp., 131 F. Supp. 2d 1347, 1356-71 (N.D. Ga. 2001) (holding that differential diagnosis cannot rule in a general causal factor, and noting in Parlodel case that “[e]xperts must do something more than just ‘rule out’ other possible causes. They must explain how they were able to ‘rule in’ the product in question”), aff’d sub nom., Rider v. Sandoz Pharm. Corp., 295 F.3d 1194 (11th Cir. 2002).


D.C. CIRCUIT

Ambrosini v. Labarraque, 101 F.3d 129, 140 (D.C.Cir.1996) (describing the appropriate use of differential diagnosis to prove specific causation)

Meister v. Med. Eng’g Corp., 267 F.3d 1123, 1129, 347 U.S. App. D.C. 361 (D.C. Cir. 2001)(“whatever factors remain after other alternative causes have been eliminated [must be] at least capable of causing the disease in question”)


STATE COURT CASES

ALASKA

John’s Heating Service v. Lamb, 46 P.3d 1024 (Alaska 2002) (“[a] differential diagnosis that fails to take serious account of other potential causes may be so lacking that it cannot provide a reliable basis for an opinion on causation,” but not in this case involving carbon monoxide poisoning)

ARIZONA

Lofgren v. Motorola, No. CV 93-05521, 1998 WL 299925, at *24 (Ariz. Super. Ct. June 1, 1998) (differential diagnosis as a method of determining the cause of disease has been “unequivocally rejected by the scientific community”)

IOWA

Ranes v. Adams Labs., Inc., 778 N.W.2d 677, 690 (Iowa 2010)(general causation for each differential should be established by adequate evidence)

KANSAS

Kuhn v. Sandoz Pharms., 14 P.3d 1170, 1173-78 (Kan. 2000) (Frye test not applicable to “pure opinion” testimony such as differential diagnosis)

LOUISIANA

Keener v. Mid-Continent Cas., 817 So. 2d 347 (La. Ct. App. 5th Cir. 2002), writ denied, 825 So. 2d 1175 (La. 2002)

MINNESOTA

Zandi v. Wyeth, 2009 Minn. App. Unpub. LEXIS 785, at *17-18 (Minn. Ct. App. July 21, 2009), petition denied, 2009 Minn. LEXIS 648 (Minn. Sept. 29, 2009)

NEW JERSEY

Creanga v. Jardal, 185 N.J. 345, 886 A.2d 633 (2005) (holding that properly conducted differential diagnosis was admissible; reversing exclusion of physician testimony in case)

OHIO

Terry v. Ottawa Cty. Bd. of Mental Retardation & Developmental Delay, 658, 847 N.E.2d 1246 (Ohio Ct. App. 2006) (“We agree with the trial court: Dr. Bernstein did not conduct a scientifically valid differential diagnosis, because his method relied primarily upon temporal relationships and because he did not rule out other possible causes. He was properly barred from testifying to specific causation.”)

TEXAS

Mitchell Energy Corp. v. Bartlett, 958 S.W.2d 430, 448 (Tex. App.–Fort Worth 1997, pet. denied) (“Dr. Basset’s failure to rule out other causes of the presence of hydrogen sulfide in appellees’ water renders his opinion ‘little more than speculation.’”)

Weiss v. Mechanical Associated Services, Inc., 989 S.W.2d 120, 126 (Tex. App.– San Antonio 1999, pet. denied) (affirming summary judgment for the defendants in a case involving injuries allegedly caused by exposure to a chemical, because “none of Weiss’ experts were able to rule out other potential causes of Weiss’ illness with reasonable certainty”)

Williams v. NGF, Inc., 994 S.W.2d 255, 257 (Tex. App.–Texarkana 1999, no pet. h.) (affirming summary judgment for defendant because plaintiffs “failed to produce evidence which excluded the possibility that . . . other flowers or chemical agents used on them were the cause of her injuries”)

Austin v. Kerr-McGee Refining Corp., 25 S.W.3d 280, 293 (Tex. App.-Texarkana 2000, no pet.) (affirming summary judgment for defendants; trial court properly excluded plaintiffs’ scientific evidence because, among other reasons, plaintiffs “failed to exclude other plausible causes with reasonable certainty”)

Martinez v. City of San Antonio, 40 S.W.3d 587, 595 (Tex. App.–San Antonio 2001, no pet.) (“The opinions of Matson and Baynes, when offered to prove Alamodome site lead caused appellants’ injuries, constitute no evidence because Matson, in arriving at his lead calculation, failed to rule out alternative sources of the lead contamination.”)

Neal v. Dow Agrosciences L.L.C., 74 S.W.3d 468, 473 n. 3 (Tex. App. – Dallas 2002, no pet.)(describing “differential diagnosis” as a patient-specific process of elimination) (citing Minnesota Min. And Mfg. Co. v. Atterbury, 978 S.W.2d 183, 194 n. 9 (Tex. App. – Texarkana 1998, pet. denied)

Coastal Tankships, USA, Inc. v. Anderson, 87 S.W.3d 591, at 609-10 (2002)(“In the toxic-tort context, a plaintiff must establish general causation for a differential diagnosis to be relevant to show specific causation.”)

UTAH

Alder v. Bayer Corp., AGFA Div., 61 P.3d 1068, 1084–85 (Utah 2002)

VERMONT

Blanchard v. Goodyear Tire & Rubber Co.,  2011 Vt. 85, 30 A.3d 1271 (2011)(holding that plaintiff’s claim that his NHL was caused by benzene was not reliably supported by differential diagnosis when a large percentage of NHL cases have no known cause)

WYOMING

Easum v. Miller, 92 P.3d 794, 802 (Wyo. 2004) (“Most circuits have held that a reliable differential diagnosis satisfies Daubert and provides a valid foundation for admitting an expert opinion. The circuits reason that a differential diagnosis is a tested methodology, has been subjected to peer review/publication, does not frequently lead to incorrect results, and is generally accepted in the medical community.”) (quoting Turner v. Iowa Fire Equip. Co., 229 F.3d 1202, 1208 (8th Cir. 2000)


COMMENTATORS

Conley & Garver,  “William C. Keady and the Law of Scientific Evidence,” 68 Miss. L.J. 39, 51 (1998) (differential diagnosis is “a mixture of science and art, far too complicated for its accuracy to be assessed quantitatively or for a meaningful error rate to be calculated”)

Wendy Michelle Ertmer, “Just What the Doctor Ordered: The Admissibility of Differential Diagnosis in Pharmaceutical Product Litigation,” 56 Vand. L. Rev. 1227 (2003)

Joe G. Hollingsworth & Eric G. Lasker, “The Case Against Differential Diagnosis: Daubert, Medical Causation Testimony, and the Scientific Method,” 37 J. Health Law 85, 98 (2004)

Edward J. Imwinkelried,, “The Admissibility and Legal Sufficiency of Testimony about Differential Diagnosis (Etiology): Of Under‑ and Over‑Estimations,” 56 Baylor L. Rev. 391, 406 (2004)

Michael B. Kent Jr., “Daubert, Doctors and Differential Diagnosis: Treating Medical Causation Testimony as Evidence,” 66 Def. Couns. J. 525 (1999)

Joseph Sanders, “Applying Daubert Inconsistently? Proof of Individual Causation in Toxic Tort and Forensic Cases,” 75 Brooklyn L. Rev. 1367 (2010)

Joseph Sanders & Julie Machal-Fulks, “The Admissibility of Differential Diagnosis Testimony to Prove Causation in Toxic Tort Cases: The Interplay of Adjective and Substantive Law,” 64 Law & Contemp. Prob. 107 (2001)

Ian S. Spechler, “Physicians at the Gates of Daubert: A Look at the Admissibility of Differential Diagnosis Testimony to Show External Causation in Toxic Tort Litigation,” 26 Rev. Litig. 739 (2007)

Teratology Society, Public Affairs Committee, “Teratology Society Public Affairs Committee Position Paper Causation in Teratology-Related Litigation,” 73 Birth Defects Research (Part A) 421, 423 (2005) (“7. Biologic plausibility is an essential element in establishing causation. *** The consideration of alternative explanations is sometimes misused by expert witnesses to mean that failure to find an alternative explanation for an outcome is proof that the exposure at issue must have caused the outcome. A conclusion that an exposure caused an outcome is, however, based on positive evidence rather than on lack of an alternative explanation.”)


[1] Wells involved a claim of birth defects caused by the use of spermicidal jelly contraceptive, which had been the subject of several studies, one of which at least yielded a statistically significant increase in detected birth defects over what was expected.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D.Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986). The problematic aspect of the evidence in Wells lay in its involving spermicidal compounds different from the one at issue in the litigation, and the multiple testing that eroded the usual interpretation of the significance probability.

[2] Brief for the United States as Amicus Curiae Supporting Respondents, in Matrixx Initiatives, Inc. v. Siracusano, 2010 WL 4624148, at *16 (“Best v. Lowe’s Home Centers, Inc., 563 F.3d 171, 178 (6th Cir. 2009) (“an ‘overwhelming majority of the courts of appeals’ agree” that differential diagnosis, a process for medical diagnosis that does not entail statistical significance tests, informs causation) (quoting Westberry v. Gislaved Gummi AB, 178 F.3d 257, 263 (4th Cir. 1999)).”

[3] In the Rule 702 hearings before Judge Jones in Hall v. Baxter Healthcare, Dr. Eric Gershwin defined idiopathic disease as what a pathetic patient suffers from when she has an idiot for a physician.

NIEHS Study – CHARGE Failure to Disclose Conflicts of Interest

June 23rd, 2014

At midnight, the Environmental Health Perspectives (EHP) posted an “in-press” paper on autism and pesticides, slated for full publication in the next few weeks.  Janie F. Shelton, Estella M. Geraghty, Daniel J. Tancredi, Lora D. Delwiche, Rebecca J. Schmidt, Beate Ritz, Robin L. Hansen, and Irva Hertz-Picciotto, “Neurodevelopmental disorders and prenatal residential Proximity to Agricultural pesticides: the CHARGE Study,” Envt’l Health Persp. (advanced publication: June 23, 2014).

The paper was embargoed until midnight, but the principal investigator, Prof. Irva Hertz-Picciotto, violated that embargo by talking about the study’s results in a YouTube video, posted two weeks ago. SeeSelective Leaking — Breaking Ingelfinger’s Rule” (June 20, 2014).

The paper is already attracting media attention. Predictably, the coverage trades on inaccurate and misleading terms, such as “links” and “increased risks.”  See, e.g., Agence France-Presse, “Study finds link between pesticides and autism,” (Yahoo news story claiming “link” in headline, but in text, noting that the study findings “do not show cause-and-effect.”); Arielle Duhaime-Ross (The Verge), “Study further confirms link between autism and pesticide exposure: Living near farms and fields can put a foetus at risk,”  (June 23, 2014 12:01 am) (filed one minute after the embargo was officially lifted, and declaring that “neurotoxins, which include everything from pesticides, to mercury and diesel, are thought to alter brain development in foetuses. Now, a new study further confirms this link by showing that pregnant women who live within a mile of farms and fields where pesticides are employed see their risk of having a child with autism increase by 60 percent — and that risk actually doubles if the exposure occurs in the third trimester”); Zoë Schlanger (Newseek), “Autism Risk Much Higher for Children of Pregnant Women Living Near Agricultural Pesticide Areas” (June 23, 2014).

There are few more incendiary issues than autism or brain damage and environmental exposures.  The media is unlikely to look very critically at this paper.  News reports talk of “links” and “increased risks,” but they do not look at methodological problems and limitations.  They should.

The media should also look at conflicts of interest (COIs). Well, in an ideal world, the media and everyone else would stop trying to use COIs as a proxy for interpreting study validity. The reality, however, is that much of the media treats corporate financial interests as sufficient reason to discount or disregard a study.  If the media want to avoid being hoisted with their own hypocritical petard, they will look closely at the undisclosed COIs in this new paper by Shelton, et al.

First, they will note that the authors disclose that they have no COIs:

“Competing financial interests: The authors have no competing financial interests.”

Second, the media will note that EHP provides explicit instructions to authors on COI disclosures:

Competing Financial Interests

EHP has a policy of full disclosure. Authors must declare all actual or potential competing finan­cial interests involving people or organizations that might reasonably be perceived as relevant. Disclosure of competing interests does not imply that the information in the article is questionable or that conclusions are biased. Decisions to pub­lish or reject an article will not be based solely on a declaration of a competing interest.

***

Employment of any author by a for-profit or nonprofit foundation or advocacy group or work as a consultant also must be indicated on the CFID form.”

EHP Instructions to authors (2013).

Third, the media will ask whether the COI disclosure (“none”) was proper.  The study is one in a series of papers that comes out of research funded by the federal government, THE CHARGE STUDY: CHILDHOOD AUTISM RISKS FROM GENETICS AND THE ENVIRONMENT (2R01ES015359-06). Journalists may want to look, in the first instance, to the principal investigator, Irva Hertz-PicciottoHertz-Picciotto is an epidemiologist at the University of California, Davis, where she is the chief of the Division of Environmental and Occupational Health, Department of Public Health Sciences.

Fourth, the media may want to ask whether Dr. Hertz-Picciotto’s COI disclosure complied with the journal’s requirements.  Recall that EHP requires authors to disclose work or consultancy for a “nonprofit foundation or advocacy group… .” Dr. Hertz-Picciotto sits on the advisory board of Autism Speaks, an advocacy group. More telling, Hertz-Picciotto also serves on the advisory board of the radically anti-chemical Healthy Child, Healthy World organization, located in California (12100 Wilshire Blvd. Suite 800, Los Angeles CA 90025).  According to its website, Healthy Child Healthy World is a California non-profit corporation that advocates to:

“  •  Demand corporate accountability
•  Engage communities for collective action
•  Support safer chemicals and products
•  Influence legislative and regulatory reform.”

Both organizations would seem to come under the EHP COI disclosure policy, but these memberships are not disclosed in the on-line article. Certainly, these affiliations are every bit as potentially enlightening about the principal investigator’s motivations and methodological choices as corporate sponsorship. Of course, it is possible that Dr. Hertz-Picciotto made these disclosures, but the EHP editors chose not to make them public.  If so, shame on the editors.

Most important, the media should provide critical review of the substance of the Shelton paper, and certainly more than sound bites on COIs or “links.” For one thing, even a quick review shows that there are four exposure periods (pre-conception, and three trimesters of pregnancy), two outcome variables (autism spectrum disorder and developmental delay), five exposure substances, and three exposure proximities, for 120 comparisons.  The statistical analysis in the paper uses an alpha of 0.05, which provides a study-wise Type I error rate, and cannot be used to evaluate any one of the 120 comparisons.  The paper’s use of “statistical significance” terminology should be taken with a grain of salt.[1] For another thing, many of the risk factors identified in other studies are not addressed here. See, e.g., Xin Zhang, Cong-Chao Lv, Jiang Tian, Ru-Juan Miao, Wei Xi, Irva Hertz-Picciotto, and Lihong Qi, “Prenatal and Perinatal Risk Factors for Autism in China,” 40 J. Autism Dev. Disord. 1311 (2010) (“In the adjusted analysis, nine risk factors showed significant association with autism: maternal second-hand smoke exposure, maternal chronic or acute medical conditions unrelated to pregnancy, maternal unhappy emotional state, gestational complications, edema, abnormal gestational age (<35 or >42 weeks), nuchal cord, gravidity >1, and advanced paternal age at delivery (>30 year-old)). Ultimately, a more demanding inquiry may be required to investigate the extent to which anti-pesticide advocacy groups have actually created an apparent increase in autism rates by informational, political, and environmental campaigns.


[1] See United States v. Harkonen, No. C 08–00164 MHP, 2010 WL 2985257, at *1 (N.D. Cal. July 27, 2010), aff’d, 510 F. App’x 633, 636 (9th Cir. Mar. 4, 2013)(affirming wire fraud conviction for author of press release who failed to disclose that endpoint was not prespecified, and failed to adjust for multiple comparisons), cert. denied, ___ U.S. ___ (Dec. 16, 2013).

 

Selective Leaking — Breaking Ingelfinger’s Rule

June 20th, 2014

The government wants us to believe that Snowden is a very evil man because he is a “leaker,” but the government leaks the information that it wants the world to have, and keeps confidential the rest.  The double standard is obvious.

Scientific publishing has its own double standard as well.  In 1969, Franz J. Ingelfinger, as editor of the prestigious New England Journal of Medicine (NEJM), set out two conditions for publication in the Journal:

(1) an embargo on articles and their content, when slated for publication in the NEJM, and

(2) a prohibition against duplicative publication or presentation of the substance of the article in any other journal or media source.

These conditions became known as the Ingelfinger rule, a rule that authors were willing to agree to in advance because they received a prestigious publication in the NEJM, and attention from the media because of that publication. The “rule” has been under relentless criticism, but has been defended in a modified form by subsequent NEJM editors. See Arnold S. Relman, “The Ingelfinger Rule,” 305 New Engl. J. Med. 824 (1981); Arnold S. Relman, “More on the Ingelfinger Rule,” 318 New Engl. J. Med. 1125 (1988); Marcia Angell & Jerome P. Kassirer, “The Ingelfinger Rule Revisited,” 325 New Engl. J. Med. 1371 (1991).  Most journals have followed the lead of the NEJM by implementing a similar set of conditions on publication.

The Ingelfinger Rule could be quite pointy when thrust into a researcher’s face.  I recall well how my cousin, conducting research on viral diseases, ran into the Rule, when his laboratory uncovered important information about the transmission of HIV. He and his colleagues wrote up their work, which was accepted by the NEJM, but the Centers for Disease Control felt that the information needed to be made public immediately.  The NEJM threatened to withdraw publication, and the intervention of a medical school dean was ultimately required to broker a compromise that allowed the authors and the NEJM to keep the publication.  See Joseph E. Fitzgibbon, Sunanda Gaur, Lawrence D. Frenkel, Fabienne Laraque, Brian R. Edlin, and Donald T. Dubin, “Transmission from one child to another of Human Immunodeficiency Virus type 1 with a zidovudine-resistance mutation,” 329 New Engl. J. Med. 1835 (1993).

In a recent blog post, scientist Dr. David Schwartz, writes about a soon-to-be published observational study of autism and environmental exposures.  See David Schwartz, “New Study Expected to Impugn Pesticides, But Is It Legit?” (June 20, 2014).  Dr. Schwartz explains:

“A new epidemiological study linking autism with pesticide exposure is expected to surface soon, according to our sources in the scientific community. The study was commissioned by Childhood Autism Risks from Genes and Environment (CHARGE). We anticipate that the mainstream media and some in the scientific community will latch onto the study because the rising rate of autism is alarming, and the public is understandably searching for answers. But we would caution the scientific community, the media, and the public to approach the study with skepticism instead of automatically buying into the results.

Speaking about the results, ahead of publication in a recent video, (posted on June 10, 2014) the senior author, Dr. Hertz-Picciotto, states: “there were associations with several classes of pesticides.” She goes on to state: “this is actually the third study to show some link with the organophosphates and autism risk.” I was surprised to see a study author talking publicly about the results of an embargoed study, since other scientists and journalists are precluded from talking about the study until after its publication.”

Dr. Irva Hertz-Picciotto is the principal investigator of the study, which is funded by the federal government, THE CHARGE STUDY: CHILDHOOD AUTISM RISKS FROM GENETICS AND THE ENVIRONMENT (2R01ES015359-06).  University of California at Davis is the funded institution. The study is due to be published in the Environmental Health Perspectives, (EHP) but according to Dr. Schwartz, the paper is still under embargo, and it does not appear on the EHP website, even in the advance of publication page. Unlike Dr. Hertz-Picciotto, Dr. Schwartz observed the embargo and did not comment upon the substance of the paper.  What is regrettable is that the EHP tolerates this selective leaking of the paper’s content, by its author, in apparent violation of its embargo policy. Perhaps everyone should join in and disregard the policy to keep the discussion balanced and to permit all sides to be heard.

Autism is a serious concern, which has been the subject of a great deal of biased and confounded research, and advocacy in courtrooms and elsewhere. The thimerosal-autism scare is still playing out in Vaccine court. Whatever the merits or demerits of Hertz-Picciotto’s study, soon to be published, what is disturbing is the cavalier breaking of the embargo by the principal investigator, on YouTube of all places.  Given the anxiety and concern over autism, scrupulous adherence to the the journal’s policies would have seemed prudent, to give serious scientific journalists a chance to comment critically on the paper at issue.

Sand in My Shoe – CERTainly

June 17th, 2014

Late last week, the California Court of Appeals reversed a dismissal on the pleadings in a claimed silicosis case. Uriarte v. Scott Sales Co., NO. B244257, California Ct. App. (2d Dist. Div. June 13, 2014)(certified for publication).

The defendants supplied silica sand to plaintiff’s employer for use as sandblasting media. Plaintiff, Mr. Francisco Uriarte, alleging that he developed lung fibrosis from sandblasting, sued the defendants.  Rather than defending on the ground of adequate warning, common knowledge, employer knowledge, plaintiff’s knowledge, and the like, defendants moved to dismiss on the pleadings on the grounds of the component parts doctrine, by which a “manufacturer of a component part is not liable for injuries caused by the finished product into which the component has been incorporated unless the component itself was defective and caused harm.” Id.  The California Court of Appeals reversed, and held that the component parts doctrine does not apply.  Fairly predictable, and probably correct to leave the defendants to their important, substantive defenses.

The appeal, however, is noteworthy for another reason, which received no comment from the California appellate court. The Council for Education and Research on Toxics (“CERT”), along with a list of physicians and scientists, filed an amicus brief in support of reversal. The individual amici[1] and CERT sought an opportunity to participate as

“a public benefit organization whose charitable purposes are education and research regarding toxic substances… . The other amici are all physicians. epidemiologists, scientists, and scholars of science and the history of science and public health.”

Amicus Brief for CERT at 1 (Oct. 10, 2013).  This participation is their right, but the amici avow that:

“None of the amici has any financial or other similar interest in the outcome of this lawsuit.”

Id. at 1.  This claim is not so clear. The Appendix to the Amicus Brief provides background on the individual amici. Although it may well be correct that none has a financial interest in the Uriarte case itself, most of the amici have been active as testifying expert witnesses, exclusively or nearly so for claimants, in cases just like the appeal.  The individual amici have been financially compensated for their witnessing. The brief fails to mention the amici’s advocacy roles, their testifying for remuneration, or their positional, political, and professional conflicts of interest.

The appearance of impropriety, however, is much greater than suffering the court system to have expert witnesses become advocates in disguise as neutral scientists, in a silicosis case. The CERT organization, one of the amici, is a non-profit California corporation, EIN: 42-1571530, founded in 2003, with a business address at:

401 E Ocean Blvd., Ste. 800, Long Beach, California 90802-4967,

and a telephone number: 

1-877-TOX-TORT

The person answering a telephone call to this number identified “The Metzger Law Group,” which makes CERT seem like the alter ego of The Metzger Law Group.

CERT’s mission statement? Furthering scientific understanding of toxins. But lawyer Ralph Metzger is noted as the contact person for CERT!  That is the same Ralphael Metzger, with the same Metzger Law Group, at the same Long Beach, California, address, who is the attorney for Mr. Uriarte in this case. Metzger apparently controls CERT, and he appears to be involved in CERT as a corporate officer.

Representing a party, and being a corporate officer in an amicus that attempts to influence an appellate court as a neutral entity, would seem to offend fundamental fairness in the appellate process.  In Uriarte, there was probably no harm, but surely the participation of amici in appellate proceedings needs to be policed more vigilantly.


[1] Richard W. Clapp*, Ronald Crystal, David A. Eastman*, Arthur L. Frank*, Robert J. Harrison*, Ronald Melnick*, Lee Newman, Stephen M. Rappaport*, Joseph Ross, and Janet Weiss*.  An asterisk indicates that the amicus was also an amicus in a brief filed by CERT, in the Milward case. SeeCERT” (July 9, 2013).

Substituting Risk for Specific Causation

June 15th, 2014

Specious, Speculative, Spurious, and Sophistical

Some legal writers assert that all evidence is ultimately “probable,” but that assertion appears to be true only to the extent that the evidentiary support for any claim can be mapped on scale from 0 to 1, much as probability is.  Probability thus finds its way into discussions of burdens of persuasion as requiring the claim to be shown more probably than not, and expert witness certitude as requiring “reasonable degree of scientific probability.”

There is a contrary emphasis in the law on “actual truth,” which is different from “mere probability.”  The rejection of probabilism can be seen in some civil cases, in which courts have emphasized the need for individualistic data and conclusions, beyond generalizations that might be made about groups that clearly encompass the individual at issue. For example, the Supreme Court has held that charging more for funding a woman’s pension than a man’s is discriminatory because not all women will outlive all men, or the men’s average life expectancy. City of Los Angeles Dep’t of Water and Power v. Manhart, 435 U.S. 702, 708 (1978) (“Even a true generalization about a class is an  insufficient reason for disqualifying an individual to whom the generalization does not apply.”). See also El v. Southeastern Pennsylvania Transportation Authority, 479 F.3d 232, 237 n.6 (3d Cir. 2007) (“The burden of persuasion … is the obligation to convince the factfinder at trial that a litigant’s necessary propositions of fact are indeed true.”).

Specific causation is the soft underbelly of the toxic tort world, in large measure because courts know that risk is not specific causation. In the context of risk of disease, which is usually based upon a probabilistic group assessment, courts occasionally distinguish between risk and specific causation. SeeProbabilism Case Law” (Jan. 28, 2013) (collecting cases for and against probabilism).

In In re Fibreboard Corp., 893 F. 2d 706, 711-12 (5th Cir. 1990), the court rejected a class action approach to litigating asbestos personal injury claims because risk could not substitute for findings of individual causation:

“That procedure cannot focus upon such issues as individual causation, but ultimately must accept general causation as sufficient, contrary to Texas law. It is evident that these statistical estimates deal only with general causation, for ‘population-based probability estimates do not speak to a probability of causation in any one case; the estimate of relative risk is a property of the studied population, not of an individual’s case.’ This type of procedure does not allow proof that a particular defendant’s asbestos ‘really’ caused a particular plaintiff’s disease; the only ‘fact’ that can be proved is that in most cases the defendant’s asbestos would have been the cause.”

Id. at 711-12 (citing Steven Gold, “Causation in Toxic Torts: Burdens of Proof, Standards of Persuasion, and Statistical Evidence,” 96 Yale L.J. 376, 384, 390 (1986). See also Guinn v. AstraZeneca Pharms., 602 F.3d 1245, 1255 (11th Cir. 2010) (“An expert, however, cannot merely conclude that all risk factors for a disease are substantial contributing factors in its development. ‘The fact that exposure to [a substance] may be a risk factor for [a disease] does not make it an actual cause simply because [the disease] developed.’”) (internal citation omitted).

Specific causation is the soft underbelly of the toxic tort world, in large measure because courts know that risk is not specific causation. The analytical care of the Guinn case and others is often abandoned when it will stand in the way of compensation. The conflation of risk and (specific) causation is prevalent precisely because in many cases there is no scientific or medical way to discern what antecedent risks actually played a role in causing an individual’s disease.  Opinions about specific causation are thus frequently devoid of factual or logical support, and are propped up solely by hand waving about differential etiology and inference to the best explanation.

In the scientific world, most authors recognize that risk, even if real and above baseline, regardless of magnitude, does not support causal attribution in a specific case.[1]  Sir Richard Doll, who did so much to advance the world’s understanding of asbestosis as a cause of lung cancer, issued a caveat about the limits of specific causation inference. Richard Doll, “Proof of Causality: Deduction from Epidemiological Observation,” 45 Perspectives in Biology & Medicine 499, 500 (2002) (“That asbestos is a cause of lung cancer in this practical sense is incontrovertible, but we can never say that asbestos was responsible for the production of the disease in a particular patient, as there are many other etiologically significant agents to which the individual may have been exposed, and we can speak only of the extent to which the risk of the disease was increased by the extent of his or her exposure.”)

Similarly, Kenneth Rothman, a leading voice among epidemiologists, cautioned against conflating epidemiologic inferences about groups with inferences about causes in individuals. Kenneth Rothman, Epidemiology: An Introduction 44 (Oxford 2002) (“An elementary but essential principal that epidemiologists must keep in mind is that a person may be exposed to an agent and then develop disease without there being any causal connection between exposure and disease.”  … “In a courtroom, experts are asked to opine whether the disease of a given patient has been caused by a specific exposure.  This approach of assigning causation in a single person is radically different from the epidemiologic approach, which does not attempt to attribute causation in any individual instance.  Rather, the epidemiologic approach is to evaluate the proposition that the exposure is a cause of the disease in a theoretical sense, rather than in a specific person.”) (emphasis added).

The late David Freedman, who was the co-author of the chapters on statistics in all three editions of the Reference Manual on Scientific Evidence, was also a naysayer when it came to transmuting risk into cause:

“The scientific connection between specific causation and a relative risk of two is doubtful. *** Epidemiologic data cannot determine the probability of causation in any meaningful way because of individual differences.”

David Freedman & Philip Stark, “The Swine Flu Vaccine and Guillaine-Barré Syndrome:  A Case Study in Relative Risk and Specific Causation,” 64 Law & Contemporary Problems 49, 61 (2001) (arguing that proof of causation in a specific case, even starting with a relative risk of four, was “unconvincing”; citing Manko v. United States, 636 F. Supp. 1419, 1437 (W.D. Mo. 1986) (noting relative risk of 3.89–3.92 for GBS from swine-flu vaccine), aff’d in part, 830 F.2d 831 (8th Cir. 1987)).

Graham Colditz, who testified for plaintiffs in the hormone therapy litigation, similarly has taught that an increased risk of disease cannot be translated into the “but-for” standard of causation.  Graham A. Colditz, “From epidemiology to cancer prevention: implications for the 21st Century,” 18 Cancer Causes Control 117, 118 (2007) (“Knowledge that a factor is associated with increased risk of disease does not translate into the premise that a case of disease will be prevented if a specific individual eliminates exposure to that risk factor. Disease pathogenesis at the individual level is extremely complex.”)

Another epidemiologist, who wrote the chapter in the Federal Judicial Center’s Reference Manual on Scientific Evidence, on epidemiology, put the matter thus:

“However, the use of data from epidemiologic studies is not without its problems. Epidemiology answers questions about groups, whereas the court often requires information about individuals.

Leon Gordis, Epidemiology 362 (5th ed. 2014) (emphasis in original).

=========================================================

In New Jersey, an expert witness’s opinion that lacks a factual foundation is termed a “net opinion.” Polzo v. County of Essex, 196 N.J. 569, 583 (2008) (explaining New Jersey law’s prohibition against “net opinions” and “speculative testimony”). Under federal law, Rule 702, such an opinion is simply called inadmissible.

Here is an interesting example of a “net opinion” from an expert witness, in the field of epidemiology, who has testified in many judicial proceedings:

 

                                                                                          November 12, 2008

George T. Brugess, Esq.
Hoey & Farina, Attorneys at Law
542 South Dearborn Street, Suite 200
Chicago, IL 60605

Ref: Oscar Brooks v. Ingram Barge and Jantran Inc.

* * * *

Because [the claimant] was employed 28 years, he falls into the greater than 20 years railroad employment category (see Table 3 of Garshick’s 2004 paper) which shows a significant risk for lung cancer that ranges from 1.24 to 1.50. This means that his diesel exposure was a significant factor in his contracting lung cancer. His extensive smoking was also a factor in his lung cancer, and diesel exposure combined with smoking is an explanation for the relatively early age, 61 years old, of his diagnosis.

Now assuming that diesel exposure truly causes lung cancer, what was the basis for this witness (David F. Goldsmith, PhD) to opine that diesel exposure was a “significant factor” in the claimant’s developing lung cancer?  None really.  There was no basis in the report, or in the scientific data, to transmute an exposure that yielded a risk ratio of 1.24 to 1.50 for lung cancer, in a similarly exposed population to diesel emissions, into a “significant factor.” The claimant’s cancer may have arisen from background, baseline risk.  The cancer may have arisen from the risk due to smoking, which would have been on the order of a 2,000% increase, or so.  The cancer may have arisen from the claimed carcinogenicity of diesel emissions, on the order of 25 to 50%, which was rather insubstantial compared with his smoking risk.  Potentially, the cancer arose from a combination of the risk from both diesel emissions and tobacco smoking. In the population of men who looked like Mr. Oscar Brooks, by far, the biggest reduction in incidence would be achieved by removing tobacco smoking.

There were no biomarkers that identified the claimant’s lung cancer as having been caused by diesel emissions.  The expert witness’s opinion was nothing more than an ipse dixit that equated a risk, and a rather small risk, with specific causation.  Notice how a 24% increased risk from diesel emissions was a “significant factor,” but the claimant’s smoking history was merely “a factor.”

Goldsmith’s report on specific causation was a net opinion that exemplifies what is wrong with a legal system that encourages and condones baseless expert witness testimony. In Agent Orange, Judge Weinstein pointed out that the traditional judicial antipathy to probabilism would mean no recovery in many chemical and medicinal exposure cases.  If the courts lowered their scruples to permit recovery on a naked statistical inference of greater than 50%, from relative risks greater than two, some cases might remain viable (but alas not the Agent Orange case itself). Judge Weinstein was, no doubt, put off by the ability of defendants, such as tobacco companies, to avoid liability because plaintiffs would never have more than evidence of risk.  In the face of relative risks often in excess of 30, with attributable risks in excess of 95%, this outcome was disturbing.

Judge Weinstein’s compromise was a pragmatic solution to the problem of adjudicating specific causation on the basis of risk evidence. Although as noted above, many scientists rejected any use of risk to support specific causation inferences, some scientists agreed with this practical solution.  Ironically, David Goldsmith, the author of the report in the Oscar Brooks case, supra, was one such writer who had embraced the relative risk cut off:

“A relative risk greater than 2.0 produces an attributable risk (sometimes called attributable risk percent10) or an attributable fraction that exceeds 50%.  An attributable risk greater than 50% also means that ‘it is more likely than not’, or, in other words, there is a greater than 50% probability that the exposure to the risk factor is associated with disease.”

David F. Goldsmith & Susan G. Rose, “Establishing Causation with Epidemiology,” in Tee L. Guidotti & Susan G. Rose, eds., Science on the Witness Stand:  Evaluating Scientific Evidence in Law, Adjudication, and Policy 57, 60 (OEM Press 2001).

In the Brooks case, Goldsmith did not have an increased risk even close to 2.0. The litigation industry ultimately would not accept anything other than full compensation for attributable risks greater than 0%.


[1] See, e.g., Sander Greenland, “Relation of the Probability of Causation to Relative Risk and Doubling Dose:  A Methodologic Error that Has Become a Social Problem,” 89 Am. J. Pub. Health 1166, 1168 (1999)(“[a]ll epidemiologic measures (such as rate ratios and rate fractions) reflect only the net impact of exposure on a population”); Joseph V. Rodricks & Susan H. Rieth, “Toxicological Risk Assessment in the Courtroom:  Are Available Methodologies Suitable for Evaluating Toxic Tort and Product Liability Claims?” 27 Regulatory Toxicol. & Pharmacol. 21, 24-25 (1998)(noting that a population risk applies to individuals only if all persons within the population are the same with respect to the influence of the risk on outcome); G. Friedman, Primer of Epidemiology 2 (2d ed. 1980)(epidemiologic studies address causes of disease in populations, not causation in individuals)

 

Hysterical Histortions

June 12th, 2014

Ramses Delafontaine is a young, aspiring historian. In his graduate thesis, Historicizing the Forensification of History: A Study of Historians as Expert Witnesses in Tobacco Litigation in the United States of America (Univ. Ghent 2013), discusses my commentary on Marxist historians, David Rosner and Gerald Markowitz, and suggests that I claim that lawyers without historical training or experience can do the job of historians.  Id. at 98-100.

Given their training and skills in documenting and recounting narratives, lawyers do, indeed, often do the job of historians, and they often do it very well. Of course, lawyers are often guided, inspired, and assisted by professional historians. Sometimes that guidance is necessary. Lawyers’ narratives, unlike historians’, are also subject to judicial control in the form of evidentiary rules about speculation, relevance, reliability, authentication, and trustworthiness.

Regurgitating Historical Evidence

American courts have thus appropriately limited the use of expert witnesses to present historical narratives in judicial proceedings.  See In re Fosamax Prods. Liab. Litig., 645 F. Supp. 2d 164, 192 (S.D.N.Y. 2009) (‘‘[A]n expert cannot be presented to the jury solely for the purpose of constructing a factual narrative based upon record evidence.’’) (internal citations omitted).[1] In Fosamax, Judge Keenan excluded Dr. Susan Parisian’s proffered narrative account of the development and approval of a bisphosphonate medication for osteoporosis in a case involving a claim of osteonecrosis of the jaw (“phossy jaw”).  Judge Keenan detailed the problems that arise from using expert witnesses to present partisan historical narratives:

“In detailing the factual basis for her opinions, Dr. Parisian’s report presents a narrative of select regulatory events through the summary or selective quotation from internal Merck documents, regulatory filings, and the deposition testimony of Merck employees.

The Court agrees with Merck that, to the extent such evidence is admissible, it should be presented to the jury directly. Dr. Parisian’s commentary on any documents and exhibits in evidence will be limited to explaining the regulatory context in which they were  created, defining any complex or specialized terminology, or drawing inferences that would not be apparent without the benefit of experience or specialized knowledge. She will not be permitted to merely read, selectively quote from, or ‘regurgitate’ the evidence.”

Id.[2]

Ramses Delafontaine is wrong, however, to opine that my rants against Rosner and Markowitz suggest that I have ruled out any role for historians in litigation.   Lisa K. Walker is an historian, who trained at the University of California, Berkeley.  In the welding fume litigation, plaintiffs’ counsel weaved a complex narrative of conspiracy allegations, based in large measure upon the absence of evidence. At the request of Metropolitan Life Insurance Company, Professor Walker researched the dates of publication for various editions of a booklet, Health Protection in Welding, which formed the basis for the plaintiffs’ speculations. Walker found and analyzed eight separate editions, and dated each by internal and external references.  Based upon her research, Walker submitted a declaration, which ultimately was immensely helpful to the resolution of the issues. See In re Welding Rod Prods. Liab. Litig., Case No. 1:03-CV-17000 (MDL Docket No. 1535) (N.D.Ohio Nov. 24, 2004; In re Welding Fume Prods. Liab. Litig., 2007 WL 1087605 (N.D. Ohio April 9, 2007)  (O’Malley, J.) (granting summary judgment in favor of Metropolitan Life Insurance Company).

Although Metropolitan Life should not have had to disprove the allegations, Walker’s research showed the plaintiffs’ historical speculation to be clearly wrong. Sometimes historians can and do contribute valuably to the resolution of legal issues, but the issues are usually more modest than the ones that social and labor historians want to see resolved in favor of their pet theories.

Delafontaine also has a website on the role of historians as expert witnesses in United States tobacco cases.  One of his theses is that “historians who have been involved as expert witnesses for the tobacco industry have been in it for the money and have sold their professional integrity as a historian and an academic.” Delafontaine’s approach is a bit one-sided in that he sees only defendants’ expert witnesses as being “in it for the money,” despite the substantial billings of plaintiffs’ expert witnesses, and their ideological biases. Somehow Delafontaine has missed how even the feel-good advocacy of anti-tobacco activists occasionally outruns its evidentiary headlights.  See, e.g., Michael Siegel, “Is the tobacco control movement misrepresenting the acute cardiovascular health effects of secondhand smoke exposure? An analysis of the scientific evidence and commentary on the implications for tobacco control and public health practice,” 4 Epidem. Persp. & Innov. 12 (2007).


[1] See also In re Prempro Prods. Liab. Litig., 554 F.Supp. 2d 871, 880, 886 (E.D.Ark.2008) (overturning a punitive damages award based on Dr. Parisian’s testimony in part because she ‘‘did not explain the documents, provide summaries, or tie them in to her proposed regulatory testimony’’ and ‘‘did not provide analysis, opinion, or expertise’’); Highland Capital Management, L.P. v. Schneider, 379 F.Supp. 2d 461, 469 (S.D.N.Y.2005)(‘‘[A]n expert cannot be presented to the jury solely for the purpose of constructing a factual narrative based upon record evidence.’’); In re Rezulin Products Liab. Litig., 309 F.Supp. 2d 531, 546 (S.D.N.Y.2004) (rejecting portion of expert report presenting history of Rezulin for no purpose but to ‘‘provid[e] an historical commentary of what happened,’’ along with subjective assessments of intent, motives, and states of mind); In re Diet Drugs Prods. Liab. Litig., MDL No. 1203, 2000 WL 876900, at *9 (E.D.Pa. June 20, 2000) (same); Taylor v. Evans, 1997 WL 154010, at *2 (S.D.N.Y. Apr.1, 1997) (rejecting portions of expert report on the ground that the testimony consisted of ‘‘a narrative of the case which a lay juror is equally capable of constructing’’).

[2] Dr. Parisian appears to be a serial narrative abuser, who has been repeatedly but not consistently excluded. See Scheinberg v. Merck & Co. 924 F.Supp. 2d 477, 497 (S.D.N.Y. 2013); Pritchett v. I-Flow Corp., 2012 WL 1059948, at *7 (D. Colo. Mar. 28, 2012); Miller v. Stryker Instruments, 2012 WL 1718825, at *10-12 (D. Ariz. Mar. 29, 2012) (excluding narrative testimony); Kaufman v. Pfizer Pharms., Inc., 2011 WL 7659333, at *6-10 (S.D. Fla. Aug. 4, 2011), reh’g denied, 2011 WL 10501233 (S.D. Fla. Aug. 10, 2011)(narrative testimony); Hines v. Wyeth, 2011 WL 2680842, at *7 (S.D.W. Va. July 8, 2011), reh’g granted in part, 2011 WL 2730908, at *2 (S.D.W. Va. July 13, 2011); In re Heparin Prods. Liab. Litig., 2011 WL 1059660, at *8 (N.D. Ohio March 21, 2011); Lopez v. I-Flow Inc., 2011 WL 1897548, at *9-10 (D. Ariz. Jan. 26, 2011) (narrative testimony); In re Trasylol Prods. Liab. Litig., 709 F.Supp.2d 1323, 1351 (S.D. Fla. 2010)(tendentious narrative testimony) (“Plainly stated, Dr. Parisian is an advocate, presented with the trappings of an expert but with no expectation or intention of abiding by the opinion constraints of Rule 702.”), reh’g denied, 2010 WL 2541892 (S.D. Fla. June 22, 2010); In re Gadolinium-Based Contrast Agents Prods. Liab. Litig., 2010 WL 1796334, at *13 (N.D. Ohio May 4, 2010); Bessemer v. Novartis Pharms. Corp., 2010 WL 2300222 (N.J. Super. Law Div. April 30, 2010); In re Prempro Prods. Liab. Litig., 554 F. Supp. 2d 871, 879-87 (E.D. Ark. 2008)(reversing judgment on grounds of erroneous admission of narrative testimony), aff’d  in relevant part, 586 F.3d 547, 571 (8th Cir. 2009). Occasionally, Dr. Parisian slips through the gate.  See, e.g., Block v. Woo Young Medical Co., 937 F.Supp.2d 1028, 1044-47 (2013)

Goodman v Viljoen – Meeting the Bayesian Challenge Head On

June 11th, 2014

Putting Science On Its Posterior

Plaintiffs’ and Defendants’ counsel both want the scientific and legal standard to be framed as a very high posterior probability of the truth of a claim. Plaintiffs want the scientific posterior probability to be high because they want to push the legal system in the direction of allowing weak or specious claims that are not supported by sufficient scientific evidence to support a causal conclusion.  By asserting that the scientific posterior probability for a causal claim is high, and that the legal and scientific standards are different, they seek to empower courts and juries to support judgments of causality that are deemed inconclusive, speculative, or worse, by scientists themselves.

Defendants want the scientific posterior probability to be high, and claim that the legal standard should be at least as high as the scientific standard.

Both Plaintiffs and Defendants thus find common cause in committing the transposition fallacy by transmuting the coefficient of confidence, typically 95%, into a minimally necessary posterior probability for scientific causal judgments.  “One wanders to the left, another to the right ; both are equally in error, but are seduced by different delusions.”[1]

In the Goodman v. Viljoen[2] case, both sides, plaintiffs and defendants, embraced the claim that science requires a high posterior probability, and that the p-value provided evidence of the posterior probability of the causal claim at issue.  The error came mostly from the parties’ clinical expert witnesses and from the lawyers themselves; the parties’ statistical expert witnesses appeared to try to avoid the transposition fallacy. Clearly, no text would support the conflation of confidence with certainty. No scientific text, treatise, or authority was cited for the notion that scientific “proof” required 95% certainty. This notion was simply an opinion of testifying witnesses.

The principal evidence that antenatal corticosteroid (ACS) therapy can prevent cerebral palsy (CP) came from a Cochrane review and meta-analysis[3] of clinical trials.  The review examined a wide range of outcomes, only one of which was CP.  The trials were apparently not designed to assess CP risk, and they varied significantly in case definition, diagnostic criteria, and length of follow up for case ascertainment. Of the five included studies, four ascertained CP at follow up from two to six years, and the length of follow up was unknown in the fifth study.

Data were sparse in the Cochrane review, as expected for a relatively rare outcome.  The five studies encompassed 904 children, with 490 in the treatment group, and 414 in the control group. There was a total of 48 CP cases, with 20 in the treatment, and 28 in the control, groups. Blinding was apparently not maintained over the extended reporting period.

Professor Andrew Willan, plaintiffs’ testifying expert witness on statistics, sponsored a Bayesian statistical analysis, with which he concluded that there was between a 91 and 97% probability that there was an increased risk of CP from not providing ACS in pre-term labor (or, a decreased risk of CP from administering ACS).[4] Willan’s posterior probabilities was for any increased risk, based upon the Cochrane data.  Willan’s calculations were not provided in his testimony, and no information about his prior probability, was given. The data came from clinical trials, but the nature of the observations and the analyses made these trials little more than observational studies conducted within the context of clinical trials designed to look at other outcomes. The Bayesian analysis did not account for the uncertainty in the case definitions, variations in internal validity and follow up, and biases in the clinical trials. Willan’s posterior probabilities thus described a maximal probability for general causation, which surely needed to be discounted for validity and bias issues.

There was a further issue of external validity. The Goodman twins developed CP from having sustained periventricular leukomalacia (PVL), which is one among several mechanistic pathways by which CP can develop in pre-term infants.  The Cochrane data did not address PVL, and the included trials were silent as to whether any of the CP cases involved PVL mechanisms.  There was no basis for assuming that ACS reduced risk of CP from all mechanisms equally, or even at all.[5] The Willan posterior probabilities did not address the external validity issues as they pertained to the Goodman case itself.

Although Dr. Viljoen abandoned the challenge to the Bayesian analysis at trial, his statistical expert witness, Dr. Robert Platt went further to opine that he agreed with Willan’s calculations.  To agree with his calculations, and the posterior probabilities that came out of those calculations, Platt had to have agreed with the analyses themselves. This agreement seems ill considered given that elsewhere in his testimony, Platt appears to advance important criticisms of the Cochrane data in the form of validity and bias issues.

Certainly, Platt’s concession about the correctness of Willan’s calculations greatly undermined Dr. Viljoen’s position with the trial and appellate court. Dr. Viljoen maintained those criticisms throughout the trial, and on appeal.  See, e.g., Defendant (Appellant) Factum, 2012 CCLTFactum 20936, at ¶14(a)

(“(a) antenatal corticosteroids have never been shown to reduce the incidence or effect of PVL”); id. at ¶14(d)(“at best, even taking the Bayesian approach at face value, the use of antenatal corticosteroids showed only a 40% reduction in the incidence of cerebral palsy, but not PVL”).

How might have things gone better for Dr. Vijoen? For one thing, Platt’s concession about the correctness of Willan’s calculations had to be explained and qualified as conceding only the posterior probability on the doubtful and unproven assumptions made by Willan. Willan’s posterior, as big as it was, represented only an idealized maximal posterior probability, which in reality had to be deeply discounted by important uncertainties, biases, and validity concerns.  The inconclusiveness of the data were “provable” on either a frequentist or a Bayesian analysis.


[1] Horace, in Wood, Dictionary of Quotations 182 (1893).

[2] Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[3] Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews, at 8, Issue 3. Art. No. CD004454 (2006)

[4] Notes of Testimony of Andrew Willan at 34 (April 9, 2010) (concluding that ACS reduces risk of CP, with a probability of 91 to 97 percent, depending upon whether random effects or fixed effect models are used).

[5] See, e.g., Olivier Baud, Laurence Laurence Foix l’Hélias, et al., “Antenatal Glucocorticoid- Treatment and Cystic Periventricular Leukomalacia in Very Premature Infants,” 341 New Engl. J. Med. 1190, 1194 (1999) (“Our results suggest that exposure to betamethasone but not dexamethasone is associated with a decreased risk of cystic periventricular leukomalacia.”).

 

Goodman v Viljoen – Statistical Fallacies from Both Sides

June 8th, 2014

There was a deep irony to the Goodman[1] case.  If a drug company, in 1995, marketed antenatal corticosteroid (ACS) for the prevention of cerebral palsy (CP) in the United States, the government might well have prosecuted the company for misbranding.  The company might also be subject to a False Claims Act case as well. No clinical trial had found ACS efficacious for the prevention of CP at the significance level typically required by the FDA; no meta-analysis had found ACS statistically significantly better than placebo for this purpose.  In the Goodman case, however, failure to order a full course of ACS was malpractice with respect to the claimed causation of CP in the Goodman twins.

The Goodman case also occasioned a well-worn debate over the difference between scientific and legal evidence, inference, and standards of “proof.” The plaintiffs’ case rested upon a Cochrane review of ACS with respect to various outcomes. For CP, the Cochrane meta-analyzed only clinical trial data, and reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”[2]

The defendant, Dr. Viljoen, appeared to argue that the Cochrane meta-analysis must be disregarded because it did not provide a showing of efficacy for ACS in preventing CP, at a significance probability less than 5 percent.  Here is the trial court’s characterization of Dr. Viljoen’s argument:

“[192] The argument that the Cochrane data concerning the effects of ACS on CP must be ignored because it fails to reach statistical significance rests on the flawed premise that legal causation requires the same standard of proof as medical/scientific causation. This is of course not the case; the two standards are in fact quite different. The law is clear that scientific certainty is not required to prove causation to the legal standard of proof on a balance of probabilities (See: Snell v. Farrell, [1990] 2 S.C.R. 311, at para. 34). Accordingly, the defendant’s argument in this regard must fail and for the purposes of this court, I accept the finding of the Cochrane analysis that ACS reduces the instance [sic] of CP by 40%.”

“Disregard” seems extreme for a meta-analysis that showed a 40% reduction in risk of a serious central nervous system disorder, with p = 0.065.  Perhaps Dr. Viljoen might have tempered his challenge some by arguing that the Cochrane analysis was insufficient.  One problem with Dr. Viljoen’s strident argument about statistical significance was that it overshadowed the more difficult, qualitative arguments about threats to validity in the Cochrane finding from loss to follow up in the aggregated trial data. These threats were probably stronger arguments against accepting the Cochrane “trend” as a causal conclusion. Indeed, the validity and the individual studies and the meta-analyses, along with questions about the accuracy of data, were not reflected in Bayesian analysis.

Another problem is that Dr. Viljoen’s strident assertion that p < 0.05 was absolutely necessary fed plaintiffs’ argument that the defendant was attempting to change the burden of proof for plaintiffs from greater than 50% to 95% or greater.  Given the defendant’s position, great care was required to prevent the trial court from committing the transposition fallacy.

Justice Walters rejected the suggestion that a meta-analysis with a p-value of 6.5% should be disregarded, but the court’s discussion skirts the question whether and how the Cochrane data can be sufficient to support a conclusion of ACS efficacy. Aside from citing a legal case, however, Justice Walters provided no basis for suggesting that the scientific standard of proof was different from the legal standard. From the trial court’s opinion, the parties or their expert witnesses appeared to conflate “confidence,” a technical term when used to describe intervals or random error around sample statistics, with “level of certainty” in the obtained result.

Justice Walters is certainly not the first judge to fall prey to the fallacious argument that the scientific burden of proof is 95%.[3]  The 95% is, of course, the coefficient of confidence for the confidence interval that is based upon a p-value of 5%. No other explanation for why 95% is a “scientific” standard of proof was offered in Goodman; nor is it likely that anyone could point to an authoritative source for the claim that scientists actually adjudge facts and theories by this 95 percent probability level.

Justice Walters’ confusion was led by the transposition fallacy, which confuses posterior and significance probabilities.  Here is a sampling from Her Honor’s opinion, first from Dr. Jon Barrett, one of the plaintiffs’ expert witnesses, an obstetrician and fetal maternal medicine specialist at Sunnybrook Hospital, in Toronto, Ontario:

“[85] Dr. Barrett’s opinion was not undermined during his lengthy cross-examination. He acknowledged that the scientific standard demands 95% certainty. He is, however, prepared to accept a lower degree of certainty. To him, 85 % is not merely a chance outcome.

                                                                                        * * *

[87] He acknowledged that scientific evidence in support of the use of corticosteroids has never shown statistical significance with respect to CP. However, he explained it is very close at 93.5%. He cautioned that if you use a black and white outlook and ignore the obvious trends, you will falsely come to the conclusion that there is no effect.”

Dr. Jon (Yoseph) Barrett is a well-respected physician, who specializes in high-risk pregnancies, but his characterization of a black-white outlook on significance testing as leading to a false conclusion of no effect was statistically doubtful.[4]  Dr. Barrett may have to make divinely inspired choices in surgery, but in a courtroom, expert witnesses are permitted to say that they just do not know. Failure to achieve statistical significance, with p < 0.05, does support a conclusion that there is no effect.

Professor Andrew Willan was plaintiffs’ testifying expert witness on statistics.  Here is how Justice Walters summarized Willan’s testimony:

“[125] Dr. Willan described different statistical approaches and in particular, the frequentist or classical approach and the Bayesian approach which differ in their respective definitions of probability. Simply, the classical approach allows you to test the hypothesis that there is no difference between the treatment and a placebo. Assuming that there is no difference, allows one to make statements about the probability that the results are not due to chance alone.

To reach statistical significance, a standard of 95% is required. A new treatment will not be adopted into practice unless there is less than a 5% chance that the results are due to chance alone (rather than due to true treatment effect).

[127] * * * The P value represents the frequentist term of probability. For the CP analysis [from the Cochrane meta-analysis], the P value is 0.065. From a statistical perspective, that means that there is a 6.5% chance that the differences that are being observed between the treatment arm versus the non-treatment arm are due to chance rather than the treatment, or conversely, a 93.5% chance that they are not.”

Justice Walters did not provide transcript references for these statements, but they are clear examples of the transposition fallacy. The court’s summary may have been unfair to Professor Willan, who seems to have taken care to avoid the transposition fallacy in his testimony:

“And I just want to draw your attention to the thing in parenthesis where it says, “P = 0.065.” So, basically that is the probability of observing data this extremely, this much in favor of ACS given, if, if in fact the no [sic, null] hypothesis was true. So, if, if the no hypothesis was true, that is there was no difference, then the probability of observing this data is only 6.5 percent.”

Notes of Testimony of Andrew Willan at 26 (April , 2010). In this quote, Professor Willan might have been more careful to point out that the significance probability of 6.5%  is a cumulative probability by describing the data observed “this extremely” and more. Nevertheless, Willan certainly made clear that the probability measure was based upon assuming the correctness of the null hypothesis. The trial court, alas, erred in stating the relevant statistical concepts.

And then there was the bizarre description by Justice Walters, of the Cochrane data, as embodying a near-uniform distribution represented by the Cochrane data:

“[190] * * * The Cochrane analysis found that ACS reduced the risk of CP (in its entirety) by 40%, 93.5% of the time.”

The trial court did not give the basis for this erroneous description of the Cochrane ACS/CP data.[5] To be sure, if the Cochrane result were true, then 40% reduction might be the expected value for all trials, but it would be a remarkable occurrence for 93.5% of the trials to obtain the same risk ratio as the one observed in the meta-analysis.

The defendant’s expert witness on statistical issues, Prof. Robert Platt, similarly testified that the significance probability reported by the Cochrane was dependent upon an assumption of the null hypothesis of no association:

“What statistical significance tells us, and I mentioned at the beginning that it refers to the probability of a chance finding could occur under the null-hypothesis of no effect. Essentially, it provides evidence in favour of there being an effect.  It doesn’t tell us anything about the magnitude of that effect.”

Notes of Testimony of Robert Platt at 11 (April 19, 2010)

Perhaps part of the confusion resulted from Prof. Willan’s sponsored Bayesian analysis, which led him to opine that the Cochrane data permitted him to state that there was a 91 to 97 percent probability of an effect, which might have appeared to the trial court to be saying the same thing as interpretation of the Cochrane’s p-value of 6.5%.  Indeed, Justice Walters may have had some assistance in this confusion from the defense statistical expert witness, Prof. Platt, who testified:

“From the inference perspective the p-value of 0.065 that we observe in the Cochrane review versus a 91 to 97 percent probability that there is an effect, those amount to the same thing.”

Notes of Testimony of Robert Platt at 50 (April 19, 2010).  Now the complement of the p-value, 93.5%, may have fallen within the range of posterior probabilities asserted by Professor Willan, but these probabilities are decidedly not the same thing.

Perhaps Prof. Platt was referring only to the numerical equivalence, but his language, “the same thing,” certainly could have bred misunderstanding.  The defense apparently attacked the reliability of the Bayesian analysis before trial, only to abandon the challenge by the time of trial.  At trial, defense expert witness Prof. Platt testified that he did not challenge Willan’s Bayesian analysis, or the computation of posterior probabilities.  Platt’s acquiescence in Willan’s Bayesian analysis is unfortunate because the parties never developed testimony exactly as to how Willan arrived at his posterior probabilities, and especially as to what prior probability he employed.

Professor Platt went on to qualify his understanding of Willan’s Bayesian analysis as providing a posterior probability that there is an effect, or in other words, that the “effect size” is greater than 1.0.  At trial, the parties spent a good deal of time showing that the Cochrane risk ratio of 0.6 represented the decreased risk for CP of administering a full course of ACS, and that this statistic could be presented as an increased CP risk ratio of 1.7, for not having administered a full course of ACS.  Platt and Willan appeared to agree that the posterior probability described the cumulative posterior probabilities for increased risks above 1.0.

“[T]he 91% is a probability that the effect is greater than 1.0, not that it is 1.7 relative risk.”

Notes of Testimony of Robert Platt at 51 (April 19, 2010); see also Notes of Testimony of Andrew Willan at 34 (April 9, 2010) (concluding that ACS reduces risk of CP, with a probability of 91 to 97 percent, depending upon whether random effects or fixed effect models are used).[6]

One point on which the parties’ expert witnesses did not agree was whether the failure of the Cochrane’s meta-analysis to achieve statistical significance was due solely to the sparse data aggregated from the randomized trials. Plaintiffs’ witnesses appeared to have testified that had the Cochrane been able to aggregate additional clinical trial data, the “effect size” would have remained constant, and the p-value would have shrunk, ultimately to below the level of 5 percent.  Prof. Platt, testifying for the defense, appropriately criticized this hand-waving excuse:

“Q. and the probability factor, the P value, was 0.065, which the previous witness had suggested is an increase in probability of our reliability on the underlying data.  Is it reasonable to assume that this data that a further increase in the sample size will achieve statistical significance?

A. No, that’s not a reasonable assumption….”

Notes of Testimony of Robert Platt at 29 (April 19, 2010).

Positions on Appeal

Dr. Viljoen continued to assert the need for significance on appeal. As appellant, he challenged the trial court’s finding that the Cochrane review concluded that there was a 40% risk reduction. See Goodman v. Viljoen, 2011 ONSC 821, at ¶192 (CanLII) (“I accept the finding of the Cochrane analysis that ACS reduces the instance of CP by 40%”). Dr. Viljoen correctly pointed out that the Cochrane review never reached such a conclusion. Appellant’s Factum, 2012 CCLTFactum 20936, ¶64.  It was the plaintiffs’ expert witnesses, not the Cochrane reviewers, who reached the conclusion of causality from the Cochrane data.

On appeal, Dr. Viljoen pressed the point that his expert witnesses described statistical significance in the Cochrane analysis would have been “a basic and universally accepted standard” for showing that ACS was efficacious in preventing CP or PVL. Id. at ¶40. The appellant’s brief then commits to the very error that Dr. Barrett complained would follow from a finding that did not have statistical significance; Dr. Viljoen maintained that the “trend” of reduced CP reduced CD rates from ACS administration “is the same as a chance occurrence.” Defendant (Appellant), 2012 CCLTFactum 20936, at ¶40; see also id. at ¶14(e) (arguing that the Cochrane result for ACS/CP “should be treated as pure chance given it was not a statistically significant difference”).

Relying upon the Daubert decision from the United States, as well as Canadian cases, Dr. Viljoen framed one of his appellate issues as whether the trial court had “erred in relying upon scientific evidence that had not satisfied the benchmark of statistical significance”:

“101. Where a scientific effect is not shown to a level of statistical significance, it is not proven. No study has demonstrated a reduction in cerebral palsy with antenatal corticosteroids at a level of statistical significance.

102. The Trial Judge erred in law in accepting that antenatal corticosteroids reduce the risk of cerebral palsy based on Dr. Willan’s unpublished Bayesian probability analysis of the 48 cases of cerebral palsy reviewed by Cochrane—an analysis prepared for the specific purpose of overcoming the statistical limitations faced by the Plaintiffs on causation.”

Defendant (Appellant), 2012 CCLTFactum 20936. The use of the verb “proven” is problematic because it suggests a mathematical demonstration, which is never available for empirical propositions about the world, and especially not for the biological world.  The use of a mathematical standard begs the question whether the Cochrane data were sufficient to establish a scientific conclusion of the efficacy of ACS in preventing CP.

In opposing Dr. Viljoen’s appeal, the plaintiffs capitalized upon his assertion that science requires a very high level of posterior probability for establishing a causal claim, by simply agreeing with it. See Plaintiffs’ (Respondents’) Factum,  2012 CCLTFactum 20937, at ¶31 (“The scientific method requires statistical significance at a 95% level.”).  By accepting the idealized notion that science somehow requires 95% certainty (as opposed to 95% confidence levels as a test for assessing random error), the plaintiffs made the defendant’s legal position untenable.

In order to keep the appellate court thinking that the defendant was imposing an extra-legal, higher burden of proof upon plaintiffs, the plaintiffs went so far as to misrepresent the testimony of their own expert witness, Professor Willan, as having committed the transposition fallacy:

“49. Dr. Willan provided the frequentist explanation of the Cochrane analysis on CP:

a. The risk ratio (RR) is .060 which means that there is a 40% risk reduction in cerebral palsy where there has been administration of antenatal corticosteroids;

b. The upper limit of the confidence interval (CI) barely crosses 1 so it just barely fails to meet the rigid test of statistical significance;

c. The p value represents the frequentist term of probability;

d. In this case the p value is .065;

e. From a statistical perspective that means that there is a 6.5% chance that the difference observed in CP rates is due to chance alone;

f. Conversely there is a 93.5% chance that the result (the 40% reduction in CP) is due to a true treatment effect of ACS.”

2012 CCLTFactum 20937, at ¶49 (citing Evidence of Dr. Willan, Respondents’ Compendium, Tab 4, pgs. 43-52).

Although Justice Doherty dissented from the affirmance of the trial court’s judgment, he succumbed to the parties’ misrepresentations about scientific certainty, and their prevalent commission of the transposition fallacy. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36 (“Scientists will draw a cause and effect relationship only when a result follows at least 95 per cent of the time. The results reported in the Cochrane analysis fell just below that standard.”), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

The statistical errors on both sides redounded to the benefit of the plaintiffs.


[1] Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[2] Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews, at 8, Issue 3. Art. No. CD004454 (2006).

[3] See, e.g., In re Ephedra Prods. Liab. Litig., 393 F.Supp. 2d 181, 191, 193 (S.D.N.Y. 2005) (fallaciously arguing that the use of a critical value of less than 5% of significance probability increased the “more likely than not” burden of proof upon a civil litigant.  Id. at 188, 193.  See also Michael O. Finkelstein, Basic Concepts of Probability and Statistics in the Law 65 (2009) (criticizing the Ephedra decision for confusing posterior probability with significance probability).

[4] I do not have the complete transcript of Dr. Barrett’s testimony, but the following excerpt from April 9, 2010, at page 100, suggests that he helped lead Justice Walters into error: “When you say statistical significance, if you say that something is statistically significance, it means you’re, for the scientific notation, 95 percent sure. That’s the standard we use, 95 percent sure that that result could not have happened by chance. There’s still a 5 percent chance it could. It doesn’t mean for sure, but 95 percent you’re sure that the result you’ve got didn’t happen by chance.”

[5] On appeal, the dissenting judge erroneously accepted Justice Walters’ description of the Cochrane review as having supposedly reported a 40% reduction in CP incidence, 93.5% of the time, from use of ACS. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36, leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[6] The Bayesian analysis did not cure the attributability problem with respect to specific causation.