Schachtman Law » Expert Witnesses

TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Daubert’s Error Rate

June 16th, 2015

In Daubert, the Supreme Court came to the realization that expert witness opinion testimony was allowed under the governing statute, Federal Rule of Evidence 702, only when that witness’s “scientific, technical, or other specialized knowledge” would help the fact finder. Knowledge clearly connotes epistemic warrant, and some of the Court’s “factors” speak directly to this warrant, such as whether the claim has been tested, and whether the opinion has an acceptable rate of error. The Court, however, continued to allow some proxies for that warrant, in the form of “general acceptance,” or “peer review.”

The “rate of error” factor has befuddled some courts in their attempt to apply the statutory requirements of Rule 702, especially when statistical evidence is involved. Some litigants have tried to suggest that a statistically significant result suffices alone to meet the demands of Rule 702, but this argument is clearly wrong. See, e.g., United States v. Vitek Supply Corp., 144 F.3d 476, 480, 485–86 (7th Cir. 1998) (stating that the purpose of the inquiry into rate of error is to determine whether tests are “accurate and reliable”) (emphasis added). See also “Judicial Control of the Rate of Error in Expert Witness Testimony” (May 28, 2015). The magnitude of tolerable actual or potential error rate remains, however, a judicial mystery[1].

Sir Austin Bradford Hill described ruling out bias, confounding, and chance (or random error) as essential prerequisites to considering his nine factors used to assess whether an association is causal:

“Disregarding then any such problem in semantics we have this situation. Our observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance. What aspects of that association should we especially consider before deciding that the most likely interpretation of it is causation.”

Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965). The better reasoned cases agree. See, e.g., Frischhertz v. SmithKline Beecham Corp., 2012 U.S. Dist. LEXIS 181507, *6 (E.D.La. 2012) (“The Bradford-Hill criteria can only be applied after a statistically significant association has been identified.”) (citing and quoting among other sources, Federal Judicial Center, Reference Manual on Scientific Evidence, 599 & n.141 (3d. ed. 2011)).

Citing the dictum in Matrixx Initiatives[2] as though it were a holding is not only ethically dubious, but also ignores the legal and judicial context of the Court’s statements[3]. There are, after all, some circumstances such as cases of death by blunt-force trauma, or bullet wounds, when epidemiological and statistical evidence is not needed. The Court did not purport to speak to all causation assessments; nor did it claim that it was addressing only instances in which there were “expected cases,” and “base-line risks,” in diseases that have an accepted occurrence and incidence among unexposed persons. It is, of course, in exactly those cases that statistical consideration of bias, confounding, and chance are essential before Bradford Hill’s factors can be parsed.

Lord Rutherford[4] is often quoted as having said that “[i]f your experiment needs statistics, you ought to have done a better experiment.” Today, physics and chemistry have dropped their haughty disdain for statistics in the face of their recognition that some processes can be understood only as stochastic and rate driven. In biology, we are a long way from being able to describe the most common disease outcomes as mechanistic genetic or epigenetic events. Statistical analyses, with considerations of random and systematic error, will be with us for a long time, whether the federal judiciary acknowledges this fact or not.

* * * * * * * * * * * * *

Cases Discussing Error Rates in Rule 702 Decisions

SCOTUS

Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593 (1993) (specifying the “the known or potential rate of error” as one of several factors in assessing the scientific reliability or validity of proffered expert witness’s opinion)

Kumho Tire Co. v. Carmichael, 526 U.S. 137, 151 (1999) (suggesting that reliability in the form of a known and an acceptable error rate is an important consideration for admissibility)

US Court of Appeals

FIRST CIRCUIT

United States v. Shea, 957 F. Supp. 331, 334–45 (D.N.H. 1997) (rejecting criminal defendant’s objection to government witness’s providing separate match and error probability rates)

SECOND CIRCUIT

Rabozzi v. Bombardier, Inc., No. 5:03-CV-1397 (NAM/DEP), 2007 U.S. Dist. LEXIS 21724, at *7, *8, *20 (N.D.N.Y. Mar. 27, 2007) (excluding testimony from civil engineer about boat design, in part because witness failed to provide rate of error)

Sorto-Romero v. Delta Int’l Mach. Corp., No. 05-CV-5172 (SJF) (AKT), 2007 U.S. Dist. LEXIS 71588, at *22–23 (E.D.N.Y. Sept. 24, 2007) (excluding engineering opinion that defective wood-carving tool caused injury because of lack of error rate)

In re Ephedra Products Liability Litigation, 393 F. Supp. 2d 181, 184 (S.D.N.Y. 2005) (confusing assessment of random error with probability that statistical estimate of true risk ratio was correct)

Roane v. Greenwich Swim Comm., 330 F. Supp. 2d 306, 309, 319 (S.D.N.Y. 2004) (excluding mechanical engineer, in part because witness failed to provide rate of error)

Nook v. Long Island R.R., 190 F. Supp. 2d 639, 641–42 (S.D.N.Y. 2002) (excluding industrial hygienist’s opinion in part because witness was unable to provide a known rate of error).

United States v. Towns, 19 F. Supp. 2d 67, 70–72 (W.D.N.Y. 1998) (permitting clinical psychologist to opine about defendant’s mens rea and claimed mental illness causing his attempted bank robbery, in part because the proffer of opinion maintained that the psychologist would provide an error rate)

Meyers v. Arcudi, 947 F. Supp. 581 (D. Conn. 1996) (excluding polygraph in civil action in part because of error rate)

THIRD CIRCUIT

United States v. Ewell, 252 F. Supp. 2d 104, 113–14 (D.N.J. 2003) (rejecting criminal defendant’s objection to government’s failure to quantify laboratory error rate)

Soldo v. Sandoz Pharmaceuticals Corp., 244 F. Supp. 2d 434, 568 (W.D. Pa. 2003) (excluding plaintiffs’ expert witnesses in part because court, and court-appointed expert witnesses, were unable to determine error rate).

Pharmacia Corp. v. Alcon Labs., Inc., 201 F. Supp. 2d 335, 360 (D.N.J. 2002) (excluding ; error too high).

FOURTH CIRCUIT

United States v. Moreland, 437 F.3d 424, 427–28, 430–31 (4th Cir. 2006) (affirming district court’s allowance of forensic chemist’s testimony that could not provide error rate because reviews of witness’s work found it to be free of error)

Buckman v. Bombardier Corp., 893 F. Supp. 547, 556–57 (E.D.N.C. 1995) (ruling that an expert witness may opine about comparisons between boat engines in rough water but only as a lay witness, because the comparison tests were unreliable, with a high estimated rate of error)

FIFTH CIRCUIT

Albert v. Jordan, Nos. 05CV516, 05CV517, 05CV518, 05CV519, 2007 U.S. Dist. LEXIS 92025, at *2–3 (W.D. La. Dec. 14, 2007) (allowing testimony of vocational rehabilitation expert witness, over objection, because witness provided “reliable” information, with known rate of error)

SIXTH CIRCUIT

United States v. Leblanc, 45 F. App’x 393, 398, 400 (6th Cir. 2002) (affirming exclusion of child psychologist, whose testimony about children’s susceptibility to coercive interrogation was based upon “‘soft science’ . . . in which ‘error is . . . rampant’.” (quoting the district court))

United States v. Sullivan, 246 F. Supp. 2d 696, 698–99 (E.D. Ky. 2003) (admitting expert witness’s opinion on the unreliability of eyewitness identification; confusing error rate of witness’s opinion with accuracy of observations made based upon order of presentation of photographs of suspect)

SEVENTH CIRCUIT

United States v. Vitek Supply Corp., 144 F.3d 476, 480, 485–86 (7th Cir. 1998) (affirming denial of defendant’s Rule 702 challenge based in part upon error rates; the purpose of the inquiry into rate of error is to determine whether tests are “accurate and reliable”; here the government’s expert witnesses used adequate controls and replication to ensure an acceptably low rate of error)

Phillips v. Raymond Corp., 364 F. Supp. 2d 730, 732–33, 740-41 (N.D. Ill. 2005) (excluding biomechanics expert witness who had not reliably tested his claims in a way to produce an accurate rate of error)

EIGHTH CIRCUIT

Bone Shirt v. Hazeltine, 461 F.3d 1011, 1020 (8th Cir. 2006) (affirming district court’s ruling to admit testimony of expert witness’s regression analysis in vote redistricting case); see id. at 1026 (Gruender, J., concurring) (expressing concern with the questioned testimony’s potential rate of error because it is “difficult to weigh this factor in Daubert’s analysis if ‘the effect of that error is unknown’.” (quoting court below, Bone Shirt v. Hazeltine, 336 F. Supp. 2d 976, 1002 (D.S.D. 2004))

United States v. Beasley, 102 F.3d 1440, 1444, 1446–48 (8th Cir. 1996) (confusing random error with general error rate) (affirming admissibility of expert witness testimony based upon DNA testing, because such testing followed acceptable standards in testing for contamination and “double reading”)

NINTH CIRCUIT

United States v. Chischilly, 30 F.3d 1144, 1148, 1152, 1154–55 (9th Cir. 1994) (affirming admissibility of testimony based upon DNA match in sex crime, noting that although error rate of error was unquantified, the government had made a sufficient showing of rarity of false positives to support an inference of low error rate)

Cascade Yarns, Inc. v. Knitting Fever, Inc., No. C10–861RSM, 2012 WL 5194085, at *7 (W.D. Wash. Oct. 18. 2012) (excluding expert witness opinion because error rate was too high)

United States v. Microtek Int’l Dev. Sys. Div., Inc., No. 99-298-KI, 2000 U.S. Dist. LEXIS 2771, at *2, *10–13, *15 (D. Or. Mar. 10, 2000) (excluding polygraph data based upon showing that claimed error rate came from highly controlled situations, and that “real world” situations led to much higher error (10%) false positive error rates)

TENTH CIRCUIT

Miller v. Pfizer, Inc., 356 F.3d 1326, 1330, 1334 (10th Cir. 2004) (affirming exclusion of plaintiffs’ expert witness, Dr. David Healy, based upon district court’s findings, made with the assistance of court-appointed expert witnesses, that Healy’s opinion was based upon studies that lacked sufficient sample size, adequate controls, and freedom from study bias, and thus prone to unacceptable error rate)

ELEVENTH CIRCUIT

Quiet Tech. DC-8, Inc. v. Hurel-Duboi U.K., Ltd., 326 F.3d 1333, 1343–45 (11th Cir. 2003) (affirming trial court’s admission of defendant’s aerospace engineer’s testimony, when the lower court had found that the error rate involved was “relatively low”; rejecting plaintiff’s argument that the witness had entered data incorrectly on ground that the asserted error would not affect the validity of the witness’s opinions)

Wright v. Case Corp., No. 1:03-CV-1618-JEC, 2006 U.S. Dist. LEXIS 7683, at *14 (N.D. Ga. Feb. 1, 2006) (granting defendant’s motion to exclude plaintiff’s mechanical engineering expert, because the expert’s alternative designs for the seat safety bar were not reliable due to potential feasibility issues, and because the associated error rate was therefore unquantifiable but potentially very high)

Benkwith v. Matrixx Initiatives, Inc., 467 F. Supp. 2d 1316, 1326, 1330, 1332 (M.D. Ala. 2006) (granting defendant’s motion to exclude testimony of an expert in the field of epidemiology regarding Zicam nasal spray’s causing plaintiff’s anosmia, because the opinions had not been tested and a rate of error could not be provided).

D.C. CIRCUIT

Ambrosini v. Upjohn Co., No. 84-3483 (NHJ), 1995 U.S. Dist. LEXIS 21318, at *16, *22–24 (D.D.C. Oct. 18, 1995) (finding that plaintiff’s teratology expert was not permitted to testify, because the methodology used was found to be unreliable and could not yield an accurate error rate)

[1] Jed S. Rakoff, “Science and the Law: Uncomfortable Bedfellows,” 38 Seton Hall L. Rev. 1379, 1382–83 (2008) (observing that an error rate of 13 percent in polygraph interpretation would likely be insufficiently reliable to support admissibility of testimony based upon polygraph results).

[2] Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, 1319 (2011) (suggesting that courts “frequently permit expert testimony on causation based on evidence other than statistical significance”).

[3] See, e.g., WLF Legal Backgrounder on Matrixx Initiatives (June 20, 2011); “The Matrixx – A Comedy of Errors”; Matrixx Unloaded (Mar. 29, 2011)”; “The Matrixx Oversold” (April 4, 2011); “De-Zincing the Matrixx.”

[4] Ernest Rutherford, a British chemist, investigated radioactivity. He won the Nobel Prize in chemistry, in 1908.

Posted in Expert Witnesses, Rule 702 | Comments Off on Daubert’s Error Rate

Judicial Control of the Rate of Error in Expert Witness Testimony

May 28th, 2015

In Daubert, the Supreme Court set out several criteria or factors for evaluating the “reliability” of expert witness opinion testimony. The third factor in the Court’s enumeration was whether the trial court had considered “the known or potential rate of error” in assessing the scientific reliability of the proffered expert witness’s opinion. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593 (1993). The Court, speaking through Justice Blackmun, failed to provide much guidance on the nature of the errors subject to gatekeeping, on how to quantify the errors, and on to know how much error was too much. Rather than provide a taxonomy of error, the Court lumped “accuracy, validity, and reliability” together with a grand pronouncement that these measures were distinguished by no more than a “hen’s kick.” Id. at 590 n.9 (1993) (citing and quoting James E. Starrs, “Frye v. United States Restructured and Revitalized: A Proposal to Amend Federal Evidence Rule 702,” 26 Jurimetrics J. 249, 256 (1986)).

The Supreme Court’s failure to elucidate its “rate of error” factor has caused a great deal of mischief in the lower courts. In practice, trial courts have rejected engineering opinions on stated grounds of their lacking an error rate as a way of noting that the opinions were bereft of experimental and empirical evidential support[1]. For polygraph evidence, courts have used the error rate factor to obscure their policy prejudices against polygraphs, and to exclude test data even when the error rate is known, and rather low compared to what passes for expert witness opinion testimony in many other fields[2]. In the context of forensic evidence, the courts have rebuffed objections to random-match probabilities that would require that such probabilities be modified by the probability of laboratory or other error[3].

When it comes to epidemiologic and other studies that require statistical analyses, lawyers on both sides of the “v” frequently misunderstand p-values or confidence intervals to provide complete measures of error, and ignore the larger errors that result from bias, confounding, study validity (internal and external), inappropriate data synthesis, and the like[4]. Not surprisingly, parties fallaciously argue that the Daubert criterion of “rate of error” is satisfied by expert witness’s reliance upon studies that in turn use conventional 95% confidence intervals and measures of statistical significance in p-values below 0.05[5].

The lawyers who embrace confidence intervals and p-values as their sole measure of error rate fail to recognize that confidence intervals and p-values are means of assessing only one kind of error: random sampling error. Given the carelessness of the Supreme Court’s use of technical terms in Daubert, and its failure to engage in the actual evidence at issue in the case, it is difficult to know whether the Court intended to suggest that random error was the error rate it had in mind[6]. The statistics chapter in the Reference Manual on Scientific Evidence helpfully points out that the inferences that can be drawn from data turn on p-values and confidence intervals, as well as on study design, data quality, and the presence or absence of systematic errors, such as bias or confounding. Reference Manual on Scientific Evidence at 240 (3d 2011) [Manual]. Random errors are reflected in the size of p-values or the width of confidence intervals, but these measures of random sampling error ignore systematic errors such as confounding and study biases. Id. at 249 & n.96.

The Manual’s chapter on epidemiology takes an even stronger stance: the p-value for a given study does not provide a rate of error or even a probability of error for an epidemiologic study:

“Epidemiology, however, unlike some other methodologies—fingerprint identification, for example—does not permit an assessment of its accuracy by testing with a known reference standard. A p-value provides information only about the plausibility of random error given the study result, but the true relationship between agent and outcome remains unknown. Moreover, a p-value provides no information about whether other sources of error – bias and confounding – exist and, if so, their magnitude. In short, for epidemiology, there is no way to determine a rate of error.”

Manual at 575. This stance seems not entirely justified given that there are Bayesian approaches that would produce credibility intervals accounting for sampling and systematic biases. To be sure, such approaches have their own problems and they have received little to no attention in courtroom proceedings to date.

The authors of the Manual’s epidemiology chapter, who are usually forgiving of judicial error in interpreting epidemiologic studies, point to one United States Court of Appeals case that fallaciously interpreted confidence intervals magically to quantify bias and confounding in a Bendectin birth defects case. Id. at 575 n. 96[7]. The Manual could have gone further to point out that, in the context of multiple studies, of different designs and analyses, cognitive biases involved in evaluating, assessing, and synthesizing the studies are also ignored by statistical measures such as p-values and confidence intervals. Although the Manual notes that assessing the role of chance in producing a particular set of sample data is “often viewed as essential when making inferences from data,” the Manual never suggests that random sampling error is the only kind of error that must be assessed when interpreting data. The Daubert criterion would appear to encompass all varieties or error, not just random error.

The Manual’s suggestion that epidemiology does not permit an assessment of the accuracy of epidemiologic findings misrepresents the capabilities of modern epidemiologic methods. Courts can, and do, invoke gatekeeping approaches to weed out confounded study findings. See “Sorting Out Confounded Research – Required by Rule 702” (June 10, 2012). The “reverse Cornfield inequality” was an important analysis that helped establish the causal connection between tobacco smoke and lung cancer[8]. Olav Axelson studied and quantified the role of smoking as a confounder in epidemiologic analyses of other putative lung carcinogens.[9] Quantitative methods for identifying confounders have been widely deployed[10].

A recent study in birth defects epidemiology demonstrates the power of sibling cohorts in addressing the problem of residual confounding from observational population studies with limited information about confounding variables. Researchers looking at various birth defect outcomes among offspring of women who used certain antidepressants in early pregnancy generally found no associations in pooled data from Iceland, Norway, Sweden, Finland, and Denmark. A putative association between maternal antidepressant use and a specific kind of cardiac defect (right ventricular outflow tract obstruction or RVOTO) did appear in the overall analysis, but was reversed when the analysis was limited to the sibling subcohort. The study found an apparent association between RVOTO defects and first trimester maternal exposure to selective serotonin reuptake inhibitors, with an adjusted odds ratio of 1.48 (95% C.I., 1.15, 1.89). In the adjusted analysis for siblings, the study found an OR of 0.56 (95% C.I., 0.21, 1.49) in an adjusted sibling analysis[11]. This study and many others show how creative analyses can elucidate and quantify the direction and magnitude of confounding effects in observational epidemiology.

Systematic bias has also begun to succumb to more quantitative approaches. A recent guidance paper by well-known authors encourages the use of quantitative bias analysis to provide estimates of uncertainty due to systematic errors[12].

Although the courts have failed to articulate the nature and consequences of erroneous inference, some authors would reduce all of Rule 702 (and perhaps 704, 403 as well) to a requirement that proffered expert witnesses “account” for the known and potential errors in their opinions:

“If an expert can account for the measurement error, the random error, and the systematic error in his evidence, then he ought to be permitted to testify. On the other hand, if he should fail to account for any one or more of these three types of error, then his testimony ought not be admitted.”

Mark Haug & Emily Baird, “Finding the Error in Daubert,” 62 Hastings L.J. 737, 739 (2011).

Like most antic proposals to revise Rule 702, this reform vision shuts out the full range of Rule 702’s remedial scope. Scientists certainly try to identify potential sources of error, but they are not necessarily very good at it. See Richard Horton, “Offline: What is medicine’s 5 sigma?” 385 Lancet 1380 (2015) (“much of the scientific literature, perhaps half, may simply be untrue”). And as Holmes pointed out[13], certitude is not certainty, and expert witnesses are not likely to be good judges of their own inferential errors[14]. Courts continue to say and do wildly inconsistent things in the course of gatekeeping. Compare In re Zoloft (Setraline Hydrochloride) Products, 26 F. Supp. 3d 449, 452 (E.D. Pa. 2014) (excluding expert witness) (“The experts must use good grounds to reach their conclusions, but not necessarily the best grounds or unflawed methods.”), with Gutierrez v. Johnson & Johnson, 2006 WL 3246605, at *2 (D.N.J. November 6, 2006) (denying motions to exclude expert witnesses) (“The Daubert inquiry was designed to shield the fact finder from flawed evidence.”).

[1] See, e.g., Rabozzi v. Bombardier, Inc., No. 5:03-CV-1397 (NAM/DEP), 2007 U.S. Dist. LEXIS 21724, at *7, *8, *20 (N.D.N.Y. Mar. 27, 2007) (excluding testimony from civil engineer about boat design, in part because witness failed to provide rate of error); Sorto-Romero v. Delta Int’l Mach. Corp., No. 05-CV-5172 (SJF) (AKT), 2007 U.S. Dist. LEXIS 71588, at *22–23 (E.D.N.Y. Sept. 24, 2007) (excluding engineering opinion that defective wood-carving tool caused injury because of lack of error rate); Phillips v. Raymond Corp., 364 F. Supp. 2d 730, 732–33 (N.D. Ill. 2005) (excluding biomechanics expert witness who had not reliably tested his claims in a way to produce an accurate rate of error); Roane v. Greenwich Swim Comm., 330 F. Supp. 2d 306, 309, 319 (S.D.N.Y. 2004) (excluding mechanical engineer, in part because witness failed to provide rate of error); Nook v. Long Island R.R., 190 F. Supp. 2d 639, 641–42 (S.D.N.Y. 2002) (excluding industrial hygienist’s opinion in part because witness was unable to provide a known rate of error).

[2] See, e.g., United States v. Microtek Int’l Dev. Sys. Div., Inc., No. 99-298-KI, 2000 U.S. Dist. LEXIS 2771, at *2, *10–13, *15 (D. Or. Mar. 10, 2000) (excluding polygraph data based upon showing that claimed error rate came from highly controlled situations, and that “real world” situations led to much higher error (10%) false positive error rates); Meyers v. Arcudi, 947 F. Supp. 581 (D. Conn. 1996) (excluding polygraph in civil action).

[3] See, e.g., United States v. Ewell, 252 F. Supp. 2d 104, 113–14 (D.N.J. 2003) (rejecting defendant’s objection to government’s failure to quantify laboratory error rate); United States v. Shea, 957 F. Supp. 331, 334–45 (D.N.H. 1997) (rejecting objection to government witness’s providing separate match and error probability rates).

[4] For a typical judicial misstatement, see In re Zoloft Products, 26 F. Supp.3d 449, 454 (E.D. Pa. 2014) (“A 95% confidence interval means that there is a 95% chance that the ‘‘true’’ ratio value falls within the confidence interval range.”).

[5] From my experience, this fallacious argument is advanced by both plaintiffs’ and defendants’ counsel and expert witnesses. See also Mark Haug & Emily Baird, “Finding the Error in Daubert,” 62 Hastings L.J. 737, 751 & n.72 (2011).

[6] See David L. Faigman, et al. eds., Modern Scientific Evidence: The Law and Science of Expert Testimony § 6:36, at 359 (2007–08) (“it is easy to mistake the p-value for the probability that there is no difference”)

[7] Brock v. Merrell Dow Pharmaceuticals, Inc., 874 F.2d 307, 311-12 (5^th Cir. 1989), modified, 884 F.2d 166 (5^th Cir. 1989), cert. denied, 494 U.S. 1046 (1990). As with any error of this sort, there is always the question whether the judges were entrapped by the parties or their expert witnesses, or whether the judges came up with the fallacy on their own.

[8] See Joel B Greenhouse, “Commentary: Cornfield, Epidemiology and Causality,” 38 Internat’l J. Epidem. 1199 (2009).

[9] Olav Axelson & Kyle Steenland, “Indirect methods of assessing the effects of tobacco use in occupational studies,” 13 Am. J. Indus. Med. 105 (1988); Olav Axelson, “Confounding from smoking in occupational epidemiology,” 46 Brit. J. Indus. Med. 505 (1989); Olav Axelson, “Aspects on confounding in occupational health epidemiology,” 4 Scand. J. Work Envt’l Health 85 (1978).

[10] See, e.g., David Kriebel, Ariana Zeka1, Ellen A Eisen, and David H. Wegman, “Quantitative evaluation of the effects of uncontrolled confounding by alcohol and tobacco in occupational cancer studies,” 33 Internat’l J. Epidem. 1040 (2004).

[11] Kari Furu, Helle Kieler, Bengt Haglund, Anders Engeland, Randi Selmer, Olof Stephansson, Unnur Anna Valdimarsdottir, Helga Zoega, Miia Artama, Mika Gissler, Heli Malm, and Mette Nørgaard, “Selective serotonin reuptake inhibitors and ventafaxine in early pregnancy and risk of birth defects: population based cohort study and sibling design,” 350 Brit. Med. J. 1798 (2015).

[12] Timothy L.. Lash, Matthew P. Fox, Richard F. MacLehose, George Maldonado, Lawrence C. McCandless, and Sander Greenland, “Good practices for quantitative bias analysis,” 43 Internat’l J. Epidem. 1969 (2014).

[13] Oliver Wendell Holmes, Jr., Collected Legal Papers at 311 (1920) (“Certitude is not the test of certainty. We have been cock-sure of many things that were not so.”).

[14] See, e.g., Daniel Kahneman & Amos Tversky, “Judgment under Uncertainty: Heuristics and Biases,” 185 Science 1124 (1974).

Posted in Cognitive Biases, Expert Witnesses, Reference Manual on Scientific Evidence, Rule 702, Scientific Evidence | Comments Off on Judicial Control of the Rate of Error in Expert Witness Testimony

Can an Expert Witness Be Too Biased to Be Allowed to Testify

May 20th, 2015

The Case of Barry Castleman

Barry Castleman has been a fixture in asbestos litigation for over three decades. By all appearances, he was the creation of the litigation industry. Castleman received a bachelor of science degree in chemical engineering in 1968, and a master’s degree in environmental engineering, in 1972. In 1975, he started as a research assistant to plaintiffs’ counsel in asbestos litigation, and in 1979, he commenced his testimonial adventures as a putative expert witness for plaintiffs’ counsel. Enrolled in a doctoral program, Castleman sent chapters of his thesis to litigation industry mentors for review and edits. In 1985, Castleman received a doctorate degree, with the assistance of a Ron Motley fellowship. See John M. Fitzpatrick, “Digging Deep to Attack Bias of Plaintiff Experts,” DRI Products Liability Seminar (2013).

Castleman candidly testified, on many occasions, that he was not an epidemiologist, a biostatistician, a toxicologist, a physician, a pathologist, or any other kind of healthcare professional. He is not a trained historian. Understandably, courts puzzled over exactly what someone like Castleman should be allowed to testify about. Many courts limited or excluded Castleman from remunerative testimonial roles[1]. Still, in the face of his remarkably inadequate training, education, and experience, Castleman persisted, and often prevailed, in making a living at testifying about the historical “state of the art” of medical knowledge about asbestos over time.

The result was often not pretty. Castleman worked not just as an expert witness, but also as an agent of plaintiffs’ counsel to suppress evidence. “The Selikoff – Castleman Conspiracy” (May 13, 2011). As a would-be historian, Castleman was controlled and directed by the litigation industry to avoid inconvenient evidence. “Discovery into the Origin of Historian Expert Witnesses’ Opinions” (Jan. 30, 2012). Despite his covert operations, and his exploitation of defendants’ internal documents, Castleman complained more than anyone about the scrutiny created by his self-chosen litigation roles. In 1985, pressed for materials he had considered in formulating his “opinions,” Castleman wrote a personal letter to the judge, the Hon. Hugh Gibson of Galveston, Texas, to object to lawful discovery into his activities:

“1. It threatens me ethically through demands that I divulge material submitted in confidence, endangering my good name and reputation.
2. It exposes me to potential liability arising from the release of correspondence and other materials provided to me by others who assumed I would honor their confidence.
3. It jeopardizes my livelihood in that material requested reveals strategies of parties with whom I consult, as well as other materials of a confidential nature.
4. It is far beyond the scope of relevant material to my qualifications and the area of expert testimony offered.
5. It is unprecedented in 49 prior trials and depositions where I have testified, in federal and state courts all over the United States, including many cases in Texas. Never before have I had to produce such voluminous and sensitive material in order to be permitted to testify.
6. It is excessively and unjustifiably intrusive into my personal and business life.
7. I have referenced most of the information I have in my 593-page book, “Asbestos: Medical and Legal Aspects.” The great majority of the information I have on actual knowledge of specific defendants has come from the defendants themselves.
8. All information that I have which is relevant to my testimony and qualifications has been the subject of numerous trials and depositions since 1979.”

Castleman Letter to Hon. Hugh Gibson (Nov. 5, 1985).

Forty years later, Castleman is still working for the litigation industry, and courts are still struggling to figure out what role he should be allowed as a testifying expert witness.

Last year, the Delaware Supreme Court had to order a new trial for R. T. Vanderbilt, in part because Castleman had blurted out non-responsive, scurrilous hearsay statements that:

(1) employees of Johns-Manville (a competitor of R.T. Vanderbilt) had called employees of Vanderbilt “liars;”

(2) R.T. Vanderbilt spent a great amount of money on studies and activities to undermine federal regulatory action on talc; and

(3) R.T. Vanderbilt was “buying senators and lobbying the government.”

The Delaware court held that Castleman’s gratuitous, unsolicited testimony on cross-examination was inadmissible, and that his conduct required a new trial. R.T. Vanderbilt Co. v. Galliher, No. 510, 2013, 2014 WL 3674180 (Del. July 24, 2014).

Late last year, a federal court ruled, pre-trial, that Castleman may testify over Rule 702 objections because he “possesses ‘specialized knowledge’ regarding the literature relating to asbestos available during the relevant time periods,” and that his testimony “could be useful to the jury as a ‘sort of anthology’ of the copious available literature.” Krik v. Crane Co., No. 10-cv-7435, – F. Supp. 2d -, 2014 WL 5350463, *3 (N.D. Ill. Oct. 21, 2014). Because Castleman was little more than a sounding board for citing and reading sections of the historical medical literature, the district court prohibited him from testifying as to the accuracy of any conclusions in the medical literature. Id.

Last week, another federal court took a different approach to keeping Castleman in business. In ruling on defendant’s Rule 702 objections to Castleman, the court held:

“I agree with defendant that plaintiffs have made no showing that Castleman is qualified to explain the meaning and significance of medical literature. Further, there is no suggestion in Krik that Castleman is qualified as an expert in that respect. To the extent that plaintiffs want Castleman simply to read excerpts from medical articles, they do not explain how doing so could be helpful to the jury. Accordingly, I am granting defendant’s motion as it relates to Castleman’s discussion of the medical literature.

***

However, Castleman’s report also includes discussions of articles in trade journals and government publications, which, presumably, would not require medical expertise to understand or summarize.”

Suoja v. Owens-Illinois, Inc., 2015 U.S. Dist. LEXIS 63170, at *3 (W.D.Wisc. May 14, 2015). Judge Barbara Crabb thus disallowed medical state of the art testimony from Castleman, but permitted him to resume his sounding board role for non-medical and other historical documents referenced in his Rule 26 report.

The strange persistence of Barry Castleman, and the inconsistent holdings of dozens of opinions strewn across the asbestos litigation landscape, raise the question whether someone so biased, so entrenched in a litigation role, so lacking in the requisite expertise, should simply be expunged from the judicial process. Rather than struggling to find some benign, acceptable role for Barry Castleman, perhaps courts should just say no. “How Testifying Historians Are Like Lawn-Mowing Dogs” (May 24, 2010).

[1] See, e.g., Van Harville v. Johns-Manville Sales Corp., CV-78-642-H (S.D. Ala 1979); In re Related Asbestos Cases, 543 F. Supp. 1142, 1149 (N.D. Cal. 1982) (rejecting Castleman’s bid to be called an “expert”) (holding that the court was “not persuaded that Mr. Castleman, as a layperson, possesses the expertise necessary to read complex, technical medical articles and discern which portions of the articles would best summarize the authors’ conclusions”); Kendrick v. Owens-Corning Fiberglas Corp., No. C-85-178-AAm (E.D. Wash. 1986); In re King County Asbestos Cases of Levinson, Friedman, Vhugen, Duggan, Bland and Horowitz, No. 81-2-08702-7, (Washington St. Super. Ct. for King Cty.1987); Franze v. Celotex Corp., C.A. No. 84-1316 (W.D. Pa.); Dunn v. Hess Oil Virgin Islands Corp., C.A. No. 1987-238 (D.V.I. May 16, 1989) (excluding testimony of Barry Castleman); Rutkowski v. Occidental Chem. Corp., No. 83 C 2339, 1989 WL 32030, at *1 (N.D. Ill. Feb. 16, 1989) (“Castleman lacks the medical background and experience to evaluate and analyze the articles in order to identify which parts of the articles best summarize the authors’ conclusions.”); In re Guam Asbestos Litigation, 61 F.3d 910, 1995 WL 411876 (9th Cir. 1995) (Kozinski, J., dissenting) (“I would also reverse because Barry Castleman was not qualified to testify as an expert witness on the subject of medical state of the art or anything else; he appears to have read a number of articles for the sole purpose of turning himself into an expert witness. Reductio ad absurdum.”); McClure v. Owens Corning Fiberglas Corp. 188 Ill. 2d 102, 720 N.E.2d 242 (1999) (rejecting probativeness of Castleman’s testimony about company conduct).

Posted in Expert Witnesses | Comments Off on Can an Expert Witness Be Too Biased to Be Allowed to Testify

Adverse Liver Events and Causal Claims Against Black Cohosh

April 6th, 2015

Liver toxicity in pharmaceutical products liability cases is one of the more difficult categories of cases for judicial gatekeeping because of the possibility of idiosyncratic liver toxicity. Sometimes a plaintiff will exploit this difficulty and try to recover for an acute liver reaction.

Susan Grant began to take a black cohosh herbal remedy in 2002, and within a year, developed autoimmune hepatitis, which required her to undergo a liver transplant. She and her husband sued the seller of black cohosh for substantial damages. Grant v. Pharmavite, LLC, 452 F. Supp 2d 903 (D. Neb. 2006). Granted enlisted two expert witnesses, Michael Corbett, Ph.D, a toxicologist, and her treating gastroenterologist, Michael Sorrell, M.D. The defense relying upon liver expert, Phillip Guzelian, M.D., challenged the admissibility of plaintiffs’ expert witnesses’ opinions under the federal rules.

Struggling with the law, Senior Judge Strom observed that Nebraska law requires expert witness opinion testimony on causation. Id. at 906. Of course, in this diversity action, federal law controlled on the scope and the requirements of expert witness opinion testimony.

And in a similarly offbeat way, Judge Strom suggested that plaintiffs’ expert witnesses need not have opinion supported by evidence:

“While it is not necessary that an opinion be backed by scientific research, it is necessary that an expert’s testimony, which contradicts all of the research, at minimum address and distinguish the contradictory research in order to support the expressed opinion.”

Id. at 907 (emphasis added). Senior Judge Strom thus suggests had there been no published research at all, then Dr. Corbett could just make up an opinion, not backed by scientific research. This is, of course, seriously wrong, but fortunately it amounts only to obiter detritus, because Judge Strom believed that given the available studies, the testifying expert witnesses had to do more than simply criticize the studies that disagreed with their subjective opinion.

Michael Corbett, Ph.D, a consultant in “chemical toxicology,” from Omaha, Nebraska, criticized existent studies, which generally failed to identify liver toxicity, but he failed to conduct his own studies. Id. at 907. And Corbett also failed to explain why he rejected the great weight of medical publications that found that black cohosh was not hepatotoxic. Id. Michael Sorrell, M.D., started out as Ms. Grant’s treating gastroenterologist, but became a litigation expert witness. He was generally unaware of the randomized clinical trials of black cohosh, or any study that, or group of scientists who, supported his opinion. Id. at 909.

To Dr. Sorrell’s credit, he did attempt to write up a case report, which was published after the termination of the case. Unfortunately for Dr. Sorrell and his colleagues, Ms. Grant and her lawyers were less than forthcoming about her medical history, which included medications and lifestyle variables that were apparently not shared with Dr. Sorrell. Id. at 909.

You know that the quality of gatekeeping due process is strained when judges fail to cite key studies sufficiently to permit their readers to find the scientific evidence. Between Google Scholar and PubMed, however, you can find Dr. Sorrell’s case report, which was published in 2005, before Judge Strom issued his Rule 702 opinion. Josh Levitsky, Tyron A. Alli, James Wisecarver, and Michael F. Sorrell, “Fulminant liver failure associated with the use of black cohosh,” 50 Digestive Dis. Sci. 538 (2005). If nothing else, Judge Strom provoked an erratum from Dr. Sorrell and colleagues:

“After the article was published, it was brought to the authors’ attention through legal documentation and testimony that the patient admitted to consuming alcohol and had been taking other medications at the time of her initial presentation of liver failure. From these records, she reported drinking no more than six glasses of wine per week. In addition, up until presentation, she was taking valacyclovir 500 mg daily for herpes prophylaxis for 2 years, an occasional pseudoephedrine tablet, calcium carbonate 500 mg three times daily, iron sulfate 325 mg daily and ibuprofen up to three times weekly. She had been taking erythromycin tablets but discontinued those 3 months prior to presentation.

The authors regret the omission of this information from the original case report. While this new information is important to include as a correction to the history, it does not change the authors’ clinical opinion … .”

The erratum omits that Ms. Grant was taking Advil (ibuprofen) at the time of her transplantation, and that she had been taking erythromycin for 2.5 years, stopping just a few months before her acute liver illness. The Valtrex use shows that Ms. Grant had a chronic herpes infection. In the past, plaintiff took such excessive doses of ibuprofen that she developed anemia. Grant v. Pharmavite, LLC, 452 F. Supp 2d at 909 n.1. Hardly an uncomplicated case report to interpret for causality and an interesting case history of confirmation bias. Remarkably, the journal charges $39.95 to download the erratum, as much as the case report itself!

And how has the plaintiff’s claim fared in the face of the evolving scientific record since Judge Strom’s opinion?

Not well.

See, e.g., Peter W Whiting, Andrew Clouston and Paul Kerlin, “Black cohosh and other herbal remedies associated with acute hepatitis,” 177 Med. J. Australia 432 (2002); Cohen SM, O’Connor AM, Hart J, et al. Autoimmune hepatitis associated with the use of black cohosh: a case study. 11 Menopause 575 (2004); Christopher R. Lynch, Milan E. Folkers, and William R. Hutson, “Fulminant hepatic failure associated with the use of black cohosh: A case report,” 12 Liver Transplantation 989 (2006); Elizabeth C-Y Chow, Marcus Teo, John A Ring and John W Chen, “Liver failure associated with the use of black cohosh for menopausal symptoms,” 188 Med. J. Australia 420 (2008); Gail B. Mahady, Tieraona Low Dog, Marilyn L. Barrett, Mary L. Chavez, Paula Gardiner, Richard Ko, Robin J. Marles, Linda S. Pellicore, Gabriel I. Giancaspro, and Dandapantula N. Sarma, “United States Pharmacopeia review of the black cohosh case reports of hepatotoxicity,” 15 Menopause 628 (2008) (toxicity only possible on available evidence); D. Joy, J. Joy, and P. Duane, “Black cohosh: a cause of abnormal postmenopausal liver function tests,” 11 Climacteric 84 (2008); Lily Dara, Jennifer Hewett, and Joseph Kartaik Lim, “Hydroxycut hepatotoxicity: A case series and review of liver toxicity from herbal weight loss supplements,” 14 World J. Gastroenterol. 6999 (2008); F. Borrelli & E. Ernst, “Black cohosh (Cimicifuga racemosa): a systematic review of adverse events,” Am. J. Obstet. & Gyn. 455 (2008); Rolf Teschke & A. Schwarzenboeck, “Suspected hepatotoxicity by Cimicifugae racemosae rhizoma (black cohosh, root): critical analysis and structured causality assessment,” 16 Phytomedicine 72 (2009); Stacie E. Geller, Lee P. Shulman, Richard B. van Breemen, Suzanne Banuvar, Ying Zhou, Geena Epstein, Samad Hedayat, Dejan Nikolic, Elizabeth C. Krause, Colleen E. Piersen, Judy L. Bolton, Guido F. Pauli, and Norman R. Farnsworth, “Safety and Efficacy of Black Cohosh and Red Clover for the Management of Vasomotor Symptoms: A Randomized Controlled Trial,” 16 Menopause 1156 (2009) (89 women randomized to four groups; no hepatic events in trial not powered to detect them); Rolf Teschke, “Black cohosh and suspected hepatotoxicity: inconsistencies, confounding variables, and prospective use of a diagnostic causality algorithm. A critical review,” 17 Menopause 426 (2010) (“The presented data do not support the concept of hepatotoxicity in a primarily suspected causal relationship to the use of BC and failure to provide a signal of safety concern, but further efforts have to be undertaken to dismiss or to substantiate the existence of BC hepatotoxicity as a special disease entity. The future strategy should be focused on prospective causality evaluations in patients diagnosed with suspected BC hepatotoxicity, using a structured, quantitative, and hepatotoxicity-specific causality assessment method.”); Fabio Firenzuoli, Luigi Gori, and Paolo Roberti di Sarsina, “Black Cohosh Hepatic Safety: Follow-Up of 107 Patients Consuming a Special Cimicifuga racemosa rhizome Herbal Extract and Review of Literature,” 2011 Evidence-Based Complementary & Alternative Med. 1 (2011); Rolf Teschke, Wolfgang Schmidt-Taenzer and Albrecht Wolff, “Spontaneous reports of assumed herbal hepatotoxicity by black cohosh: is the liver-unspecific Naranjo scale precise enough to ascertain causality?” 20 Pharmacoepidemiol. & Drug Safety 567 (2011) (causation unlikely or excluded); Rolf Teschke, Alexander Schwarzenboeck, Wolfgang Schmidt-Taenzer, Albrecht Wolff, and Karl-Heinz Hennermann, “Herb induced liver injury presumably caused by black cohosh: A survey of initially purported cases and herbal quality specifications,” 11 Ann. Hepatology 249 (2011).

Posted in Causation, Expert Witnesses, Rule 702 | Comments Off on Adverse Liver Events and Causal Claims Against Black Cohosh

Cherry Picking; Systematic Reviews; Weight of the Evidence

April 5th, 2015

In a paper prepared for one of Professor Margaret Berger’s symposia on law and science, Lisa Bero, a professor of clinical pharmacy in the University of California San Francisco’s School of Pharmacy identified a major source of error in published reviews of putative health effects:

“The biased citation of studies in a review can be a major source of error in the results of the review. Authors of reviews can influence their conclusions by citing only studies that support their preconceived, desired outcome.”

Lisa Bero, “Evaluating Systematic Reviews and Meta-Analyses,” 14 J. L. & Policy 569, 576 (2006). Biased citation, consideration, and reliance are major sources of methodological error in courtroom proceedings as well. Sometimes astute judges recognize and bar expert witnesses who would pass off their opinions, as well considered, when they are propped up only by biased citation. Unfortunately, courts have been inconsistent, sometimes rewarding cherry picking of studies by admitting biased opinions[1], sometimes unhorsing the would-be expert witnesses by excluding their opinions[2].

Given that cherry picking or “biased citation” is recognized in the professional community as a rather serious methodological sin, judges may be astonished to learn that both phrases, “cherry picking” and “biased citation” do not appear in the third edition of the Reference Manual on Scientific Evidence. Of course, the Manual could have dealt with the underlying issue of biased citation by affirmatively promoting the procedure of systematic reviews, but here again, the Manual falls short. There is no discussion of systematic review in the chapters on toxicology[3], epidemiology[4], or statistics[5]. Only the chapter on clinical medicine discusses the systematic review, briefly[6]. The absence of support for the procedures of systematic reviews, combined with the occasional cheerleading for “weight of the evidence,” in which expert witnesses subjectively include and weight studies to reach pre-ordained opinions, tends to undermines the reliability of the latest edition of the Manual[7].

[1] Spray-Rite Serv. Corp. v. Monsanto Co., 684 F.2d 1226, 1242 (7th Cir. 1982) (failure to consider factors identified by opposing side’s expert did not make testimony inadmissible).

[2] In re Zoloft, 26 F. Supp. 3d 449 (E.D. Pa. 2014) (excluding perinatal epidemiologist, Anick Bérard, for biased cherry picking of data points); In re Accutane, No. 271(MCL), 2015 WL 753674, 2015 BL 59277 (N.J.Super. Law Div. Atlantic Cty. Feb. 20, 2015) (excluding opinions Drs. Arthur Kornbluth and David Madigan because of their authors’ unjustified dismissal of studies that contradicted or undermined their opinions); In re Bextra & Celebrex Mktg. Sales Practices & Prods. Liab. Litig., 524 F.Supp. 2d 1166, 1175–76, 1179 (N.D.Cal.2007) (holding that expert witnesses may not ‘‘cherry-pick[ ]’’ observational studies to support a conclusion that is contradicted by randomized controlled trials, meta-analyses of such trials, and meta-analyses of observational studies; excluding expert witness who ‘‘ignores the vast majority of the evidence in favor of the few studies that support her conclusion’’); Grant v. Pharmative, LLC, 452 F. Supp. 2d 903, 908 (D. Neb. 2006) (excluding expert witness opinion testimony that plaintiff’s use of black cohash caused her autoimmune hepatitis) (“Dr. Corbett’s failure to adequately address the body of contrary epidemiological evidence weighs heavily against admission of his testimony.”); Downs v. Perstorp Components, Inc., 126 F. Supp. 2d 1090,1124-29 (E.D. Tenn. 1999) (expert’s opinion raised seven “red flags” indicating that his testimony was litigation biased), aff’d, 2002 U.S. App. Lexis 382 (6th Cir. Jan. 4, 2002).

[3] Bernard D. Goldstein & Mary Sue Henifin, “Reference Guide on Toxicology,” in Reference Manual on Scientific Evidence 633 (3d ed. 2011).

[4] Michael D. Green, D. Michal Freedman, and Leon Gordis, “Reference Guide on Epidemiology,” in Reference Manual on Scientific Evidence 549 (3d ed. 2011).

[5] David H. Kaye & David A. Freedman, “Reference Guide on Statistics,” in Reference Manual on Scientific Evidence 209 (3d ed. 2011).

[6] John B. Wong, Lawrence O. Gostin, and Oscar A. Cabrera, “Reference Guide on Medical Testimony,” in Federal Judicial Center and National Research Council, Reference Manual on Scientific Evidence 687 (3d ed. 2011).

[7] See Margaret A. Berger, “The Admissibility of Expert Testimony,” in Reference Manual on Scientific Evidence 11, 20 & n.51 (3d ed. 2011) (posthumously citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11, 26 (1st Cir. 2011), with approval, for reversing exclusion of expert witnesses who advanced “weight of the evidence” opinions).

Posted in Expert Witnesses, Reference Manual on Scientific Evidence, Scientific Evidence | Comments Off on Cherry Picking; Systematic Reviews; Weight of the Evidence

Johnson of Accutane – Keeping the Gate in the Garden State

March 28th, 2015

Nelson Johnson is the author of Boardwalk Empire: The Birth, High Times, and Corruption of Atlantic City (2010), a rattling good yarn, which formed the basis for a thinly fictionalized story of Atlantic City under the control of mob boss (and Republican politician) Enoch “Nucky” Johnson. HBO transformed Johnson’s book into a multi-season series, with Steve Buscemi playing Nucky Johnson (Thompson in the series). Robert Strauss, “Judge Nelson Johnson: Atlantic City’s Godfather — A Q&A with Judge Nelson Johnson,” New Jersey Monthly (Aug. 16, 2010).

Nelson Johnson is also known as the Honorable Nelson Johnson, a trial court judge in Atlantic County, New Jersey, where he inherited some of the mass tort docket of Judge Carol Higbee. Judge Higbee has since ascended to the Appellate Division of the New Jersey Superior Court. One of the litigations Judge Johnson presides over is the mosh pit of isotretinoin (Accutane) cases, involving claims that the acne medication causes irritable bowel syndrome (IBS) and Crohn’s disease (CD). Judge Johnson is not only an accomplished writer of historical fiction, but he is also an astute evaluator of the facts and data, and the accompanying lawyers’ rhetoric, thrown about in pharmaceutical products liability litigation.

Perhaps more than his predecessor ever displayed, Judge Johnson recently demonstrated his aptitude for facts and data in serving as a gatekeeper of scientific evidence, as required by the New Jersey Supreme Court, in Kemp v. The State of New Jersey, 174 NJ 412 (2002). Faced with a complex evidentiary display on the validity and reliability of the scientific evidence, Judge Johnson entertained extensive briefings, testimony, and oral argument. When the dust settled, the court ruled that the proffered testimony of Dr, Arthur Kornbluth and Dr. David Madigan did not meet the liberal New Jersey test for admissibility. In re Accutane, No. 271(MCL), 2015 WL 753674, 2015 BL 59277 (N.J.Super. Law Div. Atlantic Cty. Feb. 20, 2015). And in settling the dust, Judge Johnson dispatched several bogus and misleading “lines of evidence,” which have become standard ploys to clog New Jersey and other courthouses.

Case Reports

As so often is the case when there is no serious scientific evidence of harm in pharmaceutical cases, plaintiffs in the Accutane litigation relied heavily upon case and adverse event reports. Id. at *11. Judge Johnson was duly unimpressed, and noted that:

“[u]nsystematic clinical observations or case reports and adverse event reports are at the bottom of the evidence hierarchy.”

Id. at *16.

Bootstrapped, Manufactured Evidence

With respect to case reports that are submitted to the FDA’s Adverse Event Reporting System (FAERS), Judge Johnson acknowledged the “serious limitations” of the hearsay anecdotes that make up such reports. Despite the value of AERs in generating signals for future investigation, Judge Johnson, citing FDA’s own description of the reporting system, concluded that the system’s anecdotal data are “not evidentiary in a court of law.” Id. at 14 (quoting FDA’s description of FAERS).

Judge Johnson took notice of another fact; namely, the industry litigation creates evidence that it then uses to claim causal connections in the courtroom. Plaintiffs’ lawyers in pharmaceutical cases routinely file Medwatch adverse event reports, which thus inflate the “signal,” they claim supports the signal of harm from medication use. This evidentiary bootstrapping machine was hard at work in the isotretinoin litigation. See Derrick J. Stobaugh, Parakkal Deepak, and Eli D. Ehrenpreis, “Alleged Isotretinoin-Associated Inflammatory Bowel Disease: Disproportionate reporting by attorneys to the Food and Drug Administration Adverse Event Reporting System,” 69 J. Am. Acad. Dermatol. 398 (2013) (“Attorney-initiated reports inflate the pharmacovigilance signal of isotretinoin-associated IBD in the FAERS.”). Judge Johnson gave a wry hat tip to plaintiffs’ counsel’s industry, by acknowledging that the litigation industry itself had inflated this signal-generating process:

“The legal profession is a bulwark of our society, yet the courts should never underestimate the resourcefulness of some attorneys.”

In re Accutane, 2015 WL 753674, at *15.

Bias and Confounding

The epidemiologic studies referenced by the parties had identified a fairly wide range of “risk factors” for irritable bowel syndrome, including many prevalent factors in Westernized countries such as prior appendectomy, breast-feeding as an infant, stress, Vitamin D deficiency, tobacco or alcohol use, refined sugars, dietary animal fat, fast food. In re Accutane, 2015 WL 753674, at *9. The court also noted that there were four medications known to be risk factors for IBD: aspirin, nonsteroidal anti-inflammatory medications (NSAIDs), oral contraceptives, and antibiotics.

In reviewing the plaintiffs’ expert witnesses’ methodology, Judge Johnson found that they had been inordinately, and inappropriately selective in the studies chosen for reliance. The challenged witnesses had discounted and discarded most of the available studies in favor of two studies that were small, biased, and not population based. Indeed, one of the studies evidenced substantial selection bias by using referrals to obtain study participants, a process deprecated by the trial court as “cherry picking the subjects.” Id. at *18. “The scientific literature does not support reliance upon such insignificant studies to arrive at conclusions.” Id.

Animal Studies

Both sides in the isotretinoin cases seemed to concede the relative unimportance of animal studies. The trial court discussed the limitations on animal studies, especially the absence of a compelling animal model of human irritable bowel syndrome. Id. at *18.

Cherry Picking and Other Crafty Stratagems

With respect to the complete scientific evidentiary display, plaintiffs asserted that their expert witnesses had considered everything, but then failed to account for most of the evidence. Judge Johnson found this approach deceptive and further evidence of a cherry-picking, pathological methodology:

‘‘Finally, coursing through Plaintiffs’ presentation is a refrain that is a ruse. Repeatedly, counsel for the Plaintiffs and their witnesses spoke of ‛lines of evidence”, emphasizing that their experts examined ‛the same lines of evidence’ as did the experts for the Defense. Counsels’ sophistry is belied by the fact that the examination of the ‘lines of evidence’ by Plaintiffs’ experts was highly selective, looking no further than they wanted to—cherry picking the evidence—in order to find support for their conclusion-driven testimony in support of a hypothesis made of disparate pieces, all at the bottom of the medical evidence hierarchy.’’

Id. at *21.

New Jersey Rule of Evidence 703

The New Jersey rules of evidence, like the Federal Rules, imposes a reasonableness limit on what sorts of otherwise inadmissible evidence an expert witness may rely upon. See “RULE OF EVIDENCE 703 — Problem Child of Article VII” (Sept. 9, 2011). Although Judge Johnson did not invoke Rule 703 specifically, he was clearly troubled by plaintiffs’ expert witnesses’ reliance upon an unadjusted odds ratio from an abstract, which did not address substantial confounding from a known causal risk factor – antibiotics use. Judge Johnson concluded that the reliance upon the higher, unadjusted risk figure, contrary to the authors’ own methods and conclusions, and without a cogent explanation for so doing was “pure advocacy” on the part of the witnesses. In re Accutane, 2015 WL 753674, at *17; see also id. at *5 (citing Landrigan v. Celotex Corp., 127 N.J. 404, 417 (1992), for the proposition that “when an expert relies on such data as epidemiological studies, the trial court should review the studies, as well as other information proffered by the parties, to determine if they are of a kind on which such experts ordinarily rely.”).

Discordance Between Courtroom and Professional Opinions

One of plaintiffs’ expert witnesses, Dr. Arthur Kornbluth actually had studied putative association between isotretinoin and CD before he became intensively involved in litigation as an expert witness. In re Accutane, 2015 WL 753674, at *7. Having an expert witness who is a real world expert can be a plus, but not when that expert witness maintains a double standard for assessing causal connections. Back in 2009, Kornbluth published an article, “Ulcerative Colitis Practice Guidelines in Adults” in The American Journal of Gastroenterology. Id. at *10. This positive achievement became a large demerit when cross-examination at the Kemp hearing revealed that Kornbluth had considered but rejected the urgings of a colleague, Dr. David Sachar, to comment on isotretinoin as a cause of irritable bowel syndrome. In front of Judge Johnson, Dr. Kornbluth felt no such scruples. Id. at *11. Dr. Kornbluth’s stature in the field of gastroenterology, along with his silence on the issue in his own field, created a striking contrast with his stridency about causation in the courtroom. The contrast raised the trial court’s level of scrutiny and skepticism about his causal opinions in the New Jersey litigation. Id. (citing and quoting Soldo v. Sandoz Pharms. Corp, 244 F. Supp. 2d 434, 528 (W.D. Pa. 2003) (“Expert opinions generated as the result of litigation have less credibility than opinions generated as the result of academic research or other forms of ‘pure’ research.”) (“The expert’s motivation for his/her study and research is important. … We may not ignore the fact that a scientist’s normal work place is the lab or field, not the courtroom or the lawyer’s office.”).

Meta-Analysis

Meta-analysis has become an important facet of pharmaceutical and other products liability litigation[1]. Fortunately for Judge Johnson, he had before him an extremely capable expert witness, Dr. Stephen Goodman, to explain meta-analysis generally, and two meta-analyses performed on isotretinoin and irritable bowel outcomes. In re Accutane, 2015 WL 753674, at *8. Dr. Goodman explained that:

“the strength of the meta-analysis is that no one feature, no one study, is determinant. You don’t throw out evidence except when you absolutely have to.”

Id. Dr. Goodman further explained that plaintiffs’ expert witnesses’ failure to perform a meta-analysis was telling meta-analysis “can get us closer to the truth.” Id.

Some Nitpicking

Specific Causation

After such a commanding judicial performance by Judge Johnson, nitpicking on specific causation might strike some as ungrateful. For some reason, however, Judge Johnson cited several cases on the appropriateness of expert witnesses’ reliance upon epidemiologic studies for assessing specific causation or for causal apportionment between two or more causes. In re Accutane, 2015 WL 753674, at *5 (citing Landrigan v. Celotex Corp., 127 N.J. 404 (1992), Caterinicchio v. Pittsburgh Corning, 127 N.J. 428 (1992), and Dafler v. Raymark Inc., 259 N.J. Super. 17, 36 (App. Div. 1992), aff’d. o.b. 132 N.J. 96 (1993)). Fair enough, but specific causation was not at issue in the Accutane Kemp hearing, and the Landrigan and Caterinicchio cases are irrelevant to general causation.

In both Landrigan and Caterincchio, the defendants moved for directed verdicts by arguing that, assuming arguendo that asbestos causes colon cancer, the plaintiffs’ expert witnesses had not presented a sufficient opinion to support that Landrigan’s and Caterinnichio’s colon cancers were caused by asbestos. See “Landrigan v. The Celotex Corporation, Revisited” (June 4, 2013). General causation was thus never at issue, and the holdings never addressed the admissibility of the expert witnesses’ causation opinions. Only sufficiency of the opinions that equated increased risks, less than 2.0, to specific causation was at issue in the directed verdicts, and the appeals taken from the judgments entered on those verdicts.

Judge Johnson, in discussing previous case law suggests that the New Jersey Supreme Court reversed and remanded the Landrigan case for trial, holding that “epidemiologists could help juries determine causation in toxic tort cases and rejected the proposition that epidemiological studies must show a relative risk factor of 2.0 before gaining acceptance by a court.” In re Accutane, 2015 WL 753674, at *5, citing Landrigan, 127 N.J. at 419. A close and fair reading of Landrigan, however, shows that it was about a directed verdict, 127 N.J. at 412, and not a challenge to the use of epidemiologic studies generally, or to their use to show general causation.

Necessity of Precise Biological Mechanism

In the Accutane hearings, the plaintiffs’ counsel and their expert witnesses failed to provide a precise biological mechanism of the cause of IBD. Judge Johnson implied that any study that asserted that Accutane caused IBD ‘‘would, of necessity, require an explication of a precise biological mechanism of the cause of IBD and no one has yet to venture more than alternate and speculative hypotheses on that question.’’ In re Accutane, 2015 WL 753674, at *8. Conclusions of causality, however, do not always come accompanied by understood biological mechanisms, and Judge Johnson demonstrated that the methods and evidence relied upon by plaintiffs’ expert witnesses could not, in any event, allow them to draw causal conclusions.

Interpreting Results Contrary to Publication Authors’ Interpretations

There is good authority, no less than the United States Supreme Court in Joiner, that there is something suspect in expert witnesses’ interpreting a published study’s results in contrary to the authors’ publication. Judge Johnson found that the plaintiffs’ expert witnesses in the Accutane litigation had inferred that two studies showed increased risk when the authors of those studies had concluded that their studies did not appear to show an increased risk. Id. at *17. There will be times, however, when a published study may have incorrectly interpreted its own data, when “real” expert witnesses can, and should, interpret the data appropriately. Accutane was not such a case. In In re Accutane, Judge Johnson carefully documented and explained how the plaintiffs’ expert witnesses’ supposed reinterpretation was little more than attempted obfuscation. His Honor concluded that the witnesses’ distortion of, and ‘‘reliance upon these two studies is fatal and reveals the lengths to which legal counsel and their experts are willing to contort the facts and torture the logic associated with Plaintiffs’ hypothesis.’’ Id. at *18.

[1] “The Treatment of Meta-Analysis in the Third Edition of the Reference Manual on Scientific Evidence” (Nov. 14, 2011) (The Reference Manual fails to come to grips with the prevalence and importance of meta-analysis in litigation, and fails to provide meaningful guidance to trial judges).

Posted in Causation, Expert Witnesses, Rule 702, Rule 703, Scientific Evidence | Comments Off on Johnson of Accutane – Keeping the Gate in the Garden State

The Joiner Finale

March 23rd, 2015

“This is the end
Beautiful friend
This is the end
My only friend, the end”

Jim Morrison, “The End” (c. 1966)

* * * * * * * * * *

The General Electric Co. v. Joiner, 522 U.S. 136 (1997), case was based upon polychlorinated biphenyl exposures (PCB), only in part. The PCB part did not hold up well legally in the Supreme Court; nor was the PCB lung cancer claim vindicated by later scientific evidence. See “How Have Important Rule 702 Holdings Held Up With Time?” (Mar. 20, 2015).

The Supreme Court in Joiner reversed and remanded the case to the 11th Circuit, which then remanded the case back to the district court to address claims that Mr. Joiner had been exposed to furans and dioxins, and that these other chemicals had caused, or contributed to, his lung cancer, as well. Joiner v. General Electric Co., 134 F.3d 1457 (11th Cir. 1998) (per curiam). Thus the dioxins were left in the case even after the Supreme Court ruled.

After the Supreme Court’s decision, Anthony Roisman argued that the Court had addressed an artificial question when asked about PCBs alone because the case was really about an alleged mixture of exposures, and he held out hope that the Joiners would do better on remand. Anthony Z. Roisman, “The Implications of G.E. v. Joiner for Admissibility of Expert Testimony,” 1 Res Communes 65 (1999).

Many Daubert observers (including me) are unaware of the legal fate of the Joiners’ claims on remand. In the only reference I could find, the commentator simply noted that the case resolved before trial.[1] I am indebted to Michael Risinger, and Joseph Cecil, for pointing me to documents from PACER, which shed some light upon the Joiner “endgame.”

In February 1998, Judge Orinda Evans, who had been the original trial judge, and who had sustained defendants’ Rule 702 challenges and granted their motions for summary judgments, received and reopened the case upon remand from the 11th Circuit. In March, Judge Evans directed the parties to submit a new pre-trial order by April 17, 1998. At a status conference in April 1998, Judge Evans permitted the plaintiffs additional discovery, to be completed by June 17, 1998. Five days before the expiration of their additional discovery period, the plaintiffs moved for additional time; defendants opposed the request. In July, Judge Evans granted the requested extension, and gave defendants until November 1, 1998, to file for summary judgment.

Meanwhile, in June 1998, new counsel entered their appearances for plaintiffs – William Sims Stone, Kevin R. Dean, Thomas Craig Earnest, and Stanley L. Merritt. The docket does not reflect much of anything about the new discovery other than a request for a protective order for an unpublished study. But by October 6, 1998, the new counsel, Earnest, Dean, and Stone (but not Merritt) withdrew as attorneys for the Joiners, and by the end of October 1998, Judge Evans entered an order to dismiss the case, without prejudice.

A few months later, in February 1999, the parties filed a stipulation, approved by the Clerk, dismissing the action with prejudice, and with each party to bear its own coasts. Given the flight of plaintiffs’ counsel, the dismissals without and then with prejudice, a settlement seems never to have been involved in the resolution of the Joiner case. In the end, the Joiners’ case fizzled perhaps to avoid being Frye’d.

And what has happened since to the science of dioxins and lung cancer?

Not much.

In 2006, the National Research Council published a monograph on dioxin, which took the controversial approach of focusing on all cancer mortality rather than specific cancers that had been suggested as likely outcomes of interest. See David L. Eaton (Chairperson), Health Risks from Dioxin and Related Compounds – Evaluation of the EPA Reassessment (2006). The validity of this approach, and the committee’s conclusions, were challenged vigorously in subsequent publications. Paolo Boffetta, Kenneth A. Mundt, Hans-Olov Adami, Philip Cole, and Jack S. Mandel, “TCDD and cancer: A critical review of epidemiologic studies,” 41 Critical Rev. Toxicol. 622 (2011) (“In conclusion, recent epidemiologicalevidence falls far short of conclusively demonstrating a causal link between TCDD exposure and cancer risk in humans.”

In 2013, the Industrial Injuries Advisory Council (IIAC), an independent scientific advisory body in the United Kingdom, published a review of lung cancer and dioxin. The Council found the epidemiologic studies mixed, and declined to endorse the compensability of lung cancer for dioxin-exposed industrial workers. Industrial Injuries Advisory Council – Information Note on Lung cancer and Dioxin (December 2013). See also Mann v. CSX Transp., Inc., 2009 WL 3766056, 2009 U.S. Dist. LEXIS 106433 (N.D. Ohio 2009) (Polster, J.) (dioxin exposure case) (“Plaintiffs’ medical expert, Dr. James Kornberg, has opined that numerous organizations have classified dioxins as a known human carcinogen. However, it is not appropriate for one set of experts to bring the conclusions of another set of experts into the courtroom and then testify merely that they ‘agree’ with that conclusion.”), citing Thorndike v. DaimlerChrysler Corp., 266 F. Supp. 2d 172 (D. Me. 2003) (court excluded expert who was “parroting” other experts’ conclusions).

[1] Morris S. Zedeck, Expert Witness in the Legal System: A Scientist’s Search for Justice 49 (2010) (noting that, after remand from the Supreme Court, Joiner v. General Electric resolved before trial)

Posted in Causation, Expert Witnesses, Rule 702 | Comments Off on The Joiner Finale

Plaintiffs Sue Friendly Physicians for Access to Opinions and to Data

February 14th, 2015

Access to underlying data continues to be a difficult and contentious issue in litigation and in the world of public health. Healthcare providers have made intemperate demands upon pharmaceutical manufacturers to produce underlying data from clinical trials, but often the same providers object the loudest when asked to share underlying data from their own observational or experimental studies.

Last month saw a judicial rejection of an effort by New York researchers to resist production of underlying data. Plaintiffs in the World Trade Center health effects litigation sought to compel their treating physicians at the Mt. Sinai Health System to testify to their opinions about general and specific causation of the plaintiffs’ alleged injuries. Previously, in discovery depositions of the plaintiffs’ treating physicians as fact witnesses, Mt. Sinai Hospital’s attorneys objected to plaintiffs’ counsel’s efforts to elicit causation opinions not recorded in the plaintiffs’ personal medical records. Plaintiffs then moved to compel the treating physicians to give opinions on causation, and to produce underlying data from their published papers on the claimed health effects of exposure to World Trade Center dust. The hospital opposed these motions on grounds that its neutrality in the litigation would be compromised by compulsory testimony and production of data.

In a short opinion, Judge Alvin K. Hellerstein refused plaintiffs’ motions to compel testimony, but granted their motion to compel production of underlying data. In re World Trade Center Lower Manhattan Disaster Site Litig., ___ F.R.D. ___, 2015 WL 220988, at *1-2 (S.D.N.Y. Jan. 15, 2015). Judge Hellerstein acknowledged that, in rare instances, federal trial courts have compelled experts to provide opinion testimony. Carter–Wallace, Inc. v. Otte, 474 F.2d 529, 536 (2d Cir.1972) (“The weight of authority holds that, although it is not the usual practice, a court does have the power to subpoena an expert witness and … require him [or her] to state whatever opinions he [or she] may have previously formed.”).

Judge Hellerstein cited the five factors articulated in Kaufman v. Edelstein, as governing the exercise of discretion in compelling expert testimony:

(1) “[T]he degree to which the expert is being called because of his knowledge of facts relevant to the case rather than in order to give opinion testimony”;

(2) “the difference between testifying to a previously formed or expressed opinion and forming a new one”;

(3) “the possibility that, for other reasons, the witness is a unique expert”;

(4) “the extent to which the calling party is able to show the unlikelihood that any comparable witness will willingly testify”; and

(5) “the degree to which the witness is able to show that he has been oppressed by having continually to testify[.]”

2015 WL 220988, at *2-3, citing and quoting Kaufman v. Edelstein, 539 F.2d 811, 822 (2d Cir.1976).

Considering these factors, Judge Hellerstein refused to enforce the subpoena ad testificandum against the Mt. Sinai physicians. Plaintiffs have their own retained expert witness, Dr. Tee Guidotti, who was prepared to testify about both specific and general causation of the plaintiffs’ claimed injuries. The underlying data gathered by Mt. Sinai physicians and scientists, however, were another matter:

“However, it is undisputed that the scope and detail of the research conducted by the Mt. Sinai WTC Health Program is unparalleled. This litigation can only benefit from a full record and the inclusion of the relevant data underlying the research conducted by the Mt. Sinai WTC Health Program is appropriate. Accordingly, Mt. Sinai is ordered to produce all Mt. Sinai WTC Health Program data pursuant to the protocol established in In re World Trade Center Disaster Site Litigation, 21–mc–100, accounting for the redaction of personal identifying information and other sensitive patient material.”

Id. at *4; see also id. at *1.

Cynical observers may wonder whether the Mt. Sinai opposition to the subpoenas and motions to compel was posturing. The hospital and many of its physicians have been outspoken advocates on many occupational and environmental issues. Perhaps like Brer Rabbit, they were protesting not to be thrown in the briar patch. Or maybe, they realized that they could not resist the subpoena for data unless they also declined to testify about their opinions. In any event, Judge Hellerstein maintained the right of expert witnesses to hold their opinions to themselves and to avoid participating in the litigation system, while ensuring that the data are available to all. The plaintiffs may well have been clever by halves in bringing their motion.

Posted in Data Sharing, Expert Witnesses | Comments Off on Plaintiffs Sue Friendly Physicians for Access to Opinions and to Data

Sander Greenland on “The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics”

February 8th, 2015

Sander Greenland is one of the few academics, who has served as an expert witness, who has written post-mortems of his involvement in various litigations[1]. Although settling scores with opposing expert witnesses can be a risky business[2], the practice can provide important insights for judges and lawyers who want to avoid the errors of the past. Greenland correctly senses that many errors seem endlessly recycled, and that courts could benefit from disinterested commentary on cases. And so, there should be a resounding affirmation from federal and state courts to the proclaimed “need for critical appraisal of expert witnesses in epidemiology and statistics,” as well as in many other disciplines.

A recent exchange[3] with Professor Greenland led me to revisit his Wake Forest Law Review article. His article raises some interesting points, some mistaken, but some valuable and thoughtful considerations about how to improve the state of statistical expert witness testimony. For better and worse[4], lawyers who litigate health effects issues should read it.

Other Misunderstandings

Greenland posits criticisms of defense expert witnesses[5], who he believes have misinterpreted or misstated the appropriate inferences to be drawn from null studies. In one instance, Greenland revisits one of his own cases, without any clear acknowledgment that his views were largely rejected.[6] The State of California had declared, pursuant to Proposition 65 ( the Safe Drinking Water and Toxic Enforcement Act of 1986, Health and Safety Code sections 25249.5, et seq.), that the State “knew” that di(2-ethylhexyl)phthalate, or “DEHP” caused cancer. Baxter Healthcare challenged the classification, and according to Greenland, the defense experts erroneously interpreted inclusive studies with evidence supporting a conclusion that DEHP does not cause cancer.

Greenland argues that the Baxter expert’s reference[7] to an IARC working group’s classification of DEHP as “not classifiable as to its carcinogenicity to humans” did not support the expert’s conclusion that DEHP does not cause cancer in human. If Baxter’s expert invoked the IARC working group’s classification for complete exoneration of DEHP, then Greenland’s point is fair enough. In his single-minded attack on Baxter’s expert’s testimony, however, Greenland missed a more important point, which is that the IARC’s determination that DEHP is not classifiable as to carcinogenicity is directly contradictory of California’s epistemic claim to “know” that DEHP causes cancer. And Greenland conveniently omits any discussion that the IARC working group had reclassified DEHP from “possibly carcinogenic” to “not classifiable,” in the light of its conclusion that mechanistic evidence of carcinogenesis in rodents did not pertain to humans.[8] Greenland maintains that Baxter’s experts misrepresented the IARC working group’s conclusion[9], but that conclusion, at the very least, demonstrates that California was on very shaky ground when it declared that it “knew” that DEHP was a carcinogen. California’s semantic gamesmanship over its epistemic claims is at the root of the problem, not a misstep by defense experts in describing inconclusive evidence as exonerative.

Greenland goes on to complain that in litigation over health claims:

“A verdict of ‛uncertain’ is not allowed, yet it is the scientific verdict most often warranted. Elimination of this verdict from an expert’s options leads to the rather perverse practice (illustrated in the DEHP testimony cited above) of applying criminal law standards to risk assessments, as if chemicals were citizens to be presumed innocent until proven guilty.”

39 Wake Forest Law Rev. at 303. Despite Greenland’s alignment with California in the Denton case, the fact of the matter is that a verdict of “uncertain” was allowed, and he was free to criticize California for making a grossly exaggerated epistemic claim on inconclusive evidence.

Perhaps recognizing that he may be readily be seen as an advocate for coming to the defense of California on the DEHP issue, Greenland protests that:

“I am not suggesting that judgments for plaintiffs or actions against chemicals should be taken when evidence is inconclusive.”

39 Wake Forest Law Rev. at 305. And yet, his involvement in the Denton case (as well as other cases, such as silicone gel breast implant cases, thimerosal cases, etc.) suggest that he is willing to lend aid and support to judgments for plaintiffs when the evidence is inconclusive.

Important Advice and Recommendations

These foregoing points are rather severe limitations to Greenland’s article, but lawyers and judges should also look to what is good and helpful here. Greenland is correct to call out expert witnesses, regardless of party of affiliation, who opine that inconclusive studies are “proof” of the null hypothesis. Although some of Greenland’s arguments against the use of significance probability may be overstated, his corrections to the misstatements and misunderstandings of significance probability should command greater attention in the legal community. In one strained passage, however, Greenland uses a disjunction to juxtapose null hypothesis testing with proof beyond a reasonable doubt[10]. Greenland of course understands the difference, but the context would lead some untutored readers to think he has equated the two probabilistic assessments. Writing in a law review for lawyers and judges might have led him to be more careful. Given the prevalence of plaintiffs’ counsel’s confusing the 95% confidence coefficient with a burden of proof akin to beyond a reasonable doubt, great care in this area is, indeed, required.

Despite his appearing for plaintiffs’ counsel in health effects litigation, some of Greenland’s suggestions are balanced and perhaps more truth-promoting than many plaintiffs’ counsel would abide. His article provides an important argument in favor of raising the legal criteria for witnesses who purport to have expertise to address and interpret epidemiologic and experimental evidence[11]. And beyond raising qualification requirements above mere “reasonable pretense at expertise,” Professor Greenland offers some thoughtful, helpful recommendations for improving expert witness testimony in the courts:

“Begin publishing projects in which controversial testimony (a matter of public record) is submitted, and as space allows, published on a regular basis in scientific or law journals, perhaps with commentary. An online version could provide extended excerpts, with additional context.
Give courts the resources and encouragement to hire neutral experts to peer-review expert testimony.
Encourage universities and established scholarly societies (such as AAAS, ASA, APHA, and SER) to conduct workshops on basic epidemiologic and statistical inference for judges and other legal professionals.”

39 Wake Forest Law Rev. at 308.

Each of these three suggestions is valuable and constructive, and worthy of an independent paper. The recommendation of neutral expert witnesses and scholarly tutorials for judges is hardly new. Many defense counsel and judges have argued for them in litigation and in commentary. The first recommendation, of publishing “controversial testimony” is part of the purpose of this blog. There would be great utility to making expert witness testimony, and analysis thereof, more available for didactic purposes. Perhaps the more egregious testimonial adventures should be republished in professional journals, as Greenland suggests. Greenland qualifies his recommendation with “as space allows,” but space is hardly the limiting consideration in the digital age.

Causation

Professor Greenland correctly points out that causal concepts and conclusions are often essentially contested[12], but his argument might well be incorrectly taken for “anything goes.” More helpfully, Greenland argues that various academic ideals should infuse expert witness testimony. He suggests that greater scholarship, with acknowledgment of all viewpoints, and all evidence, is needed in expert witnessing. 39 Wake Forest Law Rev. at 293.

Greenland’s argument provides an important corrective to the rhetoric of Oreskes, Cranor, Michaels, Egilman, and others on “manufacturing doubt”:

“Never force a choice among competing theories; always maintain the option of concluding that more research is needed before a defensible choice can be made.”

Id. Despite his position in the Denton case, and others, Greenland and all expert witnesses are free to maintain that more research is needed before a causal claim can be supported. Greenland also maintains that expert witnesses should “look past” the conclusions drawn by authors, and base their opinions on the “actual data” on which the statistical analyses are based, and from which conclusions have been drawn. Courts have generally rejected this view, but if courts were to insist upon real expertise in epidemiology and statistics, then the testifying expert witnesses should not be constrained by the hearsay opinions in the discussion sections of published studies – sections which by nature are incomplete and tendentious. See “Follow the Data, Not the Discussion” (May 2, 2010).

Greenland urges expert witnesses and legal counsel to be forthcoming about their assumptions, their uncertainty about conclusions:

“Acknowledgment of controversy and uncertainty is a hallmark of good science as well as good policy, but clashes with the very time limited tasks faced by attorneys and courts”

39 Wake Forest Law Rev. at 293-4. This recommendation would be helpful in assuring courts that the data may simply not support conclusions sufficiently certain to be submitted to lay judges and jurors. Rosen v. Ciba-Geigy Corp., 78 F.3d 316, 319, 320 (7th Cir. 1996) (“But the courtroom is not the place for scientific guesswork, even of the inspired sort. Law lags science; it does not lead it.”) (internal citations omitted).

Threats to Validity

One of the serious mistakes counsel often make in health effects litigation is to invite courts to believe that statistical significance is sufficient for causal inferences. Greenland emphasizes that validity considerations often are much stronger, and more important considerations than the play of random error[13]:

“For very imperfect data (e.g., epidemiologic data), the limited conclusions offered by statistics must be further tempered by validity considerations.”

* * * * * *

“Examples of validity problems include non-random distribution of the exposure in question, non-random selection or cooperation of subjects, and errors in assessment of exposure or disease.”

39 Wake Forest Law Rev. at 302 – 03. Greenland’s abbreviated list of threats to validity should remind courts that they cannot sniff a p-value below five percent and then safely kick the can to the jury. The literature on evaluating bias and confounding is huge, but Greenland was a co-author on an important recent paper, which needs to be added to the required reading lists of judges charged with gatekeeping expert witness opinion testimony about health effects. See Timothy L. Lash, et al., “Good practices for quantitative bias analysis,” 43 Internat’l J. Epidem. 1969 (2014).

[1] For an influential example of this sparse genre, see James T. Rosenbaum, “Lessons from litigation over silicone breast implants: A call for activism by scientists,” 276 Science 1524 (1997) (describing the exaggerations, distortions, and misrepresentations of plaintiffs’ expert witnesses in silicone gel breast implant litigation, from perspective of a highly accomplished scientist physician, who served as a defense expert witness, in proceedings before Judge Robert Jones, in Hall v. Baxter Healthcare Corp., 947 F. Supp. 1387 (D. Or. 1996). In one attempt to “correct the record” in the aftermath of a case, Greenland excoriated a defense expert witness, Professor Robert Makuch, for stating that Bayesian methods are rarely used in medicine or in the regulation of medicines. Sander Greenland, “The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics,” 39 Wake Forest Law Rev. 291, 306 (2004). Greenland heaped adjectives upon his adversary, “ludicrous claim,” “disturbing, “misleading expert testimony,” and “demonstrably quite false.” See “The Infrequency of Bayesian Analyses in Non-Forensic Court Decisions” (Feb. 16, 2014) (debunking Prof. Greenland’s claims).

[2] One almost comical example of trying too hard to settle a score occurs in a footnote, where Greenland cites a breast implant case as having been reversed in part by another case in the same appellate court. See 39 Wake Forest Law Rev. at 309 n.68, citing Allison v. McGhan Med. Corp., 184 F.3d 1300, 1310 (11th Cir. 1999), aff’d in part & rev’d in part, United States v. Baxter Int’l, Inc., 345 F.3d 866 (11th Cir. 2003). The subsequent case was not by any stretch of the imagination a reversal of the earlier Allison case; the egregious citation is a legal fantasy. Furthermore, Allison had no connection with the procedures for court-appointed expert witnesses or technical advisors. Perhaps the most charitable interpretation of this footnote is that it was injected by the law review editors or supervisors.

[3] See “Significance Levels are Made a Whipping Boy on Climate Change Evidence: Is .05 Too Strict? (Schachtman on Oreskes)” (Jan. 4, 2015).

[4] In addition to the unfair attack on Professor Makuch, see supra, n.1, there is much that some will find “disturbing,” “misleading,” and even “ludicrous,” (some of Greenland’s favorite pejorative adjectives) in the article. Greenland repeats in brief his arguments against the legal system’s use of probabilities of causation[4], which I have addressed elsewhere.

[5] One of Baxter’s expert witnesses appeared to be the late Professor Patricia Buffler.

[6] See 39 Wake Forest Law Rev. at 294-95, citing Baxter Healthcare Corp. v. Denton, No. 99CS00868, 2002 WL 31600035, at *1 (Cal. App. Dep’t Super. Ct. Oct. 3, 2002) (unpublished); Baxter Healthcare Corp. v. Denton, 120 Cal. App. 4th 333 (2004)

[7] Although Greenland cites to a transcript, the citation is to a judicial opinion, and the actual transcript of testimony is not available at the citation give.

[8] See Denton, supra.

[9] 39 Wake Forest L. Rev. at 297.

[10] 39 Wake Forest L. Rev. at 305 (“If it is necessary to prove causation ‛beyond a reasonable doubt’–or be ‛compelled to give up the null’ – then action can be forestalled forever by focusing on any aspect of available evidence that fails to conform neatly with the causal (alternative) hypothesis. And in medical and social science there is almost always such evidence available, not only because of the ‛play of chance’ (the focus of ordinary statistical theory), but also because of the numerous validity problems in human research.”

[11] See Peter Green, “Letter from the President to the Lord Chancellor regarding the use of statistical evidence in court cases” (Jan. 23, 2002) (writing on behalf of The Royal Statistical Society; “Although many scientists have some familiarity with statistical methods, statistics remains a specialised area. The Society urges you to take steps to ensure that statistical evidence is presented only by appropriately qualified statistical experts, as would be the case for any other form of expert evidence.”).

[12] 39 Wake Forest Law Rev. at 291 (“In reality, there is no universally accepted method for inferring presence or absence of causation from human observational data, nor is there any universally accepted method for inferring probabilities of causation (as courts often desire); there is not even a universally accepted definition of cause or effect.”).

[13] 39 Wake Forest Law Rev. at 302-03 (“If one is more concerned with explaining associations scientifically, rather than with mechanical statistical analysis, evidence about validity can be more important than statistical results.”).

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on Sander Greenland on “The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics”