Schachtman Law » Evidence

TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Daubert Retrospective – Statistical Significance

January 5th, 2019

The holiday break was an opportunity and an excuse to revisit the briefs filed in the Supreme Court by parties and amici, in the Daubert case. The 22 amicus briefs in particular provided a wonderful basis upon which to reflect how far we have come, and also how far we have to go, to achieve real evidence-based fact finding in technical and scientific litigation. Twenty-five years ago, Rules 702 and 703 vied for control over errant and improvident expert witness testimony. With Daubert decided, Rule 702 emerged as the winner. Sadly, most courts seem to ignore or forget about Rule 703, perhaps because of its awkward wording. Rule 702, however, received the judicial imprimatur to support the policing and gatekeeping of dysepistemic claims in the federal courts.

As noted last week,¹ the petitioners (plaintiffs) in Daubert advanced several lines of fallacious and specious argument, some of which was lost in the shuffle and page limitations of the Supreme Court briefings. The plaintiffs’ transposition fallacy received barely a mention, although it did bring forth at least a footnote in an important and overlooked amicus brief filed by American Medical Association (AMA), the American College of Physicians, and over a dozen other medical specialty organizations,² all of which both emphasized the importance of statistical significance in interpreting epidemiologic studies, and the fallacy of interpreting 95% confidence intervals as providing a measure of certainty about the estimated association as a parameter. The language of these associations’ amicus brief is noteworthy and still relevant to today’s controversies.

The AMA’s amicus brief, like the brief filed by the National Academies of Science and the American Association for the Advancement of Science, strongly endorsed a gatekeeping role for trial courts to exclude testimony not based upon rigorous scientific analysis:

“The touchstone of Rule 702 is scientific knowledge. Under this Rule, expert scientific testimony must adhere to the recognized standards of good scientific methodology including rigorous analysis, accurate and statistically significant measurement, and reproducibility.”³

Having incorporated the term “scientific knowledge,” Rule 702 could not permit anything less in expert witness testimony, lest it pollute federal courtrooms across the land.

Elsewhere, the AMA elaborated upon its reference to “statistically significant measurement”:

“Medical researchers acquire scientific knowledge through laboratory investigation, studies of animal models, human trials, and epidemiological studies. Such empirical investigations frequently demonstrate some correlation between the intervention studied and the hypothesized result. However, the demonstration of a correlation does not prove the hypothesized result and does not constitute scientific knowledge. In order to determine whether the observed correlation is indicative of a causal relationship, scientists necessarily rely on the concept of “statistical significance.” The requirement of statistical reliability, which tends to prove that the relationship is not merely the product of chance, is a fundamental and indispensable component of valid scientific methodology.”⁴

And then again, the AMA spelled out its position, in case the Court missed its other references to the importance of statistical significance:

“Medical studies, whether clinical trials or epidemiologic studies, frequently demonstrate some correlation between the action studied … . To determine whether the observed correlation is not due to chance, medical scientists rely on the concept of ‘statistical significance’. A ‘statistically significant’ correlation is generally considered to be one in which statistical analysis suggests that the observed relationship is not the result of chance. A statistically significant correlation does not ‘prove’ causation, but in the absence of such a correlation, scientific causation clearly is not proven.⁹”⁵

In its footnote 9, in the above quoted section of the brief, the AMA called out the plaintiffs’ transposition fallacy, without specifically citing to plaintiffs’ briefs:

“It is misleading to compare the 95% confidence level used in empirical research to the 51% level inherent in the preponderance of the evidence standard.”⁶

Actually the plaintiffs’ ruse was much worse than misleading. The plaintiffs did not compare the two probabilities; they equated them. Some might call this ruse, an outright fraud on the court. In any event, the AMA amicus brief remains an available, citable source for opposing this fraud and the casual dismissal of the importance of statistical significance.

One other amicus brief touched on the plaintiffs’ statistical shanigans. The Product Liability Advisory Council, National Association of Manufacturers, Business Roundtable, and Chemical Manufacturers Association jointly filed an amicus brief to challenge some of the excesses of the plaintiffs’ submissions.⁷ Plaintiffs’ expert witness, Shanna Swan, had calculated type II error rates and post-hoc power for some selected epidemiologic studies relied upon by the defense. Swan’s complaint had been that some studies had only 20% probability (power) to detect a statistically significant doubling of limb reduction risk, with significance at p < 5%.⁸

The PLAC Brief pointed out that power calculations must assume an alternative hypothesis, and that the doubling of risk hypothesis had no basis in the evidentiary record. Although the PLAC complaint was correct, it missed the plaintiffs’ point that the defense had set exceeding a risk ratio of 2.0, as an important benchmark for specific causation attributability. Swan’s calculation of post-hoc power would have yielded an even lower probability for detecting risk ratios of 1.2 or so. More to the point, PLAC noted that other studies had much greater power, and that collectively, all the available studies would have had much greater power to have at least one study achieve statistical significance without dodgy re-analyses.

1 “The Advocates’ Errors in Daubert” (Dec. 28, 2018).

2 American Academy of Allergy and Immunology, American Academy of Dermatology, American Academy of Family Physicians, American Academy of Neurology, American Academy of Orthopaedic Surgeons, American Academy of Pain Medicine, American Association of Neurological Surgeons, American College of Obstetricians and Gynecologists, American College of Pain Medicine, American College of Physicians, American College of Radiology, American Society of Anesthesiologists, American Society of Plastic and Reconstructive Surgeons, American Urological Association, and College of American Pathologists.

3 Brief of the American Medical Association, et al., as Amici Curiae, in Support of Respondent, in Daubert v. Merrell Dow Pharmaceuticals, Inc., U.S. Supreme Court no. 92-102, 1993 WL 13006285, at *27 (U.S., Jan. 19, 1993)[AMA Brief].

4 AMA Brief at *4-*5 (emphasis added).

5 AMA Brief at *14-*15 (emphasis added).

6 AMA Brief at *15 & n.9.

7 Brief of the Product Liability Advisory Council, Inc., National Association of Manufacturers, Business Roundtable, and Chemical Manufacturers Association as Amici Curiae in Support of Respondent, as Amici Curiae, in Support of Respondent, in Daubert v. Merrell Dow Pharmaceuticals, Inc., U.S. Supreme Court no. 92-102, 1993 WL 13006288 (U.S., Jan. 19, 1993) [PLAC Brief].

8 PLAC Brief at *21.

Posted in Evidence, Expert Witnesses, Rule 702, Rule 703, statistical evidence, Statistical Power | Comments Off on Daubert Retrospective – Statistical Significance

The Judicial Labyrinth for Scientific Evidence

October 3rd, 2018

The real Daedalus (not the musician), as every school child knows, was the creator of the Cretan Labyrinth, where the Minotaur resided. The Labyrinth had been the undoing of many Greeks and barbarians, until an Athenian, Theseus, took up the challenge of slaying the Minotaur. With the help of Ariadne’s thread, Theseus solved the labyrinthic puzzle and slayed the Minotaur.

Theseus and the Minotaur on 6th-century black-figure pottery (Wikimedia Commons 2005)

Dædalus is also the Journal of the American Academy of Arts and Sciences. The Academy has been, for over 230 years, addressing issues issues in both the humanities and in the sciences. In the fall 2018 issue of Dædalus (volume 147, No. 4), the Academy has published a dozen essays by noted scholars in the field, who report on the murky interface of science and law in the courtrooms of the United States. Several of the essays focus on sorry state of forensic “science” in the criminal justice system, which has been the subject of several critical official investigations, only to be dismissed and downplayed by both the Obama and Trump administrations. Other essays address the equally sorry state of judicial gatekeeping in civil actions, with some limited suggestions on how the process of scientific fact finding might be improved. In any event, this issue, “Science & the Legal System,” is worth reading even if you do not agree with the diagnoses or the proposed therapies. There is still room for a collaboration between a modern day Daedalus and Ariadne to help us find the way out of this labyrinth.

Introduction

Shari Seidman Diamond & Richard O. Lempert, “Introduction” (pp. 5–14)

Connecting Science and Law

Sheila Jasanoff, “Science, Common Sense & Judicial Power in U.S. Courts” (pp. 15-27)

Linda Greenhouse, “The Supreme Court & Science: A Case in Point,” (pp. 28–40)

Shari Seidman Diamond & Richard O. Lempert, “When Law Calls, Does Science Answer? A Survey of Distinguished Scientists & Engineers,” (pp. 41–60)

Accomodation or Collision: When Science and Law Meet

Jules Lobel & Huda Akil, “Law & Neuroscience: The Case of Solitary Confinement,” (pp. 61–75)

Rebecca S. Eisenberg & Robert Cook-Deegan, “Universities: The Fallen Angels of Bayh-Dole?” (pp. 76–89)

Jed S. Rakoff & Elizabeth F. Loftus, “The Intractability of Inaccurate Eyewitness Identification” (pp. 90–98)

Jennifer L. Mnookin, “The Uncertain Future of Forensic Science” (pp. 99–118)

Joseph B. Kadane and Jonathan J. Koehler, “Certainty & Uncertainty in Reporting Fingerprint Evidence” (pp. 119–134)

Communicating Science in Court

Nancy Gertner & Joseph Sanders, “Alternatives to Traditional Adversary Methods of Presenting Scientific Expertise in the Legal System” (pp. 135–151)

Daniel L. Rubinfeld & Joe S. Cecil, “Scientists as Experts Serving the Court” (pp. 152–163)

Valerie P. Hans and Michael J. Saks, “Improving Judge & Jury Evaluation of Scientific Evidence” (pp. 164–180)

Continuing the Dialogue

David Baltimore, David S. Tatel & Anne-Marie Mazza, “Bridging the Science-Law Divide” (pp. 181–194)

Posted in Evidence, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on The Judicial Labyrinth for Scientific Evidence

N.J. Supreme Court Uproots Weeds in Garden State’s Law of Expert Witnesses

August 8th, 2018

The United States Supreme Court’s decision in Daubert is now over 25 years old. The idea of judicial gatekeeping of expert witness opinion testimony is even older in New Jersey state courts. The New Jersey Supreme Court articulated a reliability standard before the Daubert case was even argued in Washington, D.C. See Landrigan v. Celotex Corp., 127 N.J. 404, 414 (1992); Rubanick v. Witco Chem. Corp., 125 N.J. 421, 447 (1991). Articulating a standard, however, is something very different from following a standard, and in many New Jersey trial courts, until very recently, the standard was pretty much anything goes.

One counter-example to the general rule of dog-eat-dog in New Jersey was Judge Nelson Johnson’s careful review and analysis of the proffered causation opinions in cases in which plaintiffs claimed that their use of the anti-acne medication isotretinoin (Accutane) caused Crohn’s disease. Judge Johnson, who sits in the Law Division of the New Jersey Superior Court for Atlantic County held a lengthy hearing, and reviewed the expert witnesses’ reliance materials.¹ Judge Johnson found that the plaintiffs’ expert witnesses had employed undue selectivity in choosing what to rely upon. Perhaps even more concerning, Judge Johnson found that these witnesses had refused to rely upon reasonably well-conducted epidemiologic studies, while embracing unpublished, incomplete, and poorly conducted studies and anecdotal evidence. In re Accutane, No. 271(MCL), 2015 WL 753674, 2015 BL 59277 (N.J.Super. Law Div., Atlantic Cty. Feb. 20, 2015). In response, Judge Johnson politely but firmly closed the gate to conclusion-driven duplicitous expert witness causation opinions in over 2,000 personal injury cases. “Johnson of Accutane – Keeping the Gate in the Garden State” (Mar. 28, 2015).

Aside from resolving over 2,000 pending cases, Judge Johnson’s judgment was of intense interest to all who are involved in pharmaceutical and other products liability litigation. Judge Johnson had conducted a pretrial hearing, sometimes called a Kemp hearing in New Jersey, after the New Jersey Supreme Court’s opinion in Kemp v. The State of New Jersey, 174 N.J. 412 (2002). At the hearing and in his opinion that excluded plaintiffs’ expert witnesses’ causation opinions, Judge Johnson demonstrated a remarkable aptitude for analyzing data and inferences in the gatekeeping process.

When the courtroom din quieted, the trial court ruled that the proffered testimony of Dr., Arthur Kornbluth and Dr. David Madigan did not meet the liberal New Jersey test for admissibility. In re Accutane, No. 271(MCL), 2015 WL 753674, 2015 BL 59277 (N.J.Super. Law Div. Atlantic Cty. Feb. 20, 2015). And in closing the gate, Judge Johnson protected the judicial process from several bogus and misleading “lines of evidence,” which have become standard ploys to mislead juries in courthouses where the gatekeepers are asleep. Recognizing that not all evidence is on the same analytical plane, Judge Johnson gave case reports short shrift.

“[u]nsystematic clinical observations or case reports and adverse event reports are at the bottom of the evidence hierarchy.”

Id. at *16. Adverse event reports, largely driven by the very litigation in his courtroom, received little credit and were labeled as “not evidentiary in a court of law.” Id. at 14 (quoting FDA’s description of FAERS).

Judge Johnson recognized that there was a wide range of identified “risk factors” for irritable bowel syndrome, such as prior appendectomy, breast-feeding as an infant, stress, Vitamin D deficiency, tobacco or alcohol use, refined sugars, dietary animal fat, fast food. In re Accutane, 2015 WL 753674, at *9. The court also noted that there were four medications generally acknowledged to be potential risk factors for inflammatory bowel disease: aspirin, nonsteroidal anti-inflammatory medications (NSAIDs), oral contraceptives, and antibiotics. Understandably, Judge Johnson was concerned that the plaintiffs’ expert witnesses preferred studies unadjusted for potential confounding co-variables and studies that had involved “cherry picking the subjects.” Id. at *18.

Judge Johnson had found that both sides in the isotretinoin cases conceded the relative unimportance of animal studies, but the plaintiffs’ expert witnesses nonetheless invoked the animal studies in the face of the artificial absence of epidemiologic studies that had been created by their cherry-picking strategies. Id.

Plaintiffs’ expert witnesses had reprised a common claimants’ strategy; namely, they claimed that all the epidemiology studies lacked statistical power. Their arguments often ignored that statistical power calculations depend upon statistical significance, a concept to which many plaintiffs’ counsel have virulent antibodies, as well as an arbitrarily selected alternative hypothesis of association size. Furthermore, the plaintiffs’ arguments ignored the actual point estimates, most of which were favorable to the defense, and the observed confidence intervals, most of which were reasonably narrow.

The defense responded to the bogus statistical arguments by presenting an extremely capable clinical and statistical expert witness, Dr. Stephen Goodman, to present a meta-analysis of the available epidemiologic evidence.

Meta-analysis has become an important facet of pharmaceutical and other products liability litigation[1]. Fortunately for Judge Johnson, he had before him an extremely capable expert witness, Dr. Stephen Goodman, to explain meta-analysis generally, and two meta-analyses he had performed on isotretinoin and irritable bowel outcomes.

Dr. Goodman explained that the plaintiffs’ witnesses’ failure to perform a meta-analysis was telling when meta-analysis can obviate the plaintiffs’ hyperbolic statistical complaints:

“the strength of the meta-analysis is that no one feature, no one study, is determinant. You don’t throw out evidence except when you absolutely have to.”

In re Accutane, 2015 WL 753674, at *8.

Judge Johnson’s judicial handiwork received non-deferential appellate review from a three-judge panel of the Appellate Division, which reversed the exclusion of Kornbluth and Madigan. In re Accutane Litig., 451 N.J. Super. 153, 165 A.3d 832 (App. Div. 2017). The New Jersey Supreme Court granted the isotretinoin defendants’ petition for appellate review, and the issues were joined over the appropriate standard of appellate review for expert witness opinion exclusions, and the appropriateness of Judge Johnson’s exclusions of Kornbluth and Madigan. A bevy of amici curiae joined in the fray.²

Last week, the New Jersey Supreme Court issued a unanimous opinion, which reversed the Appellate Division’s holding that Judge Johnson had “mistakenly exercised” discretion. Applying its own precedents from Rubanick, Landrigan, and Kemp, and the established abuse-of-discretion standard, the Court concluded that the trial court’s ruling to exclude Kornbluth and Madigan was “unassailable.” In re Accutane Litig., ___ N.J. ___, 2018 WL 3636867 (2018), Slip op. at 79.³

The high court graciously acknowledged that defendants and amici had “good reason” to seek clarification of New Jersey law. Slip op. at 67. In abandoning abuse-of-discretion as its standard of review, the Appellate Division had relied upon a criminal case that involved the application of the Frye standard, which is applied as a matter of law. Id. at 70-71. The high court also appeared to welcome the opportunity to grant review and reverse the intermediate court reinforce “the rigor expected of the trial court” in its gatekeeping role. Id. at 67. The Supreme Court, however, did not articulate a new standard; rather it demonstrated at length that Judge Johnson had appropriately applied the legal standards that had been previously announced in New Jersey Supreme Court cases.⁴

In attempting to defend the Appellate Division’s decision, plaintiffs sought to characterize New Jersey law as somehow different from, and more “liberal” than, the United States Supreme Court’s decision in Daubert. The New Jersey Supreme Court acknowledged that it had never formally adopted the dicta from Daubert about factors that could be considered in gatekeeping, slip op. at 10, but the Court went on to note what disinterested observers had long understood, that the so-called Daubert factors simply flowed from a requirement of sound methodology, and that there was “little distinction” and “not much light” between the Landrigan and Rubanick principles and the Daubert case or its progeny. Id at 10, 80.

Curiously, the New Jersey Supreme Court announced that the Daubert factors should be incorporated into the New Jersey Rules 702 and 703 and their case law, but it stopped short of declaring New Jersey a “Daubert” jurisdiction. Slip op. at 82. In part, the Court’s hesitance followed from New Jersey’s bifurcation of expert witness standards for civil and criminal cases, with the Frye standard still controlling in the criminal docket. At another level, it makes no sense to describe any jurisdiction as a “Daubert” state because the relevant aspects of the Daubert decision were dicta, and the Daubert decision and its progeny were superseded by the revision of the controlling statute in 2000.⁵

There were other remarkable aspects of the Supreme Court’s Accutane decision. For instance, the Court put its weight behind the common-sense and accurate interpretation of Sir Austin Bradford Hill’s famous articulation of factors for causal judgment, which requires that sampling error, bias, and confounding be eliminated before assessing whether the observed association is strong, consistent, plausible, and the like. Slip op. at 20 (citing the Reference Manual at 597-99), 78.

The Supreme Court relied extensively on the National Academies’ Reference Manual on Scientific Evidence.⁶ That reliance is certainly preferable to judicial speculations and fabulations of scientific method. The reliance is also positive, considering that the Court did not look only at the problematic epidemiology chapter, but adverted also to the chapters on statistical evidence and on clinical medicine.

The Supreme Court recognized that the Appellate Division had essentially sanctioned an anything goes abandonment of gatekeeping, an approach that has been all-too-common in some of New Jersey’s lower courts. Contrary to the previously prevailing New Jersey zeitgeist, the Court instructed that gatekeeping must be “rigorous” to “prevent[] the jury’s exposure to unsound science through the compelling voice of an expert.” Slip op. at 68-9.

Not all evidence is equal. “[C]ase reports are at the bottom of the evidence hierarchy.” Slip op. at 73. Extrapolation from non-human animal studies is fraught with external validity problems, and such studies “far less probative in the face of a substantial body of epidemiologic evidence.” Id. at 74 (internal quotations omitted).

Perhaps most chilling for the lawsuit industry will be the Supreme Court’s strident denunciation of expert witnesses’ selectivity in choosing lesser evidence in the face of a large body of epidemiologic evidence, id. at 77, and their unprincipled cherry picking among the extant epidemiologic publications. Like the trial court, the Supreme Court found that the plaintiffs’ expert witnesses’ inconsistent use of methodological criteria and their selective reliance upon studies (disregarding eight of the nine epidemiologic studies) that favored their task masters was the antithesis of sound methodology. Id. at 73, citing with approval, In re Lipitor, ___ F.3d ___ (4th Cir. 2018) (slip op. at 16) (“Result-driven analysis, or cherry-picking, undermines principles of the scientific method and is a quintessential example of applying methodologies (valid or otherwise) in an unreliable fashion.”).

An essential feature of the Supreme Court’s decision is that it was not willing to engage in the common reductionism that has “all epidemiologic studies are flawed,” and which thus privileges cherry picking. Not all disagreements between expert witnesses can be framed as differences in interpretation. In re Accutane will likely stand as a bulwark against flawed expert witness opinion testimony in the Garden State for a long time.

1 Judge Nelson Johnson is also the author of Boardwalk Empire: The Birth, High Times, and Corruption of Atlantic City (2010), a spell-binding historical novel about political and personal corruption.

2 In support of the defendants’ positions, amicus briefs were filed by the New Jersey Business & Industry Association, Commerce and Industry Association of New Jersey, and New Jersey Chamber of Commerce; by law professors Kenneth S. Broun, Daniel J. Capra, Joanne A. Epps, David L. Faigman, Laird Kirkpatrick, Michael M. Martin, Liesa Richter, and Stephen A. Saltzburg; by medical associations the American Medical Association, Medical Society of New Jersey, American Academy of Dermatology, Society for Investigative Dermatology, American Acne and Rosacea Society, and Dermatological Society of New Jersey, by the Defense Research Institute; by the Pharmaceutical Research and Manufacturers of America; and by New Jersey Civil Justice Institute. In support of the plaintiffs’ position and the intermediate appellate court’s determination, amicus briefs were filed by political action committee the New Jersey Association for Justice; by the Ironbound Community Corporation; and by plaintiffs’ lawyer Allan Kanner.

3 Nothing in the intervening scientific record called question upon Judge Johnson’s trial court judgment. See, e.g., I.A. Vallerand, R.T. Lewinson, M.S. Farris, C.D. Sibley, M.L. Ramien, A.G.M. Bulloch, and S.B. Patten, “Efﬁcacy and adverse events of oral isotretinoin for acne: a systematic review,” 178 Brit. J. Dermatol. 76 (2018).

4 Slip op. at 9, 14-15, citing Landrigan v. Celotex Corp., 127 N.J. 404, 414 (1992); Rubanick v. Witco Chem. Corp., 125 N.J. 421, 447 (1991) (“We initially took that step to allow the parties in toxic tort civil matters to present novel scientific evidence of causation if, after the trial court engages in rigorous gatekeeping when reviewing for reliability, the proponent persuades the court of the soundness of the expert’s reasoning.”).

5 The Court did acknowledge that Federal Rule of Evidence 702 had been amended in 2000, to reflect the Supreme Court’s decision in Daubert, Joiner, and Kumho Tire, but the Court did not deal with the inconsistencies between the present rule and the 1993 Daubert case. Slip op. at 64, citing Calhoun v. Yamaha Motor Corp., U.S.A., 350 F.3d 316, 320-21, 320 n.8 (3d Cir. 2003).

6 See Accutane slip op. at 12-18, 24, 73-74, 77-78. With respect to meta-analysis, the Reference Manual’s epidemiology chapter is still stuck in the 1980s and the prevalent resistance to poorly conducted, often meaningless meta-analyses. See “The Treatment of Meta-Analysis in the Third Edition of the Reference Manual on Scientific Evidence” (Nov. 14, 2011) (The Reference Manual fails to come to grips with the prevalence and importance of meta-analysis in litigation, and fails to provide meaningful guidance to trial judges).

Posted in Evidence, Expert Witnesses, Frye, Meta-analysis, Reference Manual on Scientific Evidence, Rule 702, Scientific Evidence, statistical evidence, Statistical Power | Comments Off on N.J. Supreme Court Uproots Weeds in Garden State’s Law of Expert Witnesses

P-Values: Pernicious or Perspicacious?

May 12th, 2018

Professor Kingsley R. Browne, of the Wayne State University Law School, recently published a paper that criticized the use of p-values and significance testing in discrimination litigation. Kingsley R. Browne, “Pernicious P-Values: Statistical Proof of Not Very Much,” 42 Univ. Dayton L. Rev. 113 (2017) (cited below as Browne). Browne amply documents the obvious and undeniable, that judges, lawyers, and even some ill-trained expert witnesses, are congenitally unable to describe and interpret p-values properly. Most of Browne’s examples are from the world of anti-discrimination law, but he also cites a few from health effects litigation as well. Browne also cites from many of the criticisms of p-values in the psychology and other social science literature.

Browne’s efforts to correct judicial innumeracy are welcomed, but they take a peculiar turn in this law review article. From the well-known state of affairs of widespread judicial refusal or inability to discuss statistical concepts accurately, Browne argues for what seem to be two incongruous, inconsistent responses. Rejecting the glib suggestion of former Judge Posner that evidence law is not “fussy” about evidence, Browne argues that federal evidence law requires courts to be “fussy” about evidence, and that Rule 702 requires courts to exclude expert witnesses, whose opinions fail to “employ[] in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Browne at 143 (quoting from Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152 (1999). Browne tells us, with apparently appropriate intellectual rigor, that “[i]f a disparity that does not provide a p-value of less than 0.05 would not be accepted as meaningful in the expert’s discipline, it is not clear that the expert should be allowed to testify – on the basis of his expertise in that discipline – that the disparity is, in fact, meaningful.” Id.

In a volte face, Browne then argues that p-values do “not tell us much,” basically because they are dependent upon sample size. Browne suggests that the quantitative disparity between expected value and observed proportion or average can be assessed without the use of p-values, and that measuring a p-value “adds virtually nothing and just muddies the water.” Id. at 152. The prevalent confusion among judges and lawyers seems sufficient in Browne’s view to justify his proposal, as well as his further suggestion that Rule 403 should be invoked to exclude p-values:

“The ease with which reported p-values cause a trier of fact to slip into the transposition fallacy and the difficulty of avoiding that lapse of logic, coupled with the relatively sparse information actually provided by the p-value, make p-values prime candidates for exclusion under Federal Rule of Evidence 403. *** If judges, not to mention the statistical experts they rely on, cannot use the information without falling into fallacious reasoning, the likelihood that the jury will misunderstand the evidence is very high. Since the p-value actually provides little useful relevant information, the high risk of misleading the jury greatly exceeds its scant probative value, so it simply should not be presented to the jury.”

Id. at 152-53.

And yet, elsewhere in the same article, Browne ridicules one court and several expert witnesses who have argued in favor of conclusions that were based upon p-values up to 50%.¹ The concept of p-values cannot be so flexible as to straddle the extremes of having no probative value, and yet capable of rendering an expert witness’s opinions ludicrous. P-values quantify an estimate of random error, even if that error rate varies with sample size. To be sure, the measure of random error depends upon the specified model and assumption of a null hypothesis, but the crucial point is that the estimate (whether mean, proportion, risk ratio, risk difference, etc.) is rather meaningless without some further estimate of random variability of the estimate. Of course, random error is not the only type of error, but the existence of other potential systematic errors is hardly a reason to ignore random error.

In the science of health effects, many applications of p-values have given way to the use of confidence intervals, which arguably provide more direct assessments of both sample estimates, along with ranges of potential outcomes that are reasonably compatible with the sample estimates. Remarkably, Browne never substantively discusses confidence intervals in his article.

Under the heading of other problems with p-values and significance testing, Browne advances four additional putative problems with p-values. First, Browne asserts with little to no support that “[t]he null hypothesis is unlikely a priori.” Id. at 155. He fails to tell us why the null hypothesis of no disparity is not a reasonable starting place in the absence of objective evidence of a prior estimate. Furthermore, a null hypothesis of no difference will have legal significance in claims of health effects, or of unlawful discrimination.

Second, Browne argues that significance testing will lead to “[c]onflation of statistical and practical (or legal) significance” in the minds of judges and jurors. Id. at 156-58. This charge is difficult to sustain. The actors in legal cases can probably best appreciate practical significance and its separation from statistical significance, most readily. If a large class action showed that the expected value of a minority’s proportion was 15%, and the observed proportion was 14.8%, p < 0.05, most innumerate judges and jurors would sense that this disparity was unimportant and that no employer would fine tune its discriminatory activities so closely as to achieve such a meaningless difference.

Third, Browne reminds us that the validity and the interpretation of a p-value turns on the assumption that the statistical model is perfectly specified. Id. at 158-59. His reminder is correct, but again, this aspect of p-values (or confidence intervals) is relatively easy to explain, as well as to defend or challenge. To be sure, there may be legitimate disputes about whether an appropriate model was used (say binomial versus hypergeometric), but such disputes are hardly the most arcane issues that judges and jurors will face.

Fourth, Browne claims that “the alternative hypothesis is seldom properly specified.” Id. at 159-62. Unless analysts are focused on measuring pre-test power or type II error, however, they need not advance an alternative hypothesis. Furthermore, it is hardly a flaw with significance testing that it does not account for systematic bias or confounding.

Browne does not offer an affirmative response such as urging courts to adopt a Bayesian program. A Bayesian response to prevalent blunders in interpreting statistical significance would introduce perhaps even more arcane and hard-to-discern blunders in court proceedings. Browne also leaves courts without a meaningful approach to evaluate random error other than to engage in crude comparisons between two means or proportions. The recommendations in this law review article appear to be a giant step, backwards, into an epistemic void.

1See Browne at 146, citing In re Photochromic Lens Antitrust Litig., 2014 WL 1338605 (M.D. Fla. April 3, 2014) (reversing magistrate judge’s exclusion of an expert witness who had advanced claims based upon p-value of 0.50); id. at 147 n. 116, citing In re High-Tech Employee Antitrust Litig., 2014 WL 1351040 (N.D. Cal. 2014).

Posted in Evidence, Rule 702, statistical evidence | Comments Off on P-Values: Pernicious or Perspicacious?

Johnson & Johnson Leaves Them in the Dust – Echeverria Verdict Unraveled

October 24th, 2017

It was a tough week for the talc litigation industry. On October 17, the Missouri Court of Appeals reversed a large verdict for plaintiffs because a St. Louis trial court unconstitutionally had asserted personal jurisdiction over Johnson & Johnson. In essence, the Missouri appellate court just said no to forum shopping. Fox v. Johnson & Johnson, Mo. Ct. App., No. ED104580 (Oct. 17, 2017). And on Friday, October 20, a California trial court, on sober second thought, granted judgment notwithstanding the verdict, and in the alternative, a new trial in the recent Escheverria case, which had resulted in plaintiffs’ awards approaching half a billion dollars. See Orders regarding Defendants Combined Motion for New Trial and Judgment Notwithstanding the Verdict, Echeverria v. Johnson & Johnson, Inc., Case No. BC628228, JCCP No. 4872, Calif. Super. Ct., Los Angeles Cty. (Oct. 20, 2017) [cited below as Echeverria op.] See also Daniel Siegal, “J&J Wins Battle Against $417M Talc Award, But War Not Over,” Law360 (Oct. 23, 2017).

The trial court issued an opinion, over 50 pages long, which carefully reviewed the parties’ contentions. Only some of the issues considered by the trial court are discussed below.

Differential Etiology

Differential etiology resembles the biological process of solid waste management; both employ the process of elimination.

Most diseases in humans have a large “idiopathic” or “cause unknown” component. The differential methodology purports to take all the known causes and rule out the ones that are improbable in a given case. As a matter of logic, this is what is known as an iterative disjunctive syllogism. If you start with:

A or B or C.

And you show not B;

and then, not C.

you are left with A.

This argument is, of course, a perfectly valid syllogism. If the premises are true, then the conclusion must be true. The problem is that the initial premise, to be accurate for many if not most human chronic diseases, must include a disjunct, U, or “cause unknown.” And once U is added to the first line of the syllogism, rarely is there a way to exclude it.

Sometimes the “cause unknown” component may be very small. For instance, in human malignant mesothelioma, the overwhelming majority of occupational cases do have a known cause: amphibole asbestos. When sufficient amphibole asbestos fiber exposure has been shown, there is usually no serious issue of individual attribution left for debate. The base rate of (idiopathic) mesothelioma is very low, and the relative risk from occupational amphibole asbestos exposure is extraordinarily large.

Ovarian cancer, which is the subject of the Escheverria case, is a very different story. The rate of idiopathic cases – no known causes – is much higher, and may even make up a majority of cases. The so-called differential etiology method never gets down to a conclusion that it is the talc (assuming arguendo that talc causes ovarian cancer). You always have talc or unknown cause in the conclusion.

In Escheverria, the plaintiffs’ lawyers called only one expert witness on specific causation, Echeverria’s treating physician, Dr. Annie Yessaian (“Yessaian”). Yessaian advanced a “differential etiology” analysis, which she claimed allowed her to conclude that talc was “more probable than not” a cause of plaintiff’s ovarian cancer. Echeverria op. at 5. Upon careful review, the trial court realized that Yessaian had never properly applied the iterative disjunctive syllogism, or differential etiology, to reach a valid conclusion. Despite a good deal of hand waving, Yessaian never ruled out other causes of the plaintiff’s ovarian cancer. Echeverria op. at 30.

The plaintiff’s menarche was at age 11, and so she had had a large number of ovulatory cycles. She was obese, and over 60 years old at the time of diagnosis. Yessaian did not rule these factors out; rather she testified without foundation that these factors were “less likely than not” causes of plaintiff’s ovarian cancer¹. Echeverria op. at 31. The trial court noted that these potential causes had never been eliminated from the list of differentials; Yessaian had simply “discounted” them by ipse dixit. As for the “U,” or unknown causes that are clearly at play in many if not most ovarian cancers, Yessaian admitted that Escheverria’s cancer “probably” resulted from some unknown risk factor; but then, out of thin air, she testified that the probability of idiopathic causation was less than 50%. The trial court concluded that Yessian’s ruling in and ruling out decisions were ultimately nothing more than conjecture, and the plaintiff had never properly shown specific causation. Id. at 26-27, 31.

Relative Risk Less than Two

Yessaian’s specific causation opinion cratered further as a result of her inability to identify any specific biomarker or “fingerprint” of causation. The plaintiffs’ expert witnesses had argued that chronic inflammation is the mechanism by which talc causes ovarian cancer, but there was no histopathologic evidence of inflammation in association with ovarian tissue that had given rise to the cancer.

The relative risk argument is one way to attribute specific causation, and circumvent idiopathic causes by quantifying the contribution of the specific causal factor (again assuming it really is such) vis-a-vis the baseline risk of disease from unknown causes. The plaintiff, however, had called an expert witness on epidemiology, Jack Siemiatycki, who had explained that a risk ratio of 2.0 is “the point at which the probability of causation, which is the probability that a given agent causes a specific disease, exceeds 50 percent ….” Escheverria op. at 5. The defense epidemiologic expert, Dr. Douglas Weed, similarly testified and elaborated on the concept of probability of causation and attributable risk.²

The plaintiffs’ counsel attempted to extricate themselves from this arithmetic quagmire by arguing that there was “multiple causation,” and interaction among causes. Escheverria op. at 41-42. Yessaian, however, had disavowed even the most obvious concurrent causes (ovulatory cycles and age), and put all her markers down on talc. There was no evidence of multiple causation to muck up the analysis. Of course, the talc epidemiologic studies were all multivariate analyses that measured associations of talc and ovarian cancer in the presence of co-variates, such as age at menarche, and age at diagnosis.

Furthermore, Yessian was constrained by her acknowledgement that histologic type of ovarian cancer is highly relevant, and that none of the studies of serous ovaran cancer (the type diagnosed in Ms. Escheverria) reported out risk ratios in excess of 2.0. Escheverria op. at 28-29. Yessaian could not escape the inexorable math, and testimony about probability of causation from Jack Siemiatycki. Id at 29.³

Their case in extremis, the plaintiffs’ counsel argued⁴ that epidemiologic studies were not needed to prove causation, which might be true in a case involving a known mechanism with highly specific biomarkers to identify the causal mechanism that had taken place in the claimant. Having cited and relied extensively upon epidemiologic studies, Yessaian was hoisted with own her petard; the trial court found the assertion that there was an alternative path to specific causation to be absent from the record and quite incredible.

State of the Art

The duty to warn is constrained by what is known or should have been known at the time of marketing, what lawyers sometimes call “state of the art.” The trial court reasoned that since Eva Echeverria developed her serous ovarian cancer in 2007, the relevant scientific state of knowledge was censored at the time of plaintiff’s diagnosis. Any warning given after 2007 could not have prevented plaintiffs’ disease. (In truth, the relevant censoring date was likely well before 2007, but an earlier date would not have made a difference in the judicial outcome.)

There was no serious claim that the defendants had “secret” knowledge other than what was known in the scientific community. Plaintiffs’ expert witness on epidemiology, Jack Siemiatycki, co-chaired the IARC working group that concluded and published in 2007, that talc was a possible cause of ovarian cancer, a finding that rejected a higher classification, such as “probable” or “known.” IARC Monograph for Carbon Black, Titanium Dioxide & Talc, vol 93 (2010); Robert Baan, et al., “Carcinogenicity of carbon black, titanium dioxide, and talc,” 7 Lancet Oncology 295 (2006)⁵. In Escheverria, Siemiatycki testified in accordance with his public scientific work, and his service on the IARC working group, and he conceded that in 2007, there was no known causal connection between talc and human ovarian cancer. Notably, the defense lawyers failed to convert this state-of-the-art issue into a dispositive judgment because they had failed to ask for a binding jury instruction on the issue. Escheverria op. at 32.

For the trial court, the absence of scientific knowledge up to and including 2007, the year of Escheverria’s diagnosis, was also relevant to the existence vel non of malice that would support the imposition of punitive damages. Looking at the evidence in the light most favorable to the plaintiff, the trial court found that there was a scientific debate whether talc causes ovarian cancer, which debate would not allow the imputation of scienter to the defendants to permit the jury to infer that the defendants had acted with malice. Escheverria op. at 35. Given that no one in the medical or scientific community had asserted a relevant causal conclusion in or before 2007, the trial court’s conclusion is unassailable. The court’s analysis, however, begs the question why a lay jury is permitted to find any breach of a duty to warn, in the face of an engaged scientific community that uniformly refused to advance a causal conclusion in the relevant time frame.

New Trial on General and Specific Causation

The trial court did not belabor the analysis of general causation beyond pointing out that there were substantial uncertainties for many of the Bradford Hill considerations, such as consistency, strength, and exposure-response. With respect to specific causation, all the problems discussed on the motion for judgment notwithstanding the verdict were also relevant to finding that the plaintiff failed to establish specific causation by a preponderance of the evidence. Escheverria op. at 40.

The trial court identified several grounds for the grant of a new trial, but one ground involved improper argument by plaintiffs’ counsel, who has repeatedly resorted to the same argument in previous cases. Forewarned, the defense sought a ruling in limine to exclude all evidence of lobbying and communications with federal agencies over regulations and regulatory classifications of talc. In a pretrial ruling, the trial court permitted the use of company documents about attempts to influence the National Toxicology Program (NTP) and the IARC for the limited purpose of notice to defendants that scientific organizations were considering whether to label talc as a carcinogen. Escheverria op. at 45.

Perhaps the trial court was being charitable in assessing what the lobbying evidence would be used for, but the plaintiffs did not need evidence of lobbying to prove “notice.” Early, often, and deliberately, the plaintiffs’ lawyers used evidence of lobbying for purposes well beyond the permissible, limited relevancy of notice. Escheverria’s counsel, Allen Smith argued, in opening and in closing that the defendants had “fended off” the National Toxicology Program (NTP), and that “if Johnson & Johnson would have just stayed out of it, let the scientists do their work at the U.S. government, the NTP would have listed talc as a carcinogen as far back as 2000.” So lobbying activities were not used as evidence of notice at all, but rather for arguing an inference of malice and outrageous misconduct from the prevention of regulation. Escheverria op. at 46.

Predictable.

1 Yessaian did advert to a study that she interpreted as failing to establish an association between obesity and ovarian cancer, but for the other risk factors of age and ovulatory cycles, the plaintiff’s expert witness offered no basis at all.

2 The trial court studiously avoided reference to the defense expert witness on epidemiology. See “Echeverria Talc Trial – Crossexamination on Alleged Expert Witness Misconduct” (Oct. 21, 2017).

3 citing well-known relative risk of two cases, Daubert v. Merrell Dow Pharms., Inc., 43 F. 3d 1311, 1321 (9th Cir. 1995); In re Lipitor (Atorvastatin Calcium) Mktg., Sales Prac. & Prod. Liab. Litig., 185 F. Supp. 3d 786, 791-92; Marder v. G.D. Searle & Co., 630 F. Supp. 1087, 1092 (D.Md. 1986), aff’d mem. on other grounds sub nom. Wheelahan v. G.D.Searle & Co., 814 F.2d 655 (4th Cir. 1987) (per curiam).

4 citing the dubious In re Neurontin Marketing, Sales Practices & Prods. Liab. Litig., 612 F. Supp. 2d 116, 132 (D. Mass. 2009), aff’d, 712 F.3d 21 (1st Cir. 2013).

5 Unfortunately, even the IARC classification of “probably” carcinogenic to humans is actually fairly meaningless exercises in semantics, not science. A close reading of the IARC Preamble definition of probable reveals that probable does not mean greater than 50%: “The terms probably carcinogenic and possibly carcinogenic have no quantitative significance and are used simply as descriptors of different levels of evidence of human carcinogenicity, with probably carcinogenic signifying a higher level of evidence than possibly carcinogenic.”

Posted in Causation, Evidence, Scientific Evidence | Comments Off on Johnson & Johnson Leaves Them in the Dust – Echeverria Verdict Unraveled

Quackers & Cheese – Trump Picks Kennedy to Study Vaccine Safety

January 11th, 2017

Science necessarily involves a willingness to follow evidence to whatever conclusions are warranted, if conclusions properly can be had. When it comes to vaccination conspiracies, Democrats have it in their political DNA to distrust pharmaceutical companies that research, develop, and manufacture vaccines. The current Republican party, which has been commandeered by theocrats and populists, see vaccination as federal government aggrandizement, and resist vaccination policy as contrary to God’s will. Science is often the loser in the cross-fire.

And so we now have the public spectacle of watching the left and the right join in similar scientific apostasies. Consider how both McCain and Obama both suggested that vaccines and autism were related in the 2008 election. (Although both candidates were to some extent slippery in their suggestions, which might have been appropriate given how little they knew about the controversies.) And consider Michelle Bachmann was converted to a similar view about the HPV vaccine on the basis of a woman’s anecdote about her child. And then on the far left, you have the uplifting story of Robert F. Kennedy Jr, and his brief on how thimerosal supposedly causes autism.

So it should be no surprise that Donald Trump, a Birther, a Mirther, a mid-night Twitterer, should embrace the anti-vaccination movement. Trump has made it clear that he rejects evidence-based policy, and so no one should expect him to embrace a scientific policy that is driven by high-quality scientific evidence. According to Kennedy, Trump wants Kennedy to head up a “commission on vaccine safety and scientific integrity.” Michael D. Shear, Maggie Haberman & Pam Belluckjan, “Anti-Vaccine Activist Says Trump Wants Him to Lead Panel on Immunization Safety,” N.Y. Times (Jan. 10, 2017); Domenico Montanaro, “Despite The Facts, Trump Once Again Embraces Vaccine Skeptics,” National Public Radio (Jan. 10, 2017).

Who needs the National Academy of Medicine when you can put a yutzball lawyer in charge of a “commission”?

Some of the media refer to Robert F. Kennedy Jr. as a vaccine skeptic, but their terminology is grossly inaccurate and misleading. Kennedy is a vaccine denier; he has engaged in a vitriolic campaign against the safety and efficacy of vaccines. He has aligned himself with the most extreme deniers of science, medicine, and public safety, including the likes of Andrew Wakefield and Jenny McCarthy. Kennedy has not merely engaged hyperbolic rhetoric against vaccines, he has used his radio show on the lawsuit industry’s Ring of Fire, to advance his campaign against public health as well as to shill for the lawsuit industry on other issues. See “RFK, Jr.: Science Shows That Autism — Mercury Link Exists – PT. ½,” Ring of Fire (Mar 8, 2011).

Kennedy should not be characterized as a skeptic, when he is a shrill ideologue, for whom science has no method that he is bound to respect. Back in July 2005, Kennedy published an article, “Deadly Immunity,” in both Rolling Stone and on Slate’s website. The article was a hateful screed against Big Pharma and government health agencies for an alleged conspiracy to hide the autism risks of thimerosal preservatives in vaccines. Several years later, on January 16, 2011, Salon retracted the article. See “https://en.wikipedia.org/wiki/Deadly_Immunity” entry in Wikipedia. See also Phil Plait, “Robert F. Kennedy Jr.: Anti-Vaxxer,” Slate (June 5 2013) (describing Kennedy as a full-blown anti-vaccination conspiracy theorist); Rahul K. Parikh, M.D., “Inside the vaccine-and-autism scare: A pediatrician traces the rise of the anti-vaccine movement that falsely linked thimerosal with autism and turned parents away from the most lifesaving medicine in history,” Salon (Sept. 22, 2008); Keith Kloor, “Is Robert F. Kennedy Jr. Anti-Science?” Discover Magazine (June 1, 2013); Steven Novella, “RFK Jr.s Autism Conspiracy Theory,” (Jun 20 2007).

Back in 2008, President Obama apparently considered Robert Kennedy for a cabinet-level position, but on sober reflection, thought better of it. See Steven Novella, “Politics and Science – The RFK Jr. Test,” (Nov. 07 2008). The Wall Street Journal, joined by many others, are now urging Trump to think harder and better about the issue, perhaps with some evidence as well. See Alex Berezow & Hank Campbell, “Ignore Anti-Vaccine Hysteria, Mr. Trump: Robert F. Kennedy Jr.’s conspiracy theories have no place in the White House,” Wall Street J. (Jan. 10, 2017).

Posted in Causation, Cognitive Biases, Conflicts of Interest, Evidence, Public Nuisances, Scientific Evidence, Scientific Misconduct | Comments Off on Quackers & Cheese – Trump Picks Kennedy to Study Vaccine Safety

Epistemic Nihilism and Ancient Documents

December 28th, 2016

This year, the Judicial Conference Advisory Committee on Evidence Rules proposed abrogation of the current “ancient documents” exception to the rule against hearsay, Rule 803(16). The proposal would, if adopted, become effective on December 1, 2017. See James A. King & Kirsten Fraser, “Say Goodbye to the ‘Ancient Documents’ Rule,” A.B.A. Trial Evidence (Feb. 17, 2016).

Under the Federal Rules of Evidence, an old document, one over 20 years old, found in a place where one would expect to find it is treated as “authentic.” The finder of fact, judge or jury, may accept the document for what it purports to be simply because of its age and the manner of its discovery. Rule 901(b)(8). The Federal Rules, however, go further and permit the document, authenticated as a so-called “ancient document,” to be accepted for the truth of the statements it contains. Rule 803(16).

The rationale offered by the Deans of Evidence Law for this remarkable exception to the rule against hearsay is that old documents predate the legal controversies in which they might later be used in evidence, and that we might not have any other admissible evidence relevant to events in the distant past (greater than twenty years). “[A]ge affords assurance that the writing antedates the present controversy” wrote the Federal Rules of Evidence Advisory Committee. Fed. R. Evid. 803(16) advisory committee note.

Pithy and pathetic. The proffered rationale was not valid when Rule 803(16) was initially drafted or promulgated, and it is not valid today. Old does not suggest or equate with reliability, and the present controversy is not the only source of bias and error in past statements that happened to be put in writing.

First, contrary to the conjecture of the common law, many documents were and are created with a view to influence potential controversies decades later. Mergers, acquisitions, and divestitures of companies are often governed by documents that explicitly acknowledge controversies that could stretch into the distant future.

Second, sometimes the exact motivation to falsify, fabricate, or fudge is not exactly the same as later will exist in later litigation, but it is similar. So when a young patient misrepresents his smoking history to a physician, he may simply be trying to avoid a disapproving lecture from his healthcare provider. Years later when he has developed lung cancer, and he is trying to blame anything but smoking in a lawsuit, he will rely upon the distorted report of his tobacco consumption. Of course, in many other situations, the motive to create misleading documents will arise from the expectation of the possibility or probability of future litigation over intellectual or real property rights, insurance contracts, etc.

Professor Daniel J. Capra of Fordham Law School, for many years the Reporter to the Judicial Conference Advisory Committee on Evidence Rules, has not been shy about the slim to none justification for Rule 803(16). In a podcast interview with Professor Ed Cheng, Capra laid out the case against Rule 803(16), and its evanescent rationale. See Excited Utterance website, Daniel Capra, “Electronically Stored Information and the Ancient Documents Exception” (Aug. 22, 2016). Preliminary Draft of Proposed Amendments to the Federal Rules of Bankruptcy Procedure and the Federal Rules of Evidence (Aug. 2015). Capra had previously deconstructed Rule 803(16) in a law review article, with the usual scholarly apparatus of footnotes and review of historical sources. Daniel J. Capra, “Electronically Stored Information and the Ancient Documents Exception to the Hearsay Rule Exception to the Hearsay Rule: Fix It Before People Find Out About It,” 17 Yale J.L. & Tech. 1 (2015) [cited as Capra].

As Capra puts it, the justification for Rule 803(16) was “never very convincing in the first place,” Capra at 1, 5, and “a radical and irrational hearsay exception – an error of the common law,” Capra at 11. The equating of a document’s authenticity with the trustworthiness of assertions contained within the document is “curious.” Capra at 9. Curiously, Capra balks at complete abrogation.

The law professor may have well said that the rule is capricious. Still, on gossamer grounds, Capra argues to retain the rule for hardcopy old evidence, but to abandon it for electronically stored information. Capra seems to buckle under the prospect of abrogating a rule, which leads him to “split the baby.” As trenchant as Capra’s critique is, most of his defense of retaining the Rule for hardcopy documents is as unsupported as the original Rule 803(16). Capra, for instance, suggests that “the likelihood of finding a hardcopy document that is twenty years old and also relevant to an existing litigation is quite small.” Capra at 5. Capra offers no empirical evidence for this startling assertion. Similarly, Capra claims that Rule 803(16) is rarely invoked, but he cites only to the paucity of cases that discuss this exception. Capra at 12. Capra thus claims negative evidence for the infrequent use, but the failure of judges to discuss this rule in published decisions may well be the result mechanical simplicity of the rule, which rarely leads to post-trial motions and appeals.

The original rational for Rule 803(16), and a rallying cry for its retention, is the supposed necessity to have some evidence to make out the events of times past. The objectors, however, generally fail to make out their case that the residual hearsay rule (Rule 807), as well as the business record and other exceptions, do not accomplish the twin goals of providing some evidence of past events while maintaining some semblance of reliability in fact finding.

The Rule Committee’s proposal has been met with an organized campaign from the lawsuit industry, both through the self-aggrandizingly named American Association for Justice (AAJ) and many of its members.. A rough count suggests that about 82 out of 218 comments came from asbestos plaintiffs’ lawyers. See, e.g., Comment from Robert Jacobs, American Association of Justice; Comment from Larry Tawwater, American Association for Justice (AAJ); Testimony of Marc P. Weingarten; Comment from Robert Paul, NA; Comment from Mark Gallagher, NA; Comment from Joseph Rice, NA.

The lawsuit industry lawyers argue that they will be deprived of the ability to show scienter or knowledge of a risk in latent disease litigation in which disease outcomes are lagged several decades after first exposure. Their argument, however, misses the point that many documents in company files, while not admissible for their truth, will be evidence of “notice” of a potential hazard, and the documents would be admissible for “state of mind,” and not the truth, in any event. Given the changes in epistemic standards for establishing causation, it is unlikely that really ancient documents will move the fact finder any closer to the truth of the actual fact of asserted causation. See “Time to Retire Ancient Documents As Hearsay Exception” (Aug. 23, 2015); “Ancient Truths” (May 5, 2016).

The asbestos plaintiffs’ lawyers thus argue that they would be deprived of important evidence such as the “Sumner Simpson” documents in asbestos cases. See Threadgill v. Armstrong World Indus., Inc., 928 F.2d 1366 (3d Cir. 1991). The plaintiffs’ argument rests upon an epistemic mistake. If knowledge is true, justified belief, then they do not need Rule 803(16) to show that the Sumner Simpson documents contain true statements; it is enough that they show that the authors believed what they said. In other words, the plaintiffs need only show that the documents reflect the declarants’ state of mind. Whether the statements are justified as true will require a complex mixture of current evidence about what health effects can be shown to be caused by the exposures of old, and what justifications were valid in today’s knowledge that were available and embraced by the declarants in the ancient documents.

Insurance coverage plaintiffs’ counsel have argued that they need Rule 803(16) to meet their burden of proof, and that insureds rely on both internal and external records. A policyholder’s internal records might include financial statements, annual reports, meeting minutes, check registers, contracts referencing insurance, insurance policies referencing other insurance policies, and/or accounting ledgers. Comment from Sherilyn Pastor, NA.

Most of the special pleading of these interest groups is wide of the mark. Old statements may be relevant and admissible for the speaker’s or reader’s statement of mind, and thus not hearsay. Old reliable documents can still be admitted under the residual exception, Rule 807, or under the business records exception, 803(6). Statements made in the making of contracts are operative facts of offer and acceptance, “speech acts,” and not offered for their truth.

The fact that a document is old may perhaps add to its authenticity, but in many technical, scientific, and medical contexts, the “ancient” provenance actually makes the content unlikely to be true. As such, the rule as now in effect is capable of much mischief and undermines accurate fact finding. The pace of change of technical and scientific opinion and understanding is too fast to indulge this exception that permits out-dated, false statements of doubtful validity to confuse the finder of fact. With respect to statements or claims to scientific knowledge, the Federal Rules of Evidence have evolved towards a system of evidence-based opinion, and away from naked opinion based upon the apparent authority or prestige of the speaker. Similarly, the age of the speaker or of the document provides no warrant for the truth of the document’s content. Of course, the statements in authenticated ancient documents remain relevant to the declarant’s state of mind, and nothing in the proposed amendment would affect this use of the document. As for the contested truth of the document’s content, there will usually be better, more recent, and sounder scientific evidence to support the ancient document’s statements if those statements are indeed correct. In the unlikely instance that more recent, more exacting evidence is unavailable, and the trustworthiness of the ancient document’s statements can be otherwise established, then the statements would probably be admissible pursuant to other exceptions to the rule against hearsay, as noted by the Committee. The proposed abrogation of this exception to the rule against hearsay should be welcomed; it is long overdue. And if it Capra is correct that ancient hardcopy rarely exists, and that the ancient document rule is rarely invoked, then abrogation cannot have the effect of defeating expectations and reliance upon this dubious mode of proof.

Of course, witnesses who are the declarants in the ancient documents may have died or moved away, but that is precisely why the law generally has statutes of limitations. When the law has generously created discovery rules, it should not then promulgate unreasonable, unreliable rules of evidence simply because it has extended the life on what otherwise would be stale claims.

Posted in Evidence | Comments Off on Epistemic Nihilism and Ancient Documents

Excited Utterance Podcast Series on Evidence Law

August 25th, 2016

As a graduate student, I was impressed by the extent to which scholars traveled to other schools to present draft papers and obtain feedback from other faculties and graduate students. As a student, these presentations were interesting opportunities to engage with leading scholars and learn from their new ideas, as well as their mistakes. Law school faculties back in the 1970s seemed like a much less collegial community of scholars, who rarely shared their ideas before publication, and thus did not receive the benefit of feedback from other scholars.

The isolation of legal scholarship has been mitigated in good law schools with the introduction of invited lectures and presentations, often at weekly seminars or luncheons. These meetings can be exciting and inspiring, but obviously participation is limited, and the financial and travel time restraints can be burdensome.

Edward Cheng, who teaches evidence and related subjects at Vanderbilt Law School, has introduced an interesting idea: scholarly podcasts on legal topics in his field of interest. Professor Cheng’s stated hope is that he can produce and provide podcasts, on scholarly topics in the law of evidence, which replicate the faculty seminar for a broader audience.

To be sure, there have been podcasts about specific legal cases, such as the famously successful “Undisclosed” podcast on the Adnan Syed case, which can honestly share in the credit in helping expose corruption and dishonesty in the prosecution of Mr. Syed, and in helping Mr. Syed obtain a new trial. Professor Cheng’s planned podcast series, “Excited Utterance: The Evidence and Proof Podcast,” will be on evidentiary topics more of interest to legal scholars, students, and practitioners. His stated goal is to focus on legal scholarship on evidence law and “to provide a weekly virtual workshop in the world of evidence throughout the academic year” to a broader audience, more efficiently than the sporadic visiting lectures that any one school can sponsor on evidentiary topics.

The project seems worth the effort in theory, and we will see what it produces in practice. The fall 2016 schedule for Cheng’s Excited Utterance podcasts is set out below; and the first one, by Daniel Chapra, is already available at iTunes, and at the Excited Utterance website.

Daniel Capra, “Electronically Stored Information and the Ancient Documents Exception” (Aug. 22, 2016)

Michael Pardo, “Group Agency and Legal Proof, or Why the Jury Is An It” (Aug. 29, 2016)

Mary Fan, “Justice Visualized” (Sept. 5, 2016)

Sachin Pandya, “The Constitutional Accuracy of Legal Presumptions” (Sept. 12, 2016)

Christopher Slobogin, “Gatekeeping Science” (Sept. 19, 2016)

Mark Spottswood, “Unraveling the Conjunction Paradox” (Sept. 26, 2016)

Deryn Strange, “Memory Errors in Alibi Generation” (Oct. 3, 2016)

Sandra Guerra Thompson, “Cops in Lab Coats” (Oct. 10, 2016)

Maggie Wittlin, “Hindsight Evidence” (Oct. 17, 2016)

Stephanos Bibas, “Designing Plea Bargaining from the Ground Up” (Oct. 24, 2016)

Erin Murphy, “Inside the Cell: The Dark Side of Forensic DNA” (Oct. 31, 2016)

Pamela R. Metzger, “Confrontation as a Rule of Production” (Nov. 7, 2016)

Nancy S. Marder, “Juries and Lay Participation: American Perspectives and Global Trends” (Nov. 14, 2016)

Jay Koehler, “Testing for Accuracy in the Forensic Sciences” (Nov. 21, 2016)

Posted in Evidence, Expert Witnesses | Comments Off on Excited Utterance Podcast Series on Evidence Law

Ancient Truths

May 5th, 2016

David Sackett, in some paternity disputes called the “father of evidence-based medicine,” supposedly once claimed that:

“Half of what you’ll learn in medical school will be shown to be either dead wrong or out of date within five years of your graduation; the trouble is that nobody can tell you which half–so the most important thing to learn is how to learn on your own.”

See Ivan Oransky, “So how often does medical consensus turn out to be wrong?” Retraction Watch (July 11, 2011). Sackett’s meta-statement was itself certainly not “evidence based,” but his point is well taken. Time ultimately erodes the authority of the truthiest sounding claims to medical knowledge. Sara Teichholtz, “The Differential: Half of What You’re Learning is Wrong,” (Dec. 14, 2013). Only lawyers and theologians would think that a statement in an old document or text, once authenticated, has some claim on us as the “truth.”

The Federal Rules of Evidence provide an exception to the rule against hearsay for statements made in ancient documents, those at least twenty years old. Rule 803(16). In 2015, the Judicial Conference’s Committee on Rules of Practice and Procedure proposed retiring the ancient document hearsay rule.[1] The exception created for documents authenticated as “ancient” (> 20 years old) is so inimical to the truth-finding function of trials, that courts strain to avoid finding the documents “authenticated.” See, e.g., Kalamazoo River Study Group v. Menasha Corp., 228 F.3d 648 (6th Cir. 2000).

The proposal to abolish this dangerous exception to the rule against hearsay has engendered resistance from some quarters over its ability to eliminate otherwise admissible evidence in cases involving long-past events, such as environmental or occupational disease litigation. The resistance, however, is misguided. The Committee’s proposal would not affect the authenticity presumption of an “ancient document,” and such documents could still be used to show state of mind, intention, motive, or notice. If the asserted statement in the old document is actually true, then there is likely much more recent, robust evidence to support the statement. The rule as it now stands is capable of a great deal of mischief. The fact that a document has survived intact in a place where one would expect to find it may add to its presumptive authenticity, but in many technical, scientific, and medical contexts, the “ancient” provenance actually makes the content likely to be false. Technical and scientific facts and opinions have changed too quickly to endorse statements simply because of they were written down somewhere, over 20 years ago. See “Time to Retire Ancient Documents As Hearsay Exception” (Aug. 23, 2015).

Although many in the legal academy have voiced opposition to the proposal[2], one law professor, Daniel Capra, has astutely observed that we will soon have a flood of easily authenticated documents of doubtful veracity, called websites, and other electronic documents, which have reached the age of evidentiary majority. Daniel J. Capra, “Electronically Stored Information and the Ancient Documents Exception to the Hearsay Rule: Fix It Before People Find Out About It,” 17 Yale J.L. & Tech 1 (2015). The truth of a proposition requires more than the lapse of 20 years since some nincompoop wrote it down.

[1] Preliminary Draft of Proposed Amendments to the Federal Rules of Bankruptcy Procedure and the Federal Rules of Evidence (Aug. 2015); See also Debra Cassens Weiss, “Federal judiciary considers dumping ‘ancient documents’ rule,” ABA Journal Online (Aug. 19, 2015).

[2] Peter Nicolas, “Saving an Old Friend From Extinction: A Proposal to Amend Rather Than to Abrogate the Ancient Documents Hearsay Exception,” 63 UCLA L. Rev. Disc. 172 (2015).

Posted in Evidence, State of the Art | Comments Off on Ancient Truths