TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

David Bernstein on the Daubert Counterrevolution

April 19th, 2013

David Bernstein has posted a draft of an important new article on the law of expert witnesses, in which he documents the widespread resistance to judicial gatekeeping of expert witness opinion testimony among federal judges.  Bernstein, “The Daubert Counterrevolution” (Mar. 11, 2013).  Professor Bernstein has posted his draft article, set to be published in the Notre Dame Law Review, both on the Social Science Research Network, and on his law school’s website.

Professor Bernstein correctly notes that the Daubert case, and the subsequent revision to Federal Rule of Evidence 702, marked important changes in the law of expert witnesses.  These changes constituted important reforms, which in my view were as much evolutionary, as revolutionary.  Even before the Daubert case, the law was working to find ways to improve expert witness testimony, and to downplay “authoritative” opining in favor of well-documented ways of knowing.  After all, Rule 702, with its emphasis on “knowledge,” was part of the Federal Rules of Evidence, as adopted in 1975.  Pub. L. 93–595, § 1, Jan. 2, 1975, 88 Stat. 1926 (effective July 1, 1975).  Since their first adoption, Rule 702 required that expert witnesses’ knowledge be helpful to the trier of fact.  By implication, the rule has always suggested that hunches, speculation, and flights of fancy did not belong in the court room.

Professor Bernstein certainly acknowledges that Daubert did not spring out of a vacuum.  Critics of judicial decisions on expert witnesses had agitated for decades to limit expert witness conduct by standards and guidances that operate in the scientific community itself.  The Supreme Court’s serial opinions on Rule 702 (Daubert, Joiner, Kumho Tire, and Weisgram) reflect the need for top-down enforcement of a rule, on the books since 1975, while many lower courts were allowing “anything goes.”

What is perhaps surprising, but well documented by Professor Bernstein, is that after four opinions from the Supreme Court, and a revision in the operative statute itself (Rule 702), some lower federal courts have engaged in a rearguard action against expert witness gatekeeping.  Professor Bernstein rightfully settles on the First Circuit’s decision in Milward as exemplifying a trend to disregard the statutory language and mandate for gatekeeping.  For Bernstein, Milward represents the most recent high-water mark of counterrevolution, with its embrace of errors and fallacies in the name of liberal, if not libertine, admissibility.

I suppose that I would go a step further than Professor Bernstein and label the trend he identifies as “reactionary.”  What is clear is that many courts have been willing to flout the statutory language of Rule 702, in favor of old case law, and evasive reasoning on expert witness admissibility.  Indeed, the Supreme Court itself joined the trend in Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011), when it unanimously affirmed the reversal of a trial court’s Rule 12(b)(6) dismissal of a securities fraud class action.  The corporate defendant objected that the plaintiffs failed to plead statistical significance in alleging causation between Zicam and the loss of the sense of smell.  The Supreme Court, however, made clear that causation was not required to make out a claim of securities fraud.  It was, and would be, sufficient for the company’s product to have raised sufficient regulatory concerns, which in turn would bring regulatory scrutiny and action that would affect the product’s marketability.

Not content to resolve a relatively simple issue of materiality, for which causation and statistical significance were irrelevant, the Supreme Court waxed on, in obiter dicta, about causation and statistical significance, perhaps unwittingly planting seeds for those who would eviscerate Rule 702.  See Matrixx Unloaded (Mar. 29, 2011).  Although the Supreme Court disclaimed any intention to address expert witness admissibility in a case that was solely about the sufficiency of pleading allegations, it cited three cases for the proposition that statistical significance was not necessary for assessing biological causation:

“We note that courts frequently permit expert testimony on causation based on evidence other than statistical significance. See, e.g., Best v. Lowe’s Home Centers, Inc., 563 F. 3d 171, 178 (6th Cir 2009); Westberry v. Gislaved Gummi AB, 178 F. 3d 257, 263–264 (4th Cir. 1999) (citing cases); Wells v. Ortho Pharmaceutical Corp., 788 F. 2d 741, 744–745 (11th Cir. 1986). We need not consider whether the expert testimony was properly admitted in those cases, and we do not attempt to define here what constitutes reliable evidence of causation.”

Id. at 1319.

Remarkably, two of the three cases were about specific causation, arrived at using so-called “differential etiology,” which presupposed the establishment of general causation.  These cases never involved general causation or statistical reasoning, but rather simply the process of elimination (iterative disjunctive syllogism).  The citation to the third case, Wells, was a notorious pre-Daubert, pre-Rule 702 revision case, revealed disappointing scholarship.  Wells involved at least one study that purported to find a statistically significant association.  What was problematic about Wells was its failure to consider the complete evidentiary picture, and to evaluate study validity, bias, and confounding, as well as significance probability.  See Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 1 (Nov. 12, 2012).

Wells was an important precursor to Daubert in that it brought notoriety and disrepute to how federal courts (and state courts as well) were handling expert witness evidence.  Significantly, Wells was a bench trial, where the trial judge opined that plaintiffs’ expert witnesses seemed more credible based upon atmospherics rather than on their engagement with the actual evidence. See Marc S. Klein, “After Daubert:  Going Forward with Lessons from the Past,” 15 Cardozo L. Rev. 2219, 2225-26 (1994) (quoting trial testimony of trial testimony of Dr. Bruce Buehler: “I am sorry sir, I am not a statistician . . . I don’t understand confidence levels. I never use them. I have to use the author’s conclusions.” Transcript of Jan. 9, 1985, at 358, Wells v. Ortho Pharm. Corp., 615 F. Supp. 262 (N.D. Ga. 1985).

Given the Supreme Court’s opinion in Matrixx, the reactionary movement among lower courts is unsurprising.  Lower courts have now cited and follow the Matrixx dicta on statistical significance in expert witness gatekeeping, despite the Supreme Court’s clear pronouncement that it did not intend to address Rule 702.  See In re Chantix (Varenicline) Prods. Liab. Litig., 2012 U.S. Dist. LEXIS 130144, at *22 (N.D. Ala. 2012); Cheek v. Wyeth Pharm. Inc., 2012 U.S. Dist. LEXIS 123485 (E.D. Pa. Aug. 30, 2012).

Professor Bernstein’s article goes a long way towards documenting the disregard for law and science in this movement.  The examples of reactionary decisions could easily be multiplied. Take for instance, the recent Rule 702 gatekeeping decision in litigation over Celexa and Lexapro, two antidepressant medications.  Judge Rodney W. Sippel denied the defense motions to exclude plaintiffs’ principal expert witness, Dr. David Healy.  In re Celexa & Lexapro Prods. Liab. Litig.,  ___ F.3d ___, 2013 WL 791780 (E.D. Mo. 2013).  In attempting to support its decision, the court announced that:

1. Cherry picking of studies, and data within studies, is acceptable for expert witnesses.  Id. at *5, *7, *8.

2. Outdated law applies, regardless of being superseded by later Supreme Court decisions, and the statutory revision in Rule 702.  Id. at *2 (citing pre-Joiner case:   “The exclusion of an expert’s opinion is proper only if it is so fundamentally unsupported that it can offer no assistance to the jury.” Wood v. Minn. Mining & Mfg. Co., 112 F.3d 306, 309 (8th Cir.1997) (internal quotation marks and citation omitted).”

3.  The Bradford Hill factors can be disregarded.  Id. at *6 (citing In re Neurontin Mktg., Sales Practices, and Prod. Liab. Litig., 612 F. Supp. 2d 116, 133 (D. Mass. 2009) (MDL 1629), and In re Viagra, 572 F.2d 1071 (D. Minn. 2008)).

These features of the Celexa decision are hardly novel.  As Professor Bernstein shows in his draft article, disregard of Rule 702’s actual language, and of the post-Daubert Supreme Court decisions, is prevalent.  See, e.g., In re Avandia Marketing, Sales Practices & Prod. Liab. Litig., 2011 WL 13576 (E.D. Pa. 2011)(announcing that MDL district judge was bound to apply a “Third Circuit” approach to expert witness gatekeeping, which focused on the challenged expert witnesses’ methodology, not their conclusions, in contravention of Joiner, and of Rule 702 itself).

The Celexa decision pushes the envelope on Bradford Hill.  The two decisions cited downplayed Bradford Hill’s considerations, but did not dismiss them.  In re Neurontin Mktg., Sales Practices, and Prod. Liab. Litig., 612 F. Supp. 2d 116, 133 (D. Mass. 2009) (MDL 1629)(“Although courts have not embraced the Bradford Hill criteria as a litmus test of general causation, both parties repeatedly refer to the criteria, seemingly agreeing that it is a useful launching point and guide. Accordingly, this Court will begin its inquiry by evaluating Plaintiffs’ evidence of an association between Neurontin and suicide-related events, the starting point for an investigation under the criteria.”);  In re Viagra Prods. Liab. Litig., 572 F.Supp.2d at 1081 (“The Court agrees that the Bradford Hill criteria are helpful for determining reliability but rejects Pfizer’s suggestion that any failure to satisfy those criteria provides independent grounds for granting its Daubert Motion.”).

Of course, Sir Austin’s considerations were merely those that he identified in a speech to a medical society.  They were not put forward in a scholarly article; nor are his considerations the last word on the subject.  Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295 (1965).

Even as a précis, given almost 50 years ago, Hill’s factors warrant some consideration rather than waving them off as not a litmus test (whatever that means), followed by complete disregard for any of the important considerations in evaluating the causality of an association.

There was brief bright spot in this fairly dim judicial decision.  The district judge refused to exclude Dr. Healy on grounds that his opinion about particular studies differed from the authors’ own interpretations.  In re Celexa & Lexapro Prods. Liab. Litig.,  ___ F.3d ___, 2013 WL 791780, at *5 (E.D. Mo. 2013) (Sippel, J.).

That is the correct approach, even though there is language in Joiner that suggests that the authors’ views are dominant.  See Follow the Data Not the Discussion.  But the refusal to discount Healy’s opinions on this ground was done without any real inquiry whether Healy had offered a valid, competing interpretation of the data in the published studies.

At the core of the reactionary movement identified by Professor Bernstein is an unwillingness, or an inability, to engage with the scientific evidence that is at issue in various Rule 702 challenges.  Let’s hope that Bernstein’s article induces closer attention to the law and the science in future judicial gatekeeping.

Posted in Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on David Bernstein on the Daubert Counterrevolution

Professor Faigman on the Dual Goals of the Daubert Revolution

March 9th, 2013

Academic commentators on Daubert and its progeny tend to fall into two camps:  acolytes and heretics.  The acolytes have generally supported the changes brought about by Daubert and the ultimate statutory embrace of active expert witness gatekeeping.  The heretics have maintained a rearguard action against Daubert, and Rule 702; they have tried to undermine gatekeeping at every turn.

Among the chief acolytes is David Faigman, whose books and articles have contributed substantially to the discussions and debates about the law of scientific evidence and expert witnesses.  Professor Faigman’s recent article is an important contribution to the law review literature on Daubert.  David L. Faigman, “The Daubert Revolution and the Birth of Modernity:  Managing Scientific Evidence in the Age of Science,” 102 U.C. Davis Law Rev. 101 (2013) [“Revolution”].  It is well worth reading.

Professor Faigman declares himself “a fan” of Daubert, and embraces the revolution in expert witness law heralded by the Supreme Court’s 1993 decision.  Id. at 103.  He emphasizes that the decision, quickly approaching its 20th anniversary, was truly revolutionary in how the federal courts engaged with expert witness opinion testimony, and that the consequences of the revolution are still taking shape.  Id.

Faigman acknowledges that Daubert and its progeny, and the statutory embrace of gatekeeping in Rule 702, at the end of the last millennium, were important developments in ensuring the epistemic warrant of federal courts’ judgments.  Some authors, hostile to the gatekeeping enterprise, have suggested that this aspect of Daubert resulted from persistent pressures from the defense bar and industry to limit plaintiffs’ access to the courts.  Faigman does not address such suggestions, and I believe that they are cynical and incorrect.  The federal courts, by the mid-1980’s, were deeply embarrassed by the scientific community’s opprobrium, meted out over notorious decisions, such as Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).  See also In re Air Crash Disaster at New Orleans, 795 F.2d 1230, 1234 (5th Cir. 1986) (“Our message to our able trial colleagues: it is time to take hold of expert testimony in federal trials.”).  Daubert and its progeny were, in my view, the judicial response to the scientific community’s criticisms.

Faigman’s thesis in this paper, however, lies elsewhere.  He argues that the Supreme Court’s excursions into expert evidence law, in Daubert and in the later cases, were intended primarily to give trial courts greater control over their dockets by being able to excluding dubious testimony and to grant summary dispositions.  Revolution at 104  Scientific verisimilitude was secondary to docket control.  Id. at 105.

Faigman’s thesis is plausible and should be taken seriously.  The first three cases in the “Revolution,” Daubert, Joiner, and Kumho Tire, were all tort cases with “mass tort” overtones.  Daubert was one of many Bendectin cases.  Joiner was a case involving occupational PCB exposures.  If the tenuous scientific opinions were deemed “admissible,” there were sure to be many more such cases.  And Kumho Tire was a case involving dubious allegations of a defect in a tire, the sort of allegations that plague American industry because they are so easy to manufacture, and so costly to defend.

Faigman builds an impressive case for the proposition that the justices really were trying to give trial courts managerial power to control their own dockets by filtering out essential, but deficient, expert witness testimony. Id. at 118.  After all, if the Supreme Court were really interested in improving judicial use of scientific evidence, why would it have created an abuse of discretion standard for reviewing Rule 702 determinations? The abuse standard signals that decisions either way are tolerable if they are accompanied by the right verbiage and procedural steps.

Faigman also points out that the abuse-of-discretion standard deprives the appellate courts of any meaningful review of the validity of scientific opinion testimony. The claims and conclusions advanced by expert witnesses in individual cases will often be of interest and importance to scientists, policy makers, plaintiffs, defendants, beyond the confines of the individual case.  The appellate courts are in a better position to ascertain validity questions, and maintain consistency in them, as a matter of law.  Freed of the pressures of trial courts, and with input from amici curiae, the appellate court can evaluate validity issues more deliberately with a view to harmonizing competing factors across many cases.  The scientific issues are, in any event, often non-case specific, or they have the tendency to recur in many cases of the same type.  Id. at 131.

Faigman’s thesis sheds light upon who the heretics are, and why they have worked so hard to undermine expert witness gatekeeping.  At stake is not only greater scientific validity, but also summary disposition of litigation rent-seeking. Rule 702 gatekeeping challenges judges and commentators to identify their priorities:  commitment to scientific principles or to litigation as an alternative to regulation and legislation on behalf of a special constituency.

There are some ironies inherent in Faigman’s thesis.  The trial bench has been reluctant to exercise its gatekeeping function as a method of docket control.  Instead, it has moved towards greater use of pre-trial consolidations in multi-district litigations to achieve economies of scale.  The MDL trend, however, has its problems.  Placing responsibility for expert witness gatekeeping in the MDL court may be counter to its “pre-trial” rationale of the MDL statute.  Furthermore, exercising gatekeeping across hundreds or thousands of cases heightens and highlights the anxieties, fears, distaste, and institutional incompetence for deciding scientific issues. The move toward MDL handling has had the apparent result of diluting the gatekeeping mandate and reducing the use of summary dispositions.

The procedural and the validity goals of Daubert are quite independent.  Validity may have been, as Faigman argues, a secondary goal for the Justices, but it was a worthy goal in and of itself.  I believe Professor Faigman would agree.  In describing the Supreme Court’s path on validity, Faigman notes that there were two competing models of expert witness admissibility determinations that vied for acceptance:  Frye, and then Daubert.  Id. at 105. He likens Frye to nose counting among the “relevant” scientific community for support of the witness’s methodology.  All a trial judge need do is identify the relevant community and then to count the noses.  Daubert represented a possible alternative:

“to charge judges with the responsibility to consider the methods and principles underlying proffered expert opinion and have them make the validity determination.”

Id. at 105.  Making trial judges responsible for warranting the validity of scientific evidence, and ultimately all expert witness opinion testimony, was one of the important changes that resulted in the Revolution and its embrace of “good grounds” or epistemic validity:

“[p]roposed testimony must be supported by appropriate validation.”

Daubert v. Merrell Dow Pharm., 509 U.S. 579, 590 (1993)

Professor Faigman correctly observes that, although lawyers and lower court judges have obsessed over the so-called Daubert factors, the actual holding of Daubert was “the requirement that an expert’s testimony pertain to ‘scientific knowledge’ establishes a standard of evidentiary reliability.” Revolution at 111 (quoting Daubert, 509 U.S. at 590).  Despite the improvident dictum about focus on methodology and not on conclusions, the Supreme Court, in Daubert, had made clear that there are necessary implications of Rule 702’s requirement that expert witness testimony relate to specialized “knowledge”:

“This entails a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue.”

Daubert, 509 U.S. at 592-93.

Professor Faigman writes to point out the erroneous interpretations and distortions of Daubert, its progeny, and Rule 702:

“The holding of Daubert is the requirement that judges find as a preliminary fact that the methods and principles underlying proffered expert testimony are sufficiently valid to support that testimony. The four ‘Daubert factors’ were offered as guidelines to help courts assess expert testimony.”

Revolution at 114.

Faigman’s writing is a useful reminder to those judges and commentators who would simplify and abridge the entire gatekeeping project into one or another dictum found in Daubert (or Joiner or Kumho Tire), and who ignore the actual holding of the cases, or the mandate of the subsequent statute. For those writers who try to evade the difficult scientific determinations and discriminations inherent in evaluating causal claims and other scientific opinions, Faigman reminds us that Justice Breyer, in his concurrence in Joiner, was not shy about pointing out that gatekeeping:

“will sometimes ask judges to make subtle and sophisticated determinations about scientific methodology and its relation to the conclusions an expert witness seeks to offer.”

General Electric Company v. Joiner, 522 U.S. 136, 118 S. Ct. 512, 520 (1997) (Breyer, J., concurring).  I take Faigman’s essay as an eloquent importuning of the judiciary to heed Justice Breyer, to stop whining, and to start learning.

It is a measure of Professor Faigman’s concern for the accuracy and validity of scientific testimony that he cannot bring himself to address a third way:  ignore validity, reliability, sufficiency, and simply allow expert witnesses to battle out.

This third way was what really prevailed before Daubert in much of civil litigation over health effects.  The Frye rule was rarely if ever applied to such cases, and most states excepted the opinion testimony of physicians, in any event.  Before Frye, we had whatever was dished up by ready, willing, able (and sufficiently glib) testifiers.  To be sure, expert witnesses had to be qualified, but the threshold was astonishingly low.  In Pennsylvania, for instance, the standard is that the putative “expert” must have “a reasonable pretense of expertise.” See, e.g., Ruzzi v. Butler Petroleum Co., 527 Pa. 1, 9-10 (1991); Kuisis v. Baldwin-Lima-Hamilton Corp.,457 Pa. 321, 319 A.2d 914 (1974)(“the witness must have a reasonable pretension to specialized knowledge on the subject under investigation”).  The federal courts were not far behind. Ferebee v. Chevron Chem. Co., 552 F. Supp. 1297 (D.D.C. 1982), aff’d, 736 F.2d 1529 (D.C. Cir.), cert. denied, 469 U.S. 1062 (1984).

Indeed, there is a pervasive, reactionary movement afoot, among judges and academic commentators to return to the wild, woolly days, celebrated in Ferebee’s famous dictum:

“On questions … which stand at the frontier of current medical and epidemiological inquiry, if experts are willing to testify that such a link exists, it is for the jury to decide whether to credit such testimony.”

Ferebee, 736 F.2d at 1534.  This third way then is simply to delegate to the expert witnesses themselves to assess the “weight of the evidence,” and offer up their opinions, without any scrutiny from the courts as to the validity or sufficiency of the bases for those opinions. This retrograde step is not just the stuff of naive law student musings. See, e.g., Note, “Admitting Doubt: A New Standard for Scientific Evidence,” 123 Harv. L. Rev. 2021 (2010). Reactionaries in the Academy and in the judiciary are intent to reduce gatekeeping to a weak test of relevancy, without any determination of content validity.

Posted in Expert Witnesses, Rule 702, Scientific Evidence, Trial Procedure | Comments Off on Professor Faigman on the Dual Goals of the Daubert Revolution

Leaving Las Vegas

February 24th, 2013

The Journal of the National Cancer Institute recently published a curious article about what appears to be unpublished research that suggests a non-asbestos environmental cause of malignant mesothelioma in Clark County, Nevada.  Leslie Harris O’Hanlon, “Researchers Explore Possible Link Between Mesothelioma and Dust Emissions in Southern Nevada,” J. Nat’l Cancer Instit., doi: 10.1093/jnci/djt033,  published ahead of print (Feb. 12, 2013).

The researcher appears to have been Francine Baumann , an epidemiologist at the University of Hawaii Cancer Center, who has worked with Michele Carbone, on occasion.  Analyzing Nevada’s cancer registry data from 1995 to 2008, Baumann found what she believed to be an increase in earlier age at diagnosis, and a reduced ratio of male-to-female cases for Clark County.   She interpreted these data to show that an environmental exposure was at work, but she professed ignorance of what the exposure might be.

The article also quotes the Nevada state epidemiologist, Ihsan Azzam, M.D., Ph.D., as saying:

“We analyzed the data and used the same data set as the researcher and came to completely different conclusions and findings. Their interpretation of data and their representation of it is wrong.”

The article presents no data or statistical analysis.  Given that Baumann’s work is unpublished, and apparently contradicted, it is curious that the Journal would publish any story about it.  Some of the raw data can be found online at Nevada Central Cancer Registry, including an online database, and Reports From The Office of Public Health Informatics and Epidemiology.

The O’Hanlon article is even more curious considering the nature of the research.  There are 16 counties in Nevada,  so Baumann presumably was canvassing counties without a pre-specified hypothesis as to whether Clark County was different from the others, or from the national rates.  This seems like post-hoc data dredging, but the Journal does not provide sufficient information to assess the validity of Baumann’s work.

The O’Hanlon article bizarrely talks about an unknown environmental cause in Clark County, but does not mention erionite, a zeolite.  The article discusses erionite-associated mesothelioma in Turkey, and an investigation into erionite occurrences in the United States.  Remarkably, O’Hanlon fails to mention that erionite occurs in Clark County, and in many other counties, throughout Nevada.  The NIOSH Science Blog fills in the missing information by showing how widespread erionite deposits are throughout Nevada.  See David Weissman, MD, and Max Kiefer, MS, CIH, “Erionite: An Emerging North American Hazard,” (Nov. 22, 2011).  Of course, the widespread deposits argue against erionite as a causal explanation for the putative environmental trigger in Clark County.  See also Arthur J. Gude & Richard Sheppard, “Wooly Erionite from the Reese River Zeolite Deposit, Lander County, Nevada, and its Relationship to Other Erionites,” 29 Clays and Clay Minerals, 378-384 (1981); Keith Papke, “Erionite and Other Associated Zeolites in Nevada,” Bulletin 79, Nevada Bureau of Mines and Geology (1972).

Erionite occurs in several mineralogical forms, including non-fibrous and various fibrous forms.  The erionite associated with environmental cases in Turkey has been studied and found to be fibrous, but there are many variations in fibers, including length, and length-to-diameter aspect ratio.  Erionite is a zeolite mineral and has the ability to absorb metal ions, including chromate, uranyl, and other ions, which may be an independent source of potential carcinogenicity.

There are many reasons to leave Las Vegas, but Dr. Baumann probably has not found a new one.

Posted in Causation, Risk and Risk Factor, Scientific Evidence | Comments Off on Leaving Las Vegas

Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

December 27th, 2012

A recent student law review article discusses reanalyses of epidemiologic studies, an important, and overlooked topic in the jurisprudence of scientific evidence.  Alexander J. Bandza, “Epidemiological-Study Reanalyses and Daubert: A Modest Proposal to Level the Playing Field in Toxic Tort Litigation,” 39 Ecology L. Q. 247 (2012).

In the Daubert case itself, the Ninth Circuit, speaking through Judge Kozinksi, avoided the methodological issues raised by Shanna Swan’s reanalysis of Bendectin epidemiologic studies, by assuming arguendo its validity, and holding that the small relative risk yielded by the reanalysis would not support a jury verdict of specific causation. Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1317–18 (9th Cir. 1995).

There is much that can, and should, be said about reanalyses in litigation and in the scientific process, but Bandza never really gets down to the business at hand. His 36 page article curiously does not begin to address reanalysis until the bottom of the 20th page. The first half of the article, and then some, reviews some time-worn insights and factoids about scientific evidence. Finally, at page 266, the author introduces and defines reanalysis:

“Reanalysis occurs ‘when a person other than the original investigator obtains an epidemiologic data set and conducts analyses to evaluate the quality, reliability or validity of the dataset, methods, results or conclusions reported by the original investigator’.”

Bandza at 266 (quoting Raymond Neutra et al., “Toward Guidelines for the Ethical Reanalysis and Reinterpretation of Another’s Research,” 17 Epidemiology 335, 335 (2006).

Bandza correctly identifies some of the bases for judicial hostility to re-analyses. For instance, some courts are troubled or confused when expert witnesses disagree with, or reevaluate, the conclusions of a published article. The witnesses’ conclusions may not be published or peer reviewed, and thus the proffered testimony fails one of the Daubert factors.  Bandza correctly notes that peer review is greatly overrated by judges. Bandza at 270. I would add that peer review is an inappropriate proxy for validity, a “test,” which reflects a distrust of the unpublished.  Unfortunately, this judicial factor ignores the poor quality of much of what is published, and the extreme variability in the peer review process. Judges overrate peer review because they are desperate for a proxy for validity of the studies relied upon, which will allow them to pass their gatekeeping responsibility on to the jury. Furthermore, the authors’ own conclusions are hearsay, and their qualifications are often not fully before the court.  What is important is the opinion of the expert witness who can be cross-examined and challenged.  See “FOLLOW THE DATA, NOT THE DISCUSSION.” What counts is the validity of the expert witness’s reasoning and inferences.

Bandza’s article, which by title advertises itself to be about re-analyses, gives only a few examples of re-analyses without much detail.  He notes concerns that reanalyses may impugn the reputation of published scientists, and burden them with defending their data.  Who would have it any other way? After this short discussion, the article careens into a discussion of “weight of the evidence” (WOE) methodology. Bandza tells us that the rejection of re-analyses in judicial proceedings “implicitly rules out using the weight-of-the-evidence methodology often appropriate for, or even necessary to, scientific analysis of potentially toxic substances.” Bandza at 270.  This argument, however, is one sustained non-sequitur.  WOE is defined in several ways, but none of the definitions require or suggest the incorporation of re-analyses. Re-analyses raise reliability and validity issues regardless whether an expert witness incorporates them into a WOE assessment. Yet Bandza tells us that the rejection of re-analyses “Implicitly Ignores the Weight-of-the-Evidence Methodology Appropriate for the Scientific Analysis of Potentially Toxic Substances.” Bandza at 274. This conclusion simply does not follow from the nature of WOE methodology or reanalyses.

Bandza’s ipse dixit raises the independent issue whether WOE methodology is appropriate for scientific analysis. WOE is described as embraced or used by regulatory agencies, but that description hardly recommends the methodology as the basis for a scientific, as opposed to a regulatory, conclusion.  Furthermore, Bandza ignores the ambiguity and variability of WOE by referring to it as a methodology, when in reality, WOE is used to describe a wide variety of methods of reasoning to a conclusion. Bandza cites Douglas Weed’s article on WOE, but fails to come to grips with the serious objections raised by Weed in his article to the use of WOE methodologies.  Douglas Weed, “Weight of Evidence: A Review of Concept and Methods,” 25 Risk Analysis 1545, 1546–52 (2005) (describing the vagueness and imprecision of WOE methodologies). See also “WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name.”

Bandza concludes his article with a hymn to the First Circuit’s decision in Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir. 2011). Plaintiffs’ expert witness, Dr. Martyn Smith claimed to have performed a WOE analysis, which in turn was based upon a re-analysis of several epidemiologic studies. True, true, and immaterial.  The re-analyses were not inherently a part of a WOE approach. Presumably, Smith re-analyzed some of the epidemiologic studies because he felt that the data as presented did not support his desired conclusion.  Given the motivations at work, the district court in Milward was correct to look skeptically and critically at the re-analyses.

Bandza notes that there are procedural and evidentiary safeguards in federal court against unreliable or invalid re-analyses of epidemiologic studies.  Bandza at 277. Yes, there are safeguards but they help only when they are actually used. The First Circuit in Milward reversed the district court for looking too closely at the re-analyses, spouting the chestnut that the objections went to the weight not the admissibility of the evidence.  Bandza embraces the rhetoric of the Circuit, but he offers no description or analysis of the liberties that Martyn Smith took with the data, or the reasonableness of Smith’s reliance upon the re-analyzed data.

There is no necessary connection between WOE methodologies and re-analyses of epidemiologic studies.  Re-analyses can be done properly to support or deconstruct the conclusions of published papers.  As Bandza points out, some re-analyses may go on to be peer reviewed and published themselves.  Validity is the key, and WOE methodologies have little to do with the process of evaluating the original or the re-analyzed study.

 

 

Posted in Causation, Data Sharing, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

Litmus Tests

December 27th, 2012

Rule 702 is, or is not, a litmus test for expert witness opinion admissibility.  Relative risk is, or is not, a litmus test for specific causation.  Statistical significance is, or is not, a litmus test for reasonable reliance upon the results of a study.  It is relatively easy to find judicial opinions on either side of the litmus divide.  Compare National Judicial College, Resource Guide for Managing Complex Litigation at 57 (2010) (Daubert is not a litmus test) with Cryer v. Werner Enterprises, Inc., Civ. Action No. 05-S-696-NE, Mem. Op. & Order at 16 n. 63 (N.D. Ala. Dec. 28, 2007) (describing the Eleventh Circuit’s restatement of Rule 702’s “litmus test” for the methodological reliability of proffered expert witness opinion testimony).

The “litmus test“ is one sorry, overworked metaphor.  Perhaps its appeal has to do with a vague collective memory that litmus paper is one of those “things of science,” which we used in high school chemistry, and never had occasion to use again. Perhaps, litmus tests have the appeal of “proofiness.”

The reality is different. The litmus test is a semi-quantitative test for acidity or alkalinity.  Neutral litmus is purple.  Under acidic conditions, litmus turns red; under basic conditions, it turns blue.  For some time, scientists have used pH meters when they want a precise quantification of acidity or alkalinity.  Litmus paper is a fairly crude test, which easily discriminates  moderate acidity from alkalinity (say pH 4 from pH 11), but is relatively useless for detecting an acidity at pH or 6.95, or alkalinity at 7.05.

So what exactly are legal authors trying to say when they say that some feature of a test is, or is not, a “litmus test”? The litmus test is accurate, but not precise at the important boundary at neutrality.  The litmus test color can be interpreted for degree of acidity or alkalinity, but it is not the preferred method to obtain a precise measurement. Saying that a judicial candidate’s views on abortion are a litmus test for the Senate’s evaluation of the candidate makes sense, given the relative binary nature of the outcome of a litmus test, and the polarization of political views on abortion. Apparently, neutral views or views close to neutrality on abortion are not a desideratum for judicial candidates.  A cruder, binary test is exactly what is desired by politicians.

The litmus test that is used for judicial candidates does not seem to work so well when used to describe scientific or statistical inference.  The litmus test is well understood, but fairly obsolete in modern laboratory practice.  When courts say things, such as statistical significance is not a litmus test for acceptability of a study’s results, clearly they are correct because measure of random error is only one aspect of judging a body of evidence for, or against, an association.  Yet courts seem to imply something else, at least at times:

statistical significance is not an important showing in making a case that an exposure is reliably associated with a particular outcome.

Here courts are trading in half truths.  Statistical significance is quantitative, and the choice of a level of significance is not based upon immutable law. So like the slight difference between a pH of 6.95 and 7.05, statistical significance tests have a boundary issue.  Nonetheless, a consideration of random error cannot be dismissed or overlooked on the theory that significance level is not a “litmus test.”  This metaphor obscures and attempts to excuse sloppy thinking.  It is time to move beyond this metaphor.

Posted in Rule 702, Scientific Evidence, statistical evidence | Comments Off on Litmus Tests

Lumpenepidemiology

December 24th, 2012

Judge Helen Berrigan, who presides over the Paxil birth defects MDL in New Orleans, has issued a nicely reasoned Rule 702 opinion, upholding defense objections to plaintiffs expert witnesses, Paul Goldstein, Ph.D., and Shira Kramer, Ph.D. Frischhertz v SmithKline Beecham EDLa 2012 702 MSJ Op.

The plaintiff, Andrea Frischhertz, took GSK’s Paxil, a selective serotonin reuptake inhibitor (SSRI), for depression while pregnant with her daughter, E.F. The parties agreed that E.F. was born with a deformity of her right hand.  Plaintiffs originally claimed that E.F. had a heart defect, but their expert witnesses appeared to give up this claim at deposition, as lacking evidential support.

Adhering to Daubert’s Epistemiologic Lesson

Like many other lower federal courts, Judge Berrigan focused her analysis on the language of Daubert v. Merrell Dow Pharmaceuticals Inc., 509 U.S. 579 (1993), a case that has been superseded by subsequent cases and a revision to the operative statute, Rule 702.  Fortunately, the trial court did not lose sight of the key epistemological teaching of Daubert, which is based upon Rule 702:

“Regarding reliability, the [Daubert] Court said: ‘the subject of an expert’s testimony must be “scientific . . . knowledge.” The adjective “scientific” implies a grounding in the methods and procedures of science. Similarly, the word “knowledge” connotes more than subjective belief or unsupported speculation’.”

Slip Op. at 3 (quoting Daubert, 509 U.S. at 589-590).

There was not much to the plaintiffs’ expert witnesses’ opinion beyond speculation, but many other courts have been beguiled by speculation dressed up as “scientific … knowledge.”  Dr. Goldstein relied upon whole embryo culture testing of SSRIs, but in the face overwhelming evidence, Dr. Goldstein was forced to concede that this test may generate hypotheses about, but cannot predict, human risk of birth defects.  No doubt this concession made the trial court’s decision easier, but the result would have been required regardless of Dr. Goldstein’s exhibition of truthfulness at deposition.

Statistical Association – A Good Place to Begin

More interestingly, the trial court rejected the plaintiffs’ expert witnesses’ efforts to leapfrog finding a statistically significant association to parsing the so-called Bradford Hill factors:

“The Bradford-Hill criteria can only be applied after a statistically significant association has been identified. Federal Judicial Center, Reference Manual on Scientific Evidence, 599, n.141 (3d. ed. 2011) (“In a number of cases, experts attempted to use these guidelines to support the existence of causation in the absence of any epidemiologic studies finding an association . . . . There may be some logic to that effort, but it does not reflect accepted epidemiologic methodology.”). See, e.g., Dunn v. Sandoz Pharms., 275 F. Supp. 2d 672, 678 (M.D.N.C. 2003). Here, Dr. Goldstein attempted to use the Bradford-Hill criteria to prove causation without first identifying a valid statistically significant association. He first developed a hypothesis and then attempted to use the Bradford-Hill criteria to prove it. Rec. Doc. 187, Exh. 2, depo. Goldstein, p. 103. Because there is no data showing an association between Paxil and limb defects, no association existed for Dr. Goldstein to apply the Bradford-Hill criteria. Hence, Dr. Goldstein’s general causation opinion is not reliable.”

Slip op. at 6.

The trial court’s rejection of Dr. Goldstein’s attempted end run is particularly noteworthy given the Reference Manual’s weak-kneed attempt to suggest that this reasoning has “some logic” to it.  The Manual never articulates what “logic” commends Dr. Goldstein’s approach; nor does it identify any causal relationship ever established with such paltry evidence in the real world of science. The Manual does cite several legal cases that excused or overlooked the need to find a statistically significant association, and even elevated such reasoning into legally acceptable, admissibility method.  See Reference Manual on Scientific Evidence at 599 n. 141 (describing cases in which purported expert witnesses attempted to use Bradford Hill factors in the absence of a statistically significant association; citing Rains v. PPG Indus., Inc., 361 F. Supp. 2d 829, 836–37 (S.D. Ill. 2004); ); Soldo v. Sandoz Pharms. Corp., 244 F. Supp. 2d 434, 460–61 (W.D. Pa. 2003).  The Reference Manual also cited cases, without obvious disapproval, which completely dispatched with any necessity of considering any of the Bradford Hill factors, or the precondition of a statistically significant association.  See Reference Manual at 599 n. 144 (citing Cook v. Rockwell Int’l Corp., 580 F. Supp. 2d 1071, 1098 (D. Colo. 2006) (“Defendants cite no authority, scientific or legal, that compliance with all, or even one, of these factors is required. . . . The scientific consensus is, in fact, to the contrary. It identifies Defendants’ list of factors as some of the nine factors or lenses that guide epidemiologists in making judgments about causation. . . . These factors are not tests for determining the reliability of any study or the causal inferences drawn from it.“).

Shira Kramer Takes Her Lumpings

The plaintiffs’ other key expert witness, Dr. Shira Kramer, was a more sophisticated and experienced obfuscator.  Kramer attempted to provide plaintiffs with a necessary association by “lumping” all birth defects together in her analysis of epidemiologic data of birth defects among children of women who had ingested Paxil (or other SSRIs).  Given the clear evidence that different birth defects arise at different times, based upon interference with different embryological processes, the trial court discerned this “lumping” of end points to be methodologically inappropriate.  Slip op. at 8 (citing Chamber v. Exxon Corp., 81 F. Supp. 2d 661 (M.D. La. 2000), aff’d, 247 F.3d 240 (5th Cir. 2001) (unpublished).

Without her “lumping”, Dr. Kramer was left with only a weak, inconsistent claim of biological plausibility and temporality. Finding that Dr. Kramer’s opinion had outrun her headlights, Judge Berrigan, excluded Dr. Kramer as an expert witness, and granted GSK summary judgment.

Merry Christmas!

 

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on Lumpenepidemiology

Bad Gatekeeping or Missed Opportunity – Allen v. Martin Surfacing

November 30th, 2012

Sometimes when federal courts permit dubious causation opinion testimony over Rule 702 objections, the culprit is bad lawyering by the opponent of the proffered testimony.  Allen v. Martin Surfacing, 263 F.R.D. 47 (D. Mass. 2009), may be an important example.

THE CLAIMS

Daniel Allen was the former football coach of the College of The Holy Cross, in Worcester, Massachusetts.  In spring of 2001, defendant Martin Surfacing refinished the gymnasium floor at the college.  Coach Allen was exposed to solvent fumes, including toluene fumes, during defendant’s work, as well as for a couple of months afterwards.   While exposed, Allen experienced “dizziness, headaches, and disorientation.” 263 F.R.D. at 51.  After the gym floor resurfacing was completed, Allen experienced other symptoms, such as fatigue, muscle weakness, and fasciculations in his lower limbs.  In January 2002, at the age of 45, Allen was diagnosed with amyotrophic lateral sclerosis (ALS).  Id. Allen’s condition progressed, and he died three years later, in May 2004.  Id. at 52.

Allen’s family sued for wrongful death.  The parties’ apparently agreed on the following:

  • ALS occurs as a sporadic ALS, as well as “familial ALS,”
  • the cause of sporadic ALS is unknown,
  • Allen developed and died of sporadic ALS,
  • no air sampling established overexposure to any chemical,
  • there were no reliable exposure models to quantify Allen’s exposures,
  • there are no known causes of sporadic ALS, and
  • toluene did not cause Allen’s ALS

Remarkably, defendant lost the Rule 702 challenge to plaintiffs’ expert witnesses’ opinion testimony.  It is easy to suspect that the district judge was sleep at the gate, and that his gatekeeping was deficient.  A close read of the opinion supports the view that this was not Rule 702’s finest moment, but much more was going on to get to admissibility.

First, the plaintiffs’ counsel cleverly avoided running into a wall by avoiding a claim that toluene caused Allen’s ALS. Instead, plaintiffs’ claimed that toluene accelerated the onset of the disease.  This claim was equally dubious, but it allowed the expert witnesses to avoid a mountain of medical opinion, authoritative and well-supportive, that there is no known cause of sporadic ALS.

Second, the plaintiffs’ counsel took the initiative by filing an affirmative motion to admit the testimony of their expert witnesses.  Rather than ceding the initiative to the defendant, the plaintiffs seized the initiative and had the first and last word on admissibility.  As a result, plaintiffs were able to present and frame their witnesses’ opinions sympathetically rather than defensively.

Third, the plaintiffs had the good fortune of the defendant’s counsel’s apparent failure to find the key fallacies, invalidities, and flaws in plaintiffs’ questionable expert witness opinions.

The Allen case teaches that sometimes good lawyering can win a losing case.

The plaintiffs’ counsel retained and presented an array of expert witnesses who might be the usual suspects in a district court’s exclusion of expert witness testimony:

None of these four expert witnesses was a specialist in ALS or ALS causation; none was a neurologist; none had ever addressed ALS causation in a peer-reviewed article.  All four witnesses were frequent testifiers in tort litigation, and some have are repeat offenders when it comes to offering questionable or excludable opinion testimony.  Somehow, the defense dashed this opportunity by retaining only one expert, Dean M. Hashimoto, M.D., J.D., M.P.H., who was also not a specialist in ALS, who was not a neurologist, and who had never published anything on ALS.  And to make matters worse, the defense proceeded to challenge the plaintiffs’ expert witnesses for lack of qualifications!

The defense’s challenges to qualifications takes up a good deal of Judge Saylor’s published opinion, which illustrates the maxim that judges have short attention spans, and you should not waste the opportunity of a motion on an issue that is so easily decided against you.  The scientific issues are difficult and the temptation to avoid them is great.  By leading with an issue that will almost certainly lose, the defense wasted a valuable advocacy opportunity to show the court the fallacious reasoning in the plaintiffs’ case.  By submitting reports from only one expert witness, who had all the deficiencies claimed in the plaintiffs’ set of witnesses, the defense exhibited a duplicity that must have seriously undermined its credibility for the entire set of Rule 702 motion issues.

THE WITNESSES

Dr. Christine Oliver has been testifying in asbestos and other occupational lung disease cases for decades.  She is a pulmonary physician on staff at Massachusetts General Hospital, in Boston, and an associate professor of clinical medicine at the Harvard Medical School.  She is board certified in internal medicine and in occupational medicine (American Board of Preventive Medicine), and her clinical interests are asthma occupational lung disease, and health hazards of construction work.  If the defense had presented real expert witnesses in ALS causation, Dr. Oliver’s expertise would have seemed quite irrelevant.  Dr. Oliver has, as well as I can determine, never researched or published on ALS causation.  She has, however, published on “multiple chemical sensitivity,” which should give a disinterested court some pause.  See L. Christine Oliver and Alison Johnson, “Multiple Chemical Sensitivity: Reflections” (Nov. 4, 2011).

Richard Clapp, professor emeritus at the Boston University School of Public Health, is a known purveyor of dubious courtroom testimony. See, e.g., Sutera v. The Perrier Group of America Inc., 986 F.Supp. 655 (D. Mass. 1997).  He is a frequent testifier and a charter member of the surreptitiously funded SKAPP organization.  Clapp is a non-physician epidemiologist, who has never published on ALS.

Marcia Ratner Ph.D. may be best known for her possession of mace and an unlicensed gun, but she does occasionally show up in civil litigation as an expert witness.  See “Quincy District Court News,” Patriot Ledger June 09, 2010 (reporting that Ratner pleaded guilty to criminal possession of mace and a firearm).

Ratner is a postdoctoral researcher at Boston University, where she works as a neurotoxicologist.  She does not appear to have ever published a peer-reviewed paper on ALS or ALS causation.  Plaintiffs’ counsel claimed that she was researching a new drug with therapeutic potential for ALS treatment, although they were quite sketchy about details.  Ratner does not appear to hold any NIH grants for ALS drug research.

[Please see update on the discussion of Dr. Ratner at http://schachtmanlaw.com/gatekeeping-in-allen-v-martin-surfacing-postscript/]

William Ewing, an industrial hygienist, frequently testifies in asbestos litigation.  He offered no opinion on causation.

Against this field of witnesses, the defense punted on presenting its own witness with relevant expertise. Dr. Dean M. Hashimoto, the defense’s sole witness on causation, is a physician, lawyer, and has a master’s degree in occupational health.  Hashimoto has no specialized training in ALS or clinical neurology, although he serves on the Massachusetts Workers’ Compensation Board. A pubmed search  shows that Hashimoto has never published on the neurology or causation of ALS.

CAUSATION

The plaintiffs had a huge problem to avoid:  ALS has no known cause.  Counsel table could be filled up with textbooks and review articles, but perhaps the following, lengthy quote from the National Institute for Neurological Disorders and Stroke website suffices to make the point:

“What causes ALS?

The cause of ALS is not known, and scientists do not yet know why ALS strikes some people and not others. An important step toward answering that question came in 1993 when scientists supported by the National Institute of Neurological Disorders and Stroke (NINDS) discovered that mutations in the gene that produces the SOD1 enzyme were associated with some cases of familial ALS. This enzyme is a powerful antioxidant that protects the body from damage caused by free radicals. Free radicals are highly reactive molecules produced by cells during normal metabolism. If not neutralized, free radicals can accumulate and cause random damage to the DNA and proteins within cells. Although it is not yet clear how the SOD1 gene mutation leads to motor neuron degeneration, researchers have theorized that an accumulation of free radicals may result from the faulty functioning of this gene. In support of this, animal studies have shown that motor neuron degeneration and deficits in motor function accompany the presence of the SOD1 mutation.

Studies also have focused on the role of glutamate in motor neuron degeneration. Glutamate is one of the chemical messengers or neurotransmitters in the brain. Scientists have found that, compared to healthy people, ALS patients have higher levels of glutamate in the serum and spinal fluid. Laboratory studies have demonstrated that neurons begin to die off when they are exposed over long periods to excessive amounts of glutamate. Now, scientists are trying to understand what mechanisms lead to a buildup of unneeded glutamate in the spinal fluid and how this imbalance could contribute to the development of ALS.

Autoimmune responses—which occur when the body’s immune system attacks normal cells—have been suggested as one possible cause for motor neuron degeneration in ALS. Some scientists theorize that antibodies may directly or indirectly impair the function of motor neurons, interfering with the transmission of signals between the brain and muscles.

In searching for the cause of ALS, researchers have also studied environmental factors such as exposure to toxic or infectious agents. Other research has examined the possible role of dietary deficiency or trauma. However, as of yet, there is insufficient evidence to implicate these factors as causes of ALS.

Future research may show that many factors, including a genetic predisposition, are involved in the development of ALS.”

NINDS – “Amyotrophic Lateral Sclerosis (ALS) Fact Sheet.”

As a result, the plaintiffs adopted a strategy of confession and avoidance; they renounced any claim that they were asserting a causal claim.  Instead, they insisted that they were “merely” claiming that toluene exposure had accelerated the onset of sporadic ALS in Coach Allen.  This mere claim, however, was actually a causal claim in disguise, and the district judge was taken in by the ruse.  If plaintiffs were claiming that toluene can accelerate the onset of ALS by a meaningful period of time (years), then they were making a causal claim, legally and scientifically.  A shift in the age of onset of a sporadic disease is a causal claim, and it requires supporting evidence, not hand waving.

PLAUSIBLE MECHANISM

One scientist could postulate a reasonable mechanism even for a sporadic disease.  Professional journals and textbooks are filled with such speculation.  These postulations are part of science in that they inform research hypotheses and funding, but they are not conclusions of causality.  The quote above from the NINDS discusses the lack of an anti-oxidizing enzyme and glutamate toxicity as potential mechanisms in familial ALS, but even there, the authors are appropriately modest in avoiding a claim to know the pathogenesis of familial ALS.

The plaintiffs’ approach was to take the suggestion of a mechanism, misrepresent it as a known mechanism, and then claim that toluene activated glutamate toxicity and exercised an oxidizing effect on neurons. The plaintiffs’ team had no basis for claiming that short-term exposure to solvents, or toluene specifically, translated into a toxicity to the relevant human motor neurons that are involved in ALS.  It is a long stretch from suggesting a mechanism to documenting the mechanism to be actually at work in producing, or accelerating, a disease in humans.

A typical statement, from the Yale School of Medicine, Division of Neurology, in 2012:

“Why the motor neurons begin to die is still unknown. Recent evidence, however, have implicated glutamate excitotoxicity, free radical toxicity, and mitochondrial dysfunction as possible mechanisms, and this is an area of active research.”

“Amyotrophic Lateral Sclerosis (ALS)” (emphasis added).   See also Adams and Victor’s Principles of Neurology 1157-58 (7th ed. 2001) (noting that the pathogenesis of ALS and similar motor neuron diseases is not known).

The district judge seemed mesmerized by Ratner’s having providing a biologically plausible theory for tying ALS progression to toluene exposure.  263 F.R.D. at 60.  Judge Saylor stated that the defense did not address any flaw in Ratner’s methodology other than to point out that her theory was not supported by epidemiology.  The court seemed to equate providing a plausible theory with establishing a scientific conclusion.  More to the point, the court was truly asleep at its gatekeeping task because Ratner’s theory actually presupposed that she knew that Coach Allen was going to develop ALS in any event, only not as early as 2001.  The court faulted the defense for not showing that Ratner’s (and the other plaintiffs’ witnesses’) theory was unreliable, but the burden was on the plaintiffs to show reliability.  Id.  The court not only faulted the defense for carrying a burden it did not have, but it overlooked the very telling criticisms of Ratner’s theories of acceleration and mechanism.

EXCUSES – EPIDEMIOLOGY

Plaintiffs’ expert witnesses had a welter of excuses as to why there was no epidemiologic data to support their theories.  The absence of statistical significance, according to plaintiffs’ expert witnesses does not mean that a study should be disregarded.  Id. at 58.  Their claim is superficially true, but a study not disregarded does not necessarily support a causal inference, either alone or conjunction with other such studies. Similarly, plaintiffs’ claim that flawed studies should not be disregarded is also a half truth.  A flawed study may lead to a much better one, which can support valid inferences.  Flawed studies are thus part of the scientific process because they may lead to a self-correcting triangulation of the truth, but there is little to recommend relying upon flawed studies to support scientific conclusions of causality.  Nevertheless, the district court appeared to swallow these half truths, whole.

Ratner also advanced a claim that the acceleration theory had not been subjected to epidemiologic analysis because of “funding limitations, as most funding goes toward finding treatment or cures for the disease, not towards finding what accelerates the course of the disease.”  Id. at 59 n. 14.  The district court repeats this excuse without critical thought.  If a commonly used solvent such as toluene accelerated the onset of a terrible disease such as ALS by decades, such a putative effect would be amenable to epidemiologic analysis and would be a source of incredible concern and funding efforts by the NIH, NINDS, NIEHS, and other granting agencies and organizations.  Despite excusifying verbiage, Ratner maintained that there were no epidemiologic data that refuted her novel acceleration.  Id. at 59.  Of course, if her excuses were taken seriously, then this absence of refutation was fairly irrelevant, but in any event, this supposed absence could not support the reliability of Ratner’s inferences or conclusions.

The defense focused on the lack of short-term exposures in epidemiologic studies, and also the lack of statistical significance in some studies.  What appears to have been missing from both sides was a comprehensive analysis of the available epidemiologic data.  If long-term exposure were associated with earlier age of onset of ALS, or even a greater risk of ALS, then it would have given some support to Ratner’s novel theory.  The defense appeared to punt on the epidemiology by claiming its irrelevance.  It might have been helpful to point out internal as well as external validity issues to the court.

As for both sides citing different studies, and no side presenting a comprehensive view of the epidemiologic evidence, the court could have given some consideration to the ethical considerations of the incomplete presentation:

“Basis of Expert Medical Testimony

The testimony of an expert medical witness should be founded on a thorough and critical review of the pertinent medical and scientific facts, available data, and relevant literature.”

Ethical Guidelines for Occupational and Environmental Medicine Physicians Serving as Expert Witnesses (Oct. 25, 2007).

DIFFERENTIAL DIAGNOSIS

The plaintiffs’ claim that they were not asserting causation was disingenuous.  As noted above, acceleration of onset is a form of causation.  Of course, exposure to a neurotoxic material, with some symptoms, might have made Allen more aware of other symptoms, and so the time to diagnosis was abbreviated.  The plaintiffs, however, were claiming more than earlier ascertainment; they claimed the toluene exposure caused an underlying disease process to accelerate.

Oliver actually went further and performed an invalid differential etiologic analysis. Oliver reviewed medical records and claimed to have applied “differential diagnosis to the review.”  Id. at 63. This claim was quite bogus because there was no dispute that Allen had and died of ALS, but the district court was beguiled.  Having ruled out family history, Oliver claimed to then rule out other “putative causes” of ALS:  “pesticides and agricultural chemicals containing solvents, 60-hertz magnetic fields, and welding fumes.”  Id. at 63.  In one fell swoop, Oliver created several known causes to be ruled out, and then ruled them out in Allen’s case.  This is remarkable given that NINDS and most of medical sciences does not recognize any known or putative causes of sporadic ALS, and that Oliver failed to rule out the one potential cause that some scientists take seriously:  cigarette smoking.  See, e.g., Hao Wang, Éilis J. O’Reilly, Marc G. Weisskopf, Giancarlo Logroscino, Marji L. McCullough, Michael Thun, Arthur Schatzkin, Laurence N. Kolonel, Alberto Ascherio, “Smoking and risk of amyotrophic lateral sclerosis: a pooled analysis of 5 prospective cohorts” 68 Arch. Neurol. 207 (2011); A. Alonso, G. Logroscino, M.A. Hernán, “Smoking and the risk of amyotrophic lateral sclerosis: a systematic review and meta-analysis,” 81 J. Neurol. Neurosurg. & Psychiatry 1249 (2010); F. Fang & W. Ye, “Smoking may be considered an established risk factor for sporadic ALS,” 74 Neurology 1927 (2010).

Of course, Oliver, and the entire plaintiffs’ expert witness team failed to rule out the most obvious, most prevalent explanation for Allen’s ALS:  unknown.

GENETIC SUSCEPTIBILITY

Ratner testified “to a reasonable degree of scientific certainty that Allen was genetically predisposed to develop ALS and would have developed and died from ALS later in his life.”  263 F.R.D. at 61.  This assertion was truly an incredible, unsupported, unverifiable, and unfalsifiable statement.  If a drug company ever made a similarly unsupported claim in an electronically transmitted document, the Department of Justice would prosecute it for wire fraud.  United States v. Harkonen, 2010 WL 2985257 (N.D. Calif. 2010).

The parties had essentially stipulated that Allen did not suffer from familial ALS, and neither Ratner nor anyone else identified any gene that was responsible for his “susceptibility.”  The district court, of course, did not report how Ratner could possibly have known that Allen was going to develop ALS, only at some unspecified date later than the date when Allen first became aware of signs and symptoms of motor neuron disease.  The district court announced that plaintiffs’ expert witnesses were not propounding “junk science,” but perhaps the heavy perfume helped masquerade the garbage.

POST HOC ERGO PROPTER HOC

The court conclusorily noted, without explanation, that the temporal relationship between exposure and disease manifestation would allow a conclusion of causality:

“Finally, after interpreting the data within a chronological context, the clinician may conclude that the patient’s disease is a neurotoxic illness.”

Id. at 61.  The court appears to accept the temporal pattern as sufficient in itself, or with other information, to support the conclusion.  This reasoning is fallacious.

AGE OF ONSET

Allen developed ALS when he was 45 years old.  Ratner reasoned that the average age of onset was 60, and Allen developed his disease “much earlier than would be expected”; therefore toluene accelerated the onset of Allen’s disease.  Id. at 61. The problem is that there is no “therefore” that can reasonably be claimed in the court’s sentence.

Most publications put the mean and median of age of ALS onset around 55 years, but even if the court were to accept Ratner’s reference to 60 as correct, surely the court recognized that half the cases therefore occurred below the age of 60.  The question of course is the variability in age of onset, and the court’s opinion is silent about the scatter or distribution of age-of-onset data.  Ratner’s reasoning was prima facie invalid unless there was additional information to show a very narrow distribution of age of onset around the mean.  It is difficult to discern whether the defense made this point, but Ratner could not have supported this counterfactual claim.

Here is what the ALS association has to say about the issue:

“Most people who develop ALS are between the ages of 40 and 70, with an average age of 55 at the time of diagnosis. However, cases of the disease do occur in persons in their twenties and thirties.”

“Who Gets ALS.”

Ratner essentially conceded that her argument was vacuous and invalid.  When confronted at her deposition about whether age of onset greater than the mean would have changed her opinion, she emphatically denied its relevance:

“My opinion would be the same even if that guy died at 60 instead of 75 and had history of this exposure … but you wouldn’t have bothered to depose me in that case… . Somebody else has moved down from where they are to here. But it may not result in a lawsuit, and I wouldn’t be here, because— I wouldn’t be here.”

Ratner Deposition at 172-3.

RULE 702 ANALYSIS

The district court recognized the novelty of Ratner’s analysis, but opined that Ratner, Oliver, and Clapp had provided sufficient cumulative evidence to support their theories.  263 F.R.D. at 61.  The trial court apparently conducted a Rule 702 hearing, over three days. Both sides filed what appears to have been extensive briefing and affidavits.  There are some huge gaps in the reasoning of the plaintiffs’ expert witnesses, and in the district court’s opinion.  Perhaps those gaps could be filled in with volumes of testimony.  My unscientific opinion is to doubt it. Although the plaintiffs should have had the burden of showing admissibility, the defendant had the practical burden of illustrating the analytical gaps, ipse dixit, fallacies, and invalid inferences that were before the court.  The defense may have indeed pointed out such problems, which were fulsomely present, but the district court’s opinion does not report the obvious defense arguments.  Without more background information, it is difficult to evaluate comprehensively the court’s or the defense’s handling of the scientific issues that were clearly before the court on the Rule 702 motions.  What is clear from what the district court reports is, however, sufficient to document an unsatisfactory judicial review of the evidence discussed.

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Bad Gatekeeping or Missed Opportunity – Allen v. Martin Surfacing

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

November 21st, 2012

In 1984, before Judge Shoob gave his verdict in the Wells case, another firm filed a birth defects case against Ortho for failure to warn in connection with its non-ionic surfactant spermicides, in the same federal district court, the Northern District of Georgia. The mother in Smith used Ortho’s product about the same time as the mother in Wells (in 1980).  The case was assigned to Judge Shoob, who recused himself.  Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1562 n.1 (N.D. Ga. 1991) (no reasons for the recusal provided).  The Smith case was reassigned to Judge Horace Ward, who entertained Ortho’s motion for summary judgment in July 1988.  Two and one-half years later, Judge Ward granted summary judgment to Ortho on grounds that the plaintiffs’ expert witnesses’ testimony was not based upon the type of data reasonably relied upon by experts in the field, and was thus inadmissible under Federal Rule of Evidence 703. 770 F. Supp. at 1681.

A prevalent interpretation of the split between Wells and Smith is that the scientific evidence developed with new studies, and that the scientific community’s views matured in the five years between the two district court opinions. The discussion in Modern Scientific Evidence is typical:

“As epidemiological evidence develops over time, courts may change their view as to whether testimony based on other evidence is admissible. In this regard it is worth comparing Wells v. Ortho Pharmaceutical Corp., 788 F.2d 741 (11th Cir. 1986), with Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561 (N.D. Ga. 1991). Both involve allegations that the use of spermicide caused a birth defect. At the time of the Wells case there was limited epidemiological evidence and this type of claim was relatively novel.  In a bench trial the court found for the plaintiff.  *** The Smith court, writing five years later, noted that, ‘The issue of causation with respect to spermicide and birth defects has been extensively researched since the Wells decision.’ Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1563 (N.D. Ga. 1991).”

1 David L. Faigman, Michael J. Saks, Joseph Sanders, and Edward K. Cheng, Modern Scientific Evidence:  The Law and Science of Expert Testimony, “Chapter 23 – Epidemiology,” § 23:4, at 213 n.12 (West 2011) (internal citations omitted).

Although Judge Ward was being charitable to his judicial colleague, this attempt to reconcile Wells and Smith does a disservice to Judge Ward’s hard work in Smith, and Judge Shoob’s errors in Wells.

Even a casual reading of Smith and Wells reveals that the injuries were completely differently.  Plaintiff Crystal Smith was born with a chromosomal defect known as Trisomy-18; Plaintiff Katie Wells was born with limb reduction deficits.   Some studies relevant to one injury had no information about the other.  Other studies, which addressed both injuries, yielded different results for the different injuries.  Although some additional studies were available to Judge Ward in 1988, this difference is hardly the compelling difference between the two cases.

Perhaps the most important difference between the cases is that in Smith, the biologically plausibility that spermicides could cause a Trisomy-18 was completely absent.  The chromosomal defect arises from a meiotic disjunction, an error in meiosis that is part of the process in which germ cells are formed.  Simply put, spermicides arrive on the scene too late to cause a Trisomy-18.  Notwithstanding the profound differences between the injuries involved in Wells and Smith, the Smith plaintiffs sought the application of collateral estoppel.  Judge Ward refused this motion, on the basis of the factual differences in the cases, as well as the availability of new evidence.  770 F.Supp. at 1562.

The difference in injuries, however, was not the only important difference between these two cases.  Wells was actually tried, apparently without any challenge under Frye, or Rules 702 or 703, to the admissibility of expert witness testimony.  There is little to no discussion of scientific validity of studies, or analysis of the requisites for evaluating associations for causality.  It is difficult to escape the conclusion that Judge Shoob decided the Wells case on the basis of superficial appearances, and that he frequently ignored validity concerns in drawing invidious distinctions between plaintiffs’ and defendant’s expert witnesses and their “credibility.”  Smith, on the other hand, was never tried.  Judge Ward entertained and granted dispositive motions for summary judgment, on grounds that the plaintiffs’ expert witnesses’ testimony was inadmissible. Legally, the cases are light years apart.

In Smith, Judge Ward evaluated the same FDA reports and decisions seen by Judge Shoob.  Judge Ward did not, however, dismiss these agency materials simply because one or two of dozens of independent scientists involved had some fleeting connection with industry. 770 F.Supp. at 1563-64.

Judge Ward engaged with the structure and bases of the expert witnesses’ opinions, under Rules 702 and 703.  The Smith case thus turned on whether expert witness opinions were admissible, an issue not considered or discussed in Wells.  As was often the case before the Supreme Court decided Daubert in 1993, Judge Ward paid little attention to Rule 702’s requirement of helpfulness or knowledge.  The court’s 702 analysis was limited to qualifications.  Id. at 1566-67.  The qualifications of the plaintiffs’ witnesses were rather marginal.  They relied upon genetic and epidemiologic studies, but they had little training or experience in these disciplines. Finding the plaintiffs’ expert witnesses to meet the low threshold for qualification to offer an opinion in court, Judge Ward focused on Rule 703’s requirement that expert witnesses reasonably rely upon facts and data that are not otherwise admissible.

The trial court in Smith struggled with how it should analyze the underpinnings of plaintiffs’ witnesses’ proffered testimony.  The court acknowledged that conflicts between expert witnesses typically raise questions of weight, not admissibility.  Id. at 1569.  Ortho had, however, challenged plaintiffs’ witnesses for having given opinions that lacked a “sound underlying methodology.” Id.  The trial court found at least one Fifth Circuit case that suggested that Rule 703 requires trial courts to evaluate the reliability of expert witnesses’ sources.  Id. (citing Soden v. Freightliner Corp., 714 F.2d 498, 505 (5th Cir. 1983). Elsewhere, the trial court also found precedent from Judge Weinstein’s opinion in Agent Orange, as well as Court of Appeals decisions involving Bendectin, all of which turned to Rule 703 as the legal basis for reviewing, and in some cases limiting or excluding expert witness opinion testimony.  Id.

The defendant’s argument under Rule 703 was strained; Ortho argued that the plaintiffs’

“experts’ selection and use of the epidemiological data is faulty and thus provides an insufficient basis upon which experts in the field of diagnosing the source of birth defects normally form their opinions. The defendant also contends that the plaintiffs’ experts’ data on genetics is not of the kind reasonably relied upon by experts in field of determining causation of birth defects.”

Id. at 1572.  Nothing in Rule 703 addresses the completeness or thoroughness of expert witnesses in their consideration of facts and data; nor does Rule 703 address the sufficiency of data or the validity vel non of inferences drawn from facts and data considered.  Nonetheless, the trial court in Smith took Rule 703 as its legal basis for exploring the epistemic warrant for plaintiffs’ witnesses’ causation opinions.

Although plaintiffs’ expert witnesses stated that they had relied upon epidemiologic studies and method, the trial court in Smith went beyond their asseverations.  The Smith trial court explored the credibility of these witnesses at a whole other level.  The court reviewed and discussed the basic structure of epidemiologic studies, and noted that the objective of such studies is to provide a statistical analysis:

“The objective of both case-control and cohort studies is to determine whether the difference observed in the two groups, if any, is ‘statistically significant’, (that is whether the difference found in the particular study did not occur by chance alone).40 However, statistical methods alone, or the finding of a statistically significant association in one study, do not establish a causal relationship.41 As one authority states:

‘Statistical methods alone cannot establish proof of a causal relationship in an association’.42

As a result, once a statistical association is found in an epidemiological study, that data must then be evaluated in a systematic manner to determine causation. If such an association is present, then the researcher looks for ‘bias’ in the study.  Bias refers to the existence of factors in the design of a study or in the manner in which the study was carried out which might distort the result.43

If a statistically significant association is found and there is no apparent ‘bias’, an inference is created that there may be a cause-and-effect relationship between the agent and the medical effect. To confirm or rebut that inference, an epidemiologist must apply five criteria in making judgments as to whether the associations found reflect a cause-and-effect relationship.44 The five criteria are:

1. The consistency of the association;

2. The strength of the association;

3. The specificity of the association;

4. The temporal relationship of the association; and,

5. The coherence of the association.

Assuming there is some statistical association, it is these five criteria that provide the generally accepted method of establishing causation between drugs or chemicals and birth defects.45 ”

The Smith court acknowledged that there were differences of opinion in weighting these five factors, but that some of them were very important to drawing a reliable inference of causality.  Id. at 1775.

A major paradigm shift thus separates Wells and Smith.  The trial court in Wells contented itself with superficial and subjective indicia of witnesses’ personal credibility; the trial in Smith delved into the methodology of drawing an appropriate scientific conclusion about causation.  Telling was the Smith court’s citation to Moultrie v. Martin, 690 F.2d 1078, 1082 (4th Cir. 1982) (“In borrowing from another discipline. a litigant cannot be selective in which principles are applied.”).  770 F.Supp. at 1575 & n.45.  Gone is the Wells retreat from engagement with science, and the dodge that the court must make a legal, not a scientific decision.

Applying the relevant principles, the Smith court found that the plaintiffs’ expert witnesses had deviated from the scientific standards of reasoning and analysis:

“It is apparent to the court that the testimony of Doctors Bussey and Holbrook is insufficiently grounded in any reliable evidence. * * * The conclusions Doctors Bussey and Holbrook reach are also insufficient as a basis for a finding of causality because they fail to consider critical information, such as the most relevant epidemiologic studies and the other possible causes of disease.81

The court finds that the opinions of plaintiffs’ experts are not based upon the type of data reasonably relied upon by experts in determining the cause of birth defects. Experts in determining birth defects rely upon a consensus in genetic or epidemiological investigations or specific generally accepted studies in these fields. While a consensus in genetics or epidemiology is not a prerequisite to a finding of causation in any and all birth defect cases, Rule 703 requires some reliable evidence for the basis of an expert’s opinion.

Experts in determining birth defects also utilize methodologies and protocols not followed by plaintiffs’ experts. Without a well-founded methodology, opinions which run contrary to the consensus of the scientific community and are not supported by any reliable data are necessarily speculative and lacking in the type of foundation necessary to be admissible.

For the foregoing reasons, the court finds that plaintiffs have failed to produce admissible evidence sufficient to show that defendant’s product caused Crystal’s birth defects.”

Id. at 1581.  Rule 703 was forced into a service to filter out methodologically specious opinions.

Not all was smooth sailing for Judge Ward.  Like Judge Shoob, Judge Ward seemed to think that a physical examination of the plaintiff provided helpful, relevant evidence, but he never articulated what the basis for this opinion was. (His Honor did note that the parties agreed that the physical examination offered no probative evidence about causation.  Id. at 1572 n.32.) No harm came of this opinion.  Judge Ward wrestled with the lack of peer review in some unpublished studies, and the existence of a study only in abstract form.  See, e.g., id. at 1579 (“a scientific study not subject to peer review has little probative value”); id. at 1578 (insightfully noting that an abstract had insufficient data to permit a reader to evaluate its conclusions).  The Smith court recognized the importance of statistical analysis, but it confused Bayesian posterior probabilities with significance probabilities:

“Because epidemiology involves evidence on causation derived from group based information, rather than specific conclusions regarding causation in an individual case, epidemiology will not conclusively prove or disprove that an agent or chemical causes a particular birth defect. Instead, its probative value lies in the statistical likelihood of a specific agent causing a specific defect. If the statistical likelihood is negligible, it establishes a reasonable degree of medical certainty that there is no cause-and-effect relationship absent some other evidence.”

The confusion here is hardly unique, but ultimately it did not prevent Judge Ward from reaching a sound result in Smith.

What intervened between Wells and Smith was not any major change in the scientific evidence on spermicides and birth defects; the sea change came in the form of judicial attitudes toward the judge’s role in evaluating expert witness opinion testimony.  In 1986, for instance, after the Court of Appeals affirmed the judgment in Wells, Judge Higginbotham, speaking for a panel of the Fifth Circuit, declared:

“Our message to our able trial colleagues: it is time to take hold of expert testimony in federal trials.”

 In re Air Crash Disaster at New Orleans, 795 F.2d 1230, 1234 (5th Cir. 1986).  By the time the motion for summary judgment in Smith was decided, that time had come.

Posted in Causation, Data Sharing, Frye, Rule 702, Rule 703, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

November 21st, 2012

While the trial court was preparing its findings of fact and conclusions of law, Ortho moved to reopen to evidence to permit additional testimony based upon three new articles.  Ortho’s motion came three months after the close of evidence, and Judge Shoob’s announcement of his verdict. The court denied this motion without mentioning what the new articles purported to show.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262, 298 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).

What is remarkable in Wells, from the vantage point of current practice, is the absence of motions directed at the proffered expert witness opinion testimony.  On the basis of Judge Shoob’s opinion, there appears to have been no Frye motion, no motions to exclude expert witnesses based upon the Federal Rules of Evidence, and no motions to strike testimony after the fact for lack of a proper basis.

Having lost the verdict in a bench trial, Ortho had little chance for success in the Court of Appeals on a claim that the evidence supporting the plaintiffs’ verdict was legally insufficient.  The traditional standard, applied by the Court of Appeals, was to sustain the trier of fact’s decision as not “clearly erroneous” when there were two “permissible” views of the evidence. 788 F.2d 741, 743 (11th Cir. 1986).  Without some legal doctrine to filter out flawed, invalid, and inadequate expert witness opinion from permissible views of an evidentiary display, the Court of Appeals was left with only a rubber stamp, which it proceeded to use with alacrity.

Ortho attempted to turn its appellate argument about the sufficiency of the evidence into a legal principle about rejecting factual findings not based upon “scientifically reliable foundations.”  Id. at 744.  The appellate court framed the issue on appeal simply as a “battle of the experts,” which Ortho had lost.  Both sides had qualified expert witnesses, and thus, according to the appellate court, “the district court was forced to make credibility determinations to ‘decide the victor’.” Id. (citing Ferebee v. Chevron Chemical Co., 736 F.2d 1529, 1535 (D.C. Cir.), cert. denied, 469 U.S. 1062 (1984)).  The Court of Appeals thus acquiesced in Judge Shoob’s superficial analysis, which attempted to resolve a scientific issue by trial atmospherics, demeanor, and subjective impressions of witness confidence rather than the validity of the studies relied upon and inferences drawn therefrom.  The possibility that Judge Shoob might have evaluated the evidentiary basis underlying the expert witnesses’ opinions was not even acknowledged.

The Court of Appeals invoked the language from Ferebee on statistical significance, despite its irrelevance to the case before it:

“We recognize, as did the Ferebee court, that ‘a cause-effect relationship need not be clearly established by animal or epidemiological studies before a doctor can testify that, in his opinion, such a relationship exists. As long as the basic methodology employed to reach such a conclusion is sound, such as use of tissue samples, standard tests, and patient examination, products liability law does not preclude recovery until a “statistically significant” number of people have been injured or until science has had the time and resources to complete sophisticated laboratory studies of the chemical. Id. at 1535-36.”

Wells, 788 F.2d at 745 (quoting Ferebee). Ferebee involved an injury that all parties agreed could be attributed to paraquat exposure without the need for epidemiologic studies; statistical analysis was not particularly germane.  In Wells, on the other hand, both sides relied upon studies that required statistical analyses for any sensible interpretation, and some of the studies actually reported statistically significant results.  The appellate court’s rhetoric was empty and irrelevant.

(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 3

November 18th, 2012

So Ortho lost the liability issue in the Wells case, which turned on historical knowledge in 1980.  How did Ortho lose medical causation?

The Wells case, as reported, is a fantastic if not terrifying insight into the mind of a judge, sitting as the trier of fact.  It is not a legal determination of the reliability or validity of the expert witness opinions under Federal Rule of Evidence 702; nor is it an assessment of the reasonableness of any expert’s reliance upon any study or data.  No Frye motions were made.  (If such motions were made, the trial court did not reference them in its opinion.) Ortho defended with its expert witnesses, and argued its case as a battle of experts.

Many of the litigation and judicial decision-making themes on state of the art carried over to medical causation.  Once Ortho lost the liability issue on such weak, controverted, and irrelevant evidence, its fate was likely sealed on causation, before Judge Shoob.  The trial court, in resolving liability, had already determined that it would allow hints, hunches, and hypotheses to substitute for knowledge.  Obscure and unpublished papers were accorded great weight, to the exclusion of peer-reviewed, published, data-driven, and carefully analyzed studies. Confounding in older studies with a mercury chemical not involved in Ortho’s product was ignored.  Any evidence of fetal harm, even in non-validated animal models with extraordinary doses, was permitted to substitute for relevant congenital malformations.  Exculpatory opinions, including those of the FDA’s Advisory Committee, were ignored.

In the litigation over medical causation, the parties agreed upon some facts and opinions.  Timing of exposure within the embryological development is crucial to an assessment of causality.  Some time windows are important, others irrelevant, to the development of specific malformations.  Therefore, not only is the time window of exposure important, but so is the type of malformation at issue. Notwithstanding this agreement, the trial court was consistently vague in discussing the specific malformations in various studies.  Not a single “effect size” is reported; and not a single p-value or confidence interval is indicated.  There is no discussion or recognition of the role multiple testing and comparison may play in diluting the meaning of a particular p-value.

Physical Examinations

The court mentioned no less than four times that some of plaintiffs’ expert witnesses had Katie Wells, and that none of the defendant’s witnesses had. The court failed, however, to articulate what was learned in such examinations that could not be appreciated from the medical records themselves.  This waving of hands about the “laying on of hands” reflects the trial court’s superficial approach to decision making.

Case Reports

The court mentioned that the plaintiffs’ expert witnesses relied upon the existence of case reports, but did not discuss what if any significance such case reports had for evaluating plaintiffs’ causal claims.  615 F. Supp. at 274.

Association Cannot Be Ruled Out  — Further Research Is Needed

The trial court imposed a highly skewed approach in its evaluation of expert witnesses on either side of the case.  One of the plaintiffs’ expert witnesses, Dr. John Holbrook had testified that “a definite relationship” existed:

“In Dr. Holbrook’s opinion, a ‘definite relationship’ exists between the use of spermicides and fetal malformations.”

615 F. Supp. at 275.

The court found Dr. Holbrook to be credible.  Id. at 276. There are several interesting aspects of how the trial court reacted to such statements from the plaintiffs’ expert witnesses.  First, the court tolerated an infuriating imprecision and inaccuracy in how the plaintiffs’ witnesses expressed their opinions. In the case of Dr. Holbrook, he was permitted to talk generally of “spermicides,” when it was clear that not all spermicides could be evaluated for causality together.  Second, the court permitted Dr. Holbrook to speak of ill-defined “definite” relationship, without declaring whether that relationship was “causal,” or something else.  When Holbrook, and the other plaintiffs’ witnesses, disagreed with the conclusions of articles upon which they relied, the trial court generally did not find that the discrepancies discredited the witnesses or their testimony.

The court’s approach to the plaintiffs’ witnesses should be compared with how it evaluated the defense expert witnesses’ testimony.  Dr. Stolley was called by the defense, and he was the only witness who had any real training and expertise in epidemiology.  Plaintiffs’ counsel questioned Dr. Stolley about statements made by Dr. Shapiro, an author of one of the studies relied upon by Dr. Stolley in his testimony.  (Remarkably, and importantly, the trial court does not report what if anything the plaintiffs’ witnesses had to say about Dr. Shapiro’s study.)

The Shapiro study concluded as follows:

“We conclude that there is no satisfactory evidence to indicate that spermicides increase the overall risk of major birth defects. It is possible, however, that spermicides increase the risk of certain specific malformations. Our own study is not large enough to exclude such a possibility. To evaluate it, further studies are needed.”

Id. at 284.  This sort of statement, an acknowledgement that further studies may be needed, is hardly an indication that the study had found an increased risk of a particular outcome.  And yet, the trial court somehow thought that it was Dr. Stolley who equivocated by having relied upon the Shapiro study.

At the FDA hearing, Dr. Shapiro described his published study:

“There is one question that is unanswered and that we fully acknowledge, and that is that all of the cohort studies so far done, or the case control studies, for that matter, lack the power to evaluate in statistical terms that there is not an appreciable increase in the risk of specific birth defects.

….

I’m not for one moment claiming that this study rules out an increase in the risk of limb reduction deformities.”

Id. at 285 (emphasis added).

Dr. Stolley disagreed with Shapiro about a relatively unimportant linguistic issue of what the magnitude is of an “appreciable” increased risk; Dr. Stolley opined that the available studies could rule out an “appreciable increase” in risk. Note that Dr. Shapiro was addressing a power issue in his study, whereas Dr. Stolley was addressing the issue with respect to all the available studies.  With respect to the supposed disagreement over “appreciable,” here is how Dr. Stolley expressed his view:

“I think that what [Shapiro] is saying is that, if there were a tiny increase in limb reduction defects, which is rare, you would need an extremely large study to absolutely rule that out and that can’t be done.

* * *

What he was trying to do is address the question of ruling out very, very tiny risks, and I think that I would agree that it’s very difficult to do that.

* * *

I think to me the data is quite conclusive in that it does not demonstrate — does not demonstrate an association of spermicides with limb reduction defects or with any other congenital anomaly.

* * *

I think that Doctor Shapiro would agree with my statement, and that is that no association between spermicides and birth defects has been shown, and I agree with him when he says a small increase cannot be ruled out. I think that’s true about a small increase in anything, but why should one even bother to do that given data that doesn’t show any substantial association as it now stands.”

Id. at 285-86.

The court found the above passages to be the basis for assigning “little weight” to Dr. Stolley’s opinion despite his “impressive credentials.”  Here is how the court described what it perceived to be an “equivocation”:

“Several times during direct examination, Dr. Stolley testified emphatically that a specific study or studies showed that spermicides are not related to birth defects. ***. In the portions of his testimony on cross-examination quoted above, however, Dr. Stolley equivocated. His interpretation of the studies discussed was that ‘no association between spermicides and birth defects has been shown’, but on cross he conceded that ‘a small increase [in birth defects] cannot be ruled out’. Finally, by disagreeing with Shapiro’s warning that an ‘appreciable increase’ cannot be ruled out, Dr. Stolley in effect was expressing greater confidence in Shapiro’s findings that Shapiro himself thought was justified. ***  In short, because of his apparent bias and his overstatements, the Court discounted Dr. Stolley’s conclusions.”

Id. at 286 (internal citations omitted). Putting aside the issue whether Dr. Stolley was assessing the Shapiro study alone or all the studies together, we can say that the trial court was clearly erroneous in finding an equivocation or an inconsistency between saying a study both failed to show an association and acknowledging that it also failed to rule out a small (or even an appreciable) increased risk.  The trial court had shifted the burden of proof to the defense to show that there was no association.

The trial court, confused by its own conflation of failing to show and failing to rule out, proceeded to dismiss Dr. Stolley’s testimony as having little or no probative value:

“Although these scientific studies are invaluable aids, Dr. Stolley’s testimony demonstrates that the studies alone do not show conclusively whether or not Katie Wells’ birth defects were caused by Ortho-Gynol Contraceptive Jelly. Further, the Court reiterates that plaintiffs’ ultimate burden was not to produce a flawless epidemiological study, but rather to show from all the evidence presented, to a reasonable degree of medical certainty, that the Product caused some or all of Katie Wells’ birth defects in 1980.”

Id. at 286. Lovely rhetoric, but yet the trial court was prepared to hold the defense to a non-legal, insuperable burden to produce an infinitely powerful study that could rule out any increased risk.  Dr. Robert Brent, the defense expert witness on teratology, similarly expressed the limitation of hypothesis testing to “prove” the hull hypothesis of no association:

“[T]here’s no way to prove that a substance is not teratogenic. It’s proving a negative, and, in fact, it’s true of water and vitamins and everything else that we are exposed to. All we can say is that after extensive evaluation, the risk appears no greater than if you didn’t expose yourself to that. That’s about as far as you can go.”

Id. at 289-90 (internal citations omitted).  If the defense expert witnesses committed any mistake it was in not explaining hypothesis testing better in their direct examinations so that statements on cross-examination were not seen as “equivocations.” This tactical mistake, however, cannot excuse or erase the profoundly and clearly erroneous findings of the trial court.

Statistical Significance

It was, after all, Justice Sotomayor’s opinion in Matrixx Initiatives, with Her Honor’s broad dicta  about statistical significance, which motivated my re-reading of the Wells case.  The Supreme Court’s opinion cited the appellate court’s decision to affirm the judgment entered upon Judge Shoob’s verdict, but the essence of the Wells case was established in the proceedings in district court.  Here is what Judge Shoob wrote about statistical significance:

“Plaintiffs’ burden of proving that Katie Wells’ defects were caused by the Product did not necessarily require them to produce scientific studies showing a statistically significant association between spermicides and congenital malformations in a large population.”

615 F. Supp. at 292.  And later, in connection with one of the defense expert witnesses, the court noted:

“In Dr. Brent’s opinion, none of these three indicators suggests that non-ionic surfactants are teratogenic. Other than the Jick study, he was aware of no studies showing a statistically significant association between spermicides and birth defects.”

Id. at 289.  Because there was at least one statistically significant outcome reported in one of the studies at issues, the Wells case cannot represent a legal precedent that established that statistically significant evidence is not necessary to a judgment of causation.  Justice Sotomayor’s opinion in dictum to the contrary is not only wrong as a scientific generalization, it is wrong as a matter of basic legal process.

(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 3

« Older Entries
Newer Entries »