TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

December 27th, 2012

A recent student law review article discusses reanalyses of epidemiologic studies, an important, and overlooked topic in the jurisprudence of scientific evidence.  Alexander J. Bandza, “Epidemiological-Study Reanalyses and Daubert: A Modest Proposal to Level the Playing Field in Toxic Tort Litigation,” 39 Ecology L. Q. 247 (2012).

In the Daubert case itself, the Ninth Circuit, speaking through Judge Kozinksi, avoided the methodological issues raised by Shanna Swan’s reanalysis of Bendectin epidemiologic studies, by assuming arguendo its validity, and holding that the small relative risk yielded by the reanalysis would not support a jury verdict of specific causation. Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1317–18 (9th Cir. 1995).

There is much that can, and should, be said about reanalyses in litigation and in the scientific process, but Bandza never really gets down to the business at hand. His 36 page article curiously does not begin to address reanalysis until the bottom of the 20th page. The first half of the article, and then some, reviews some time-worn insights and factoids about scientific evidence. Finally, at page 266, the author introduces and defines reanalysis:

“Reanalysis occurs ‘when a person other than the original investigator obtains an epidemiologic data set and conducts analyses to evaluate the quality, reliability or validity of the dataset, methods, results or conclusions reported by the original investigator’.”

Bandza at 266 (quoting Raymond Neutra et al., “Toward Guidelines for the Ethical Reanalysis and Reinterpretation of Another’s Research,” 17 Epidemiology 335, 335 (2006).

Bandza correctly identifies some of the bases for judicial hostility to re-analyses. For instance, some courts are troubled or confused when expert witnesses disagree with, or reevaluate, the conclusions of a published article. The witnesses’ conclusions may not be published or peer reviewed, and thus the proffered testimony fails one of the Daubert factors.  Bandza correctly notes that peer review is greatly overrated by judges. Bandza at 270. I would add that peer review is an inappropriate proxy for validity, a “test,” which reflects a distrust of the unpublished.  Unfortunately, this judicial factor ignores the poor quality of much of what is published, and the extreme variability in the peer review process. Judges overrate peer review because they are desperate for a proxy for validity of the studies relied upon, which will allow them to pass their gatekeeping responsibility on to the jury. Furthermore, the authors’ own conclusions are hearsay, and their qualifications are often not fully before the court.  What is important is the opinion of the expert witness who can be cross-examined and challenged.  See “FOLLOW THE DATA, NOT THE DISCUSSION.” What counts is the validity of the expert witness’s reasoning and inferences.

Bandza’s article, which by title advertises itself to be about re-analyses, gives only a few examples of re-analyses without much detail.  He notes concerns that reanalyses may impugn the reputation of published scientists, and burden them with defending their data.  Who would have it any other way? After this short discussion, the article careens into a discussion of “weight of the evidence” (WOE) methodology. Bandza tells us that the rejection of re-analyses in judicial proceedings “implicitly rules out using the weight-of-the-evidence methodology often appropriate for, or even necessary to, scientific analysis of potentially toxic substances.” Bandza at 270.  This argument, however, is one sustained non-sequitur.  WOE is defined in several ways, but none of the definitions require or suggest the incorporation of re-analyses. Re-analyses raise reliability and validity issues regardless whether an expert witness incorporates them into a WOE assessment. Yet Bandza tells us that the rejection of re-analyses “Implicitly Ignores the Weight-of-the-Evidence Methodology Appropriate for the Scientific Analysis of Potentially Toxic Substances.” Bandza at 274. This conclusion simply does not follow from the nature of WOE methodology or reanalyses.

Bandza’s ipse dixit raises the independent issue whether WOE methodology is appropriate for scientific analysis. WOE is described as embraced or used by regulatory agencies, but that description hardly recommends the methodology as the basis for a scientific, as opposed to a regulatory, conclusion.  Furthermore, Bandza ignores the ambiguity and variability of WOE by referring to it as a methodology, when in reality, WOE is used to describe a wide variety of methods of reasoning to a conclusion. Bandza cites Douglas Weed’s article on WOE, but fails to come to grips with the serious objections raised by Weed in his article to the use of WOE methodologies.  Douglas Weed, “Weight of Evidence: A Review of Concept and Methods,” 25 Risk Analysis 1545, 1546–52 (2005) (describing the vagueness and imprecision of WOE methodologies). See also “WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name.”

Bandza concludes his article with a hymn to the First Circuit’s decision in Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir. 2011). Plaintiffs’ expert witness, Dr. Martyn Smith claimed to have performed a WOE analysis, which in turn was based upon a re-analysis of several epidemiologic studies. True, true, and immaterial.  The re-analyses were not inherently a part of a WOE approach. Presumably, Smith re-analyzed some of the epidemiologic studies because he felt that the data as presented did not support his desired conclusion.  Given the motivations at work, the district court in Milward was correct to look skeptically and critically at the re-analyses.

Bandza notes that there are procedural and evidentiary safeguards in federal court against unreliable or invalid re-analyses of epidemiologic studies.  Bandza at 277. Yes, there are safeguards but they help only when they are actually used. The First Circuit in Milward reversed the district court for looking too closely at the re-analyses, spouting the chestnut that the objections went to the weight not the admissibility of the evidence.  Bandza embraces the rhetoric of the Circuit, but he offers no description or analysis of the liberties that Martyn Smith took with the data, or the reasonableness of Smith’s reliance upon the re-analyzed data.

There is no necessary connection between WOE methodologies and re-analyses of epidemiologic studies.  Re-analyses can be done properly to support or deconstruct the conclusions of published papers.  As Bandza points out, some re-analyses may go on to be peer reviewed and published themselves.  Validity is the key, and WOE methodologies have little to do with the process of evaluating the original or the re-analyzed study.

 

 

Posted in Causation, Data Sharing, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

Lumpenepidemiology

December 24th, 2012

Judge Helen Berrigan, who presides over the Paxil birth defects MDL in New Orleans, has issued a nicely reasoned Rule 702 opinion, upholding defense objections to plaintiffs expert witnesses, Paul Goldstein, Ph.D., and Shira Kramer, Ph.D. Frischhertz v SmithKline Beecham EDLa 2012 702 MSJ Op.

The plaintiff, Andrea Frischhertz, took GSK’s Paxil, a selective serotonin reuptake inhibitor (SSRI), for depression while pregnant with her daughter, E.F. The parties agreed that E.F. was born with a deformity of her right hand.  Plaintiffs originally claimed that E.F. had a heart defect, but their expert witnesses appeared to give up this claim at deposition, as lacking evidential support.

Adhering to Daubert’s Epistemiologic Lesson

Like many other lower federal courts, Judge Berrigan focused her analysis on the language of Daubert v. Merrell Dow Pharmaceuticals Inc., 509 U.S. 579 (1993), a case that has been superseded by subsequent cases and a revision to the operative statute, Rule 702.  Fortunately, the trial court did not lose sight of the key epistemological teaching of Daubert, which is based upon Rule 702:

“Regarding reliability, the [Daubert] Court said: ‘the subject of an expert’s testimony must be “scientific . . . knowledge.” The adjective “scientific” implies a grounding in the methods and procedures of science. Similarly, the word “knowledge” connotes more than subjective belief or unsupported speculation’.”

Slip Op. at 3 (quoting Daubert, 509 U.S. at 589-590).

There was not much to the plaintiffs’ expert witnesses’ opinion beyond speculation, but many other courts have been beguiled by speculation dressed up as “scientific … knowledge.”  Dr. Goldstein relied upon whole embryo culture testing of SSRIs, but in the face overwhelming evidence, Dr. Goldstein was forced to concede that this test may generate hypotheses about, but cannot predict, human risk of birth defects.  No doubt this concession made the trial court’s decision easier, but the result would have been required regardless of Dr. Goldstein’s exhibition of truthfulness at deposition.

Statistical Association – A Good Place to Begin

More interestingly, the trial court rejected the plaintiffs’ expert witnesses’ efforts to leapfrog finding a statistically significant association to parsing the so-called Bradford Hill factors:

“The Bradford-Hill criteria can only be applied after a statistically significant association has been identified. Federal Judicial Center, Reference Manual on Scientific Evidence, 599, n.141 (3d. ed. 2011) (“In a number of cases, experts attempted to use these guidelines to support the existence of causation in the absence of any epidemiologic studies finding an association . . . . There may be some logic to that effort, but it does not reflect accepted epidemiologic methodology.”). See, e.g., Dunn v. Sandoz Pharms., 275 F. Supp. 2d 672, 678 (M.D.N.C. 2003). Here, Dr. Goldstein attempted to use the Bradford-Hill criteria to prove causation without first identifying a valid statistically significant association. He first developed a hypothesis and then attempted to use the Bradford-Hill criteria to prove it. Rec. Doc. 187, Exh. 2, depo. Goldstein, p. 103. Because there is no data showing an association between Paxil and limb defects, no association existed for Dr. Goldstein to apply the Bradford-Hill criteria. Hence, Dr. Goldstein’s general causation opinion is not reliable.”

Slip op. at 6.

The trial court’s rejection of Dr. Goldstein’s attempted end run is particularly noteworthy given the Reference Manual’s weak-kneed attempt to suggest that this reasoning has “some logic” to it.  The Manual never articulates what “logic” commends Dr. Goldstein’s approach; nor does it identify any causal relationship ever established with such paltry evidence in the real world of science. The Manual does cite several legal cases that excused or overlooked the need to find a statistically significant association, and even elevated such reasoning into legally acceptable, admissibility method.  See Reference Manual on Scientific Evidence at 599 n. 141 (describing cases in which purported expert witnesses attempted to use Bradford Hill factors in the absence of a statistically significant association; citing Rains v. PPG Indus., Inc., 361 F. Supp. 2d 829, 836–37 (S.D. Ill. 2004); ); Soldo v. Sandoz Pharms. Corp., 244 F. Supp. 2d 434, 460–61 (W.D. Pa. 2003).  The Reference Manual also cited cases, without obvious disapproval, which completely dispatched with any necessity of considering any of the Bradford Hill factors, or the precondition of a statistically significant association.  See Reference Manual at 599 n. 144 (citing Cook v. Rockwell Int’l Corp., 580 F. Supp. 2d 1071, 1098 (D. Colo. 2006) (“Defendants cite no authority, scientific or legal, that compliance with all, or even one, of these factors is required. . . . The scientific consensus is, in fact, to the contrary. It identifies Defendants’ list of factors as some of the nine factors or lenses that guide epidemiologists in making judgments about causation. . . . These factors are not tests for determining the reliability of any study or the causal inferences drawn from it.“).

Shira Kramer Takes Her Lumpings

The plaintiffs’ other key expert witness, Dr. Shira Kramer, was a more sophisticated and experienced obfuscator.  Kramer attempted to provide plaintiffs with a necessary association by “lumping” all birth defects together in her analysis of epidemiologic data of birth defects among children of women who had ingested Paxil (or other SSRIs).  Given the clear evidence that different birth defects arise at different times, based upon interference with different embryological processes, the trial court discerned this “lumping” of end points to be methodologically inappropriate.  Slip op. at 8 (citing Chamber v. Exxon Corp., 81 F. Supp. 2d 661 (M.D. La. 2000), aff’d, 247 F.3d 240 (5th Cir. 2001) (unpublished).

Without her “lumping”, Dr. Kramer was left with only a weak, inconsistent claim of biological plausibility and temporality. Finding that Dr. Kramer’s opinion had outrun her headlights, Judge Berrigan, excluded Dr. Kramer as an expert witness, and granted GSK summary judgment.

Merry Christmas!

 

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on Lumpenepidemiology

Egilman Petitions the Supreme Court for Review of His Own Exclusion in Newkirk v. Conagra Foods

December 13th, 2012

Last year, the Ninth Circuit of the United States Court of Appeals affirmed a district judge’s decision to exclude Dr David S. Egilman from testifying in a consumer-exposure diacetyl case.  Newkirk v. Conagra Foods Inc., 438 Fed.Appx. 607  (9th Cir. 2011).  The plaintiff moved on, but his expert witness could not let his exclusion go.

To get the full “flavor” of this diacetyl case, read the district court’s opinion, which excluded Egilman and other witnesses, and entered summary judgment for the defense. Newkirk v. Conagra Foods, Inc., 727 F. Supp. 2d 1006  (E.D. Wash. July 2, 2010).  Here is the language that had Dr. Egilman popping mad:

“In other parts of his reports and testimony, Dr. Egilman relies on existing data, mostly in the form of published studies, but draws conclusions far beyond what the study authors concluded, or Dr. Egilman manipulates the data from those studies to reach misleading conclusions of his own. See Daubert I, 509 U.S. at 592–93, 113 S.Ct. 2786.”

727 F. Supp. 2d at 1018.

This language, cut Dr. Egilman to the kernel, and provoked him to lodge a personal appeal to the Ninth Circuit, based in part upon the economic harm done to his litigation consulting and testimonial practice. (See attached Egilman Motion Appeal Diacetyl Exclusion 2011 and Egilman Declaration Newkirk Diacetyl Appeal 2011.)  Not only did the exclusion hurt Dr. Egilman’s livelihood, but also his eleemosynary endeavors:

“The Daubert ruling eliminates my ability to testify in this case and in others. I will lose the opportunity to bill for services in this case and in others (although I generally donate most fees related to courtroom testimony to charitable organizations, the lack of opportunity to do so is an injury to me). Based on my experience, it is virtually certain that some lawyers will choose not to attempt to retain me as a result of this ruling. Some lawyers will be dissuaded from retaining my services because the ruling is replete with unsubstantiated pejorative attacks on my qualifications as a scientist and expert. The judge’s rejection of my opinion is primarily an ad hominem attack and not based on an actual analysis of what I said – in an effort to deflect the ad hominem nature of the attack the judge creates ‘straw man’ arguments and then knocks the straw men down, without ever addressing the substance of my positions.”

Egilman Declaration in Newkirk at Paragraph 11.

The Ninth Circuit affirmed Dr. Egilman’s exclusion, Newkirk v. Conagra Foods, Inc., 438 Fed. Appx. 607 (9th Cir. 2011).  See “Ninth Circuit Affirms Rule 702 Exclusion of Dr David Egilman in Diacetyl Case.

This year, the Ninth Circuit dismissed his personal appeal for lack of standing.  Egilman v. Conagra Foods, Inc., 2012 WL 3836100 (9th Cir. 2012). Previously, I suggested that the Ninth Circuit had issued a judgment from which there will be no appeal.  I may have been mistaken.  Last week, counsel for Dr. Egilman filed a petition for certiorari in the United States Supreme Court.  Smarting from the district court’s attack on his character and professionalism, Dr. Egilman is seeking the personal right to appeal an adverse Rule 702 ruling.  The Circuit split, which Dr. Egilman hopes will get him a hearing in the Supreme Court, involves the issue whether he, as a non-party witness, must intervene in the proceedings in order to preserve his right to appeal:

“Whether a nonparty to a district court proceeding has a right to appeal a decision that adversely affects his interest, as the Second, Sixth, and D.C. Circuits hold, or whether, as six other circuit courts hold, the nonparty must intervene or otherwise participate in the district court proceedings to have a right to appeal.”

Egilman Pet’n Cert Newkirk v Conagra SCOTUS at 5 (Dec. 2012).  Of course there is also a split among courts about Dr. Egilman reliability.

And who represents Dr. Egilman?  Counsel of record is Alexander A. Reinert, who teaches at Cardozo Law School, here in New York.  Dr. Egilman and Reinert have published several articles together, within the scope of Dr. Egilman’s litigation-oriented practice.[i]  In the past, I have commented upon Reinert’s work.  See, e.g., Schachtman, “Confidence in Intervals and Diffidence in the Courts” (May 8, 2012 ) (Arthur H. Bryant & Alexander A. Reinert, “The Legal System’s Use of Epidemiology,” 87 Judicature 12, 19 (2003)(“The confidence interval is intended to provide a range of values within which, at a specified level of certainty, the magnitude of association lies.”) (incorrectly citing the first edition of Rothman & Greenland, Modern Epidemiology 190 (Philadelphia 1998)). It should be interesting to see what mischief Egilman & Reinert can make in the Supreme Court.

[i] David S. Egilman & Alexander A. Reinert, “Corruption of Previously Published Asbestos Research,” 55 Arch. Envt’l Health 75 (2000); David S. Egilman & Alexander A. Reinert,“Asbestos Exposure and Lung Cancer: Asbestosis Is Not Necessary,” 30 Am. J. Indus. Med. 398 (1996); David S. Egilman & Alexander A. Reinert, “The Asbestos TLV: Early Evidence of Inadequacy,” Am. J. Indus. Med. 369 (1996);  David S. Egilman & Alexander A. Reinert,“The Origin and Development of the Asbestos Threshold Limit Value: Scientific Indifference and Corporate Influence,”  25 Internat’l J. Health Serv. 667 (1995).

Posted in Expert Witnesses, Rule 702 | Comments Off on Egilman Petitions the Supreme Court for Review of His Own Exclusion in Newkirk v. Conagra Foods

EPA Post Hoc Statistical Tests – One Tail vs Two

December 2nd, 2012

EPA 1992 Meta-Analysis of ETA & Lung Cancer – Part 2

In 1992, the U.S. Environmental Protection Agency (EPA) published a risk assessment of lung cancer (and other) risks from environmental tobacco smoke (ETS).  See Respiratory Health Effects of Passive Smoking: Lung Cancer and Other Disorders EPA/600/6-90/006F (1992).  The agency concluded that ETS causes about 3,000 lung cancer deaths each year among non-smoking adults.  See also EPA “Fact Sheet: Respiratory Health Effects of Passive Smoking,” Office of Research and Development, and Office of Air and Radiation, EPA Document Number 43-F-93-003 (Jan. 1993).

In my last post, I discussed  how various plaintiffs, including tobacco companies, challenged the EPA’s conclusions as agency action that violated administrative and statutory procedures. “EPA Cherry Picking (WOE) – EPA 1992 Meta-Analysis of ETA & Lung Cancer – Part 1” (Dec. 2. 2012). The plaintiffs further claimed that the EPA had manufactured its methods to achieve the result it desired in advance of the analyses. A federal district court agreed with the methodological challenges to the EPA’s report, but the Court of Appeals reversed on grounds that the agency’s report was not reviewable agency action.  Flue-Cured Tobacco Cooperative Stabilization Corp. v. EPA, 4 F. Supp. 2d 435 (M.D.N.C. 1998), rev’d 313 F.3d 852, 862 (4th Cir. 2002) (Widener, J.) (holding that the issuance of the report was not “final agency action”).

One of the grounds of the plaintiffs’ challenge was that the EPA had changed, without explanation, from a 95% to a 90% confidence interval.  The change in the specification of the coefficient of confidence was equivalent to a shift from a two-tailed to a one-tailed test of confidence, with alpha set at 5%.  This change, along with gerrymandering or “cherry picking” of studies, allowed the EPA to claim a statistically significant association between ETS and lung cancer. 4 F. Supp. 2d at 461.  The plaintiffs pointed to EPA’s own previous risk assessments, as well as statistical analyses by the World Health Organization (International Agency for Research on Cancer), the National Research Council, and the Surgeon General, all of which routinely use 95% intervals, and two-tailed tests of significance.  Id.

In its 1990 Draft ETS Risk Assessment, the EPA had used a 95% confidence interval, but in later drafts, changed to a 90% interval.  One of the epidemiologists on the EPA’s Scientific Advisory Board, Geoffrey Kabat, criticized this post hoc change, noting that the use of 90% intervals are disfavored and that the post hoc change in statistical methodology created the appearance of an intent to influence the outcome of the analysis. Id. (citing Geoffrey Kabat, “Comments on EPA’s Draft Report: Respiratory Health Effects of Passive Smoking: Lung Cancer and Other Disorders,” II.SAB.9.15 at 6 (July 28, 1992) (JA 12,185).

The EPA argued that its adoption of a one-tailed test of significance was justified on the basis of an a priori hypothesis that ETS is associated with lung cancer.  Id. at 451-52, 461 (citing to ETS Risk Assessment at 5–2). The court found this EPA argument hopelessly circular.  The agency postulated its a priori hypothesis, which it then took as license to dilute the statistical test for assessing the evidence.  The agency, therefore, had assumed what it wished to show, in order to achieve the result it sought.  Id. at 456.  The EPA claimed that the one-tailed test had more power, but with dozens of studies aggregated into a summary result, the court recognized that Type I error was a larger threat to the validity of the agency’s conclusions.

The EPA also advanced a muddled defense of its use of 90% confidence intervals by arguing that if it used a 95% interval, the results would have been incongruent with the one-tailed p-values.  The court recognized that this was really no discrepancy at all, but only a corollary of using either one-tailed 5% tests or 90% confidence intervals.  Id. at 461.

If the EPA had adhered to its normal methodology, there would have been no statistically significant association between ETS and lung cancer. With its post hoc methodological choice, and highly selective approach to study inclusions in its meta-analysis, the EPA was able to claim a weak statistically significant association between ETS and lung cancer.  Id. at 463.  The court found this to be a deviation from the legally required use of “best judgment possible based upon the available evidence.”  Id.

Of course, the EPA could have announced its one-tailed test from the inception of the risk assessment, and justified its use on grounds that it was attempting to reach only a precautionary judgment for purposes of regulation.  Instead, the agency tried to showcase its finding as a scientific conclusion, which only further supported the tobacco companies’ challenge to the post hoc change in plan for statistical analysis.

Although the validity issues in the EPA’s 1992 meta-analysis should have been superseded by later studies, and later meta-analyses, the government’s fraud case, before Judge Kessler, resurrected the issue:

“3344. Defendants criticized EPA’s meta-analysis of U.S. epidemiological studies, particularly its use of an ‘unconventional 90 percent confidence interval’. However, Dr. [David] Burns, who participated in the EPA Risk Assessment, testified that the EPA used a one-tailed 95% confidence interval, not a two-tailed 90% confidence interval. He also explained in detail why a one-tailed test was proper: The EPA did not use a 90% confidence interval. They used a traditional 95% confidence interval, but they tested for that interval only in one direction. That is, rather than testing for both the possibility that exposure to ETS increased risk and the possibility that it decreased risk, the EPA only tested for the possibility that it increased the risk. It tested for that possibility using the traditional 5% chance or a P value of 0.05. It did not test for the possibility that ETS protected those exposed from developing lung cancer at the direction of the advisory panel which made that decision based on its prior decision that the evidence established that ETS was a carcinogen. What was being tested was whether the exposure was sufficient to increase lung cancer risk, not whether the agent itself, that is cigarette smoke, had the capacity to cause lung cancer with sufficient exposure. The statement that a 90% confidence interval was used comes from the observation that if you test for a 5% probability in one direction the boundary is the same as testing for a 10% probability in two directions. Burns WD, 67:5-15. In fact, the EPA Risk Assessment stated, ‘Throughout this chapter, one-tailed tests of significance (p = 0.05) are used …’ .”

U.S. v. Philip Morris USA, Inc., 449 F. Supp. 2d 1, 702-03 (D.D.C., 2006) (Kessler, J.) (internal citations omitted).

Judge Kessler was misled by Dr. Burns, a frequent testifier for plaintiffs’ counsel in tobacco cases.  Burns should have known that with respect to the lower bound of the confidence interval, which is what matters for determining whether the meta-analysis excludes a risk ratio of 1.0, there is no difference between a one-tailed 95% confidence interval and a two-tailed 90% interval.  Burns’ sophistry hardly saves the EPA’s error in changing its pre-specified end point and statistical analysis, or the danger of unduly increasing the risk of Type I error in the EPA meta-analysis. See “Pin the Tail on the Significance Test” (July 14th, 2012)

Post-script

Judge Widener wrote the opinion for a panel of the United States Court of Appeals, for the Fourth Circuit, which reversed the district court’s judgment, enjoining the EPA’s report.  The Circuit’s decision did not address the scientific issues, but by holding that the agency action was not reviewable, the basis for the district court’ review of the scientific and statistical issues was removed.  For those pundits who see only self-interested behavior in judging, the author of the Circuit’s decision was a life-time smoker, who grew Burley tobacco on his farm, outside Abingdon, Virginia.  Judge Widener died on September 19, 2007, of lung cancer.

Posted in Causation, Expert Witnesses, Meta-analysis, statistical evidence | Comments Off on EPA Post Hoc Statistical Tests – One Tail vs Two

Bad Gatekeeping or Missed Opportunity – Allen v. Martin Surfacing

November 30th, 2012

Sometimes when federal courts permit dubious causation opinion testimony over Rule 702 objections, the culprit is bad lawyering by the opponent of the proffered testimony.  Allen v. Martin Surfacing, 263 F.R.D. 47 (D. Mass. 2009), may be an important example.

THE CLAIMS

Daniel Allen was the former football coach of the College of The Holy Cross, in Worcester, Massachusetts.  In spring of 2001, defendant Martin Surfacing refinished the gymnasium floor at the college.  Coach Allen was exposed to solvent fumes, including toluene fumes, during defendant’s work, as well as for a couple of months afterwards.   While exposed, Allen experienced “dizziness, headaches, and disorientation.” 263 F.R.D. at 51.  After the gym floor resurfacing was completed, Allen experienced other symptoms, such as fatigue, muscle weakness, and fasciculations in his lower limbs.  In January 2002, at the age of 45, Allen was diagnosed with amyotrophic lateral sclerosis (ALS).  Id. Allen’s condition progressed, and he died three years later, in May 2004.  Id. at 52.

Allen’s family sued for wrongful death.  The parties’ apparently agreed on the following:

  • ALS occurs as a sporadic ALS, as well as “familial ALS,”
  • the cause of sporadic ALS is unknown,
  • Allen developed and died of sporadic ALS,
  • no air sampling established overexposure to any chemical,
  • there were no reliable exposure models to quantify Allen’s exposures,
  • there are no known causes of sporadic ALS, and
  • toluene did not cause Allen’s ALS

Remarkably, defendant lost the Rule 702 challenge to plaintiffs’ expert witnesses’ opinion testimony.  It is easy to suspect that the district judge was sleep at the gate, and that his gatekeeping was deficient.  A close read of the opinion supports the view that this was not Rule 702’s finest moment, but much more was going on to get to admissibility.

First, the plaintiffs’ counsel cleverly avoided running into a wall by avoiding a claim that toluene caused Allen’s ALS. Instead, plaintiffs’ claimed that toluene accelerated the onset of the disease.  This claim was equally dubious, but it allowed the expert witnesses to avoid a mountain of medical opinion, authoritative and well-supportive, that there is no known cause of sporadic ALS.

Second, the plaintiffs’ counsel took the initiative by filing an affirmative motion to admit the testimony of their expert witnesses.  Rather than ceding the initiative to the defendant, the plaintiffs seized the initiative and had the first and last word on admissibility.  As a result, plaintiffs were able to present and frame their witnesses’ opinions sympathetically rather than defensively.

Third, the plaintiffs had the good fortune of the defendant’s counsel’s apparent failure to find the key fallacies, invalidities, and flaws in plaintiffs’ questionable expert witness opinions.

The Allen case teaches that sometimes good lawyering can win a losing case.

The plaintiffs’ counsel retained and presented an array of expert witnesses who might be the usual suspects in a district court’s exclusion of expert witness testimony:

None of these four expert witnesses was a specialist in ALS or ALS causation; none was a neurologist; none had ever addressed ALS causation in a peer-reviewed article.  All four witnesses were frequent testifiers in tort litigation, and some have are repeat offenders when it comes to offering questionable or excludable opinion testimony.  Somehow, the defense dashed this opportunity by retaining only one expert, Dean M. Hashimoto, M.D., J.D., M.P.H., who was also not a specialist in ALS, who was not a neurologist, and who had never published anything on ALS.  And to make matters worse, the defense proceeded to challenge the plaintiffs’ expert witnesses for lack of qualifications!

The defense’s challenges to qualifications takes up a good deal of Judge Saylor’s published opinion, which illustrates the maxim that judges have short attention spans, and you should not waste the opportunity of a motion on an issue that is so easily decided against you.  The scientific issues are difficult and the temptation to avoid them is great.  By leading with an issue that will almost certainly lose, the defense wasted a valuable advocacy opportunity to show the court the fallacious reasoning in the plaintiffs’ case.  By submitting reports from only one expert witness, who had all the deficiencies claimed in the plaintiffs’ set of witnesses, the defense exhibited a duplicity that must have seriously undermined its credibility for the entire set of Rule 702 motion issues.

THE WITNESSES

Dr. Christine Oliver has been testifying in asbestos and other occupational lung disease cases for decades.  She is a pulmonary physician on staff at Massachusetts General Hospital, in Boston, and an associate professor of clinical medicine at the Harvard Medical School.  She is board certified in internal medicine and in occupational medicine (American Board of Preventive Medicine), and her clinical interests are asthma occupational lung disease, and health hazards of construction work.  If the defense had presented real expert witnesses in ALS causation, Dr. Oliver’s expertise would have seemed quite irrelevant.  Dr. Oliver has, as well as I can determine, never researched or published on ALS causation.  She has, however, published on “multiple chemical sensitivity,” which should give a disinterested court some pause.  See L. Christine Oliver and Alison Johnson, “Multiple Chemical Sensitivity: Reflections” (Nov. 4, 2011).

Richard Clapp, professor emeritus at the Boston University School of Public Health, is a known purveyor of dubious courtroom testimony. See, e.g., Sutera v. The Perrier Group of America Inc., 986 F.Supp. 655 (D. Mass. 1997).  He is a frequent testifier and a charter member of the surreptitiously funded SKAPP organization.  Clapp is a non-physician epidemiologist, who has never published on ALS.

Marcia Ratner Ph.D. may be best known for her possession of mace and an unlicensed gun, but she does occasionally show up in civil litigation as an expert witness.  See “Quincy District Court News,” Patriot Ledger June 09, 2010 (reporting that Ratner pleaded guilty to criminal possession of mace and a firearm).

Ratner is a postdoctoral researcher at Boston University, where she works as a neurotoxicologist.  She does not appear to have ever published a peer-reviewed paper on ALS or ALS causation.  Plaintiffs’ counsel claimed that she was researching a new drug with therapeutic potential for ALS treatment, although they were quite sketchy about details.  Ratner does not appear to hold any NIH grants for ALS drug research.

[Please see update on the discussion of Dr. Ratner at http://schachtmanlaw.com/gatekeeping-in-allen-v-martin-surfacing-postscript/]

William Ewing, an industrial hygienist, frequently testifies in asbestos litigation.  He offered no opinion on causation.

Against this field of witnesses, the defense punted on presenting its own witness with relevant expertise. Dr. Dean M. Hashimoto, the defense’s sole witness on causation, is a physician, lawyer, and has a master’s degree in occupational health.  Hashimoto has no specialized training in ALS or clinical neurology, although he serves on the Massachusetts Workers’ Compensation Board. A pubmed search  shows that Hashimoto has never published on the neurology or causation of ALS.

CAUSATION

The plaintiffs had a huge problem to avoid:  ALS has no known cause.  Counsel table could be filled up with textbooks and review articles, but perhaps the following, lengthy quote from the National Institute for Neurological Disorders and Stroke website suffices to make the point:

“What causes ALS?

The cause of ALS is not known, and scientists do not yet know why ALS strikes some people and not others. An important step toward answering that question came in 1993 when scientists supported by the National Institute of Neurological Disorders and Stroke (NINDS) discovered that mutations in the gene that produces the SOD1 enzyme were associated with some cases of familial ALS. This enzyme is a powerful antioxidant that protects the body from damage caused by free radicals. Free radicals are highly reactive molecules produced by cells during normal metabolism. If not neutralized, free radicals can accumulate and cause random damage to the DNA and proteins within cells. Although it is not yet clear how the SOD1 gene mutation leads to motor neuron degeneration, researchers have theorized that an accumulation of free radicals may result from the faulty functioning of this gene. In support of this, animal studies have shown that motor neuron degeneration and deficits in motor function accompany the presence of the SOD1 mutation.

Studies also have focused on the role of glutamate in motor neuron degeneration. Glutamate is one of the chemical messengers or neurotransmitters in the brain. Scientists have found that, compared to healthy people, ALS patients have higher levels of glutamate in the serum and spinal fluid. Laboratory studies have demonstrated that neurons begin to die off when they are exposed over long periods to excessive amounts of glutamate. Now, scientists are trying to understand what mechanisms lead to a buildup of unneeded glutamate in the spinal fluid and how this imbalance could contribute to the development of ALS.

Autoimmune responses—which occur when the body’s immune system attacks normal cells—have been suggested as one possible cause for motor neuron degeneration in ALS. Some scientists theorize that antibodies may directly or indirectly impair the function of motor neurons, interfering with the transmission of signals between the brain and muscles.

In searching for the cause of ALS, researchers have also studied environmental factors such as exposure to toxic or infectious agents. Other research has examined the possible role of dietary deficiency or trauma. However, as of yet, there is insufficient evidence to implicate these factors as causes of ALS.

Future research may show that many factors, including a genetic predisposition, are involved in the development of ALS.”

NINDS – “Amyotrophic Lateral Sclerosis (ALS) Fact Sheet.”

As a result, the plaintiffs adopted a strategy of confession and avoidance; they renounced any claim that they were asserting a causal claim.  Instead, they insisted that they were “merely” claiming that toluene exposure had accelerated the onset of sporadic ALS in Coach Allen.  This mere claim, however, was actually a causal claim in disguise, and the district judge was taken in by the ruse.  If plaintiffs were claiming that toluene can accelerate the onset of ALS by a meaningful period of time (years), then they were making a causal claim, legally and scientifically.  A shift in the age of onset of a sporadic disease is a causal claim, and it requires supporting evidence, not hand waving.

PLAUSIBLE MECHANISM

One scientist could postulate a reasonable mechanism even for a sporadic disease.  Professional journals and textbooks are filled with such speculation.  These postulations are part of science in that they inform research hypotheses and funding, but they are not conclusions of causality.  The quote above from the NINDS discusses the lack of an anti-oxidizing enzyme and glutamate toxicity as potential mechanisms in familial ALS, but even there, the authors are appropriately modest in avoiding a claim to know the pathogenesis of familial ALS.

The plaintiffs’ approach was to take the suggestion of a mechanism, misrepresent it as a known mechanism, and then claim that toluene activated glutamate toxicity and exercised an oxidizing effect on neurons. The plaintiffs’ team had no basis for claiming that short-term exposure to solvents, or toluene specifically, translated into a toxicity to the relevant human motor neurons that are involved in ALS.  It is a long stretch from suggesting a mechanism to documenting the mechanism to be actually at work in producing, or accelerating, a disease in humans.

A typical statement, from the Yale School of Medicine, Division of Neurology, in 2012:

“Why the motor neurons begin to die is still unknown. Recent evidence, however, have implicated glutamate excitotoxicity, free radical toxicity, and mitochondrial dysfunction as possible mechanisms, and this is an area of active research.”

“Amyotrophic Lateral Sclerosis (ALS)” (emphasis added).   See also Adams and Victor’s Principles of Neurology 1157-58 (7th ed. 2001) (noting that the pathogenesis of ALS and similar motor neuron diseases is not known).

The district judge seemed mesmerized by Ratner’s having providing a biologically plausible theory for tying ALS progression to toluene exposure.  263 F.R.D. at 60.  Judge Saylor stated that the defense did not address any flaw in Ratner’s methodology other than to point out that her theory was not supported by epidemiology.  The court seemed to equate providing a plausible theory with establishing a scientific conclusion.  More to the point, the court was truly asleep at its gatekeeping task because Ratner’s theory actually presupposed that she knew that Coach Allen was going to develop ALS in any event, only not as early as 2001.  The court faulted the defense for not showing that Ratner’s (and the other plaintiffs’ witnesses’) theory was unreliable, but the burden was on the plaintiffs to show reliability.  Id.  The court not only faulted the defense for carrying a burden it did not have, but it overlooked the very telling criticisms of Ratner’s theories of acceleration and mechanism.

EXCUSES – EPIDEMIOLOGY

Plaintiffs’ expert witnesses had a welter of excuses as to why there was no epidemiologic data to support their theories.  The absence of statistical significance, according to plaintiffs’ expert witnesses does not mean that a study should be disregarded.  Id. at 58.  Their claim is superficially true, but a study not disregarded does not necessarily support a causal inference, either alone or conjunction with other such studies. Similarly, plaintiffs’ claim that flawed studies should not be disregarded is also a half truth.  A flawed study may lead to a much better one, which can support valid inferences.  Flawed studies are thus part of the scientific process because they may lead to a self-correcting triangulation of the truth, but there is little to recommend relying upon flawed studies to support scientific conclusions of causality.  Nevertheless, the district court appeared to swallow these half truths, whole.

Ratner also advanced a claim that the acceleration theory had not been subjected to epidemiologic analysis because of “funding limitations, as most funding goes toward finding treatment or cures for the disease, not towards finding what accelerates the course of the disease.”  Id. at 59 n. 14.  The district court repeats this excuse without critical thought.  If a commonly used solvent such as toluene accelerated the onset of a terrible disease such as ALS by decades, such a putative effect would be amenable to epidemiologic analysis and would be a source of incredible concern and funding efforts by the NIH, NINDS, NIEHS, and other granting agencies and organizations.  Despite excusifying verbiage, Ratner maintained that there were no epidemiologic data that refuted her novel acceleration.  Id. at 59.  Of course, if her excuses were taken seriously, then this absence of refutation was fairly irrelevant, but in any event, this supposed absence could not support the reliability of Ratner’s inferences or conclusions.

The defense focused on the lack of short-term exposures in epidemiologic studies, and also the lack of statistical significance in some studies.  What appears to have been missing from both sides was a comprehensive analysis of the available epidemiologic data.  If long-term exposure were associated with earlier age of onset of ALS, or even a greater risk of ALS, then it would have given some support to Ratner’s novel theory.  The defense appeared to punt on the epidemiology by claiming its irrelevance.  It might have been helpful to point out internal as well as external validity issues to the court.

As for both sides citing different studies, and no side presenting a comprehensive view of the epidemiologic evidence, the court could have given some consideration to the ethical considerations of the incomplete presentation:

“Basis of Expert Medical Testimony

The testimony of an expert medical witness should be founded on a thorough and critical review of the pertinent medical and scientific facts, available data, and relevant literature.”

Ethical Guidelines for Occupational and Environmental Medicine Physicians Serving as Expert Witnesses (Oct. 25, 2007).

DIFFERENTIAL DIAGNOSIS

The plaintiffs’ claim that they were not asserting causation was disingenuous.  As noted above, acceleration of onset is a form of causation.  Of course, exposure to a neurotoxic material, with some symptoms, might have made Allen more aware of other symptoms, and so the time to diagnosis was abbreviated.  The plaintiffs, however, were claiming more than earlier ascertainment; they claimed the toluene exposure caused an underlying disease process to accelerate.

Oliver actually went further and performed an invalid differential etiologic analysis. Oliver reviewed medical records and claimed to have applied “differential diagnosis to the review.”  Id. at 63. This claim was quite bogus because there was no dispute that Allen had and died of ALS, but the district court was beguiled.  Having ruled out family history, Oliver claimed to then rule out other “putative causes” of ALS:  “pesticides and agricultural chemicals containing solvents, 60-hertz magnetic fields, and welding fumes.”  Id. at 63.  In one fell swoop, Oliver created several known causes to be ruled out, and then ruled them out in Allen’s case.  This is remarkable given that NINDS and most of medical sciences does not recognize any known or putative causes of sporadic ALS, and that Oliver failed to rule out the one potential cause that some scientists take seriously:  cigarette smoking.  See, e.g., Hao Wang, Éilis J. O’Reilly, Marc G. Weisskopf, Giancarlo Logroscino, Marji L. McCullough, Michael Thun, Arthur Schatzkin, Laurence N. Kolonel, Alberto Ascherio, “Smoking and risk of amyotrophic lateral sclerosis: a pooled analysis of 5 prospective cohorts” 68 Arch. Neurol. 207 (2011); A. Alonso, G. Logroscino, M.A. Hernán, “Smoking and the risk of amyotrophic lateral sclerosis: a systematic review and meta-analysis,” 81 J. Neurol. Neurosurg. & Psychiatry 1249 (2010); F. Fang & W. Ye, “Smoking may be considered an established risk factor for sporadic ALS,” 74 Neurology 1927 (2010).

Of course, Oliver, and the entire plaintiffs’ expert witness team failed to rule out the most obvious, most prevalent explanation for Allen’s ALS:  unknown.

GENETIC SUSCEPTIBILITY

Ratner testified “to a reasonable degree of scientific certainty that Allen was genetically predisposed to develop ALS and would have developed and died from ALS later in his life.”  263 F.R.D. at 61.  This assertion was truly an incredible, unsupported, unverifiable, and unfalsifiable statement.  If a drug company ever made a similarly unsupported claim in an electronically transmitted document, the Department of Justice would prosecute it for wire fraud.  United States v. Harkonen, 2010 WL 2985257 (N.D. Calif. 2010).

The parties had essentially stipulated that Allen did not suffer from familial ALS, and neither Ratner nor anyone else identified any gene that was responsible for his “susceptibility.”  The district court, of course, did not report how Ratner could possibly have known that Allen was going to develop ALS, only at some unspecified date later than the date when Allen first became aware of signs and symptoms of motor neuron disease.  The district court announced that plaintiffs’ expert witnesses were not propounding “junk science,” but perhaps the heavy perfume helped masquerade the garbage.

POST HOC ERGO PROPTER HOC

The court conclusorily noted, without explanation, that the temporal relationship between exposure and disease manifestation would allow a conclusion of causality:

“Finally, after interpreting the data within a chronological context, the clinician may conclude that the patient’s disease is a neurotoxic illness.”

Id. at 61.  The court appears to accept the temporal pattern as sufficient in itself, or with other information, to support the conclusion.  This reasoning is fallacious.

AGE OF ONSET

Allen developed ALS when he was 45 years old.  Ratner reasoned that the average age of onset was 60, and Allen developed his disease “much earlier than would be expected”; therefore toluene accelerated the onset of Allen’s disease.  Id. at 61. The problem is that there is no “therefore” that can reasonably be claimed in the court’s sentence.

Most publications put the mean and median of age of ALS onset around 55 years, but even if the court were to accept Ratner’s reference to 60 as correct, surely the court recognized that half the cases therefore occurred below the age of 60.  The question of course is the variability in age of onset, and the court’s opinion is silent about the scatter or distribution of age-of-onset data.  Ratner’s reasoning was prima facie invalid unless there was additional information to show a very narrow distribution of age of onset around the mean.  It is difficult to discern whether the defense made this point, but Ratner could not have supported this counterfactual claim.

Here is what the ALS association has to say about the issue:

“Most people who develop ALS are between the ages of 40 and 70, with an average age of 55 at the time of diagnosis. However, cases of the disease do occur in persons in their twenties and thirties.”

“Who Gets ALS.”

Ratner essentially conceded that her argument was vacuous and invalid.  When confronted at her deposition about whether age of onset greater than the mean would have changed her opinion, she emphatically denied its relevance:

“My opinion would be the same even if that guy died at 60 instead of 75 and had history of this exposure … but you wouldn’t have bothered to depose me in that case… . Somebody else has moved down from where they are to here. But it may not result in a lawsuit, and I wouldn’t be here, because— I wouldn’t be here.”

Ratner Deposition at 172-3.

RULE 702 ANALYSIS

The district court recognized the novelty of Ratner’s analysis, but opined that Ratner, Oliver, and Clapp had provided sufficient cumulative evidence to support their theories.  263 F.R.D. at 61.  The trial court apparently conducted a Rule 702 hearing, over three days. Both sides filed what appears to have been extensive briefing and affidavits.  There are some huge gaps in the reasoning of the plaintiffs’ expert witnesses, and in the district court’s opinion.  Perhaps those gaps could be filled in with volumes of testimony.  My unscientific opinion is to doubt it. Although the plaintiffs should have had the burden of showing admissibility, the defendant had the practical burden of illustrating the analytical gaps, ipse dixit, fallacies, and invalid inferences that were before the court.  The defense may have indeed pointed out such problems, which were fulsomely present, but the district court’s opinion does not report the obvious defense arguments.  Without more background information, it is difficult to evaluate comprehensively the court’s or the defense’s handling of the scientific issues that were clearly before the court on the Rule 702 motions.  What is clear from what the district court reports is, however, sufficient to document an unsatisfactory judicial review of the evidence discussed.

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Bad Gatekeeping or Missed Opportunity – Allen v. Martin Surfacing

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

November 21st, 2012

While the trial court was preparing its findings of fact and conclusions of law, Ortho moved to reopen to evidence to permit additional testimony based upon three new articles.  Ortho’s motion came three months after the close of evidence, and Judge Shoob’s announcement of his verdict. The court denied this motion without mentioning what the new articles purported to show.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262, 298 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).

What is remarkable in Wells, from the vantage point of current practice, is the absence of motions directed at the proffered expert witness opinion testimony.  On the basis of Judge Shoob’s opinion, there appears to have been no Frye motion, no motions to exclude expert witnesses based upon the Federal Rules of Evidence, and no motions to strike testimony after the fact for lack of a proper basis.

Having lost the verdict in a bench trial, Ortho had little chance for success in the Court of Appeals on a claim that the evidence supporting the plaintiffs’ verdict was legally insufficient.  The traditional standard, applied by the Court of Appeals, was to sustain the trier of fact’s decision as not “clearly erroneous” when there were two “permissible” views of the evidence. 788 F.2d 741, 743 (11th Cir. 1986).  Without some legal doctrine to filter out flawed, invalid, and inadequate expert witness opinion from permissible views of an evidentiary display, the Court of Appeals was left with only a rubber stamp, which it proceeded to use with alacrity.

Ortho attempted to turn its appellate argument about the sufficiency of the evidence into a legal principle about rejecting factual findings not based upon “scientifically reliable foundations.”  Id. at 744.  The appellate court framed the issue on appeal simply as a “battle of the experts,” which Ortho had lost.  Both sides had qualified expert witnesses, and thus, according to the appellate court, “the district court was forced to make credibility determinations to ‘decide the victor’.” Id. (citing Ferebee v. Chevron Chemical Co., 736 F.2d 1529, 1535 (D.C. Cir.), cert. denied, 469 U.S. 1062 (1984)).  The Court of Appeals thus acquiesced in Judge Shoob’s superficial analysis, which attempted to resolve a scientific issue by trial atmospherics, demeanor, and subjective impressions of witness confidence rather than the validity of the studies relied upon and inferences drawn therefrom.  The possibility that Judge Shoob might have evaluated the evidentiary basis underlying the expert witnesses’ opinions was not even acknowledged.

The Court of Appeals invoked the language from Ferebee on statistical significance, despite its irrelevance to the case before it:

“We recognize, as did the Ferebee court, that ‘a cause-effect relationship need not be clearly established by animal or epidemiological studies before a doctor can testify that, in his opinion, such a relationship exists. As long as the basic methodology employed to reach such a conclusion is sound, such as use of tissue samples, standard tests, and patient examination, products liability law does not preclude recovery until a “statistically significant” number of people have been injured or until science has had the time and resources to complete sophisticated laboratory studies of the chemical. Id. at 1535-36.”

Wells, 788 F.2d at 745 (quoting Ferebee). Ferebee involved an injury that all parties agreed could be attributed to paraquat exposure without the need for epidemiologic studies; statistical analysis was not particularly germane.  In Wells, on the other hand, both sides relied upon studies that required statistical analyses for any sensible interpretation, and some of the studies actually reported statistically significant results.  The appellate court’s rhetoric was empty and irrelevant.

(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 3

November 18th, 2012

So Ortho lost the liability issue in the Wells case, which turned on historical knowledge in 1980.  How did Ortho lose medical causation?

The Wells case, as reported, is a fantastic if not terrifying insight into the mind of a judge, sitting as the trier of fact.  It is not a legal determination of the reliability or validity of the expert witness opinions under Federal Rule of Evidence 702; nor is it an assessment of the reasonableness of any expert’s reliance upon any study or data.  No Frye motions were made.  (If such motions were made, the trial court did not reference them in its opinion.) Ortho defended with its expert witnesses, and argued its case as a battle of experts.

Many of the litigation and judicial decision-making themes on state of the art carried over to medical causation.  Once Ortho lost the liability issue on such weak, controverted, and irrelevant evidence, its fate was likely sealed on causation, before Judge Shoob.  The trial court, in resolving liability, had already determined that it would allow hints, hunches, and hypotheses to substitute for knowledge.  Obscure and unpublished papers were accorded great weight, to the exclusion of peer-reviewed, published, data-driven, and carefully analyzed studies. Confounding in older studies with a mercury chemical not involved in Ortho’s product was ignored.  Any evidence of fetal harm, even in non-validated animal models with extraordinary doses, was permitted to substitute for relevant congenital malformations.  Exculpatory opinions, including those of the FDA’s Advisory Committee, were ignored.

In the litigation over medical causation, the parties agreed upon some facts and opinions.  Timing of exposure within the embryological development is crucial to an assessment of causality.  Some time windows are important, others irrelevant, to the development of specific malformations.  Therefore, not only is the time window of exposure important, but so is the type of malformation at issue. Notwithstanding this agreement, the trial court was consistently vague in discussing the specific malformations in various studies.  Not a single “effect size” is reported; and not a single p-value or confidence interval is indicated.  There is no discussion or recognition of the role multiple testing and comparison may play in diluting the meaning of a particular p-value.

Physical Examinations

The court mentioned no less than four times that some of plaintiffs’ expert witnesses had Katie Wells, and that none of the defendant’s witnesses had. The court failed, however, to articulate what was learned in such examinations that could not be appreciated from the medical records themselves.  This waving of hands about the “laying on of hands” reflects the trial court’s superficial approach to decision making.

Case Reports

The court mentioned that the plaintiffs’ expert witnesses relied upon the existence of case reports, but did not discuss what if any significance such case reports had for evaluating plaintiffs’ causal claims.  615 F. Supp. at 274.

Association Cannot Be Ruled Out  — Further Research Is Needed

The trial court imposed a highly skewed approach in its evaluation of expert witnesses on either side of the case.  One of the plaintiffs’ expert witnesses, Dr. John Holbrook had testified that “a definite relationship” existed:

“In Dr. Holbrook’s opinion, a ‘definite relationship’ exists between the use of spermicides and fetal malformations.”

615 F. Supp. at 275.

The court found Dr. Holbrook to be credible.  Id. at 276. There are several interesting aspects of how the trial court reacted to such statements from the plaintiffs’ expert witnesses.  First, the court tolerated an infuriating imprecision and inaccuracy in how the plaintiffs’ witnesses expressed their opinions. In the case of Dr. Holbrook, he was permitted to talk generally of “spermicides,” when it was clear that not all spermicides could be evaluated for causality together.  Second, the court permitted Dr. Holbrook to speak of ill-defined “definite” relationship, without declaring whether that relationship was “causal,” or something else.  When Holbrook, and the other plaintiffs’ witnesses, disagreed with the conclusions of articles upon which they relied, the trial court generally did not find that the discrepancies discredited the witnesses or their testimony.

The court’s approach to the plaintiffs’ witnesses should be compared with how it evaluated the defense expert witnesses’ testimony.  Dr. Stolley was called by the defense, and he was the only witness who had any real training and expertise in epidemiology.  Plaintiffs’ counsel questioned Dr. Stolley about statements made by Dr. Shapiro, an author of one of the studies relied upon by Dr. Stolley in his testimony.  (Remarkably, and importantly, the trial court does not report what if anything the plaintiffs’ witnesses had to say about Dr. Shapiro’s study.)

The Shapiro study concluded as follows:

“We conclude that there is no satisfactory evidence to indicate that spermicides increase the overall risk of major birth defects. It is possible, however, that spermicides increase the risk of certain specific malformations. Our own study is not large enough to exclude such a possibility. To evaluate it, further studies are needed.”

Id. at 284.  This sort of statement, an acknowledgement that further studies may be needed, is hardly an indication that the study had found an increased risk of a particular outcome.  And yet, the trial court somehow thought that it was Dr. Stolley who equivocated by having relied upon the Shapiro study.

At the FDA hearing, Dr. Shapiro described his published study:

“There is one question that is unanswered and that we fully acknowledge, and that is that all of the cohort studies so far done, or the case control studies, for that matter, lack the power to evaluate in statistical terms that there is not an appreciable increase in the risk of specific birth defects.

….

I’m not for one moment claiming that this study rules out an increase in the risk of limb reduction deformities.”

Id. at 285 (emphasis added).

Dr. Stolley disagreed with Shapiro about a relatively unimportant linguistic issue of what the magnitude is of an “appreciable” increased risk; Dr. Stolley opined that the available studies could rule out an “appreciable increase” in risk. Note that Dr. Shapiro was addressing a power issue in his study, whereas Dr. Stolley was addressing the issue with respect to all the available studies.  With respect to the supposed disagreement over “appreciable,” here is how Dr. Stolley expressed his view:

“I think that what [Shapiro] is saying is that, if there were a tiny increase in limb reduction defects, which is rare, you would need an extremely large study to absolutely rule that out and that can’t be done.

* * *

What he was trying to do is address the question of ruling out very, very tiny risks, and I think that I would agree that it’s very difficult to do that.

* * *

I think to me the data is quite conclusive in that it does not demonstrate — does not demonstrate an association of spermicides with limb reduction defects or with any other congenital anomaly.

* * *

I think that Doctor Shapiro would agree with my statement, and that is that no association between spermicides and birth defects has been shown, and I agree with him when he says a small increase cannot be ruled out. I think that’s true about a small increase in anything, but why should one even bother to do that given data that doesn’t show any substantial association as it now stands.”

Id. at 285-86.

The court found the above passages to be the basis for assigning “little weight” to Dr. Stolley’s opinion despite his “impressive credentials.”  Here is how the court described what it perceived to be an “equivocation”:

“Several times during direct examination, Dr. Stolley testified emphatically that a specific study or studies showed that spermicides are not related to birth defects. ***. In the portions of his testimony on cross-examination quoted above, however, Dr. Stolley equivocated. His interpretation of the studies discussed was that ‘no association between spermicides and birth defects has been shown’, but on cross he conceded that ‘a small increase [in birth defects] cannot be ruled out’. Finally, by disagreeing with Shapiro’s warning that an ‘appreciable increase’ cannot be ruled out, Dr. Stolley in effect was expressing greater confidence in Shapiro’s findings that Shapiro himself thought was justified. ***  In short, because of his apparent bias and his overstatements, the Court discounted Dr. Stolley’s conclusions.”

Id. at 286 (internal citations omitted). Putting aside the issue whether Dr. Stolley was assessing the Shapiro study alone or all the studies together, we can say that the trial court was clearly erroneous in finding an equivocation or an inconsistency between saying a study both failed to show an association and acknowledging that it also failed to rule out a small (or even an appreciable) increased risk.  The trial court had shifted the burden of proof to the defense to show that there was no association.

The trial court, confused by its own conflation of failing to show and failing to rule out, proceeded to dismiss Dr. Stolley’s testimony as having little or no probative value:

“Although these scientific studies are invaluable aids, Dr. Stolley’s testimony demonstrates that the studies alone do not show conclusively whether or not Katie Wells’ birth defects were caused by Ortho-Gynol Contraceptive Jelly. Further, the Court reiterates that plaintiffs’ ultimate burden was not to produce a flawless epidemiological study, but rather to show from all the evidence presented, to a reasonable degree of medical certainty, that the Product caused some or all of Katie Wells’ birth defects in 1980.”

Id. at 286. Lovely rhetoric, but yet the trial court was prepared to hold the defense to a non-legal, insuperable burden to produce an infinitely powerful study that could rule out any increased risk.  Dr. Robert Brent, the defense expert witness on teratology, similarly expressed the limitation of hypothesis testing to “prove” the hull hypothesis of no association:

“[T]here’s no way to prove that a substance is not teratogenic. It’s proving a negative, and, in fact, it’s true of water and vitamins and everything else that we are exposed to. All we can say is that after extensive evaluation, the risk appears no greater than if you didn’t expose yourself to that. That’s about as far as you can go.”

Id. at 289-90 (internal citations omitted).  If the defense expert witnesses committed any mistake it was in not explaining hypothesis testing better in their direct examinations so that statements on cross-examination were not seen as “equivocations.” This tactical mistake, however, cannot excuse or erase the profoundly and clearly erroneous findings of the trial court.

Statistical Significance

It was, after all, Justice Sotomayor’s opinion in Matrixx Initiatives, with Her Honor’s broad dicta  about statistical significance, which motivated my re-reading of the Wells case.  The Supreme Court’s opinion cited the appellate court’s decision to affirm the judgment entered upon Judge Shoob’s verdict, but the essence of the Wells case was established in the proceedings in district court.  Here is what Judge Shoob wrote about statistical significance:

“Plaintiffs’ burden of proving that Katie Wells’ defects were caused by the Product did not necessarily require them to produce scientific studies showing a statistically significant association between spermicides and congenital malformations in a large population.”

615 F. Supp. at 292.  And later, in connection with one of the defense expert witnesses, the court noted:

“In Dr. Brent’s opinion, none of these three indicators suggests that non-ionic surfactants are teratogenic. Other than the Jick study, he was aware of no studies showing a statistically significant association between spermicides and birth defects.”

Id. at 289.  Because there was at least one statistically significant outcome reported in one of the studies at issues, the Wells case cannot represent a legal precedent that established that statistically significant evidence is not necessary to a judgment of causation.  Justice Sotomayor’s opinion in dictum to the contrary is not only wrong as a scientific generalization, it is wrong as a matter of basic legal process.

(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 3

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 2

November 13th, 2012

How then did Ortho lose a case in which the trial judge, sitting as the trier of fact, declared that overall the studies failed to show that the spermicide caused any or all of Katie Wells’ birth defects?  First, let’s look at the liability case to the extent it depended upon scientific evidence up to the time of actual use of the product by the mother.

State of the Art

The state of the art in 1980 was based in large measure on 30 years of marketing experience.

The plaintiffs’ expert witnesses relied upon a limited number of papers and studies that could have been the basis for a determination that defendant had “knowledge,” actual or constructive, that the product caused birth defects of the type experienced by Katie Wells.

According to the court, the articles cited by some of the plaintiffs’ expert witnesses, in support for a warning on or before November 1980, suggested “the possibility of such a risk” from spermicides.  615 F. Supp. at 294. The studies themselves did not claim that there was a “known” causal relationship between spermicides and any birth defects, let alone the kind of malformations found in Katie Wells. Furthermore, the plaintiffs’ hired expert witnesses did not assert that any of the articles from this time period permitted such a conclusion.

A larger problem loomed in the interpretation of the early studies and papers.  Along with non-ionic surfactants, phenylmercuric acetate (PMA) (and also phenylmercuric nitrate) was used in spermicidal preparations.  There was some concern over the use of mercury compounds, which had known animal teratogenicity.  See FDA OTC Panel on Contraceptive and Other Vaginal Drug Products, Summary minutes of the fourth meeting November 18-19, 1973 (Adopted January 1974).  The Panel noted that a report was presented on phenylmercuric acetate (PMA), as a spermicidal agent and active ingredient of some vaginal contraceptive preparations. The report failed to find evidence of adverse effects on women or on human embryos, but it did note that teratogenic effects had been observed in animal experiments.

1.  R. Belsky, “Vaginal contraceptives–a time for reappraisal?” Population Reports:  Series H – Barrier Methods 37 (Jan. 1975).  This appears to be a review article, and none of the witnesses attributed any data or conclusions emanating from the paper.  The court described this article, as urged by some of plaintiffs’ hired experts to support a basis for warning on the product sold by Ortho and used by Katie Wells’ mother in 1980.  The alleged concern about spermicides is not specified as relating to non-ionic or mercury preparations.

2. Frank W. Oechsli, Studies of the Consequences of Contraceptive Failure (Apr. 8, 1976) (unpublished study).  According to the court, this study was the “Final Report for Contract NO1-HD-5-2816” for the Contraceptive Evaluation Branch, Center for Population Research, National Institute of Child Health and Development.   The court reports that plaintiffs’ expert witnesses cited this unpublished study as a basis for the defendant’s need to warn.

The elephant on the table for the Oechsli manuscript was why it was never published.  The plaintiffs’ expert witnesses sang the praises of NIH investigators, but obviously these investigators never thought that their work was sufficiently important or valid to publish and share with the medical and scientific community.  The lead author of this unpublished study, Frank W. Oechsli, has 17 publications listed in PubMed, none of which address potential associations between congenital malformations and spermicide use.

The availability of the Oechsli unpublished study to Ortho, before 1980, was disputed. Plaintiffs and their witnesses argued that Ortho could have obtained it by way of a FOIA request, and that it was cited in the Harlap article, infra, which was published in March 1980.  One of plaintiffs’ witnesses colorfully described Ortho as “asleep at the wheel” for not having obtained the unpublished paper. 615 F. Supp. at 277.

The defendant offered undisputed factual testimony that the NIH told the FDA Advisory Committee, considering safety and efficacy of non-ionic spermicides, that the study would not be available during the work of the Committee, which concluded in 1978.  615 F. Supp. at 280.  The NIH was aware that the FDA Committee was meeting, but never otherwise offered to share the unpublished Oechsli study.  In 1978, the FDA Committee presented its final report to the FDA.  The Committee report, which classified  non-ionic surfactants as safe, effective, and properly labeled, for use as spermicidal preparations.  Id. at 279.

In any event, Dr. Buehler, one of plaintiffs’ hired experts, described the Oechsli study as raising a “question and possible relationship” between spermicides and birth defects.  Although the defense had raised the problem of confounding with PMA, which was known to be teratogenic in animals, the court never disambiguated the plaintiffs’ broad brush references to “spermicides,” which could have referred to non-ionic compounds, mercury compounds, or both, in Oechsli’s study, as well as other of the early studies.

3.  S. Harlap, P. Shiono, S. Ramcharan, “Spontaneous foetal losses in women using different contraceptives around the time of conception,” 9 Internat’l J. Epidem. 49 (1980). The plaintiffs’ expert witnesses cited the Harlap study only because it referenced the Oechsli study.  Harlap, et al., however, reported that their data were contrary to those of the unpublished Oechsli study:  “Previous reports [citation to Oechsli] of an excess risk associated with spermicides are not borne out by this study.” Id. at 56.

4. E. S. Smith, C.S. Dafoe, J.R. Miller, P. Banister, “An epidemiological study of congenital reduction deformities of the limbs,” 31 Brit. J. Prev. & Social Med. 39 (1977). This study did evaluate limb deformities,  and compared many different maternal exposures among cases and controls, using a chi-squared analysis.  With two control groups combined, the chi-square was marginally significant for contraceptive foam or jelly, but no correction was made for the multiple comparisons. The authors noted that this exposure, foam or jelly, involved some products that were known to be teratogenic in animals (mercury compounds).

5. Warburton, Environmental Influences on Rates of Chromosome Anomalies, American Journal of Human Genetics – ABSTRACT (1980).  One of the plaintiffs’ witnesses relied upon this abstract, which identified spermicides as a possible factor in spontaneous abortions.  Dr. Holbrook suggested that this finding might mean that spermicides were teratogenic as well.  The completed paper, however, failed to confirm the suggestion in the Warburton abstract.  615 F. Supp. at 284.

 

The court found that at the time Katie Wells used Ortho’s product in fall of 1980, Ortho was negligent in failing to warn about an increased risk of birth defects that “might” arise from use of the product. 615 F. Supp. at 294 (emphasis added). Various article and studies suggested “the possibility of such a risk” in this time period. Id. (emphasis added).

The court relied heavily upon plaintiffs’ witnesses’ distorted testimony on this point.  For instance, Dr. Dick Gourley, testifying for the plaintiffs, opined that “as soon as the first study is done and any information that is provided … gives the hint that there is a possibility of a drug causing birth defects,” a labeled warning should be provided. Id. at 276.  Dr. Gourley thus testified that Ortho should have warned about an increased risk of birth defects “soon after that possibility was suggested by the Oechsli study in 1976, and no later than the publication of the Smith study in 1977.” Id.  Judge Shoob found Gourley “undogmatic,” and thus very credible! Dr. Gourley not only offered opinions that did not conform to the legal standard, he offered opinions based upon his own personal, subjective, ethical and emotive feelings. 615 F. Supp. at 277 n.19.  Unlike the treatment of defense experts who possibly strayed outside their fields of expertise, Judge Shoob regarded testimonial frolic and detours, such as Dr. Gourley’s to reflect their emphatic, and confident testimony.  Id.

The court similarly appeared to embrace plaintiffs’ expert witness, Dr. Sutherland, who opined that Ortho’s warnings were inadequate on grounds of the Oechsli unpublished paper (1976), and the Smith (1977) study, which together raised a “serious potential” that “spermicides increase the risk of fetal injury.” 615 F. Supp. at 274.  Remarkably, Judge Shoob did not require the plaintiffs to address, and to present evidence of, the spermicide at issue, or the specific congenital malformation at issue. The plaintiffs’ witnesses’ testimonies, as summarized by Judge Shoob, refer to possibilities, potentials, and hints of broad, ill-defined fetal harms, but this was held sufficient to satisfy plaintiffs’ burden of showing actual or constructive knowledge.  Having conflated hunches with knowledge, the court concluded that:

“This knowledge gave rise to a duty by defendant to warn consumers and certain health professionals of this possible risk.”

Id. at 294.

How did Ortho lose the state of the art issue?  Ortho lost when Judge Shoob allowed:

  • hints and hunches to substitute for knowledge;
  • obscure or unpublished papers to substitute for peer-reviewed, published, data-based, carefully analyzed studies;
  • that the evidence no longer had to relate to Ortho’s product, but could include another product, which contained a mercury compound known to cause birth defects in animals;
  • any fetal harm to substitute for the relevant congenital malformations; and
  • exculpatory opinions, including those of the FDA’s Advisory Committee, to be excluded from consideration.
(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 2

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 1

November 12th, 2012

In Matrixx Initiatives, Inc. v. Siracusano, 131 S.Ct. 1309 (2011), the Supreme Court, speaking through Justice Sotomayor, wandered into a discussion whether statistical significance was necessary for a determination of the causality of an association:

“We note that courts frequently permit expert testimony on causation based on evidence other than statistical significance. See, e.g., Best v. Lowe’s Home Centers, Inc., 563 F. 3d 171, 178 (6th Cir 2009); Westberry v. Gislaved Gummi AB, 178 F. 3d 257, 263–264 (4th Cir. 1999) (citing cases); Wells v. Ortho Pharmaceutical Corp., 788 F. 2d 741, 744–745 (11th Cir. 1986). We need not consider whether the expert testimony was properly admitted in those cases, and we do not attempt to define here what constitutes reliable evidence of causation.”

Id. at 1319.

As I have pointed out previously, the Court’s citation to these three cases was jarring for their irrelevance, and for the questionable scholarship involved.

The first two cases cited involved differential etiology  to assess specific causation, not general causation.  As most courts have recognized, this assessment strategy requires that general causation has already been established. See, e.g., Hall v. Baxter Healthcare, 947 F. Supp. 1387 (D. Ore. 1996).  These cases did not, therefore, even touch on the use of statistical significance to establish general causation. There was no statistical analysis in those cases, and nothing to judge significant or insignificant.

The citation to the third case, Wells, is noteworthy because the case has nothing to do with adverse event reports or the lack of statistical significance.  Wells involved a claim of birth defects caused by the use of spermicidal jelly contraceptive, which had been the subject of several studies, one of which at least yielded a statistically significant increase in detected birth defects over what was expected.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).  Wells could thus hardly be an example of a case in which there was a judgment of causation based upon a scientific study that lacked statistical significance in its findings. Of course, finding statistical significance is just the beginning of assessing the causality of an association; Wells was notorious for its poor assessment of all the determinants of scientific causation.

As I pointed out in Matrixx Unloaded, the citation to Wells was remarkable because the Wells decision has been widely criticized for its failure to evaluate the entire evidentiary display, as well as for its failure to rule out bias and confounding in the studies relied upon by the plaintiff.[i]

A few years later, another case in the same judicial district against the same defendant for the same product resulted in the grant of summary judgment.  Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1582 (N.D. Ga. 1991) (supposedly distinguishing Wells on the basis of more recent studies).  Some legal scholars have been content to point out that the science had matured with the passage of time, from the point that there was sufficient evidence to support plaintiffs’ case (in Wells) to the point that the evidence was so overwhelmingly adverse to plaintiffs that summary judgment for the defense was appropriate (in Smith).  While this suggestion has the virtue of simplicity, it fails to look at what really happened in Wells, and to analyze the evidence in front of the Wells court.

Another remarkable aspect of the Supreme Court’s citation to Wells is that the case, and all it stands for, was overruled sub silentio by the Supreme Court’s own decisions in Daubert, Joiner, Kumho Tire, and Weisgram.  And if that did not kill the concept, then there was the simple matter of a supervening statute:  the 2000 amendment of Rule 702, of Federal Rules of Evidence. Citing a case as jurisprudentially dead and discredited as Wells could have been sloppy scholarship and lawyering.  The principle of charity, however, suggests it was purposeful, and that is a frightful prospect.

Katie Wells was conceived sometime in the month of October 1980.  Her mother was using the defendant’s product, Ortho-Gynol Contraceptive Jelly, with its active ingredient, a non-ionic surfactant called p-diisobutylphenoxypolyethoxyethanol or octoxynol-9, from July until mid-November 1980.  Defendant Ortho had manufactured and marketed this product since 1950, without any warning that the product causes birth defects. 615 F. Supp. at 268-69.  Ortho’s defense consisted in denying general and specific causation, and noting that in the fall of 1980, “no published reports or studies had concluded that spermicides cause birth defects,” and that “it had received no other complaints nor had access to any other evidence suggesting a link between its Product and birth defects.” 615 F. Supp. at 269.

Key issues in Wells were whether the state of the art required a warning on product in 1980, when used, and whether, as of the time of the trial in 1984, plaintiffs could establish cause in fact.

The district judge, Judge Marvin Shoob, heard the case as the trier of fact.  Both sides waived their rights to a jury trial in part because of the number of witnesses could not be accommodated within the time scheduled for the trial.  Given that the plaintiffs called so many redundant witnesses, a reasonable observer may suspect that the plaintiffs wanted a bench trial.  Perhaps there was something about the relationship between Katie’s mother and father to give plaintiffs’ counsel pause.  Katie’s father, Mr. Gary Wells, was not a party; only her mother was claiming economic damages.  Perhaps Katie’s father’s admitted substantial use of illegal, recreational drugs, pushed plaintiffs to make this choice.  Katie’s mother admitted to using only therapeutic drugs, Decadron for bronchitis, and Amoxicillin during her pregnancy, but her father admitted to using LSD, amphetamines, methaqualone, and marijuana.  Id. at 269. The defense probably agreed, hoping to avoid an unduly sympathetic jury, and to gain a more analytical decision maker.

Jurisdiction in Wells was based upon diversity of citizenship; the court was bound to apply Georgia law under controlling choice-of-law precedent.  Georgia law requires plaintiffs to prove medical causation “reasonable degree of medical certainty.”  See Parrott v. Chatham County Hospital Authority, 145 Ga.App. 113, 115, 243 S.E.2d 269 (1978); Robertson v. Emory University Hospital, 611 F.2d 604, 608 n. 13 (5th Cir.1980); Watson v. United States, 346 F.2d 52, 54 (5th Cir.1965), cert. denied, 382 U.S. 976 (1966). 615 F. Supp. at 295. Plaintiffs, under Georgia law, must show more than a “bare possibility” of causation, as well as ruling out other theories that are equally plausible. Id. (citing Maddox v. Houston County Hospital Authority, 158 Ga.App. 283, 284, 279 S.E.2d 732 (1981)).

According to Georgia law of negligence, a manufacturer must exercise reasonable care to warn of dangers of which it has “actual or constructive knowledge,” when the product was sold and delivered.  Id. (citations omitted).

In deciding this case, Judge Shoob began defensively, on thin ice, by declaring that he had to make a “legal” decision, not a medical decision. Id. at 266.  This defensive prelude was curious because Georgia law clearly imported “knowledge” into both the duty to warn, and into resolution of the causal issue.  The requisite knowledge could come only from the scientific evidence that the parties tried to marshal.  The defensive prelude was also curious because, as we shall see, Judge Shoob allowed the plaintiffs’ expert witnesses to opine about their subjective perceptions that warnings should be required upon suspicion of harm.

It bears pointing out that, as in Ferebee, Federal Rule of Evidence 702 or 703 is never mentioned in Judge Shoob’s decision. There appears to have been no defense challenges to the qualifications or the “helpfulness” of any of the plaintiffs’ witnesses.  There are no challenges to any expert witness’s qualifications. There are no Frye challenges discussed.

On re-reading Judge Shoob’s opinion, the most salient feature is the absence of any discussion of the “point estimates” of association or “effect size” in any of the studies discussed.  Similarly, Judge Shoob fails to mention the extent of the random error in any study, either in the form of p-values of in confidence intervals.  There is an occasional hint that studies are too small to yield meaningful findings, but no power calculations or assessments are provided, and no alternative hypotheses specified.  There is no consideration of multiple comparisons, which may have diluted the usual interpretation of significance probability. Judge Shoob failed to engage in the scientific studies or the evidence, which was offered to him as the trier of fact.

Notwithstanding the failure to consider important rules of evidence, and the scientific evidence, Judge Shoob appreciated that there was a failure of proof on the part of the plaintiffs:

“Although some of the studies suggested a connection between spermicides and birth defects, overall the studies failed to show conclusively whether or not the spermicide caused any or all of the birth defects suffered by Katie Wells.”

Id. at 266. How then did Ortho loose?

(to be continued).

[i] See, e.g., James L. Mills and Duane Alexander, “Teratogens and ‘Litogens’,” 15 New Engl. J. Med. 1234 (1986); Samuel R. Gross, “Expert Evidence,” 1991 Wis. L. Rev. 1113, 1121-24 (1991) (“Unfortunately, Judge Shoob’s decision is absolutely wrong. There is no scientifically credible evidence that Ortho-Gynol Contraceptive Jelly ever causes birth defects.”). See also Editorial, “Federal Judges v. Science,” N.Y. Times, December 27, 1986, at A22 (unsigned editorial) (“That Judge Shoob and the appellate judges ignored the best scientific evidence is an intellectual embarrassment.”);  David E. Bernstein, “Junk Science in the Courtroom,” Wall St. J. at A 15 (Mar. 24,1993) (pointing to Wells as a prominent example of how the federal judiciary had embarrassed American judicial system with its careless, non-evidence based approach to scientific evidence); Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, “Science and the Law in the Wake of Daubert: A New Search for Scientific Knowledge,” 72 Texas L. Rev. 715, 733-34 (1994) (lawyers and leading scientist noting that the district judge “found that the scientific studies relied upon by the plaintiffs’ expert were inconclusive, but nonetheless held his testimony sufficient to support a plaintiffs’ verdict. *** [T]he court explicitly based its decision on the demeanor, tone, motives, biases, and interests that might have influenced each expert’s opinion. Scientific validity apparently did not matter at all.”) (internal citations omitted); Troyen A. Brennan, “Untangling Causation Issues in Law and Medicine: Hazardous Substance Litigation,” 107 Ann. Intern. Med. 741, 744-45 (1987) (describing the result in Wells as arising from the difficulties created by the Ferebee case; “[t]he Wells case can be characterized as the court embracing the hypothesis when the epidemiologic study fails to show any effect”).  Kenneth R. Foster, David E. Bernstein, and Peter W. Huber, eds., Phantom Risk: Scientific Inference and the Law 28-29, 138-39 (MIT Press 1993) (criticizing Wells decision); Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation § 6.5, at 93(1997) (noting the multiple comparisons in studies of birth defects among women who used spermicides, based upon the many reported categories of birth malformations, and the large potential for even more unreported categories); id. at § 6.5 n.3, at 271 (characterizing Wells as “notorious,” and noting that the case became a “lightning rod for the legal system’s ability to handle expert evidence.”).

Posted in Causation, Expert Witnesses, Scientific Evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 1

Evidence-Based Specific Causation

November 7th, 2012

In the last post, I discussed an important article by Professors Davidson and Guzelian, on the legal implications of evidence-based medicine (EBM).  Terence M. Davidson & Christopher P. Guzelian, “Evidence-based Medicine (EBM): The (Only) Means for Distinguishing Knowledge of Medical Causation from Expert Opinion in the Courtroom,” 47 Tort Trial & Ins. Practice L. J. 741 (2012) [cited as Davidson].

Their criticism of the deficiencies in current gatekeeping practice cries out for reform of much current judicial practice.  Education of the judiciary in EBM would be helpful to both plaintiffs and defendants in civil cases, as well as prosecutors and defendants in criminal cases.  I will leave for another day a discussion of whether the boundary between knowledge and “mere opinion” is so easily discernible.

Although the authors’ call for EBM in judicial decisions is timely and needed, I noted my dissent to their assessment of one defense expert witness’s specific causation opinion in the hormone therapy litigation. Davidson and Guzelian criticize one district judge for having admitted the challenged testimony of a defense expert witness, Dr. Blaustein, who opined that (1) estrogen + progesterone combination post-menopausal hormone replacement therapy (CHRT) has not been shown to cause breast cancer, and (2) there is no generally accepted method for determining a cause of a woman’s breast cancer.  Cross v. Wyeth Pharms., Inc., CASE NO.: 8:06-cv-429-T-23AEP, 2011 U.S. Dist. LEXIS 89078; 2011 WL 3498305 (M.D. Fla. 2011) (Merryday, J.).

Here is how Davidson & Guzelian put the matter:

“Blaustein also opined that there is no generally accepted method for diagnosing specific medical causation—that is, whether a medical intervention or treatment, even if it is known to generally cause a result ‘X’ (e.g., breast cancer), caused a particular patient’s result ‘X’. Blaustein’s statement, accepted by the judge as befitting of presentation to the jury, is ‘simply false’.”

The claim that Blaustein’s statement is “simply false” is pretty strong for both general and specific causation, and partially explains my initial dissent.  The prestige of the Women’s Health Initiative (WHI), a large, randomized, clinical meta-trial of  CHRT, with its finding of an increased risk for breast cancer among women, certainly has made Dr. Blaustein’s opinion on general causation a distinctly minority view.  There are, however, some careful authors who have challenged the findings of the WHI on grounds of internal and external validity.  See, e.g., Samuel Shapiro, Richard Farmer, Helen Seaman, J. C. Stevenson, “Does hormone replacement therapy cause breast cancer? An application of causal principles to three studies: part 2. The Women’s Health Initiative: estrogen plus progestogen,” 37 J. Family Planning & Reproductive Health Care 165, 165 (2011) (“HRT with estrogen plus progestogen may or may not increase the risk of breast cancer, but the WHI did not establish that it does.”).  In any event, given the state of the science, most defendants would hedge their position on general causation even if they stopped short of saying it was not established.  Still, the validity concerns may cause us to have some doubts that the conclusions drawn from the WHI and other studies are truly “knowledge.”

Professor Guzelian has persuaded me that their indictment of Blaustein’s opinion on specific causation is correct, at least technically.  There are, to be sure, a few genetic causes of breast cancer, such as the BrCa 1, and 2, genes, which can cause breast cancer, and which, if present in a particular woman, would constitute an adequate description of the cause of that woman’s cancer.  Blaustein claimed that there is no generally accepted method for attributing an individual woman’s breast cancer to known causes, and Davidson and Guzelian are correct that Blaustein’s claim is, therefore, “simply false.”

Given how cursory and conclusory the trial court’s opinion is, Davidson and Guzelian’s derision is, however, relatively uncharitable.  We can probably credit the plaintiff’s counsel with sufficient entrepreneurial savvy to have not pressed the claims of a woman who had a known genetic cause of breast cancer.  And we can similarly credit the defense counsel with sufficient intelligence not to have overlooked the presence of such a gene if it were present.

With genetic causes taken out of the equation, Blaustein’s opinion seems fairly unremarkable.  Even if the criticism of the trial court’s refusal to preclude Dr. Blaustein’s testimony on general causation were correct under Federal Rule of Evidence 702, Davidson and Guzelian have failed to make their case against the trial court, and Dr. Blaustein, on specific causation.

Here is what Davidson and Guzelian have to say about Blaustein’s specific causation opinion:

“EBM has documented and catalogued the best practices regarding how to diagnose whether generally applicable evidence-based conclusions hold for specific patients.51 Moreover, even if in a specific case it were plausible to assert that it is impossible to reach a specific causation conclusion for a particular patient’s condition using the scientific method, that is very different than saying that there is no scientific method for analyzing specific causation. According to EBM’s strictures, Blaustein’s proposed blanket denial of the possibility of specific causation should have been precluded from his testimony.52”

Davidson at 757. The footnotes in this passage are to the section of an article on evidence-based toxicology, which deals with attribution of an adverse drug reaction.  Philip Guzelian et al., “Evidence-Based Toxicology: A Comprehensive Framework for Causation,” 24 Human & Experimental Toxicol. 161, 190-91 (Table 9) (2005) (presenting “an overview of evidence-based specific causation diagnostic criteria”). Putting aside the very substantial differences between cancer causation and the general run of adverse drug reactions, we can see that the proposed criteria for specific causation in the referenced article contain this extremely important criterion:

“No good alternative candidate (unexplained exacerbation or recurrence of underlying illness).”

Id.  Specific causation in a breast cancer case falls outside this criterion because most cases of breast cancer (with the exception of the genetic cases mentioned above) have no identifiable risk factor.  Dr. Blaustein’s opinion on specific causation — that there is no generally accepted method for attributing breast cancer to exogenous cause(s) — seems then exactly on point, even on the EBM criteria for specific causation urged by the authors.

In Cross, the trial court does not provide any insight into the basis for the plaintiffs’ challenge to Blaustein’s specific causation opinion, and I am aware of none.  The trial court does not give us any particulars of the plaintiff’s use of CHRT or development of breast cancer, and there is no suggestion that she had an extraordinarily high risk (say > 10-fold increase, which I have never seen reported, in any event).

The plaintiffs did not attempt to infer causation from risk.  Perhaps they thought better of it, or perhaps one of their testifying epidemiologists, Dr. Graham Colditz, refused to support such an inference:

“Knowledge that a factor is associated with increased risk of disease does not translate into the premise that a case of disease will be prevented if a specific individual eliminates exposure to that risk factor. Disease pathogenesis at the individual level is extremely complex. As Rose stated, a preventive measure that brings large benefits to the community offers little to each participating individual [3]. Accordingly, epidemiology must be harnessed to identify the population level strategies that will reduce the burden of illness.”

Graham A. Colditz, “From epidemiology to cancer prevention: implications for the 21st Century,” 18 Cancer Causes Control 117, 118 (2007).  Professor Colditz’ view is hardly unique; there are many similar refusals to base an inference of specific causation upon an increased risk, whether or not that increased risk is quantified as a relative risk greater than two.  The late David Freedman, who was the co-author of the chapters on statistics in all three editions of the Reference Manual on Scientific Evidence, was also a naysayer when it came to transmuting risk into cause:

“The scientific connection between specific causation and a relative risk of two is doubtful. *** Epidemiologic data cannot determine the probability of causation in any meaningful way because of individual differences.”

David Freedman & Philip Stark, “The Swine Flu Vaccine and Guillaine-Barré Syndrome:  A Case Study in Relative Risk and Specific Causation,” 64 Law & Contemporary Problems 49, 61 (2001).

The plaintiffs’ challenge to Blaustein’s claim that there was no “generally accepted” method for specific causal attribution took the form of advancing their own “method”:  differential diagnosis. At face value, the plaintiffs’ use of differential diagnosis to advance a claim of specific causation is “simply false.”  There was no dispute about diagnosis, and no differential diagnosis at issue.  Unfortunately, courts have permitted lawyers to corrupt the meaning of differential diagnosis and contend that it covers something akin to differential etiology.

Davidson and Guzelian, in their reference back to an earlier article on evidence-based toxicology, embrace differential etiology as a method of specific case attribution.  I agree that there is really no dispute about the logically validity of such reasoning, generally.  The logic of differential etiology is simple.  If you can specify all the known causes of a disease, and eliminate all but one cause, then you have ruled in the specific cause.  Logically, this is an iterative disjunctive syllogism, also known as the process of elimination.  The syllogism requires an exhaustive statement of disjuncts, with the negation of all but one:

A v B v C v D

~A and ~B and ~ C.

Therefore, D.

In “The Adventure of the Beryl Coronet,” Sir Arthur Conan Doyle had his famous detective, Sherlock Holmes, articulate this method in ordinary English, with a bit more flair:

“It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth.”

Arthur Conan Doyle, The Penguin Complete Sherlock Holmes 315 (Penguin 1981). The process of elimination was a mainstay of Holmes’ forensic thought:

“Eliminate all other factors, and the one which remains must be the truth.”

“The Sign of the Four,” chap. 1 (“The Science of Deduction”), in Arthur Conan Doyle, The Penguin Complete Sherlock Holmes at 92 (Penguin 1981).

The problem of course is, for a disease such as breast cancer, is that one of the disjuncts has been, and will remain, for some time: the proposition that this case is “idiopathic” or “sporadic.” The plaintiffs in Cross did not advance any plausible method for eliminating this disjunct.  As a result, they can never arrive at a conclusion that CHRT was a cause of Ms. Cross’s breast cancer.  The closest they can get to their desired conclusion with this “method,” once they have eliminated genetic causes, is a conclusion that:

The case is idiopathic OR the case resulted from CHRT.

This conclusion is not really a conclusion at all, but an indeterminate statement, which would be quite unhelpful to the trier in deciding the case. Furthermore, as the case is described by the trial court’s opinion, the plaintiffs did not even attempt a quantification of the probability of each of these two disjuncts.  Thus, the plaintiffs failed to offer any substantial evidence that a jury could believe to find in their favor on specific causation.  The trial court was correct to reject the challenge to Dr. Blaustein’s specific causation opinion, but the court should have granted the defendants’ challenge to the plaintiffs’ expert witnesses, who had no method at all on the crucial element of specific causation.

Interestingly, one of the plaintiffs’ better arguments against Dr. Blaustein was that he was, as a clinician, unqualified to opine on causation.  Cross, *9.  The trial court did not elaborate on the argument other than to point out that plaintiffs had emphasized that Dr. Blaustein relied upon his “unquantifiable and untested clinical experience.” Cross, *10. The courts have been remarkably resistant to the argument that physicians are generally unqualified to interpret scientific evidence of causation.  Sadly, there is a good deal of empirical evidence to show that physicians are not particularly well trained in statistics or in interpreting clinical research.  See, e.g., Donna Windish, Stephen Huot, and Michael Green, “Medicine Residents’ Understanding of the Biostatics and Results in the Medical Literature,” 298 J. Am. Med. Ass’n 1010, 1010 (2007) (“Most residents in this study lacked the knowledge in biostatistics to interpret many of the results in published clinical research.”).

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Evidence-Based Specific Causation

« Older Entries
Newer Entries »