For your delectation and delight, desultory dicta on the law of delicts.

Judicial Innumeracy and the MDL Process

February 26th, 2011

In writing previously about the Avandia MDL Court’s handling of the defendants’ Daubert motion, I noted the trial court’s erroneous interpretation of statistical evidence.  See “Learning to Embrace Flawed Evidence – The Avandia MDL’s Daubert Opinion” (Jan. 10, 2011).  In fact, the Avandia court badly misinterpreted the meaning of a p-value, a basic concept in statistics:

“The DREAM and ADOPT studies were designed to study the impact of Avandia on prediabetics and newly diagnosed diabetics. Even in these relatively low-risk groups, there was a trend towards an adverse outcome for Avandia users (e.g., in DREAM, the p-value was .08, which means that there is a 92% likelihood that the difference between the two groups was not the result of mere chance).”

In re Avandia Marketing, Sales Practices and Product Liability Litigation, 2011 WL 13576, *12 (E.D. Pa. 2011) (internal citation omitted).  The Avandia MDL court was not, however, the first to commit this howler.  Professor David Kaye collected examples of statistical blunders from published cases in a 1986 law review, and again in his chapter on statistical evidence in the Federal Judicial Center’s Reference Manual on Scientific Evidence created a list of erroneous interpretations:

United States v. Georgia Power Co., 474 F.2d. 906, 915 (5th Cir. 1973)

National Lime Ass’n v. EPA, 627 F.2d 416, 453 (D.C. Cir. 1980)

Rivera v. City of Wichita Falls, 665 F.2d 531, 545 n.22 (5th Cir. 1982) (“A variation of two standard deviations would indicate that the probability of the observed outcome occurring purely by chance would be approximately five out of 100; that is, it could be said with a 95% certainty that the outcome was not merely a fluke.”);

Vuyanich v. Republic Nat’l Bank, 505 F. Supp. 224, 272 (N.D. Tex. 1980) (“[I]f a 5% level of significance is used, a sufficiently large t-statistic for the coefficient indicates that the chances are less than one in 20 that the true coefficient is actually zero.”), vacated, 723 F.2d 1195 (5th Cir. 1984)

Craik v. Minnesota State Univ. Bd., 731 F.2d 465, 476n.13 (8th Cir. 1984)(“[a] finding that a disparity is statistically significant at the 0.095 or 0.01 level means that there is a 5 per cent. Or 1 per cent. Probability, respectively, that the disparity is due to chance.”  See also id. at 510 (Swygert, J., dissenting)(stating that coefficients were statistically significant at 1% level, allowing him to say that “we can be 99% confident that each was different from zero.”)

Sheehan v. Daily Racing Form, Inc., 104 F.3d 940, 941 (7th Cir. 1997) (“An affidavit by a statistician . . . states that the probability that the retentions . . . are uncorrelated with age is less than 5 percent.”)

Waisome v. Port Authority, 948 F.2d 1370, 1376 (2d Cir. 1991) (“Social scientists consider a finding of two standard deviations significant, meaning there is about one chance in 20 that the explanation for a deviation could be random . . . .”)

David H. Kaye & David A. Freedman, “Reference Guide on Statistics,” in Reference Manual on Scientific Evidence 83, 122-24 (2nd ed. 2000); David H. Kaye, “Is Proof of Statistical Significance Relevant?” 61 Wash. L. Rev. 1333, 1347 (1986)(pointing out that before 1970, there were virtually no references to “statistical significance” or p-values in reported state or federal cases. 

Notwithstanding the educational efforts of the Federal Judicial Center, the innumeracy continues, and with the ascent of the MDL model for addressing mass torts, many recent howlers have come from trial judges given responsibility for overseeing the pretrial coordination of thousands of lawsuits.  In addition to the Avandia MDL Court, here are some other recent erroneous statements that can be added to Professor Kaye’s lists: 

“Scientific convention defines statistical significance as “P ≤ .05,” i.e., no more than one chance in twenty of a finding a false association due to sampling error.  Plaintiffs, however, need only prove that causation is more-probable-than-not.”

In re Ephedra Prods. Liab. Litig., 393 F.Supp.2d 181, 193 (S.D.N.Y. 2005)(confusing the standard for Type I statistical error with the burden of proof).

“More-probable-than-not might be likened to P < .5, so that preponderance of the evidence is nearly ten times less significant (whatever that might mean) than the scientific standard.”

Id. at 193 n.9 (same). 

In the Phenylpropanolamine litigation, the error was even more clearly stated, for both p-values and confidence intervals:

“P-values measure the probability that the reported association was due to chance… .”

“… while confidence intervals indicate the range of values within which the true odds ratio is likely to fall.”

In re Phenylpropanolamine Products Liab. Litig., 289 F. 2d 1230, 1236n.1 (2003)

These misstatements raise important questions about judicial competency for gatekeeping, the selection, education, and training of judges, the assignment of MDL cases to individual trial judges, and the aggregation of Rule 702 motions to a trial judge for a single, one-time decision that will control hundreds if not thousands of cases.

Recently, a student published a bold note that argued for the dismantling of judicial gatekeeping.  Note, “Admitting Doubt: A New Standards for Scientific Evidence,” 123 Harvard Law Review 2021 (2010).  With all the naiveté of someone who has never tried a jury trial, the student argued that juries are at least as good, if not better, at handling technical questions.  The empirical evidence for such a suggestion is slim, and ignores the geographic variability in jury pools.  The above instances of erroneous statistical interpretations might seem to support the student’s note, but the argument would miss two important points: 

  • these errors are put on display for all to see, and for commentators to note and correct, whereas jury decisions obscure their mistakes; and
  • judges can be singled out for their technical competencies, and given appropriate assignments (which hardly ever happens at present), and judges can be required to partake in professional continuing legal education, which might well include training in technical areas to improve their decision making.

The Federal Judicial Center, and its state court counterparts, have work to do.  Lawyers also have an obligation to help courts get difficult, technical issue right.  Finally, courts, lawyers, and commentators need to rethink how the so-called Daubert process works, and does not work, especially in the high-stakes arena of multi-district litigation.

Can Daubert Survive the Multi-District Litigation Process?

February 23rd, 2011

The so-called Daubert process, by which each side in a lawsuit may challenge and seek preclusion of the other side’s expert witnesses, arose in the setting of common-law judges making rulings in individual cases.  Indeed, the Daubert case itself, although one of many cases involving claims of birth defects allegedly caused by Bendectin, was an individual case. 

In the silicone gel breast implant (SGBI) litigation, the process evolved over time, with decisions from different judges, each of whom saw the evidence differently.  The different judges brought different insights and aptitudes to bear on the evidence, and the expert witnesses themselves may have varied in their approaches and reliance upon different studies.  This incrementalist approach, in the context of the SBGI litigation, worked to the benefit of the defendants, in part because their counsel learned about the fraudulent evidence underlying certain studies, and about serious lapses in the standard of research care on the part of some investigators whose studies were prominently relied upon by plaintiffs’ counsel.  In the case of one dubious study, one of its authors, Marc Lappe, a prominent expert witness for plaintiffs, withdrew his support from the conclusions advanced in the study.

Early decisions in the SGBI cases (shortly after the Supreme Court’s decision in Daubert, in 1993) denied the defendants’ applications to preclude plaintiffs’ expert witnesses’ opinion testimony.  Later decisions converged upon the unavoidable truth that the case for SGBIs causing atypical or typical connective tissue diseases was a house of cards, built mostly with jokers.  If the Daubert process had been censored after the first hearing, the result would have been to deem all the breast implant cases trial and jury worthy, to the detriment of the judicial process, to the public’s interest in knowing the truth about silicone biomaterials, to the defendants’ reputational and financial interests, and to the interests of the claimants who had been manipulated by their counsel and support group leaders.

The evolutionary approach taken in the SGBI litigation was indirectly supported by the late Judge Sam Pointer, who presided over the SGBI federal multi-district litigation (MDL).  Judge Pointer strongly believed that the decision to exclude expert testimony belonged to individual trial judges, who received cases on remand from the MDL 926, when the cases were ready for trial.  Judge Pointer ruled on expert witness challenges in cases set for trial before him, but he was not terribly enthusiastic about the Daubert process, and denied most of the motions in a fairly perfunctory fashion.  Because of this procedural approach, Judge Pointer’s laissez-faire attitude towards expert witness testimony did not interfere with the evolutionary process that allowed other courts to see through the dense fog in the plaintiffs’ case.

Since MDL 926, the MDL process has absorbed the ritual of each side’s challenging the other’s expert witnesses, and MDL judges view their role as including the hearing and deciding all pre-trial Daubert challenges.  It has been over 17 years since the Supreme Court decided Daubert, and in that time, the MDL model, both state and federal, has become dominant.  As a result, the Daubert process has often been truncated and abridged to a single motion, decided at one time, by one judge.  The results of this abridgement have not always been happy for ensuring reliable and accurate gatekeeping. 

The MDL process appears to have broken the promise of Rule 702 in many cases.  By putting the first and only Rule 702 gatekeeping decision in the hands of a single judge, charged with making pre-trial rulings in the entire MDL, the MDL process has sapped the gatekeeping process of its dynamic, evolutionary character.  No longer can litigants and judges learn from previous efforts, as well as from commentary by scientists and legal scholars on the prior outcomes.  For judges who lack scientific and analytical acumen, this isolation from the scientific community works to the detriment of the entire process.

To be sure, the MDL process for deciding Rule 702 is efficient.  In many cases, expensive motions, briefings, and hearings are reduced to one event.  The incorporation of expert challenges into an MDL may improve fairness in some instances by allowing well-qualified plaintiffs’ counsel to wrest control of the process from unprepared plaintiffs’ counsel who are determined to control their individual cases.  Defendants may embrace the MDL process because it permits a single, unified document production and discovery schedule of corporate executives.  Perhaps defendants see the gains from MDL process as sufficiently important to forgo the benefit of a fuller opportunity to litigate the expert witness issues.  Whatever can be said in favor of using the MDL forum to resolve expert witness challenges, it is clear that MDL procedures limit the parties’ ability to refine their challenges over time, and to incorporate new evidence and discovery gained after the first challenges are resolved.  In the SGBI litigation, for instance, the defendants learned of significant scientific malfeasance and misfeasance that undermined key studies relied upon by plaintiffs, including some studies done by apparently neutral, well-credential scientists.  The omnibus MDL Daubert motion prevents either side, or the judiciary, from learning from the first and only motion.

Another example of an evidentiary display that has changed over time comes from the asbestos litigation, where plaintiffs continue to claim that asbestos causes gastrointestinal cancer.  The first such cases were pressed by plaintiffs in the early 1980s, with the support of Dr Selikoff and his cadre of testifying physicians and scientists.  A few years ago, however, the Institutes of Medicine convened a committee to review non-pulmonary cancers and asbestos, and concluded that the studies, now accumulated over 35 years since Dr Selikoff’s ipse dixit, do not support a conclusion that asbestos causes colorectal cancer.  Institute of Medicine of the National Academies, Asbestos: Selected Health Effects (2006).

Unfortunately, many trial judges view the admissibility and sufficiency of causation opinions on asbestos and colorectal cancer as “grandfathered” by virtue of the way business has been conducted in trial courts for over three decades.  Still, defendants have gained the opportunity to invoke an important systematic review, which shows that the available evidence does not reliably support the conclusion urged by plaintiffs’ expert witnesses. 

The current approach of using the MDL as the vehicle for resolving expert witness challenges raises serious questions about how MDLs are assigned to judges, and whether those judges have the analytical or quantitative skills to resolve Daubert challenges.  Assigning an MDL to a judge, who will have to rule on the admissibility of expert witness opinion testimony she or he does not understand, does not inspire confidence in the judicial process.  At least in the ad hoc approach employed in the SGBI, the parties could size up their trial judge, and decide that they would forgo their expert challenges based upon their assessment.  Furthermore, an anomalous outcome could be corrected over a series of decisions.  The MDL process, on the other hand, frequently places the Rule 702 decision in the discretion of a single judge.  The selection criteria for that sole decision maker becomes critical.  As equity in days of old varied with the size of the Chancellor’s foot, today’s scientific equity under Rule 702 may vary with accuracy of the trial judge’s slide rule.

Toxic Litigation and Toxic Torts

February 2nd, 2011

Christopher J. Robinette, at TortsProf Blog, thoughtfully provided a link to a new paper, in press, by Professor Robert Rabin.  The paper is a short romp through the last few decades of toxic tort law.  Robert L. Rabin, “Harms from Exposure to Toxic Substances:  The Limits of Liability Law,” 38 Pepperdine L. Rev. 101 (2011), available at

Having lived and practiced law through the romp, I thought it would make for an interesting read.

Professor Rabin describes the growth and contraction of judicial activism in response to popular enthusiasm for environmental and products liability.  As part of historical review, Rabin describes the growth of strict product liability, the advent of medical monitoring, fear, and increased risk damages, and the application of class action procedures to so-called toxic torts.

The story is familiar, but here it is told with enthusiasm for the very idea of liability.  Although I may be misreading the piece, Rabin seems to share the popular enthusiasm for liability, and regrets missed opportunities to impose even greater liability.  For instance, Rabin tells us that the “signals” sent by mass tort cases involving asbestos, Agent Orange, and Dalkon Shield, were “encouraging,” while the Bendectin litigation was one of the “notable litigation failures.” Id. at 105

The reader is challenged to imagine exactly what Professor Rabin has in mind by his scorecard of successes and failures.  Why, for instance, would anyone consider the asbestos litigation encouraging?  Asbestos litigation can take credit for dozens of bankruptcies, with the erosion of the country’s industrial manufacturing capability.  Jobs have been lost.  The asbestos litigation can take further credit for:

  • disruption and destruction of insurance markets,
  • procedural innovations, such as collusive class actions that sold out future claimants,
  • collusive bankruptcies that favored powerfully positioned plaintiffs’ law firms,
  • egregious consolidations
  • magic jurisdictions known “easy law,”
  • special rules for asbestos cases that deprived defendants of their opportunity to prepare defenses

Of course, procedural peculiarities of asbestos litigation pale in comparison with the substantive abuses:

  • fraudulent product identification,
  • fraudulent diagnoses,
  • unlawful and unethical mass screenings,
  • diluted causation standards,
  • markets for junk medicine
  • speculative damages for fear and risk of unrelated diseases
  • governmental avoidance of liability for its widespread use of asbestos in shipyards, and elsewhere

A sensible reaction would be to condemn asbestos litigation, and similar enterprises, as grotesque failures, and to cede the control of  risks, to the extent they are real, to federal and state police powers.  Here, however, Professor Rabin goes to an even farther extreme:  he tells us that “regulation has played virtually no role at all in reducing risk and compensating victims.”  Id. 113.  Rabin tells us that the regulatory failure was “especially evident in the case of asbestos,” which continued to be used in marketed products, and thus “remained unregulated in any meaningful sense, until the toll of death and disease had spiraled entirely out of control.”  Id. at 113 & n 62. 

Well, most (but not all) regulations of asbestos deal with mitigating risk, actual or potential, and not with providing compensation.  So on that score, we can hardly fault EPA, OSHA, CDC, NIOSH, etc., in their handling of health risks from asbestos.  The remainder of this assessment is equally difficult to understand.  The landmark case of Borel v. Fibreboard Paper Products Corp., 493 F.2d 1076 (5th Cir. 1973), cited by Professor Rabin, came one year before asbestos-containing insulation products were banned.  To be sure, EPA and OSHA have failed to ban all uses of asbestos, but their failure is driven by a lack of scientific knowledge that extremely low exposures to asbestos, and especially to chrysotile asbestos, are of any moment at all.  Asbestosis has become a medical curiosity in the last decade or so.  Lung cancer continues of course because men and women continue to smoke tobacco products.  Mesothelioma rates have stabilized or decreased, and the orthodoxy that asbestos causes gastrointestinal cancers has been debunked by this country’s Institute of Medicine.  Tellingly, Professor Rabin cites no support for his opinion that the failure to regulate low exposures to asbestos played any role in producing a spiral of death and disease. 

And why was Bendectin litigation a failure?  A new-age style of consolidated trials of multiple claimants in federal court ended in a defense verdict on general causation.  Although a few state courts were more hospitable to the plaintiffs’ claims, the Bendectin litigation taught the federal bench and most state courts about the quality and quantity of extremist advocacy on the part of claimants.  We owe Havner and Daubert, and a host of lesser known cases to Bendectin litigation.  So although much work needs to be done, one of Bendectin’s litigation successes was the education of American courts in the ways of statistical and epidemiologic evidence.  Ultimately, the courts put their teeth into standard procedural devices, such as summary judgment and expert witness gatekeeping, to put the Bendectin claims to rest.  Before the manufacturer, Merrill Richardson achieved vindication, however, it pulled an efficacious medication from the market, despite the absence of reliable evidence to support the claims that it caused birth defects.  Perhaps Rabin suggested that Bendectin was a litigation failure because the litigation process could not shut down the unfounded allegations and claims in time to save a worthwhile medication.

Absent from Professor Rabin’s historical discussion is any mention of the silicone gel breast implant litigation, which took hold with the advocacy of expert witnesses, described by Judge Jack Weinstein as “charlatans,” B. Weinstein, “Preliminary Reflections on Administration of Complex Litigation.”   The massive, toxic litigation inspired by silicone led to billions of dollars in settlements before a few courageous judges (including Judge Weinstein) were willing to pay attention to the science in a more discriminating fashion.  Also absent from Rabin’s retrospective is any mention of the silica litigation, with its rampant fraud that has led to the defrocking of several physician witnesses.  In re Silica Products Liab. Litig., MDL No. 1553, 398 F.Supp. 2d 563 (S.D.Tex. 2005).

In his final analysis, Professor Rabin seems to acknowledge that the enthusiasm of the 1970s and early 1980s had to give way to other institutional goals, values, and considerations.  What Rabin does not say, about the abuses and excesses of toxic torts, and the toxic litigation it spawned, however, could fill volumes.