TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Mississippi High Court Takes the Bite Out of Forensic Evidence

November 3rd, 2017

The Supreme Court’s 1993 decision in Daubert changed the thrust of Federal Rule of Evidence 702, which governs the admissibility of expert witness opinion testimony in both civil and criminal cases. Before Daubert, lawyers who hoped to exclude opinions lacking in evidentiary and analytical support turned to the Frye decision on “general acceptance.” Frye, however, was an outdated rule that was rarely applied outside the context of devices. Furthermore, the meaning and application of Frye were unclear. Confusion reigned on whether expert witnesses could survive Frye challenges simply by adverting to their claimed use of a generally accepted science, such as epidemiology, even though their implementation of epidemiologic science was sloppy, incoherent, and invalid.

Daubert noted that Rule 702 should be interpreted in the light of the “liberal” goals of the Federal Rules of Evidence. Some observers rejoiced at the invocation of “liberal” values, but history of the last 25 years has shown that they really yearned for libertine interpretations of the rules. Liberal, of course, never meant “anything goes.” It is unclear why “liberal” cannot mean restricting evidence not likely to advance the truth-finding function of trials.

Criminal versus Civil

Back on April 27, 2009, then President Barack Obama announced the formation of the President’s Council of Advisors on Science and Technology (PCAST). The mission of PCAST was to advise the President and his administration on science and technology, and their policy implications. Although the PCAST was a new council, presidents have had scientific advisors and advisory committees back to Franklin Roosevelt, in 1933.

On September 20, 2016, PCAST issued an important report to President Obama, Report to the President on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods. Few areas of forensic “science,” beyond DNA matching, escaped the Council’s withering criticism. Bite-mark evidence in particular received a thorough mastication.

The criticism was hardly new. Seven years earlier, the National Academies of Science issued an indictment that forensic scientists had largely failed to establish the validity of their techniques and conclusions, and that the judiciary had “been utterly ineffective in addressing this problem.”1

The response from Obama’s Department of Justice, led by Loretta Lynch, was underwhelming.2 The Trump response was equally disappointing.3 The Left and the Right appear to agree that science is dispensable when it becomes politically inconvenient. It is a common place in the community of evidence scholars that Rule 702 is not applied with the same enthusiasm in criminal cases, to the benefit of criminal defendants, as the rule is sometimes, sporadically and inconsistently applied in civil cases. The Daubert revolution has failed the criminal justice system perhaps because courts are unwilling to lift the veil on forensic evidence, for fear they may not like what the find.4

A Grudging Look at the Scientific Invalidity of Bite Mark Evidence

Sherwood Brown was convicted of a triple murder in large measure as a result of testimony from Dr. Michael West, a forensic odontologist. West, as well as another odontologist, opined that a cut on Brown’s wrist matched the shape of a victim’s mouth. DNA testing authorized after the conviction, however, rendered West’s opinions edentulous. Samples from inside the female victim’s mouth yielded male DNA, but not that of Mr. Brown.5

Did the PCAST report leave an impression upon the highest court of Mississippi? The Supreme Court of Mississippi vacated Brown’s conviction and remanded for a new trial, in an opinion that a bitemark expert might describe as reading like a bite into a lemon. Brown v. State, No. 2017 DR 00206 SCT, Slip op. (Miss. Sup. Ct. Oct. 26, 2017). The majority could not bring themselves to comment upon the Dr. West’s toothless opinions. Three justices would have kicked the can down to the trial judge by voting to grant a new hearing without vacating Brown’s convictions. The decision seems mostly predicated on the strength of the DNA evidence, rather than the invalidity of the bite mark evidence. Mr. Brown will probably be vindicated, but bite mark evidence will continue to mislead juries, with judicial imprimatur.


1 National Research Council, Committee on Identifying the Needs of the Forensic Sciences Community, Strengthening Forensic Science in the United States: A Path Forward 53 (2009).

2 See Jordan Smith, “FBI and DoJ Vow to Continue Using Junk Science Rejected by White House Report,” The Intercept (Sept. 23, 2016); Radley Balko, “When Obama wouldn’t fight for science,” Wash. Post (Jan. 4, 2017).

3 See Radley Balko, “Jeff Sessions wants to keep forensics in the Dark Ages,” Wash. Post (April 11, 2017); Jessica Gabel Cino, “Session’s Assault on Forensic Science Will Lead to More Unsafe Convictions,” Newsweek (April 20, 2017).

4 See, e.g., Paul C. Giannelli, “Forensic Science: Daubert’s Failure,” Case Western Reserve L. Rev. (2017) (“in press”).

Multiplicity in the Third Circuit

September 21st, 2017

In Karlo v. Pittsburgh Glass Works, LLC, C.A. No. 2:10-cv-01283 (W. D. Pa.), plaintiffs claimed that their employer’s reduction in force unlawfully targeted workers over 50 years of age. Plaintiffs lacked any evidence of employer animus against old folks, and thus attempted to make out a statistical disparate impact claim. The plaintiffs placed their chief reliance upon an expert witness, Michael A. Campion, to analyze a dataset of workers agreed to have been the subject of the R.I.F. For the last 30 years, Campion has been on the faculty in Purdue University. His academic training and graduate degrees are in industrial and organizational psychology. Campion has served an editor of Personnel Psychology, and as a past president of the Society for Industrial and Organizational Psychology. Campion’s academic website page notes that he manages a small consulting firm, Campion Consulting Services1.

The defense sought to characterize Campion as not qualified to offer his statistical analysis2. Campion did, however, have some statistical training as part of his master’s level training in psychology, and his professional publications did occasionally involve statistical analyses. To be sure, Campion’s statistical acumen paled in comparison to the defense expert witness, James Rosenberger, a fellow and a former vice president of the American Statistical Association, as well as a full professor of statistics in Pennsylvania State University. The threshold for qualification, however, is low, and the defense’s attack on Campion’s qualifications failed to attract the court’s serious attention.

On the merits, the defense subjected Campion to a strong challenge on whether he had misused data. The defense’s expert witness, Prof. Rosenberger, filed a report that questioned Campion’s data handling and statistical analyses. The defense claimed that Campion had engaged in questionable data manipulation by including, in his RIF analysis, workers who had been terminated when their plant was transferred to another company, as well as workers who retired voluntarily.

Using simple z-score tests, Campion compared the ages of terminated and non-terminated employees in four subgroups, ages 40+, 45+, 50+, and 55+. He did not conduct an analysis of the 60+ subgroup on the claim that this group had too few members for the test to have sufficient power3Campion found a small z-score for the 40+ versus <40 age groups comparison (z =1.51), which is not close to statistical significance at the 5% level. On the defense’s legal theory, this was the crucial comparison to be made under the Age Discrimination in Employment Act (ADEA). The plaintiffs, however, maintained that they could make out a case of disparate impact by showing age discrimination at age subgroups that started above the minimum specified by the ADEA. Although age is a continuous variable, Campion decided to conduct z-scores on subgroups that were based upon five-year increments. For the 45+, 50+, and 55+ age subgroups, he found z-scores that ranged from 2.15 to 2.46, and he concluded that there was evidence of disparate impact in the higher age subgroups4. Karlo v. Pittsburgh Glass Works, LLC, C.A. No. 2:10-cv-01283, 2015 WL 4232600, at *11 (W.D. Pa. July 13, 2015) (McVerry, S.J.)

The defense, and apparently the defense expert witnesses, branded Campion’s analysis as “data snooping,” which required correction for multiple comparisons. In the defense’s view, the multiple age subgroups required a Bonferroni correction that would have diminished the critical p-value for “significance” by a factor of four. The trial court agreed with the defense contention about data snooping and multiple comparisons, and excluded Campion’s opinion of disparate impact, which had been based upon finding statistically significant disparities in the 45+, 50+, and 55+ age subgroups. 2015 WL 4232600, at *13. The trial court noted that Campion, in finding significant disparities in terminations in the subgroups, but not in the 40+ versus <40 analysis:

[did] not apply any of the generally accepted statistical procedures (i.e., the Bonferroni procedure) to correct his results for the likelihood of a false indication of significance. This sort of subgrouping ‘analysis’ is data-snooping, plain and simple.”

Id. After excluding Campion’s opinions under Rule 702, as well as other evidence in support of plaintiffs’ disparate impact claim, the trial court granted summary judgment on the discrimination claims. Karlo v. Pittsburgh Glass Works, LLC, No. 2:10–cv–1283, 2015 WL 5156913 (W. D. Pa. Sept. 2, 2015).

On plaintiffs’ appeal, the Third Circuit took the wind out of the attack on Campion by holding that the ADEA prohibits disparate impacts based upon age, which need not necessarily be on workers’ being over 40 years old, as opposed to being at least 40 years old. Karlo v. Pittsburgh Glass Works, LLC, 849 F.3d 61, 66-68 (3d Cir. 2017). This holding took the legal significance out of the statistical insignificance of Campion’s comparison 40+ versus <40 age-group termination rates. Campion’s subgroup analyses were back in play, but the Third Circuit still faced the question whether Campion’s conclusions, based upon unadjusted z-scores and p-values, offended Rule 702.

The Third Circuit noted that the district court had identified three grounds for excluding Campion’s statistical analyses:

(1) Dr. Campion used facts or data that were not reliable;

(2) he failed to use a statistical adjustment called the Bonferroni procedure; and

(3) his testimony lacks ‘‘fit’’ to the case because subgroup claims are not cognizable.

849 F.3d at 81. The first issue was raised by the defense’s claims of Campion’s sloppy data handling, and inclusion of voluntarily retired workers and workers who were terminated when their plant was turned over to another company. The Circuit did not address these data handling issues, which it left for the trial court on remand. Id. at 82. The third ground went out of the case with the appellate court’s resolution of the scope of the ADEA. The Circuit did, however, engage on the issue whether adjustment for multiple comparisons was required by Rule 702.

On the “data-snooping” issue, the Circuit concluded that the trial court had applied “an incorrectly rigorous standard for reliability.” Id. The Circuit acknowledged that

[i]n theory, a researcher who searches for statistical significance in multiple attempts raises the probability of discovering it purely by chance, committing Type I error (i.e., finding a false positive).”

849 F.3d at 82. The defense expert witness contended that applying the Bonferroni adjustment, which would have reduced the critical significance probability level from 5% to 1%, would have rendered Campion’s analyses not statistically significant, and thus not probative of disparate impact. Given that plaintiffs’ cases were entirely statistical, the adjustment would have been fatal to their cases. Id. at 82.

At the trial level and on appeal, plaintiffs and Campion had objected to the data-snooping charge on ground that

(1) he had engaged in only four subgroups;

(2) virtually all subgroups were statistically significant;

(3) his methodology was “hypothesis driven” and involved logical increments in age to explore whether the strength of the evidence of age disparity in terminations continued in each, increasingly older subgroup;

(4) his method was analogous to replications with different samples; and

(5) his result was confirmed by a single, supplemental analysis.

Id. at 83. According to the plaintiffs, Campion’s approach was based upon the reality that age is a continuous, not a dichotomous variable, and he was exploring a single hypothesis. A.240-241; Brief of Appellants at 26. Campion’s explanations do mitigate somewhat the charge of “data snooping,” but they do not explain why Campion did not use a statistical analysis that treated age as a continuous variable, at the outset of his analysis. The single, supplemental analysis was never described or reported by the trial or appellate courts.

The Third Circuit concluded that the district court had applied a ‘‘merits standard of correctness,’’ which is higher than what Rule 702 requires. Specifically, the district court, having identified a potential methodological flaw, did not further evaluate whether Campion’s opinion relied upon good grounds. 849 F.3d at 83. The Circuit vacated the judgment below, and remanded the case to the district court for the opportunity to apply the correct standard.

The trial court’s acceptance that an adjustment was appropriate or required hardly seems a “merits standard.” The use of a proper adjustment for multiple comparisons is very much a methodological concern. If Campion could reach his conclusion only by way of an inappropriate methodology, then his conclusion surely would fail the requirements of Rule 702. The trial court did, however, appear to accept, without explicit evidence, that the failure to apply the Bonferroni correction made it impossible for Campion to present sound scientific argument for his conclusion that there had been disparate impact. The trial court’s opinion also suggests that the Bonferroni correction itself, as opposed to some more appropriate correction, was required.

Unfortunately, the reported opinions do not provide the reader with a clear account of what the analyses would have shown on the correct data set, without improper inclusions and exclusions, and with appropriate statistical adjustments. Presumably, the parties are left to make their cases on remand.

Based upon citations to sources that described the Bonferroni adjustment as “good statistical practice,” but one that is ‘‘not widely or consistently adopted’’ in the behavioral and social sciences, the Third Circuit observed that in some cases, failure to adjust for multiple comparisons may “simply diminish the weight of an expert’s finding.”5 The observation is problematic given that Kumho Tire suggests that an expert witness must use “in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.” Kumho Tire Co. v. Carmichael, 526 U.S. 137, 150, (1999). One implication is that courts are prisoners to prevalent scientific malpractice and abuse of statistical methodology. Another implication is that courts need to look more closely at the assumptions and predicates for various statistical tests and adjustments, such as the Bonferroni correction.

These worrisome implications are exacerbated by the appellate court’s insistence that the question whether a study’s result was properly calculated or interpreted “goes to the weight of the evidence, not to its admissibility.”6 Combined with citations to pre-Daubert statistics cases7, judicial comments such as these can appear to be a general disregard for the statutory requirements of Rules 702 and 703. Claims of statistical significance, in studies with multiple exposure and multiple outcomes, are frequently not adjusted for multiple comparisons, without notation, explanation, or justification. The consequence is that study results are often over-interpreted and over-sold. Methodological errors related to multiple testing or over-claiming statistical significance are commonplace in tort litigation over “health-effects” studies of birth defects, cancer, and other chronic diseases that require epidemiologic evidence8.

In Karlo, the claimed methodological error is beset by its own methodological problems. As the court noted, adjustments for multiple comparisons are not free from methodological controversy9. One noteworthy textbook10 labels the Bonferroni correction as an “awful response” to the problem of multiple comparisons. Aside from this strident criticism, there are alternative approaches to statistical adjustment for multiple comparisons. In the context of the Karlo case, the Bonferroni might well be awful because Campion’s four subgroups are hardly independent tests. Because each subgroup is nested within the next higher age subgroup, the subgroup test results will be strongly correlated in a way that defeats the mathematical assumptions of the Bonferroni correction. On remand, the trial court in Karlo must still make his Rule 702 gatekeeping decision on the methodological appropriateness of whether Campion’s properly considered the role of multiple subgroups, and multiple anaslyses run on different models.


1 Although Campion describes his consulting business as small, he seems to turn up in quite a few employment discrimination cases. See, e.g., Chen-Oster v. Goldman, Sachs & Co., 10 Civ. 6950 (AT) (JCF) (S.D.N.Y. 2015); Brand v. Comcast Corp., Case No. 11 C 8471 (N.D. Ill. July 5, 2014); Powell v. Dallas Morning News L.P., 776 F. Supp. 2d 240, 247 (N.D. Tex. 2011) (excluding Campion’s opinions), aff’d, 486 F. App’x 469 (5th Cir. 2012).

2 See Defendant’s Motion to Bar Dr. Michael Campion’s Statistical Analysis, 2013 WL 11260556.

3 There was no mention of an effect size for the lower aged subgroups, and a power calculation for the 60+ subgroup’s probability of showing a z-score greater than two. Similarly, there was no discussion or argument about why this subgroup could not have been evaluated with Fisher’s exact test. In deciding the appeal, the Third Circuit observed that “Dr. Rosenberger test[ed] a subgroup of sixty-and-older employees, which Dr. Campion did not include in his analysis because ‘[t]here are only 14 terminations, which means the statistical power to detect a significant effect is very low’. A.244–45.” Karlo v. Pittsburgh Glass Works, LLC, 849 F.3d 61, 82 n.15 (3d Cir. 2017).

4 In the trial court’s words, the z-score converts the difference in termination rates into standard deviations. Karlo v. Pittsburgh Glass Works, LLC, C.A. No. 2:10-cv-01283, 2015 WL 4232600, at *11 n.13 (W.D. Pa. July 13, 2015). According to the trial court, Campion gave a rather dubious explanation of the meaning of the z-score: “[w]hen the number of standard deviations is less than –2 (actually–1.96), there is a 95% probability that the difference in termination rates of the subgroups is not due to chance alone” Id. (internal citation omitted).

5 See 849 F.3d 61, 83 (3d Cir. 2017) (citing and quoting from Paetzold & Willborn § 6:7, at 308 n.2) (describing the Bonferroni adjustment as ‘‘good statistical practice,’’ but ‘‘not widely or consistently adopted’’ in the behavioral and social sciences); see also E.E.O.C. v. Autozone, Inc., No. 00-2923, 2006 WL 2524093, at *4 (W.D. Tenn. Aug. 29, 2006) (‘‘[T]he Court does not have a sufficient basis to find that … the non-utilization [of the Bonferroni adjustment] makes [the expert’s] results unreliable.’’). And of course, the Third Circuit invoked the Daubert chestnut: ‘‘Vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but

admissible evidence.’’ Daubert, 509 U.S. 579, 596 (1993).

6 See 849 F.3d at 83 (citing Leonard v. Stemtech Internat’l Inc., 834 F.3d 376, 391 (3d Cir. 2016).

7 See 849 F.3d 61, 83 (3d Cir. 2017), citing Bazemore v. Friday, 478 U.S. 385, 400 (1986) (‘‘Normally, failure to include variables will affect the analysis’ probativeness, not its admissibility.’’).

8 See Hans Zeisel & David Kaye, Prove It with Figures: Empirical Methods in Law and Litigation 93 & n.3 (1997) (criticizing the “notorious” case of Wells v. Ortho Pharmaceutical Corp., 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S. 950 (1986), for its erroneous endorsement of conclusions based upon “statistically significant” studies that explored dozens of congenital malformation outcomes, without statistical adjustment). The authors do, however, give an encouraging example of a English trial judge who took multiplicity seriously. Reay v. British Nuclear Fuels (Q.B. Oct. 8,1993) (published in The Independent, Nov. 22,1993). In Reay, the trial court took seriously the multiplicity of hypotheses tested in the study relied upon by plaintiffs. Id. (“the fact that a number of hypotheses were considered in the study requires an increase in the P-value of the findings with consequent reduction in the confidence that can be placed in the study result … .”), quoted in Zeisel & Kaye at 93. Zeisel and Kaye emphasize that courts should not be overly impressed with claims of statistically significant findings, and should pay close attention to how expert witnesses developed their statistical models. Id. at 94.

9 See David B. Cohen, Michael G. Aamodt, and Eric M. Dunleavy, Technical Advisory Committee Report on Best Practices in Adverse Impact Analyses (Center for Corporate Equality 2010).

10 Kenneth J. Rothman, Sander Greenland, and Timoth L. Lash, Modern Epidemiology 273 (3d ed. 2008); see also Kenneth J. Rothman, “No Adjustments Are Needed for Multiple Comparisons,” 1 Epidemiology 43, 43 (1990)

Seventh Circuit Franks ‘Every Exposure’ Theory for Extinction

September 11th, 2017

In Krik v. Exxon Mobil Corp., no. 15-3112, 2017 WL 3768933, Slip op. (7th Cir. Aug. 31, 2017) [slip op. cited as Krik], a jury found that smoking cigarettes causes lung cancer, which is not particularly noteworthy. The plaintiff, Charles Krik, however, wanted the jury to find that asbestos exposure, either alone or with his 45 pack-year smoking history caused his lung cancer. The jury found that smoking was the sole cause. Hannah Meisel, “7th Circuit Affirms Exxon’s Trial Win In Asbestos Cancer Suit,” Law360 (Sept. 1, 2017).

Krik’s asbestos exposure was not particularly impressive, and he apparently did not have asbestosis. He claimed asbestos exposure from his four years of work aboard naval vessels, occasionally removing insulation materials, and his two weeks as an independent contractor at an Exxon Mobil refinery, where he replaced heaters supposedly insulated with asbestos. Exxon Mobil disputed whether the heaters even had asbestos in them. The naval vessels would have had asbestos insulation from many manufacturers, but Krik focused on Owens-Illinois because it is the only solvent company remaining in the plaintiffs’ asbestos-powered perpetual litigation machine.

Lung cancer in a man with minor asbestos exposure with very substantial tobacco consumption – who are you going to call? See Arthur Frank Report, 2011 WL 12192776 (2011).

Arthur Frank is a physician who counts himself among the intellectual progeny of the late Irving Selikoff. Like Selikoff, Frank is intensely interested in outcomes that help workers show that their work has caused them illness. In furthering his interests, Frank sometimes makes things up, such as the “each and every exposure” theory. Frank is also a proponent of the “big-tent” theory of causation, which attempts to keep every possible defendant in a lawsuit, bu asserting that every asbestos exposure, regardless of its intensity, duration, quantity, variety of asbestos, or fiber length, constitutes a cause of plaintiff’s lung cancer.

Defendants moved to bar Frank’s opinions under Federal Rule of Evidence 702. See Exxon Mobil’s motion, at 2013 WL 10847058. Judge Lee of the Northern District of Illinois found that Arthur Frank’s opinions, in the form of the “each and every exposure theory,” “any exposure theory,” “single fiber theory,” or “no safe level of exposure theory” was scientifically insubstantial and inadmissible under Rule 702. Krik at 2-3. Judge Lee thus ruled that Krik could not offer expert witness opinions that espoused “every exposure” is substantial.

After Judge Lees’ ruling, Krik’s case was transferred to Judge Manish Shah, for trial. Despite the earlier ruling by Judge Lee, Krik’s counsel called Dr. Frank to testify at trial, with a repackaged opinion about Krik’s “cumulative exposure” caused his lung cancer, and every constituent exposure to that cumulative exposure was causally responsible.

After a voir dire examination of Frank, Judge Shah concluded that Frank’s opinion was still untethered to any “specific quantum of exposure attributable to the defendants, but was instead based on his medical and scientific opinion that every exposure is a substantial contributing factor to the cumulative exposure that causes cancer.” Krik v. Owens‐Illinois, Inc., No. 10‐CV‐07435, 2015 WL 5050143, at *1 (N.D. Ill. Aug. 25, 2015). Frank and plaintiffs’ counsel had attempted to circumvent the earlier ruling by Judge Lee, but their ruse failed to fool Judge Shah. On appeal to the Seventh Circuit1, a panel affirmed Judge Shah’s reasoning and exclusion of Arthur Frank’s opinions. Krik at 4-5.

Arthur Frank is used to making things up, including the law. The law of causation in most jurisdictions distinguishes between substantial and insubstantial contribution, but Frank decreed: “Either it’s zero or it’s substantial; there is no such thing as not substantial.” R. 66‐3 at 23, pageID 923. Really? In Frank’s mind, even a minute, perhaps a second, of fleeting exposure, would be a substantial contributing factor to a plaintiff’s lung cancer because he has legislated insubstantial out of existence. R. 376 at 273–74, pageID 10146‐47.

Frank’s testimony presented several problems:

First, his cumulative exposure theory was no different from the previously excluded “each and every exposure” theory. Even Frank, in his deposition testimony conflated “each and every exposure” with a cumulative exposure theory.

Second, Frank’s opinion did not conform to the legal standard. In the initial ruling on Frank, Judge Lee held that plaintiff must show that asbestos was a “substantial contributing factor” to his injury2.

Third, Frank’s opinion lacked an adequate scientific foundation. Krik was tasked with showing that asbestos was a “substantial contributing factor” to his lung cancer. Krik at 7; Krik, 76 F. Supp. 3d at 747 (Lee, J.). Frank’s opinion on “every exposure” did not help him make out his case.

The trial court judges recognized, putting aside the issue of thresholds, that asbestos‐induced lung cancers are dose dependent. At the very least, any attempt to attribute a person’s lung cancer to an exposure requires a consideration of the timing and quantum of exposure. Frank, in defiance of basic common sense and basic toxicologic principles, would – if allowed by courts – treat every exposure, regardless how de minimis, as a substantial contribution to the total exposure and the total risk. Krik at 8; Krik, 76 F. Supp. 3d at 753 (Lee, J.).

The panel of the Seventh Circuit found the trial judges’ exclusion of the Frank nonsense to be well supported and well within their discretion as gatekeepers3. Krik at 14

Krik’s counsel also complained that the trial court refused to admit the so-called Helsinki document4, a 1997 statement of public policy statement of scientists who opined that “[c]umulative exposure on a probability basis should thus be considered the main criteria for the attribution of a substantial contribution by asbestos to lung cancer risk.” R. 412‐4 at 4, pageID 13657.

The problem for counsel, and for Frank, was that Frank never referred to or embraced the Helsinki statement as an “authoritative text.” If he had, he would have been roundly impeached by the statement’s pronouncement that the “likelihood that asbestos exposure has made a substantial contribution increases when the exposure increases.” Id. The Seventh Circuit held that the exclusion of this document as a stand-alone piece of evidence did not support plaintiff’s theory, and that its exclusion was not an abuse of discretion5. Krik at 15-17.


1 The appellate court noted that it reviewed de novo the question whether the trial court properly applied Rule 702. The district court’s decision to exclude or admit expert witness opinion testimony is reviewed only for “abuse of discretion.” Krik at 4 (citing C.W. ex rel. Wood v. Textron, Inc., 807 F.3d 827, 835 (7th Cir. 2015). The party proponent has the burden of showing that the challenged expert witness testimony satisfies the Rule 702 statutory requirements, by a preponderance of evidence. Id. (citing Lewis v. CITGO Petroleum Corp., 561 F.3d 698, 705 (7th Cir. 2009).

2 Krik v. Crane Co., 76 F. Supp. 3d 747, 753 (N.D. Ill. 2014) (citing Lindstrom v. A‐C Prod. Liab., 424 F.3d 488, 493 (6th Cir. 2005) (applying maritime law); Thacker v. UNR Indus., Inc., 603 N.E.2d 449, 457 (Ill. 1992) (Illinois law).

3 The panel noted that the Sixth and Ninth Circuits had ruled similarly. McIndoe v. Huntington Ingalls Inc., 817 F.3d 1170, 1177 (9th Cir. 2016); Lindstrom v. A‐C Prod. Liab., 424 F.3d 488, 493 (6th Cir. 2005) (“The requirement, however, is that the plaintiff make a showing with respect to each defendant that the defendant’s product was a substantial factor in plaintiff’s injury … . A holding to the contrary would permit imposition of liability on the manufacturer of any product with which a worker had the briefest of encounters on a single occasion.”).

5 Accord Rockman v. Union Carbide Corp., No. CV RDB‐16‐1169, 2017 WL 3022969, at *5 (D. Md. July 17, 2017); Bell v. Foster Wheeler Energy Corp., No. CV 15‐6394, 2016 WL 5847124, at *3, n.3 (E.D. La. Oct. 6, 2016), recon. denied, No. CV 15‐6394, 2017 WL 876983 (E.D. La. Mar. 6, 2017); Watkins v. Affinia Group, 2016‐Ohio‐2830, ¶ 37, 54 N.E.3d 174, 182; In reJames Wilson Assoc., 965 F.2d 160, 173 (7th Cir.1992); United States v. Dixon, 413 F.3d 520, 524–25 (5th Cir. 2005); Yates v. Ford Motor Co., 113 F. Supp. 3d 841, 862 (E.D.N.C. 2015); Betz v. Pneumo Abex, LLC, 44 A.3d 27, 47, 55 n.35 (Pa. 2012); Bostic v. Georgia‐Pacific Corp., 439 S.W.3d 332, 356–57 (Tex. 2014).

WOE — Zoloft Escapes a MDL While Third Circuit Creates a Conceptual Muddle

July 31st, 2017

Multidistrict Litigations (MDLs) can be “muddles” that are easy to get in, but hard to get out of. Pfizer and subsidiary Greenstone fabulously escaped a muddle through persistent lawyering and the astute gatekeeping of a district judge, in the Eastern District of Pennsylvania. That judge, the Hon. Cynthia Rufe, sustained objections to the admissibility of plaintiffs’ epidemiologic expert witness Anick Bérard. When the MDL’s plaintiffs’ steering committee (PSC) demanded, requested, and begged for a do over, Judge Rufe granted them one more chance. The PSC put their litigation industry eggs in a single basket, carried by statistician Nicholas Jewell. Unfortunately for the PSC, Judge Rufe found Jewell’s basket to be as methodologically defective as Bérard’s, and Her Honor excluded Jewell’s proffered testimony. Motions, paper, and appeals followed, but on June 2, 2017, the Third Circuit declared that the PSC and its clients had had enough opportunities to get through the gate. Their baskets of methodological deplorables were not up to snuff. In re Zoloft Prod. Liab. Litig., No. 16-2247 , __ F.3d __, 2017 WL 2385279, 2017 U.S. App. LEXIS 9832 (3d Cir. June 2, 2017) (affirming exclusion of Jewell’s dodgy opinions, which involved multiple methodological flaws and failures to follow any methodology faithfully) [Slip op. cited below as Zoloft].

Plaintiffs Attempt to Substitute WOE for Depressingly Bad Expert Witness Opinion

The ruse of conflating “weight of the evidence,” as used to describe the appellate standard of review for sustaining or reversing a trial court’s factual finding with a purported scientific methodology for inferring causation, was on full display by the PSC in their attack on Judge Rufe’s gatekeeping. In their appellate brief in the Court of Appeals for the Third Circuit, the PSC asserted that Jewell had used a “weight of the evidence method,” even though that phrase, “weight of the evidence” (WOE) was never used in Jewell’s litigation reports. The full context of the PSC’s argument and citations to Milward make clear a deliberate attempt to conflate WOE as an appellate judicial standard for reviewing jury fact finding and a purported scientific methodology. See Appellants’ Opening Brief at 54 (Aug. 10, 2016) [cited as PSC] (asserting that “[a]t all times, the ultimate evaluation of the weight of the evidence is a jury question”; citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11, 20 (1st Cir. 2011), cert. denied, 133 S. Ct. 63 (2012).

Having staked the ground that WOE is akin to a jury’s factual finding, and thus immune to any but the most extraordinary trial court action or appellate intervention, the PSC then pivoted to claim that Jewell’s WOE-ful method was nothing much more than an assessment of “the totality of the available scientific evidence, guided by the well-accepted Bradford-Hill criteria.” PSC at 3, 4, 7. This maneuver allowed the PSC to argue, apparently with a straight face, that WOE methodology as used by Jewell, had been generally accepted in the scientific community, as well as by the Third Circuit, in previous cases in which the court accepted the use of Bradford Hill’s considerations as a reliable method for establishing general causation. See PSC at 4 (citing Gannon v. United States, 292 F. App’x 170, 173 n.1 (3d Cir. 2008)). Jewell then simply plugged in his expertise and “40 years of experience,” and the desired conclusion of causation popped out. Id. Quod erat demonstrandum.

In pressing its point, the PSC took full advantage of loose, inaccurate language from the American Law Institute’s Restatement’s notorious comment C:

No algorithm exists for applying the Hill guidelines to determine whether an association truly reflects a causal relationship or is spurious.”

PSC at 33-34, citing Restatement (Third) of Torts: Physical and Emotional Harm § 28 cmt. c(3) (2010). Well true, but the absence of a mathematical algorithm hardly means that causal judgments are devoid of principles and standards. The PSC was undeterred, by text or by shame, from equating an unarticulated use of WOE methodology with some vague invocation of Bradford Hill’s considerations for evaluating associations for causality. See PSC at 43 (citing cases that never mentioned WOE but only Bradford Hill’s 50-plus year old heuristic as somehow supporting the claimed identity of the two approaches)1.

Pfizer Rebuffs WOE

Pfizer filed a comprehensive brief that unraveled the PSC’s duplicity. For unknown reasons, tactical or otherwise, however, Pfizer did not challenge the specifics of PSC’s equation of WOE with an abridged, distorted application of Bradford Hill’s considerations. See generally Opposition Brief of Defendants-Appellees Pfizer Inc., Pfizer International LLC, and Greenstone LLC [cited as Pfizer]. Perhaps given page limits and limited judicial attention spans, and just how woefully bad Jewell’s opinions were, Pfizer may well have decided that a more directed approach of assuming arguendo WOE’s methodological appropriateness was a more economical, pragmatic approach. A close reading of Pfizer’s brief, however, makes clear that it never conceded the validity of WOE as a scientific methodology.

Pfizer did point to the recasting of Jewell’s aborted attempt to apply Bradford Hill considerations as an employment of WOE methodology. Pfizer at 46-47. The argument reminded me of Abraham Lincoln’s famous argument:

How many legs does a dog have if you call his tail a leg?

Four.

Saying that a tail is a leg doesn’t make it a leg.”

Allen Thorndike Rice, Reminiscences of Abraham Lincoln by Distinguished Men of His Time at 242 (1909). Calling Jewell’s supposed method WOE or Bradford Hill or WOE/Bradford Hill did not cure the “fatal methodological flaws in his opinions.” Pfizer at 47.

Pfizer understandably and properly objected to the PSC’s attempt to cast Jewell’s “methodology” at such a high level of generality that any consideration of the many instances of methodological infidelity would be relegated to mere jury questions. Acquiescence in the PSC’s rhetorical move would constitute a complete abandonment of the inquiry whether Jewell had used a proper method. Pfizer at 15-16.

Interestingly, none of the amici curiae addressed the slippery WOE arguments advanced by the PSC. See generally Brief of Amici Curiae American Tort Reform Ass’n & Pharmaceutical Research and Manufacturers of America (Oct. 18, 2016); Brief of Washington Legal Fdtn. as Amicus Curiae (Oct. 18, 2016). There was no meaningful discussion of WOE as a supposedly scientific methodology at oral argument. See Transcript of Oral Argument in In re Zoloft Prod. Liab. Litig., No. 16-2247 (Jan. 25, 2017).

The Third Circuit Acknowledges that Some Methodological Infelicities, Flaws, and Fallacies Are Properly the Subject of Judicial Gatekeeping

Fortunately, Jewell’s methodological infidelities were easily recognized by the Circuit judges. Jewell treated multiple studies, which were nested within one another, and thus involved overlapping and included populations, as though they were independent verifications of the same hypothesis. When the population at issue (from the Danish cohort) was included in a more inclusive pan-Scandivanian study, the relied-upon association dissipated, and Jewell utterly failed to explain or account for these data. Zoloft at 5-6.

Jewell relied upon a study by Anick Bérard, even though he later had to concede that the study had serious flaws that invalidated its conclusions, and which flaws caused him to have a lack of confidence in the paper’s findings.2 In another instance, Jewell relied innocently upon a study that purported to report a statistically significant association, but the authors of this paper were later required by the journal, The New England Journal of Medicine, to correct the very calculated confidence interval upon which Jewell had relied. Despite his substantial mathematical prowess, Jewell missed the miscalculation and relied (uncritically) upon a finding as statistically significant when in fact it was not.

Jewell rejected a meta-analysis of Zoloft studies for questionable methodological quibbles, even though he had relied upon the very same meta-analysis, with the same methodology, in his litigation efforts involving Prozac and birth defects. Not to be corralled by methodological punctilio, Jewell conducted his own meta-analysis with two studies Huybrechts (2014) and Jimenez-Solem (2012), but failed to explain why he excluded other studies, the inclusion of which would have undone his claimed result. Zoloft at 9. Jewell purported to reanalyze and recalculate point estimates in two studies, Jimenez-Solem (2012) and Huybrechts (2014), without any clear protocol or consistency in his approach to other studies. Zoloft at 9. The list goes on, but in sum, Jewell’s handling of these technical issues did not inspire confidence, either in the district or in the appellate court.

WOE to the Third Circuit

The Circuit gave the PSC every conceivable break. Because Pfizer had not engaged specifically on whether WOE was a proper, or any kind of, scientific method, the Circuit treated the issue as virtually conceded:

Pfizer does not seem to contest the reliability of the Bradford Hill criteria or weight of the evidence analysis generally; the dispute centers on whether the specific methodology implemented by Dr. Jewell is reliable. Flexible methodologies, such as the “weight of the evidence,” can be implemented in multiple ways; despite the fact that the methodology is generally reliable, each application is distinct and should be analyzed for reliability.”

Zoloft at 18. The Court acknowledged that WOE arose only in the PSC’s appellate brief, which would have made the entire dubious argument waived under general appellate jurisdictional principles, but the Court, in a footnote, indulged the assumption, “for the sake of argument,” that WOE was Jewell’s purported method from the inception. Zoloft at 18 n. 39. Without any real evidentiary support or analysis or concession from Pfizer, the Circuit accepted that WOE analyses were “generally reliable.” Zoloft at 21.

The Circuit accepted, rather uncritically, that Jewell used a combination of WOE analysis and Bradford Hill considerations. Zoloft at 17. Although Jewell had never described WOE in his litigation report, and WOE was not a feature of his hearing testimony, the Circuit impermissibly engrafted Carl Cranor’s description of WOE as involving inference to the best explanation. Zoloft at 17 & n.37, citing Milward v. Acuity Specialty Prods. Grp., Inc., 639 F.3d 11, 17 (1st Cir. 2011) (internal quotation marks and citation omitted).

There was, however, a limit to the Circuit’s credulousness and empathy. As the Court noted, there must be some assurance that the purported Bradford Hill/WOE method is something more than a “mere conclusion-oriented selection process.” Zoloft at 20. Ultimately, the Court put its markers down for Jewell’s putative WOE methodology:

there must be a scientific method of weighting that is used and explained.”

Zoloft at 20. Calling the method WOE did not, in the final analysis, exclude Jewell from Rule 702 gatekeeping. Try as the PSC might, there was just no mistaking Jewell’s approach as anything other than a crazy patchwork quilt of numerical wizardry in aid of subjective, result-oriented conclusion mongering.

In the Court’s words:

we find that Dr. Jewell did not 1) reliably apply the ‘techniques’ to the body of evidence or 2) adequately explain how this analysis supports specified Bradford Hill criteria. Because ‘any step that renders the analysis unreliable under the Daubert factors renders the expert’s testimony inadmissible’, this is sufficient to show that the District Court did not abuse its discretion in excluding Dr. Jewell’s testimony.”

Zoloft at 28. As heartening as the Circuit’s conclusion is, the Court’s couching its observation as a finding (“we find”) is disheartening with respect to the Third Circuit’s apparent inability to distinguish abuse-of-discretion review from de novo appellate findings. Equally distressing is the Court’s invocation of Daubert factors, which were dicta in a Supreme Court case that was superseded by an amended statute over 17 years ago, in Federal Rule of Evidence 702.

On the crucial question whether Jewell had engaged in an unreliable application of methods or techniques that superficially, at a very high level of generality, claim to be generally accepted, the Court stayed on course. The Court “found” that Jewell had applied techniques, analyses, and critiques so obviously inconsistently that no amount of judicial indulgence, assumptions arguendo, or careless glosses could save Jewell and his fatuous opinions from judicial banishment. Zoloft 28-29. Returning to the correct standard of review (abuse of discretion), but the wrong governing law (Daubert instead of Rule 702), the Court announced that:

[b]ecause ‘any step that renders the analysis unreliable under the Daubert factors renders the expert’s testimony inadmissible’, this is sufficient to show that the District Court did not abuse its discretion in excluding Dr. Jewell’s testimony.”

Zoloft at 21 n.50 (citation omitted). The Court found itself unable to say simply and directly that “the MDL trial court decided the case well within its discretion.”

The Zoloft case was not the Third Circuit’s first WOE rodeo. WOE had raised its unruly head in Magistrini v. One Hour Martinizing Dry Cleaning, 180 F. Supp. 2d 584, 602 (D.N.J. 2002), aff’d, 68 F. App’x 356 (3d Cir. 2003), where an expert witness, David Ozonoff, offered what purported to be a WOE opinion. The Magistrini trial court did not fuss with the assertion that WOE was generally reliable, but took issue with how Ozonoff tried to pass off his analysis as a comprehensive treatment of the totality of the evidence. In Magistrini, Judge Hochberg noted that regardless of the rubric of the methodology, the witness must show that in conducting a WOE analysis:

all of the relevant evidence must be gathered, and the assessment or weighing of that evidence must not be arbitrary, but must itself be based on methods of science.”

Magistrini, 180 F. Supp. 2d at 602. The witness must show that the methodology is more than a “mere conclusion-oriented selection process,” and that it has a “a scientific method of weighting that is used and explained.” Id. at 607. Asserting the use of WOE was not an excuse or escape from judicial gatekeeping as specified by Rule 702.

Although the Third Circuit gave the Zoloft MDL trial court’s findings a searching review (certainly much tougher than the prescribed abuse-of-discretion review), the MDL court’s finding that Jewell “failed to consistently apply the scientific methods he articulates, has deviated from or downplayed certain well-established principles of his field, and has inconsistently applied methods and standards to the data so as to support his a priori opinion” were ultimately vindicated by the Court of Appeals. Zoloft at 10.

All’s well that ends well. Perhaps. It remains unfortunate, however, that a hypothetical method, WOE — which was never actually advocated by the challenged expert witnesses, which lacks serious support in the scientific community, and which was merely assumed arguendo to be valid — will be taken by careless readers to have been endorsed the Third Circuit.


1 Among the cases cited without any support for the PSC’s dubious contention were Gannon v. United States, 292 F. App’x 170, 173 n.1 (3d Cir. 2008); Bitler v. A.O. Smith Corp., 391 F.3d 1114, 1124-25 (10th Cir. 2004); In re Joint E. & S. Dist. Asbestos Litig., 52 F.3d 1124, 1128 (2d Cir. 1995); In re Avandia Mktg., Sales Practices & Prods. Liab. Litig., No. 2007-MD-1871, 2011 WL 13576, at *3 (E.D. Pa. Jan. 4, 2011) (“Bradford-Hill criteria are used to assess whether an established association between two variables actually reflects a causal relationship.”).

2 Anick Bérard, Sertraline Use During Pregnancy and the Risk of Major Malformations, 212 Am. J. Obstet. Gynecol. 795 (2015).