Schachtman Law

TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Lawyer and Economist Expert Witnesses Fail the t-Test

July 7th, 2016

Chad L. Staller is a lawyer and James Markham is an economist. The two testify frequently in litigation. They are principals in a litigation-mill known as the Center for Forensic Economic Studies (CFES), which has been a provider of damages opinions-for-hire for decades.

According to its website, the CFES is:

“a leading provider of expert economic analysis and testimony. Our economists and statisticians consult on matters arising in litigation, with a focus on the analysis of economic loss and expert witness testimony on damages.

We assist with discovery, uncover key data, critique opposing claims and produce clear, credible reports and expert testimony. Attorneys and their clients have relied on our expertise in thousands of cases in jurisdictions across the country.”

Modesty was never CFES’s strong suit. CFES was founded by Chad Staller’s father, the late Jerome M. Staller, who infused the run-away inflation of the early 1980s into his reports for plaintiffs in personal injury actions. When this propensity for inflation brought in a large volume of litigation consulting, Staller brought on Brian P. Sullivan. The CFES website notes that Sullivan’s “courtroom demeanor was a model of modesty and good humor, yet he was known to be merciless when cross examined by an opposing attorney.” My personal recollection is that Sullivan sweated profusely on cross-examination. In one case, in which I cross-examined him, Sullivan had added several figures incorrectly to the plaintiff’s detriment. My cross-examination irked the trial judge (Judge Dowling, who was easily irked) to the point that he interrupted me to ask why I was wasting time to point out an error that favored the defense. The question allowed me to give a short summation about how I thought the jury might want to know that the witness, Sullivan, had such difficulty in adding uncomplicated numbers.

In Butt v. v. United Brotherhood of Carpenters & Joiners of America, 2016 WL 3365772 (E.D. Pa. June 16, 2016) [cited as Butt], plaintiffs, women union members sued for alleged disparate treatment, which treatment supposedly caused them to have lower incomes than male union members. To support their claims, the women produced reports prepared by CFES’s Chad Staller and James Markham. Counsel for the union challenged the admissibility of the proffered opinions under Rule 702. The magistrate judge sustained the Rule 702 challenges, in an opinion that questioned the reliability and ability of the challenged putative expert witnesses.[1]

Staller and Markham apparently had proffered a “t-test,” which, in their opinion, showed a statistically significant disparity in male and female hours worked, “not attributable to chance.” Butt at *1. Staller and Markham failed, however, to explain or justify their use of the t-test. The sample size in their analysis included 17 women and 388 men on average across ten years. The magistrate judge noted serious reservations over the CFES analysis’s failure to specify how many men or women were employed in any given year. Plaintiffs’ counsel improvidently attempted to support the CFES analysis by adverting to the Reference Manual on Scientific Evidence (3d ed. 2011), which properly notes that the t-test is designed for small samples, but also issues the caveat that “[a] t-test is not appropriate for small samples drawn from a population that is not normal.” Butt at *1 n.2. The CFES reports, submitted without statistical analysis output, apparently did not attempt to justify the assumption of normality; nor did they proffer a non-parametric analysis.

Putting aside the plaintiffs’ expert witnesses’ failure to explain and justify its use of the t-test, the magistrate judge took issue with the assumption that a comparison of average salaries between the genders was an appropriate analysis in the first place. Butt at *2.

First, the CFES reports assigned damages beyond the years used in their data analysis, which ended in 2012. This extrapolation was especially speculative unwarranted given that union carpenter working hours were trending downward after 2009. Butt at *3. Second, and even more seriously, the magistrate judge saw that no useful comparison could be made between male and female salaries without taking into account several important additional variables such as their individual skills, the extent that individual carpenters solicited employment, or used referral systems, or accepted out-of-town employment. Butt at *3.[2] Without an appropriate multivariate analysis, the CFES reports could not conclude that the discrepancy in hours worked was caused by, rather than merely correlated with, gender. Butt at *4.[3]

[1] See Calhoun v. Yamaha Motor Corp., U.S.A., 350 F.3d 316, 322 (3d Cir. 2003) (affirming exclusion of “speculative and unreliable” expert evidence).

[2] citing Stair v. Lehigh Valley Carpenters Local Union No. 600 of United Brotherhood of Carpenters and Joiners of America, No. Civ. A. 91-1507, 1993 WL 235491, at *7, *18 (E.D. Pa. July 24, 1993) (Huyett, J.), aff’d, 43 F.3d 1463 (3d Cir. 1994) (“Many variables determine the number of hours worked by a carpenter: whether the carpenter solicits employment, whether he or she uses the referral system, whether an employer asks for that carpenter by name, whether the carpenter will accept out of town employment, and whether the carpenter has the skills requested by an employer when that employer calls the Union for a referral.”

[3] Interesting cases cited by the magistrate judge in support included Molthan v. Temple University, 778 F.2d 955, 963 (3d Cir. 1985) (“Because the considerations affecting promotion decisions may differ greatly from one department to another, statistical evidence of a general underrepresentation of women in the position of full professor adds little to a disparate treatment claim.”); Riding v. Kaufmann’s Dep’t Store, 220 F.Supp. 2d 442, 459 (W.D. Pa. 2002) (“Plaintiff’s statistical evidence is mildly interesting, but she does not put the data in context (how old were the women?) [or] tell us what to do with it or what inferences should be gathered from it…”); Brown v. Cost Co., No. Civ. A. 03-224 ERIE, 2006 WL 544296, at *3 (W.D. Pa. Mar. 3, 2006) (excluding statistical evidence proffered in support of claims of disparate treatment).

Posted in Rule 702, statistical evidence | Comments Off on Lawyer and Economist Expert Witnesses Fail the t-Test

National Academies’ Teaching Modules on Scientific Policy Issues

June 30th, 2016

Today, the National Academies of Sciences, Engineering, and Medicine announced its release of nine teaching modules to help public policy decision makers and students in professional schools understand the role of science in policy decision making.[1] The modules were developed by university faculty members for the use of other faculty who want to help their students appreciate the complexity and nuances of the evidence for and against scientific claims.

A group within the Academies’ Committee on Science, Technology and the Law supervised the development of the teaching modules, which are now publicly available at the Academies’ website. The Committee was chaired by Paul Brest, former dean and professor emeritus (active), Stanford Law School, and Saul Perlmutter, Franklin W. and Karen Weber Dabby Chair, University of California, Berkeley, and senior scientist, E.O. Lawrence Berkeley National Laboratory. The Gordon and Betty Moore Foundation and the National Biomedical Research Foundation sponsored the development of the modules.

The modules use case studies to illustrate basic scientific and statistical principles involved in contemporary scientific issues that have significant policy implications. The modules are designed to help future policy and decision makers understand and evaluate the scientific evidence that they will doubtlessly encounter. To date, nine modules have been developed and released, in the hope that they will serve as references and examples for future teaching modules.

The nine modules prepared to date are:

Models: Scientific Practice in Context

prepared by:
– Elizabeth Fisher, Professor of Environmental Law, Faculty of Law and Corpus Christi College, Oxford University
– Pasky Pascual, Environmental Protection Agency
– Wendy Wagner, Joe A. Worsham Centennial Professor, University of Texas at Austin School of Law

The Interpretation of DNA Evidence: A Case Study in Probabilities

prepared by:

– David H. Kaye, Associate Dean for Research and Distinguished Professor, The Pennsylvania State University (Penn State Law)

Translating Science into Policy: The Role of Decision Science

prepared by:

– Paul Brest, Former Dean and Professor Emeritus (active), Stanford Law School

Placing a Bet: A New Therapy for Parkinson’s Disease

prepared by:

– Kevin W. Sharer, Senior Lecturer, Harvard Business School, Harvard University

Shale Gas Development

prepared by:

– John D. Graham, Dean, School of Public and Environmental Affairs, Indiana University
– John A. Rupp, Adjunct Instructor, School of Public and Environmental Affairs, and Senior Research Scientist, Indiana Geological Survey, Indiana University
– Adam V. Maltese, Associate Professor of Science Education, School of Education, and Adjunct Faculty in Department of Geological Sciences, Indiana University

Drug-Induced Birth Defects: Exploring the Intersection of Regulation, Medicine, Science, and Law

prepared by:

– Nathan A. Schachtman, Lecturer in Law, Columbia Law School

Vaccines

prepared by:

– Arturo Casadevall, Professor and Chair, W. Harry Feinstone Department of Molecular Microbiology and Immunology, Johns Hopkins University Bloomberg School of Public Health

Forensic Pattern Recognition Evidence

prepared by:

– Simon A. Cole, Professor, Department of Criminology, Law, and Society, Director, Newkirk Center for Science and Society, University of California, Irvine
– Alyse Berthental, Ph.D. Candidate, Department of Criminology, Law, and Society, University of California, Irvine
– Jaclyn Seelagy, Scholar, PULSE (Program on Understanding Law, Science, and Evidence), University of California, Los Angeles School of Law

Scientific Evidence of Factual Causation

prepared by:

– Steve C. Gold, Professor of Law, Rutgers School of Law-Newark
– Michael D. Green, Williams Professor of Law, Wake Forest University School of Law
– Joseph Sanders, A.A. White Professor of Law, University of Houston Law Center

[1] See “Academies Release Educational Modules to Help Future Policymakers and Other Professional-School Students Understand the Role of Science in Decision Making” (June 30, 2016).

Posted in Causation, Scientific Evidence, statistical evidence | Comments Off on National Academies’ Teaching Modules on Scientific Policy Issues

Whether to Conduct Depositions of Expert Witnesses

June 23rd, 2016

In a Litigation magazine article, Gregory Joseph sets out some strong reasons to consider for not conducting depositions of expert witnesses under the revised 2010 Federal Rules of Civil Procedure (FRCP). See Gregory P. Joseph, “The Temptation to Depose Every Expert,” 40 Litigation 35 (Winter 2014) [cited below as Joseph]. Joseph points out that FRCP 26(a)(2)(B) requires parties to disclose, for all retained expert witnesses, “all opinions” and the “full factual basis” of all their opinions, among other things. The rule is exacting. All opinions includes “a complete statement of all opinions the witness will express and the basis and reasons for them.” FRCP 26(a)(2)(B)(i). And a full factual basis includes “the facts or data considered by the witness in forming” all of the opinions disclosed in the report. FRCP 26(a)(2)(B)(ii) (emphasis added).

Joseph argues that the breadth of the required disclosure, combined with sanctions for retained expert witnesses’s attempting to testify beyond the four corners of their reports, should give lawyers sufficient assurances in many instances to forego conducting depositions of expert witnesses.

Joseph notes that the FRCP creates a presumptive mandatory sanction of exclusion for undisclosed expert testimony. FRCP 37(c)(1).[1] Joseph offers other arguments beyond the supposed comfort given by the “four corners” rule set out in the FRCP. Joseph at 36-37. First, the deposition may “reopen” discovery by giving expert witnesses opportunities to expand upon the four corners of their reports. Although some courts will limit what expert witnesses can throw over the transom at depositions, a supervising magistrate or district judge may not regard the expansion upon the disclosures in the report as “sandbagging,” and thus fail exclude the arguably new opinions or bases. Joseph cites a few cases in which courts condemned the sandbagging of counsel by the offering of new opinions in depositions, but points out that exclusion is this circumstance is highly discretionary. The court is not required to exclude, and it may permit the new material, or allow the new material with an inadequate amount of additional time in deposition. So taking the deposition has risks.

Joseph argues also that depositions may educate expert witnesses about intended trial cross-examination, and help adversary counsel better prepare direct examination and anticipatory rebuttal. Furthermore, the new protections afforded expert witnesses from discovery into drafts of reports and most communications with retaining counsel take away one of the previous reasons to conduct depositions.

To be sure, some additional areas of discovery may be covered by interrogatories, Rule 34 document requests, or Rule 45 subpoenas directly to the expert witnesses. These non-deposition methods of discovery, however, will not reach valuable topics of discovery such as oral communications between retained expert witnesses and professional colleagues, consulting expert witnesses, the retaining lawyers’ clients, and other persons. The suggested alternative discovery methods also suffer in that they will provoke canned answers, written by counsel, and not the ingenuous, unrehearsed responses of expert witnesses required to give answers directly and without resort to “privileged” consultation with retaining counsel.

The revised FRCP carve out important areas of inquiry from the new protections against discovery into draft reports and with counsel. Counsel still are permitted to inquire into compensation, the retaining attorneys’ provision of “facts or data” considered by the witnesses, and retaining attorneys’ identification of assumptions “relied” upon by the witnesses. Invoices can, of course, be subpoenaed, but often oral examination is required to discover whether the invoices have been paid, whether they are contingent, or whether payment flows to the personal benefit of the expert witnesses. Inquiring into what “facts or data” were provided by retaining counsel can be attempted by written discovery, but the written responses will likely be hedged and unclear, and the responses will not distinguish which lawyer-provided “facts or data” were actually relied upon.

The FRCP clearly allow discovery into retaining attorneys’ provision of assumptions relied upon by expert witnesses, but clear, unrehearsed answers to questions about what was assumed and relied upon, as opposed to merely considered, are not likely to be forthcoming in written discovery. Furthermore, if there will be any fair opportunity to explore the significance of relying upon counsel’s assumptions, only a deposition will likely allow for the extemporaneous, first-person expression of expert witnesses’ opinions. Questions into expert witnesses’ opinions based upon hypothetical questions that contradict the assumptions given, or into opinions about the level of confidence or knowledge witnesses have about the correctness of the assumptions, are likely to be effective only in face-to-face encounters.

There are important additional reasons for taking expert witness depositions, not addressed in Joseph’s article. Litigation-savvy expert witnesses will often glibly assert that they have “considered” all the relevant studies, data, and facts. If written discovery is propounded to inquire whether a study omitted from the “consideration” list in the FRCP report was not considered, the study, if meaningful, will be added to the list in the written response with a feeble excuse that it was inadvertently omitted from the list. And the omission will likely be judged harmless because the party seeking discovery obviously knew about the omitted study already. Written discovery into what studies, data, or facts were considered but not relied upon will also yield highly rehearsed answers, and interrogatories will not permit inquiries into the fine details of key studies.

The pertinent sections of the FRCP do not require expert witness reports to distinguish what the witnesses have considered from what they have actually relied upon. Written discovery could be propounded, but again, it will not likely yield clear answers such as might be had with follow up inquiry into what was considered but not relied upon, and why was reliance rejected. The deposition upon oral examination has the benefit of permitting follow up questions into why some studies were relied upon for some parts but not others, or were considered but completely excluded from actual reliance. The opportunity to field incoherent, inconsistent rationales for inclusions and exclusions that establish expert witness cherry picking will be lost without the face-to-face encounter allowed by oral examination.

With some courts engaged in retrograde refusal to apply Rule 702 as enacted, some expert witnesses have been encouraged to employ vague, invalid, and unreliable methodologies, such as the so-called “weight of the evidence” approach. Oral examination will be necessary to establish expert witnesses’ weighting considerations, their inclusion and exclusion criteria, and to test their consistency in applying these considerations and criteria, across the entire evidentiary base for conclusions.

Concessions to Be Obtained

Written discovery is not well suited to inquire into general principles of interpreting data and studies, data integrity and validity, and validity of inference. Interrogatories are too difficult to draft in sufficient detail to permit setting up an examination that will lead to the disqualification of the expert witness under Rule 702. Obtaining concise, clear concessions about basic methodological principles is crucial to structuring persuasive cross-examinations. Of course, if the deponent balks at accepting generally accepted principles, then this testimony is filed under Rule 702 motion, rather than trial cross-examination.

Furthermore, written discovery is poorly suited to identify whether expert witnesses have subject-matter weaknesses. Interrogatories are the wrong discovery tool to conduct pop-quizzes on arcane statistical and scientific methodologies. Lawyers rightfully do not want to get into show-game style quizzes to test expert witnesses’ understanding of the esoteric, but important, methodologies used in the studies relied upon, in front of a jury. Rule 26 reports rarely announce that witnesses have had no meaningful training in statistics and that they have no idea what assumptions were made in various statistical analyses or tests in the studies that they have embraced and relied upon for their opinions.

Expert witnesses have social and professional connections not always apparent from their curriculum vitae, their Rule 26 reports, or their websites. Expert witnesses are not likely, for instance, to disclose that they are Marxists, who believe that corporations are evil and mercenary, and cannot be trusted to tell the truth in litigation.[2]

As noted, the FRCP requires disclosure of facts or data considered, which disclosure is usually inadequate to permit distinguishing what was actually relied upon in forming opinions. But what about opinions considered or relied upon? FRCP does not address reliance upon opinions; nor does Rule 703. Expert witnesses may contend that their opinions are not “based upon” others’ opinions, but that their opinions are strengthened and corroborated by the opinions of others. The FRCP do not specifically call for disclosure of opinions relied upon by retained expert witnesses, and adversary counsel can be trusted to argue that there were no obligations to disclose opinions or the identity of “authoritative” treatises and publications. If there is no entitlement to disclosure, there can be no surprise and prejudice.

Interpreting the scope of the report may not be as clear as Joseph suggests. Rule 26 reports usually contain some opinions with sufficient breadth and generality that foregoing depositions becomes a game of Russian roulette. Trial judges may not look kindly upon “scope of the report” objections, made at trial, when the objecting counsel had the opportunity to conduct an examination, and the report language is sufficiently broad to intimate the witness’s opinion at trial. Judges seem to have great hindsight vision, and they may well distrust counsel’s objections as a different sort of sandbagging. An entire strategy of restraint may be sunk by a quick, discretionary ruling on “scope of the report,” which often will favor the proponent of the witness.

Joseph is correct that many depositions fail to accomplish much, but such failures are not the result of how wonderful the revised FRCP are. Failed depositions are more likely to result from the lack of preparation, creativity and knowledge of counsel in carrying out coherent, effective depositions.

[1] See Primus v. United States, 389 F.3d 231, 234 (1^st Cir. 2004); Vaughn v. City of Lebanon, 18 F.App’x 252, 263 (6^th Cir. 2001); Musser v. Gentiva Health Services, 356 F.3d 751, 758 (7^th Cir. 2004). See also Design Strategy, Inc. v. Davis, 469 F.3d 284, 296 (2d Cir. 2006) (characterizing exclusion as discretionary, but upholding district court’s exclusion).

[2] Such as may be seen with expert witnesses who belong to the Committees of Correspondence for Democracy and Socialism, a branch of the Communist Party USA, formed in 1992, after the demise of the Soviet Union.

Posted in Expert Witness Discovery, Expert Witnesses | Comments Off on Whether to Conduct Depositions of Expert Witnesses

The IARC Announces Water Causes Cancer

June 18th, 2016

Well, drinking water very hot, or other scalding beverages, probably does cause cancer. Earlier this week, the International Agency for Research on Cancer (IARC) issued a press release that one of its working groups had reviewed the data on the carcinogencity of coffee, maté, and very hot beverages, and concluded that maté, which is often served very hot, “probably” causes esophageal cancer. IARC Press Release N° 244, “IARC Monographs evaluate drinking coffee, maté, and very hot beverages” (June 15, 2016). Very hot beverages were rated 2A, for their probably causing human esophageal cancer.

The good news is that “probably” does not mean “more likely than not” in IARC-speak, and the working group was evaluating hazard not risk.[1] IARC classifications do not attempt to quantify the magnitude of risk that may result from exposure to a classified “hazard.” Id. at Note to the Editor. Because all empirical propositions have a probability of being true, somewhere between 0 and 100%, (with P ≠ 0; P ≠ 100%), the IARC classifications of “probably” causing cancer are probably not particularly meaningful. Everything “probably” causes cancer in this sense. See Ed Yong, “Beefing With the World Health Organization’s Cancer Warnings,” The Atlantic (Oct 26, 2015).

The IARC group’s evaluation of “very hot drinks” accords with the World Health Organization’s Technical Report Series 916 on Diet, Nutrition and the Prevention of Chronic Diseases, which recommends against consumption of scalding hot temperatures. See Anahad O’Connor, “Coffee May Protect Against Cancer, W.H.O. Concludes,” N.Y. Times (June 15, 2016)[O’Connor]. As though people, other than McDonald’s coffee drinkers, needed such a recommendation. The IARC group found no conclusive evidence to implicate drinking cold maté, or maté at temperatures below scalding levels.

An IARC Decision We Can Like a Latte

The Working Group found no conclusive evidence for a carcinogenic effect of drinking coffee, and placed coffee in its category 3, “not classifiable” with respect to carcinogenicity.[2] The working group’s evaluation included over 1,000 observational and experimental studies, including randomized trials, and found no evidence to support the claims that coffee causes human cancer. The IARC also found a good deal of evidence supporting the claim that drinking coffee reduces the risk of various human cancers.

There is a Group 4, for exposures probably not carcinogenic in humans, but in its 45 years of evaluations, the IARC has found only one substance on Planet Earth, which does not cause cancer: caprolactam. Perhaps after another 1,000 studies, coffee will reach this exalted category. For now, coffee is unclassifiable with “inadequate” evidence of human carcinogenicity in the IARC’s view.

The New York Times, not particularly expertly, and without supporting citations, declared that the evidence for coffee’s health benefits could not establish actual causation of benefit because the data came from epidemiologic studies. See O’Conner. This would not be the first time that the New York Times made up things.

In 1991, the IARC evaluated coffee drinking as a “possible” human carcinogen (Group 2B), based upon limited evidence of an association with urinary bladder cancer in case-control studies, and some evidence in experimental animals.[3] This year’s evaluation of coffee as Group 3 thus represents a rare reversal of opinion, in the face of additional evidence, from the IARC.

[1] The IARC Preamble definition of probable reveals that “probable” does not mean greater than 50%. See also “The IARC Process is Broken” (May 4, 2016).

[2] See Dana Loomis, Kathryn Guyton, Yann Grosse, Béatrice Lauby-Secretan, Fatiha El Ghissassi, Véronique Bouvard, Lamia Benbrahim-Tallaa, Neela Guha, Heidi Mattock, Kurt Straifon behalf of the IARC Monograph Working Group, “Carcinogenicity of drinking coffee, mate, and very hot beverages,” Lancet Oncology (2016 in press).

[3] IARC, “Coffee, tea, mate, methylxanthines and methylglyoxal,” 51 IARC Monogr Eval Carcinog Risks Humans 1 (1991).

Posted in Causation | Comments Off on The IARC Announces Water Causes Cancer

The Webb of Unsophistication in Products Liability Law

May 29th, 2016

The Heart of the Matter

The classic early cases in products liability law were about consumers hurt by consumer products, sold by manufacturers or dealers directly to consumers. The key component of these cases was inequality of bargaining power, of knowledge about latent defects or hazards, and of control over the discovery of latent hazards or defects. American products liability law was created around consumer products. Just think of Henningsen, Escola, and MacPherson.[1] These were all consumer products for which the rhetoric about inequality of bargaining, knowledge, and control over design, manufacturing, and latent hazards sometimes makes sense. The paradigmatic model for products liability, however, frequently does not work for the three-way relationship of sales of products to large industrial employers. The model especially does not work when the product is a raw material used throughout a factory, or incorporated into another product.

Many courts have failed to come to grips with the inadequacy of the consumer model for products liability cases in instances of occupational harm to industrial employees. Courts have been trying to ram this square peg into a round hole since the early asbestos litigation (which perhaps made some sense because there was inequality between Johns Manville and most vendees), but makes no sense when John Manville is itself the purchaser.

The Tangled Webb in California Law

The Webb case received some attention after the California Court of Appeals reversed a trial court’s entry of JNOV for defendant Special Electric on the so-called sophisticated intermediary defense. See “California Supreme Court Set To Untangle Webb” (July 7, 2013); “Big Blue & The Sophisticated User and Intermediary Defenses” (Sept. 27, 2014); G. Jeff Coons, “What a Tangled Webb We Weave: Court Imposes Failure to Warn Liability On Supplier to Johns-Manville” (April 2013). Special Electric petition for review, and eventually the California Supreme Court called for full briefing and oral argument in the Webb case.

The wheels of justice grind slowly in California. Special Electric filed its opening brief on the merits, on September 10, 2013. Webb’s widow answered in December 2013, and Special Electric replied in February 2014. Several amici curiae joined the fray in April 2014. Mark A. Behrens filed a brief on behalf of the Coalition for Litigation Justice, Inc., Chamber of Commerce, NFIB Small Business Legal Center, and American Chemistry Council. The Pacific Legal Foundation also filed, as did Elementis Chemicals Inc.

After mulling over the briefs for two years, the California Supreme Court heard argument on March 1, 2016, and then in surprisingly short order, affirmed the intermediate appellate, earlier this week. The Supreme Court’s ruling upheld a Court of Appeal’s decision that reversed a judgment for defendant Special Electric, based upon a jury verdict in favor of William Webb, who was exposed to crocidolite sold by Special Electric, and which caused him to develop mesothelioma in 2011. The Supreme Court’s opinion [2] held that sophisticated intermediary doctrine was a complete legal defense, even potentially for an asbestos supplier, but declined to apply it to the benefit of Special Electric, which had misrepresented facts about crocidolite and offered no evidence that its purchaser was sophisticated about crocidolite asbestos and its unique relationship with mesothelioma. [Slip opinion cited here as Webb.] Webb v. Special Elec. Co., Inc., 2016 BL 163642, Cal., No. S209927, 5/23/16).

The majority opinion[3] fortunately was able to separate the poorly framed and supported defense by Special Electric from the basic tenets of tort law and the sophisticated intermediary defense. To the extent that anyone doubted the validity of the sophisticated intermediary defense, the Webb Court formally adopted the doctrine as the law of California, as set out in the Second and Third Restatements of Tort Law. Webb at 15-16. According to the Court, a defendant may set up sophisticated intermediary doctrine as a complete defense, to failure to warn claims for known or knowable product risks, sounding in negligence or in strict liability, when the defendant supplier:

“(1) provides adequate warnings to the product’s immediate purchaser, or sells to a sophisticated purchaser that it knows is aware or should be aware of the specific danger, and

(2) reasonably relies on the purchaser to convey appropriate warnings to downstream users who will encounter the product.”

Webb at 16 (emphasis in original).[4]

As an affirmative defense, the defendant supplier must carry its burden of showing that it adequately warned the intermediary, or that it knew the intermediary knew or should have known of the specific hazard, and that it reasonably relied upon the purchaser to transmit warnings. Id.

On appeal, the California Supreme Court held that defendant Special Electric failed to preserve its entitlement to the sophisticated intermediary defense because “it never attempted to show that it actually or reasonably relied on Johns-Manville to warn end users. Nor did Special Electric request a jury instruction or verdict form question on the sophisticated intermediary doctrine.” Webb at 23.

Alternatively, on the assumption that Special Electric preserved the defense, the Court held that this defendant failed to establish the defense as a matter of law because:

“[a]lthough the record clearly shows Johns-Manville was aware of the risks of asbestos in general, no evidence established it knew about the particularly acute risks posed by the crocidolite asbestos Special Electric supplied. In addition, plaintiffs presented evidence that at least one Special Electric salesperson told customers crocidolite was safer than other types of asbestos fiber, when the opposite was true.”

Webb at 23.

The Webb Court reviewed the Tort Restatements’ embrace of the sophisticated intermediary defense in both the Second and Third editions. The Webb Court noted that the Third Restatement demonstrated the continued validity and vitality of the defense, as had been expressed in the Section 388 of the Restatement Second of Torts.[5] The Court noted and followed the Third Restatement’s recitation of guiding considerations for invoking and sustaining the defense:

“There is no general rule as to whether one supplying a product for the use of others through an intermediary has a duty to warn the ultimate product user directly or may rely on the intermediary to relay warnings. The standard is one of reasonableness in the circumstances. Among the factors to be considered are the gravity of the risks posed by the product, the likelihood that the intermediary will convey the information to the ultimate user, and the feasibility and effectiveness of giving a warning directly to the user.”

Webb at 15 (citing Restatement 3d Torts, Products Liability, § 2, com. i, at p. 30.) Citing California precedent, the Webb Court noted that

“[t]he focus of the [sophisticated intermediary] defense . . . is whether the danger in question was so generally known within the trade or profession that a manufacturer should not have been expected to provide a warning specific to the group to which plaintiff belonged.”

Webb at 9-10 (quoting from Johnson v. American Standard, Inc. 43 Cal.4th 56, 72 (2008). The pertinent legal test is whether a reasonable supplier would have known of the intermediary’s sophistication with respect to the relevant risk. Webb at 20.[6] Of course, the existence of a pervasive regulatory control of risk creation, detection, and mitigation in the workplace would count heavily in this objective test. “Every person has a right to presume that every other perform his duty and obey the law.” Webb at 21 (internal citation omitted) (emphasis added).

The Restatement factors, however, did not support Special Electric’s invocation of the defense in a case involving:

(1) crocidolite asbestos, one of the most hazardous substances known,

(2) defendant’s affirmative and blatantly false misrepresentations of the relative safety of crocidolite relative to chrysotile asbestos,[7] and

(3) a complete failure of proof that the purchaser, Johns Manville, knew that crocidolite was especially hazardous with respect to the causation of mesothelioma.

Webb at 23-24. Factors one and two were givens for defense counsel, but factor three speaks to unnecessary coyness on the part of the defense. Showing that Johns Manville was well aware of the extraordinarily great hazard of crocidolite would have been relatively easy to do from past transcripts, articles, speeches, and litigation conduct of the Johns Manville companies. Despite the extreme hazards from uncontrolled asbestos exposures, the Webb case explained that the sophisticated intermediary defense was not per se inapplicable to asbestos cases, and went so far as to disapprove an earlier California Court of Appeals decision that refused to apply the defense in the asbestos personal injury context when no warnings had been given.[8] “Sophistication obviates the need for warnings because a sophisticated purchaser already knows or should know of the relevant risks.” Webb at 17-18.

The Webb case acknowledged that defective design claims against raw material suppliers are incoherent and invalid, whether for the raw material itself, or for downstream design defect claims against for the product with the incorporated raw material. “[A] basic raw material such as sand, gravel, or kerosene cannot be defectively designed.” Webb at 11-12 (quoting from Restatement 3d Torts, Products Liability, § 5, com. c, at p. 134).[9]

The Webb Court also evinced a healthy disrespect for the notion that tort law is only about spreading risk and compensating injured persons. The Court acknowledged that in some instances, there were competing policies of compensating persons injured by products and “encouraging conduct that can feasibly be performed.” Webb at 2. The Court also acknowledged that there were hazards to warning when none was needed or when the absence of a warning would not be a legal cause of harm:

“Because sophisticated users already know, or should know, about the product’s dangers, the manufacturer’s failure to warn is not the legal cause of any harm. A sophisticated user’s knowledge is thus the equivalent of prior notice. The defense serves public policy, because requiring warnings of obvious or generally known product dangers could invite consumer disregard and contempt for warnings in general.”

Webb at 9 (internal citations omitted) (emphasis added). Furthermore, the sophisticated intermediary defense balances the need for the worker-consumer’s safety with “the practical realities of supplying products.” Webb at 17.

The Webb decision puts California in line with the majority rule that recognizes the validity of the sophisticated intermediary defense, and embraces real-world truth that:

“[in] some cases, the buyer’s sophistication can be a substitute for actual warnings, but this limited exception only applies if the buyer was so knowledgeable about the material supplied that it knew or should have known about the particular danger.”

Webb at 17.[10] The Court noted and agreed with the Restatement Third’s observation that imposing liability upon raw material suppliers for failure to warn can be unduly and unfairly burdensome when such liability would require remote suppliers

“to develop expertise regarding a multitude of different end-products and to investigate the actual use of raw materials by manufacturers over whom the supplier has no control.”

Webb at 12 (quoting from Restatement 3d Torts, Products Liability, § 5, com. c, at p. 134).

Concurrence

Chief Justice Tani Gorre Cantil-Sakauye, along with Justice Ming W. Chin, concurred in the result, but dissented from the majority’s rationale as overly broad. The concurring justices insisted that a supplier reasonably relies upon its purchaser only when the purchaser has actual awareness of the product’s risks. Webb concurrence at 4. Even this stingier approach noted that one of the purpose of warnings is

“to enable the consumer or others who might come in contact with the product to choose not to expose themselves to the risks presented.”

Webb Concurrence at 3 (citing Restatement3d Torts, Products Liability, § 2, com. i, at p. 30). In many sophisticated intermediary contexts involving occupational exposures to fumes, vapors, and dusts, workers (consumers) cannot appreciate whether they might come in contact with the product such that they have actual risks unless the sophisticated intermediary measures its specific workplace exposures, given its actual engineering, administrative, and person protection controls.

Commentary

The Webb Court failed to address in any meaningful form how Special Electric could discharge a duty to warn Mr. Webb directly, when it sold blue asbestos to Johns-Manville, which then incorporated that fiber, along with other recycled asbestos into transite pipes. To this extent, the Webb decision carries forward the glib belief in efficacy of warnings, without any evidence or critical thought.

It is hard to imagine an industrial purchaser that was unaware of the special hazards of crocidolite by 1970, and yet Special Electric apparently failed to offer evidence on the issue whether Johns-Manville had such awareness. A court might take judicial notice of Johns-Manville sophistication, but there is not even the suggestion that Special Electric attempted to supplement the vacuous record with a request for judicial notice.

If the California Supreme Court’s recitation of the facts of the case is correct, then we are left with an unflattering inference about Special Electric’s trial strategy and execution. Perhaps Special Electric was coyly trying to avoid a downside outcome in which it was responsible for 99.99% of the verdict because its blue asbestos was by far the most important cause of Mr. Webb’s tragic disease, a disease that would have almost certainly been avoided had never had exposure to blue asbestos. The propensity of crocidolite to cause mesothelioma is orders of magnitude greater than chrysotile, which by itself may not even be a competent cause of the harm suffered by Mr. Webb.

In the final analysis, the Webb Court correctly adopted the sophisticated intermediary principle as an essential limit to tort liability, but denied its benefit to Special Electric. The sophisticated intermediary doctrine should not, however, be conceived of as an affirmative defense. The scope of the rule is defined by the rationale for its existence, and the sophisticated intermediary situation lies outside the realm and rationale of protecting, by warning, consumers against latent hazards. It is time that courts recognize that much litigation brought to its doors is really the result of labor-management issues within the workplace, and not the doings or responsibility of remote suppliers of raw materials.

[1] See, e.g, MacPherson v. Buick Motor Co., 217 N.Y. 382, 111 N.E. 1050 (1916) (holding that privity of contract did not bar suit and that product manufacturers could be liable to consumers for injuries); Henningsen v. Bloomfield Motors, Inc., 32 N.J. 358, 161 A.2d 69 (1960); Escola v. Coca Cola Bottling Co., 24 Cal. 2d 453, 150 P. 2d 436 (1944).

[2] See Steven Sellers, “California Ruling Defines Asbestos Supplier’s Duty to Warn,” BNA Product Safety & Liability Reporter (May 24, 2016).

[3] The majority opinion was written by Associate Justice Carol A. Corrigan, and joined by Associate Justices Kathryn Mickle Werdegar, Goodwin Liu, Mariano-Florentino Cuéllar and Leondra R. Kruger.

[4] See also Webb at 2 (“Under the sophisticated intermediary doctrine, the supplier can discharge this duty if it conveys adequate warnings to the material’s purchaser, or sells to a sufficiently sophisticated purchaser, and reasonably relies on the purchaser to convey adequate warnings to others, including those who encounter the material in a finished product. Reasonable reliance depends on many circumstances, including the degree of risk posed by the material, the likelihood the purchaser will convey warnings, and the feasibility of directly warning end users.”); Webb at 6 (“[T]he sophisticated intermediary doctrine provides that a supplier can discharge its duty to warn if it provides adequate warnings, or sells to a sufficiently sophisticated buyer, and reasonably relies on the buyer to warn end users about the harm.”). Webb at 17 (“If a purchaser is so knowledgeable about a product that it should already be aware of the product’s particular dangers, the seller is not required to give actual warnings telling the buyer what it already knows.”).

[5] See Webb at 15 (“The drafters intended this comment to be substantively the same as section 388, comment n, of the Restatement Second of Torts.”) (citing Restatement 3d Torts, Products Liability, § 2, com. i, reporter’s note 5, at p. 96; Humble Sand & Gravel Inc. v. Gomez, 146 S.W.3d 170, 190 (Tex. 2004). See also Webb at 9 (citing Restatement 2d Torts, § 388 (b), com. k, at pp. 306-307) (“Courts have interpreted section 388, subdivision (b), to mean that if the manufacturer reasonably believes the user will know or should know about a given product’s risk”).

[6] Relevant considerations may include the general dissemination of knowledge of relevant risks, the intermediary’s knowledge of those risks, and the intermediary’s reputation for care. Webb at 20.

[7] Webb at 3, 23.

[8] See Webb at 17-18 (disapproving of the holding in Stewart v. Union Carbide Corp., 190 Cal. App. 4th 23, 29-30 (2010)).

[9] See also Webb at 12 (quoting from Restatement 3d Torts, Products Liability, § 5, com. c, at p. 134) (“Inappropriate decisions regarding the use of such materials are not attributable to the supplier of the raw materials but rather to the fabricator that puts them to improper use.”).

[10] citing approvingly Cimino v. Raymark Industries, Inc., 151 F.3d 297, 334 (5th Cir. 1998) (holding that raw asbestos supplier did not need to warn asbestos product manufacturer Fibreboard, which was “a sophisticated, expert, and knowledgeable manufacturer” of insulation products, about asbestos risks); Higgins v. E.I. DuPont de Nemours & Co., 671 F. Supp. 1055, 1061-1062 (D. Md. 1987) (exculpating supplier when purchaser was a highly sophisticated manufacturer with knowledge from independent sources, as well as its suppliers), aff’d, 863 F.2d 1162 (4th Cir. 1988).

Posted in Asbestos, Silica, Sophisticated User | Comments Off on The Webb of Unsophistication in Products Liability Law

Credible Incredulity

May 19th, 2016

Has skepticism become a victim of political correctness and adversarial zeal?

In the last century, philosopher Bertrand Russell advanced intelligent skepticism against myriad enthusiams and mindless beliefs, political, religious, and pseudo-scientific. Russell saw unwarranted certainty as a serious intellectual offense:

“The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.”

Bertrand Russell, “The Triumph of Stupidity” (1933), Mortals and Others: Bertrand Russell’s American Essays, 1931-1935 , at 28 (1998). When many American intellectuals were still in their love swoon over Stalin, Russell chastised the Soviet dictator for his betrayal of ideals and his enslavement of Eastern European. Stalinism’s certainty about politics and science was not a virtue, but a grave sin. Or, in Russell’s words:

“One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision.”

Bertrand Russell, New Hopes for a Changing World at 4-5 (1951).

In the 21st century, ideologues of various stripes have tried to silence healthy skepticism and doubt by claiming that their critics have “manufactured doubt.”[1] This aggression against skepticism and doubt, joined with a biased conception of conflicts of interest, have become part of a concerted campaign to privilege tendentious scientific claims from critical scrutiny.

Philosopher Susan Haack, who has aligned herself on occasion with these politicized acolytes of certainty,[2] recently has pushed back, with a reminder that credulity for unwarranted claims, in all walks of life, is unethical.[3] Haack’s essay is a delightful effort to clarify what credulity is, and to explore why credulity is an epistemologic vice and a social hazard, as well as the implications for citizens and scientists of living in an evidence-based, not a faith-based world.

Drawing inspiration from the the English mathematician and philosopher, William Kingdon Clifford, Haack has adopted one of Clifford’s bon mots as her motto:

“The credulous man is father to the liar and the cheat.”[4]

Indeed! And credulous judges and juries are the parents to specious claims and shyster lawyers.

Clifford’s essay should be required reading for politicians, judges, regulators, and legislators who evaluate the claims of scientist advocates. Spurning ethical relativism, Clifford identified the key intellectual “sin” in an evidence-based world:

“It is wrong always, everywhere, and for anyone, to believe anything upon insufficient evidence.”

William K. Clifford, “The Ethics of Belief,” 29 Contemporary Rev. 289, 295 (1877).

Professor Haack should be commended for her fulsome irony for publishing in a journal of one of the world’s more credulous institutions, and for reminding us that credulity is an intellectual vice.

[1] See, e.g., David Michaels, Doubt is Their Product: How Industry’s Assault on Science Threatens Your Health (2008); Naomi Oreskes and Erik M. Conway, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming (2010).

[2] See, e.g., “Bendectin, Diclegis & The Philosophy of Science” (Oct. 26, 2013).

[3] Susan Haack, “Credulity and Circumspection: Epistemological Character and the Ethics of Belief,” 88 Proc. Am. Catholic Philosophical Ass’n 27 (2015).

[4] citing and quoting William K. Clifford, “The Ethics of Belief ” (1877), in Leslie Stephen and Sir Frederick Pollock, eds., The Ethics of Belief and Other Essays 70, 77 (London 1947).

Posted in Scientific Evidence, Scientific Misconduct | Comments Off on Credible Incredulity

Ancient Truths

May 5th, 2016

David Sackett, in some paternity disputes called the “father of evidence-based medicine,” supposedly once claimed that:

“Half of what you’ll learn in medical school will be shown to be either dead wrong or out of date within five years of your graduation; the trouble is that nobody can tell you which half–so the most important thing to learn is how to learn on your own.”

See Ivan Oransky, “So how often does medical consensus turn out to be wrong?” Retraction Watch (July 11, 2011). Sackett’s meta-statement was itself certainly not “evidence based,” but his point is well taken. Time ultimately erodes the authority of the truthiest sounding claims to medical knowledge. Sara Teichholtz, “The Differential: Half of What You’re Learning is Wrong,” (Dec. 14, 2013). Only lawyers and theologians would think that a statement in an old document or text, once authenticated, has some claim on us as the “truth.”

The Federal Rules of Evidence provide an exception to the rule against hearsay for statements made in ancient documents, those at least twenty years old. Rule 803(16). In 2015, the Judicial Conference’s Committee on Rules of Practice and Procedure proposed retiring the ancient document hearsay rule.[1] The exception created for documents authenticated as “ancient” (> 20 years old) is so inimical to the truth-finding function of trials, that courts strain to avoid finding the documents “authenticated.” See, e.g., Kalamazoo River Study Group v. Menasha Corp., 228 F.3d 648 (6th Cir. 2000).

The proposal to abolish this dangerous exception to the rule against hearsay has engendered resistance from some quarters over its ability to eliminate otherwise admissible evidence in cases involving long-past events, such as environmental or occupational disease litigation. The resistance, however, is misguided. The Committee’s proposal would not affect the authenticity presumption of an “ancient document,” and such documents could still be used to show state of mind, intention, motive, or notice. If the asserted statement in the old document is actually true, then there is likely much more recent, robust evidence to support the statement. The rule as it now stands is capable of a great deal of mischief. The fact that a document has survived intact in a place where one would expect to find it may add to its presumptive authenticity, but in many technical, scientific, and medical contexts, the “ancient” provenance actually makes the content likely to be false. Technical and scientific facts and opinions have changed too quickly to endorse statements simply because of they were written down somewhere, over 20 years ago. See “Time to Retire Ancient Documents As Hearsay Exception” (Aug. 23, 2015).

Although many in the legal academy have voiced opposition to the proposal[2], one law professor, Daniel Capra, has astutely observed that we will soon have a flood of easily authenticated documents of doubtful veracity, called websites, and other electronic documents, which have reached the age of evidentiary majority. Daniel J. Capra, “Electronically Stored Information and the Ancient Documents Exception to the Hearsay Rule: Fix It Before People Find Out About It,” 17 Yale J.L. & Tech 1 (2015). The truth of a proposition requires more than the lapse of 20 years since some nincompoop wrote it down.

[1] Preliminary Draft of Proposed Amendments to the Federal Rules of Bankruptcy Procedure and the Federal Rules of Evidence (Aug. 2015); See also Debra Cassens Weiss, “Federal judiciary considers dumping ‘ancient documents’ rule,” ABA Journal Online (Aug. 19, 2015).

[2] Peter Nicolas, “Saving an Old Friend From Extinction: A Proposal to Amend Rather Than to Abrogate the Ancient Documents Hearsay Exception,” 63 UCLA L. Rev. Disc. 172 (2015).

Posted in Evidence, State of the Art | Comments Off on Ancient Truths

The IARC Process is Broken

May 4th, 2016

Last spring, the International Agency for Research on Cancer (IARC) convened a working group that voted to classify the herbicide glyphosate as “probably carcinogenic to humans.” The vote was followed by IARC’s Press Release, a summary in The Lancet,[1] and the publication of a “monograph,” volume 112 in the IARC series.

IARC classifications of a chemical as “probably” carcinogenic to humans are actually fairly meaningless exercises in semantics, not science. A close reading of the IARC Preamble definition of probable reveals that probable does not mean greater than 50%:

“The terms probably carcinogenic and possibly carcinogenic have no quantitative significance and are used simply as descriptors of different levels of evidence of human carcinogenicity, with probably carcinogenic signifying a higher level of evidence than possibly carcinogenic.”

Despite the vacuity of the IARC’s “probability” determinations, IARC decisions have serious real-world consequences in the realm of regulation and litigation. Monsanto, the manufacturer of glyphosate herbicide, reacted strongly, expressing “outrage” and claiming that the IARC had cherry picked data to reach its conclusion. Jack Kaskey, “Monsanto ‘Outraged’ by Assessment That Roundup Probably Causes Cancer,” 43 Product Safety & Liability Reporter 416 (Mar. 30, 2015).

In the wake of the IARC classification, in the fall of 2015, the United States Environmental Protection Agency (EPA) reviewed the evidence for, and against, glysophate’s carcinogenicity. The EPA found that the IARC had deliberately failed to consider studies that did not find associations, and that the complete scientific record did not support a conclusion of human carcinogenicity. EPA Report of the Cancer Assessment Review Committee on Glyphosate (Oct. 1, 2015).

For undisclosed reasons, however, the EPA’s report was never made public until a couple of weeks ago, when it showed up briefly on the agency’s website, only to be pulled down after a day or so. See David Schultz, “EPA Panel Finds Glyphosate Not Likely to Cause Cancer,” Product Safety & Liability Reporter (May 03, 2016). No doubt the present Administration viewed a conflict between EPA and IARC, and disparaging comments about the IARC’s “process” to be national security issues. At the very least, the Administration would not want to undermine the litigation industry’s reliance upon the IARC cherry-picked report.

All joking aside, the incident highlights the problematic nature of the IARC decision process, and the reliance of regulatory agencies on the apparent authority of IARC determinations. The IARC process is toxic and should be remediated.

[1] Kathryn Z Guyton, Dana Loomis, Yann Grosse, Fatiha El Ghissassi, Lamia Benbrahim-Tallaa, Neela Guha, Chiara Scoccianti, Heidi Mattock, Kurt Straif, on behalf of the International Agency for Research on Cancer Monograph Working Group, IARC, Lyon, France, “Carcinogenicity of tetrachlorvinphos, parathion, malathion, diazinon, and glyphosate,” 16 The Lancet Oncology 490 (2015).

Posted in Causation, Scientific Evidence, Scientific Misconduct | Comments Off on The IARC Process is Broken

Reinventing the Burden of Proof

April 27th, 2016

If lawyers make antic claims that keep the courtrooms busy, law professors make antic proposals to suggest that the law is conceptually confused and misguided, to keep law reviews full.

A few years ago, an article by Professor Edward Cheng claimed that common law courts have failed to grasp the true meaning of burdens of proof. Edward K. Cheng, “Reconceptualizing the Burden of Proof,” 122 Yale L. J. 1254 (2013) [Cheng]. Every law student knows that the preponderance-of-the-evidence standard requires that the party with the burden of proof to establish each element of the claim or defense to a probability greater than 50%. Cheng acknowledges that courts know this as well (citations omitted), but then he goes on to state some remarkable assertions.

First, Cheng suggests that the legal system has engaged in a “casual recharacterization of the burden of proof into p > 0.5 and p > 0.95.” Cheng at 1258. Being charitable, let’s say “characterization” rather than “recharacterization,” for Cheng cites nothing for his suggestion that there was some prior characterization that the law mischievously changed. Cheng at 1258.

Second, Cheng claims that the failure to deal with quantified posterior probabilities is the result of an educational or psychological deficiency of judges and lawyers:

“By comparison, the criminal beyond-a-reasonable-doubt standard is akin to a probability greater than 0.9 or 0.95. Perhaps, as most courts have ruled, the prosecution is not allowed to quantify ‘reasonable doubt’, but that is only an odd quirk of the math-phobic legal system.”

Cheng at 1256 (internal citations omitted). Cheng’s “recharacterization” has given way to his own mischaracterization of the legal system. There is a pandemic math phobia in the legal system, but the refusal to quantify the burden of proof in criminal cases has nothing to do with fear or mathematical incompetence. Most cases simply do not permit any rational or principled quantification of posterior probabilities. And even if they were to allow such a cognitive maneuver, most people, and even judges, cannot map practical certainty, or something like “beyond a reaonable doubt” on to a probability scale of 0 to 1. No less than Judge Jack Weinstein, certainly a friend to the notion that “all evidence is probabilistic,” showed in his informal survey of federal judges of the Eastern District of New York, that judges have no idea of what probability corresponds to the criminal burden of proof:

US v Fatico BoP

U.S. v. Fatico, 458 F.Supp. 388 (E.D.N.Y. 1978). Judge Weinstein’s informal survey showed well enough that there is no real understanding of how to map reasonable doubt or its complement onto a scale of 0 to 1. Furthermore, for the vast majority of cases, there is simply no way to assign meaningful probabilities to events, causes, and states of mind, which make up the elements of claims and defenses in our legal system.

Third, Cheng makes much of the non-existence of absolute probabilities in legal contexts. The word “absolute” is used 14 times in his essay. This point is confusing as stated because no one, to my knowledge, has claimed that the burden of proof is an absolute probability that is stated or arrived at independently of evidence in the case. Plaintiffs and defendants can have burdens of proof and claims and defenses, respectively, but for sake of simplicity, let’s follow Cheng and describe the civil burden of proof as the plaintiff’s burden. The relevant probability is not the absolute probability P(Hπ), but rather the conditional posterior probability: P(Hπ | E).

Fourth, Cheng’s principal innovation, the introduction of a probability ratio as the true meaning and model of the burden of proof has little or no support in case law or in evidence theory. Cheng cites virtually no cases, and only a few selected publications from the world of law reviews. Cheng proposes to recast burdens of proof as a ratio of conditional probabilities of the plaintiff’s and defendant’s “stories.” If the posterior probability of the plaintiff’s story at trial’s end is P(Hπ | E)¹, and the defendant’s story is represented as P(Hδ | E), then Cheng argues that the plaintiff has carried his burden of proof whenever

P(Hπ | E) / P(Hδ | E) > 1.0

This innovation seems fundamentally wrong for several reasons. Again, assuming that the plaintiff or the State has the burden of proof, the defendant has none. If the plaintiff presents no evidence, then the numerator will be zero, and the ratio will be zero. The defendant prevails, and Cheng’s theory holds. But if the plaintiff presents some evidence and the defendant presents none, then the ratio is undefined. Alternatively, we may see the ratio in this situation as approaching infinity as a limit as the probability of the defendant’s “story” based upon his evidence approaches zero. On either interpretation of this scenario, the ratio Cheng invents is huge, and yet the plaintiff may well lose as for instance when plaintiff’s case is insufficient as a matter of law.

Cheng’s ratio theory thus fails as a descriptive theory. The theory appears to fail prescriptively as well. In most civil and criminal cases, the finder of fact is instructed that the defendant has no burden of proof and need not present any evidence at all. Even when the defendant has remained silent, and the plaintiff has presented a legally sufficient case, the fact finder may return a verdict for the defendant when the P(Hπ | E) seems too low with respect to the burden of proof.

Let’s consider an example, perhap not too far fetched in some American courtrooms. The plaintiff claims that drug A has caused him to develop Syndrome Z. Plaintiff has no clinical trial, or analytical epidemiologic, or animal evidence to support his claim. All the plaintiff can adduce is a so-called disproportionality analysis based upon the reporting of adverse events to the FDA. The defendant does not present any evidence of safety. The end point of interest in the lawsuit, Syndrome Z, was not observed in the trials, and was never looked for in any epidemiologic or toxicologic study. The defendant thus has no affirmative evidence of safety that counts for P(Hδ | E).

Assuming that the trial court does not toss this claim pretrial on a Rule 702 motion, or on a directed verdict, the defendant must address the plaintiff’s claim and the assertion that P(Hπ | E) > 0. The plaintiff supports his claim and assertion by presenting an expert witness who endorses the validity, accuracy, and probativeness of the disproportionality analysis. The defendant confronts this evidence solely on cross-examination, and not by trying to suggest that the plaintiff’s expert witness’s analysis is actually evidence of safety. The point of the cross-examination is to show that the proferred analysis is not a valid tool and lacks validity, accuracy, and probativeness.

In this situation, the plaintiff’s P(Hπ | E) might have been greater than 0.5 at the end of direct examination, but if defense counsel has done his job, then at the end of the cross-examination, the P(Hπ | E) < 0.5. Perhaps at this stage of the proceedings, P(Hπ | E) < 0.01.

The defendant, having no affirmative evidence of safety, rests without presenting any evidence. P(Hδ | E) = 0. Alas, we cannot say that P(Hδ | E) is the complement of P(Hπ | E). There is, in most cases, way too much room for ignorance, indeterminate, or unknown probability of the P(Hδ). In this hypothetical, however, there is no evidence adduced for safety at all, only very weak and unreliable evidence of harm. The ratio is undefined, but the law would allow the dismissal of the plaintiff’s case, or would affirm a rational fact finder’s return of a defense verdict. And the law should do those things.

Fifth, Cheng commits other errors along the way to arriving at his ratio theory. In one instance, he commits a serious category mistake:

“Looking at the statistical world, we immediately see that characterizing any decision rule as a 0.5 probability threshold is odd. Statisticians rarely attempt to prove the truth of a proposition or hypothesis by using its absolute probability. Instead, hypothesis testing is usually comparative. There is a null hypothesis and an alternative hypothesis, and one is rejected in favor of the other depending on the evidence observed and the consistency of that evidence with the two hypotheses.”

Cheng at 1259 (internal citations omitted; emphasis added).

Again, Cheng is correct insofar as he suggests that statisticians do not often use use absolute probabilities. Attained levels of significance probabilities, whether used in hypothesis testing or otherwise, are conditional probabilities that describe the probability of observing the sample statistic, or one more extreme, based upon the statistical model and posited null hypothesis. Indeed, many methodologically rigorous statisticians and scientists would resist placing a quantified posterior probability on the truth of a proposition or hypothesis. The measures of probability may be helpful in identifying uncertainties due to random error, or even on occasion due to bias, but these measures do not translate into assigning the quantified posterior probabilites that Cheng wants and needs to make his ratio theory work. There is nothing, however, odd about using the quantified posterior probability of greater than 50% as a metaphor.

But whence comes rejecting one hypothesis “in favor of” another, as a matter of statistics? The null hypothesis is not accepted in the hypothesis test; rather it was assumed in order to conduct the test. The inference Cheng describes would be improper. In a footnote, Cheng asserts that “classical hypothesis testing strongly favors the null hypothesis,” but this conflates attained level of significance with posterior probabilities. Cheng at 1259 n. 12. Cheng states that “the null hypothesis can be given no specific preference,” in legal contexts, id., but this statement seems to ignore what it means for a party to have a burden of proving facts needed to establish its claim or defense.

Of course, over the course of multiple studies, which look at the issue repeatedly with increasingly precise and valid experiments and studies, and which consistently fail to reject a given null hypothesis, we sometimes do, as a matter of judgment, accept the null hypothesis. This situation has little to do with the Cheng’s ratio theory, however.

1 Where P stands for probability, Hπ for the plaintiff’s “story,” Hδ for the defendant’s story, P(Hπ | E) represents the posterior probability at trial’s end of the plaintiff’s story given the evidence, and P(Hδ | E) represents the posterior probability at trial’s end of the defendant’s story given the evidence.

Posted in statistical evidence | Comments Off on Reinventing the Burden of Proof

Lipitor Diabetes MDL’s Inexact Analysis of Fisher’s Exact Test

April 21st, 2016

Muriel Bristol was a biologist who studied algae at the Rothamsted Experimental Station in England, after World War I. In addition to her knowledge of plant biology, Bristol claimed the ability to tell whether tea had been added to milk, or the tea poured first and then milk had been added. Bristol, as a scientist and a proper English woman, preferred the latter.

Ronald Fisher, who also worked at Rothamsted, expressed his skepticism over Dr. Bristol’s claim. Fisher set about to design a randomized experiment that would efficiently and effectively test her claim. Bristol was presented with eight cups of tea, four of which were prepared with milk added to tea, and four prepared with tea added to milk. Bristol, of course, was blinded to which was which, but was required to label each according to its manner of preparation. Fisher saw his randomized experiment as a 2 x 2 contingency table, from he could calculate the observed outcome (and ones more extreme if there were any more extreme outcomes) using the assumption of fixed marginal rates and the hypergeometric probability distribution. Fisher’s Exact Test was born at tea time.[1]

Fisher described the origins of his Exact Test in one of his early texts, but he neglected to report whether his experiment vindicated Bristol’s claim. According to David Salsburg, H. Fairfield Smith, one of Fisher’s colleagues, acknowledged that Bristol nailed Fisher’s Exact test, with all eight cups correctly identified. The test has gone on to become an important tool in the statistician’s armamentarium.

Fisher’s Exact, like any statistical test, has model assumptions and preconditions. For one thing, the test is designed for categorical data, with binary outcomes. The test allows us to evaluate whether two proportions are likely different by chance alone, by calculating the probability of the observed outcome, as well as more extreme outcomes.

The calculation of an exact attained significance probability, using Fisher’s approach, provides a one-sided p-value, with no unique solution to calculating a two-side attained significance probability. In discrimination cases, the one-sided p-value may well be more appropriate for the issue at hand. The Fisher’s Exact Test has thus played an important role in showing the judiciary that small sample size need not be an insuperable barrier to meaningful statistical analysis. In discrimination cases, the one-sided p-value provided by the test is not a particular problem.[2]

The difficulty of using Fisher’s Exact for small sample sizes is that the hypergeometric distribution, upon which the test is based, is highly asymmetric. The observed one-sided p-value does not measure the probability of a result equally extreme in the opposite direction. There are at least three ways to calculate the p-value:

Double the one-sided p-value.
Add the point probabilities from the opposite tail that are more extreme than the observed point probability.
Use the mid-P value; that is, add all values more extreme (smaller) than the observed point probability from both sides of the distribution, PLUS ½ of the observed point probability.

Some software programs will proceed in one of these ways by default, but their doing so does guarantee the most accurate measure of two-tailed significance probability.

In the Lipitor MDL for diabetes litigation, Judge Gergel generally used sharp analyses to cut through the rancid fat of litigation claims, to get to the heart of the matter. By and large, he appears to have done a splendid job. In course of gatekeeping under Federal Rule of Evidence 702, however, Judge Gergel may have misunderstood the nature of Fisher’s Exact Test.

Nicholas Jewell is a well-credentialed statistician at the University of California. In the courtroom, Jewell is a well-known expert witness for the litigation industry. He is no novice at generating unreliable opinion testimony. See In re Zoloft Prods. Liab. Litig., No. 12–md–2342, 2015 WL 7776911 (E.D. Pa. Dec. 2, 2015) (excluding Jewell’s opinions as scientifically unwarranted and methodologically flawed). In the Lipitor cases, some of Jewell’s opinions seemed outlandish indeed, and Judge Gergel generally excluded them. See In re Lipitor Marketing, Sales Practices and Prods. Liab. Litig., MDL No. 2:14-mn-02502-RMG, ___ F.Supp. 3d ___ (2015), 2015 WL 7422613 (D.S.C. Nov. 20, 2015) [Lipitor Jewell], reconsideration den’d, 2016 WL 827067 (D.S.C. Feb. 29, 2016) [Lipitor Jewell Reconsidered].

As Judge Gergel explained, Jewell calculated a relative risk for abnormal blood glucose in a Lipitor group to be 3.0 (95% C.I., 0.9 to 9.6), using STATA software. Also using STATA, Jewell obtained an attained significance probability of 0.0654, based upon Fisher’s Exact Test. Lipitor Jewell at *7.

Judge Gergel did not report whether Jewell’s reported p-value of 0.0654, was one- or two-sided, but he did state that the attained probability “indicates a lack of statistical significance.” Id. & n. 15. The rest of His Honor’s discussion of the challenged opinion, however, makes clear that of 0.0654 must have been a two-sided value. If it had been a one-sided p-value, then there would have been no way of invoking the mid-p to generate a two-sided p-value below 5%. The mid-p will always be larger than the one-tailed exact p-value generated by Fisher’s Exact Test.

The court noted that Dr. Jewell had testified that he believed that STATA generated this confidence interval by “flip[ping]” the Taylor series approximation. The STATA website notes that it calculates confidence intervals for odds ratios (which are different from the relative risk that Jewell testified he computed), by inverting the Fisher exact test.[3] Id. at *7 & n. 17. Of course, this description suggests that the confidence interval is not based upon exact methods.

STATA does not provide a mid p-value calculation, and so Jewell used an on-line calculator, to obtain a mid p-value of 0.04, which he declared statistically significant. The court took Jewell to task for using the mid p-value as though it were a different analysis or test. Id. at *8. Because the mid-p value will always be larger than the one-sided exact p-value from Fisher’s Exact Test, the court’s explanation does not really make sense:

“Instead, Dr. Jewell turned to the mid-p test, which would ‘[a]lmost surely’ produce a lower p-value than the Fisher exact test.”

Id. at *8. The mid-p test, however, is not different from the Fisher’s exact; rather it is simply a way of dealing with the asymmetrical distribution that underlies the Fisher’s exact, to arrive at a two-tailed p-value that more accurately captures the rate of Type I error.

The MDL court acknowledged that the mid-p approach, was not inherently unreliable, but questioned Jewell’s inconsistent, selective use of the approach for only one test.[4] Jewell certainly did not help the plaintiffs’ cause and his standing by having discarding the analyses that were not incorporated into his report, thus leaving the MDL court to guess at how much selection went on in his process of generating his opinions.. Id. at *9 & n. 19.

None of Jewell’s other calculated p-values involved the mid-p approach, but the court’s criticism begs the question whether the other p-values came from a Fisher’s Exact Test with small sample size, or other highly asymmetrical distribution. Id. at *8. Although Jewell had shown himself willing to engage in other dubious, result-oriented analyses, Jewell’s use of the mid-p for this one comparison may have been within acceptable bounds after all.

The court also noted that Jewell had obtained the “exact p-value and that this p-value was not significant.” Id. The court’s notation here, however, does not report the important detail whether that exact, unreported p-value was merely the doubled of the one-sided p-value given by the Fisher’s Exact Test. As the STATA website, cited by the MDL court, explains:

“The test naturally gives a one-sided p-value, and there are at least four different ways to convert it to a two-sided p-value (Agresti 2002, 93). One way, not implemented in Stata, is to double the one-sided p-value; doubling is simple but can result in p-values larger than one.”

Wesley Eddings, “Fisher’s exact test two-sided idiosyncrasy” (Jan. 2009) (citing Alan Agresti, Categorical Data Analysis 93 (2d ed. 2002)).

On plaintiffs’ motion for reconsideration, the MDL court reaffirmed its findings with respect to Jewell’s use of the mid-p. Lipitor Jewell Reconsidered at *3. In doing so, the court insisted that the one instance in which Jewell used the mid-p stood in stark contrast to all the other instances in which he had used Fisher’s Exact Test. The court then cited to the record to identify 21 other instances in which Jewell used a p-value rather than a mid-p value. The court, however, did not provide the crucial detail whether these 21 other instances actually involved small-sample applications of Fisher’s Exact Test. As result-oriented as Jewell can be, it seems safe to assume that not all his statistical analyses involved Fisher’s Exact Test, with its attendant ambiguity for how to calculate a two-tailed p-value.

Post-Script (Aug. 9, 2017)

The defense argument and the judicial error were echoed in a Washington Legal Foundation paper that pilloried Nicholas Jewell for the surfeit of many methodological flaws in his expert witness opinions in In re Lipitor. Unfortunately, the paper uncritically recited the defense’s theory about the Fisher’s Exact Test:

“In assessing Lipitor data, even after all of the liberties that [Jewell] took with selecting data, he still could not get a statistically-significant result employing a Fisher’s exact test, so he switched to another test called a mid-p test, which generated a (barely) statistically significant result.”

Kirby Griffis, “The Role of Statistical Significance in Daubert/Rule 702 Hearings,” at 19, Wash. Leg. Foundation Critical Legal Issues Working Paper No. 201 (Mar. 2017). See Kirby Griffis, “Beware the Weak Argument: The Rule of Thirteen,” For the Defense 72 (July 2013) (quoting Justice Frankfurter, “A bad argument is like the clock striking thirteen. It puts in doubt the others.”). The fallacy of Griffis’ argument is that it assumes that a mid-p calculation is a different statistical test from the Fisher’s Exact test, which yields a one-tailed significance probability. Unfortunately, Griffis’ important paper is marred by this and other misstatements about statistics.

[1] Sir Ronald A. Fisher, The Design of Experiments at chapter 2 (1935); see also Stephen Senn, “Tea for three: Of infusions and inferences and milk in first,” Significance 30 (Dec. 2012); David Salsburg, The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century (2002).

[2] See, e.g., Dendy v. Washington Hosp. Ctr., 431 F. Supp. 873 (D.D.C. 1977) (denying preliminary injunction), rev’d, 581 F.2d 99 (D.C. Cir. 1978) (reversing denial of relief, and remanding for reconsideration). See also National Academies of Science, Reference Manual on Scientific Evidence 255 n.108 (3d ed. 2011) (“Well-known small sample techniques [for testing significance and calculating p-values] include the sign test and Fisher’s exact test.”).

[3] See Wesley Eddings, “Fisher’s exact test two-sided idiosyncrasy” (Jan. 2009), available at <http://www.stata.com/support/faqs/statistics/fishers-exact-test/>, last visited April 19, 2016 (“Stata’s exact confidence interval for the odds ratio inverts Fisher’s exact test.”). This article by Eddings contains a nice discussion of why the Fisher’s Exact Test attained significance probability disagrees with the calculated confidence interval. Eddings points out the asymmetry of the hypergeometric distribution, which complicates arriving at an exact p-value for a two-sided test.

[4] See Barber v. United Airlines, Inc., 17 Fed.Appx. 433, 437 (7th Cir. 2001) (“Because in formulating his opinion Dr. Hynes cherry-picked the facts he considered to render an expert opinion, the district court correctly barred his testimony because such a selective use of facts fails to satisfy the scientific method and Daubert.”).

Posted in Expert Witnesses, Rule 702, statistical evidence | Comments Off on Lipitor Diabetes MDL’s Inexact Analysis of Fisher’s Exact Test

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.