Schachtman Law » Expert Witnesses

TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Tenpenny Down to Tuppence

August 22nd, 2023

Over two years ago, an osteopathic physician by the name of Sherri Tenpenny created a stir when she told the Ohio state legislature that Covid vaccines magnetize people or cause them to “interface with 5G towers.”[1] What became apparent at that time was that Tenpenny was herself a virulent disease vector of disinformation. Indeed, in its March 2021 report, the Center for Countering Digital Hate listed Tenpenny as a top anti-vaccination shyster. As a social media vector, she is ranked in the top dozen “influencers.”[2] No surprise, in addition to bloviating about Covid vaccines, someone with such quirkly non-evidence based opinions turns up in litigation as an expert witness.[3]

At the time of Tenpenny’s ludicrous testimony before the Ohio state legislature, one astute observer remarked that the AMA Ethical Guidelines specify that medical societies and medical licensing boards are responsible for maintaining high standards for medical testimony, and must assess “claims of false or misleading testimony.”[4] When the testimony is false or misleading, these bodies should discipline the offender “as appropriate.”[5]

The State Medical Board of Ohio stepped up to its responsibility. After receiving hundreds (roughly 350) complaints about Tenpenny’s testimony, the Ohio Board launched an investigation of Tenpenny, who was first licensed as an osteopathic physician in 1984.[6] The Board’s investigators tried to contact Tenpenny, who apparently evaded engaging with them. Eventually, Thomas Renz, a lawyer for Tenpenny informed the Board that Tenpenny “[d]eclin[ed] to cooperate in the Board’s bad faith and unjustified assault on her licensure, livelihood, and constitutional rights cannot be construed as an admission of any allegations against her.”

After multiple unsuccessful attempts to reach Tenpenny, the Board issued a citation, in 2022, against her for stonewalling the investigation. Tenpenny requested an administrative hearing, set for April 2023, when she would be able to submit her defense in writing. The Board refused to let Tenpenny evade questioning, and suspended her license for failure to comply with the investigation. According to the Board’s Order, “Dr. Tenpenny did not simply fail to cooperate with a Board investigation, she refused to cooperate. *** And that refusal was based on her unsupported and subjective belief regarding the Board’s motive for the investigation. Licensees of the Board cannot simply refuse to cooperate in investigations because they decide they do not like what they assume is the reason for the investigation.”[7]

According to the Board’s Order, Tenpenny has been fined $3,000, and she must satisfy the Board’s conditions before applying for reinstatement. The Ohio Board’s decision is largely based upon a procedural ruling that flowed from Tenpenny’s refusal to cooperate with the Board’s investigation. Most state medical boards have done little to nothing to address the substance of physician misconduct arising out of the COVID pandemic. This month, American Board of Internal Medicine (ABIM) announced that it was revoking the board certifications of two physicians, Drs. Paul Marik and Pierre Kory, members of the Front Line COVID-19 Critical Care Alliance, for engaging in promoting disinformation and invalid opinions about therapies for COVID-19 opinions.[8] Ron Johnson, the quack senator from Wisconsin, predictably and transparently criticized the ABIM’s action with an ad hominem attack on the ABIM as the action of a corporate cabal. Quack physicians of course have a first amendment right to say whatever, but their licensure and their board certification are contingent on basic competence. Both the state boards and the certifying private groups have the right and responsibility to revoke licenses and privileges when physicians demonstrate incompetence and callousness in the face of a pandemic. There is no unqualified right to professional licenses or certifications.

[1] Andrea Salcedo, “A doctor falsely told lawmakers vaccines magnetize people: ‘They can put a key on their forehead. It sticks’,” Washington Post (June 9, 2021); Andy Downing, “What an exceedingly dumb time to be alive,” Columbus Alive (June 10, 2021); Jake Zuckerman, “She says vaccines make you magnetized. This West Chester lawmaker invited her testimony, chair says,” Ohio Capital Journal (July 14, 2021).

[2] The Disinformation Dozen (2021),

[3] Shaw v. Sec’y Health & Human Servs., No. 01-707V, 2009 U.S. Claims LEXIS 534, *84 n.40 (Fed. Cl. Spec. Mstr. Aug. 31, 2009) (excluding expert witness opinion testimony from Tenpenny).

[4] “Epistemic Virtue – Dropping the Dime on Tenpenny” Tortini (July 18, 2021).

[5] A.M.A. Code of Medical Ethics Opinion 9.7.1.

[6] Michael DePeau-Wilson, “Doc Who Said COVID Vax Magnetized People Has License Suspended,” MedPageToday (Aug. 11, 2023); David Gorski, “The Ohio State Medical Board has finally suspended the medical license of antivax quack Sherri Tenpenny,” Science-Based Medicine (Aug, 14, 2023).

[7] In re Sherri J. Tenpenny, D.O., Case No. 22-CRF-0168 State Medical Board of Ohio (Aug. 9, 2023).

[8] David Gorski, “The American Board of Internal Medicine finally acts against two misinformation-spreading doctors,” Science-Based Medicine (Aug. 7, 2023).

Posted in Expert Witnesses, Public Nuisances, Scientific Evidence, Scientific Misconduct | No Comments »

Consensus Rule – Shadows of Validity

April 26th, 2023

Back in 2011, at a Fourth Circuit Judicial Conference, Chief Justice John Roberts took a cheap shot at law professors and law reviews when he intoned:

“Pick up a copy of any law review that you see, and the first article is likely to be, you know, the influence of Immanuel Kant on evidentiary approaches in 18th Century Bulgaria, or something, which I’m sure was of great interest to the academic that wrote it, but isn’t of much help to the bar.”[1]

Anti-intellectualism is in vogue these days. No doubt, Roberts was jocularly indulging in an over-generalization, but for anyone who tries to keep up with the law reviews, he has a small point. Other judges have rendered similar judgments. Back in 1993, in a cranky opinion piece – in a law review – then Judge Richard A. Posner channeled the liar paradox by criticizing law review articles for “the many silly titles, the many opaque passages, the antic proposals, the rude polemics, [and] the myriad pretentious citations.”[2] In a speech back in 2008, Justice Stephen Breyer noted that “[t]here is evidence that law review articles have left terra firma to soar into outer space.”[3]

The temptation to rationalize, and to advocate for reflective equilibrium between the law as it exists, and the law as we think it should be, combine to lead to some silly and harmful efforts to rewrite the law as we know it. Jeremy Bentham, Mr. Nonsense-on-Stilts, who sits stuffed in the hallway of the University of London, ushered in a now venerable tradition of rejecting tradition and common sense, in proposing all sorts of law reforms.[4] In the early 1800s, Jeremy Bentham, without much in the way of actual courtroom experience, deviled the English bench and bar with sweeping proposals to place evidence law on what he thought was a rational foundation. As with his naïve utilitarianism, Bentham’s contributions to jurisprudence often ignored the realities of human experience and decision making. The Benthamite tradition of anti-tradition is certainly alive and well in the law reviews.

Still, I have a soft place in my heart for law reviews. Although not peer reviewed, law reviews provide law students a tremendous opportunity to learn about writing and scholarship through publishing the work of legal scholars, judges, thoughtful lawyers, and other students. Not all law review articles are non-sense on stilts, but we certainly should have our wits about us when we read immodest proposals from the law professoriate.

* * * * * * * * * *

Professor Edward Cheng has written broadly and insightfully about evidence law, and he certainly has the educational training to do so. Recently, Cheng has been bemused by the expert paradox, which wonders how lay persons, without expertise, can evaluate and judge issues of the admissibility, validity, and correctness of expert opinion. The paradox has long haunted evidence law, and it is at center stage in the adjudication of expert admissibility issues, as well as the trial of technical cases. Recently, Cheng has proposed a radical overhaul to the law of evidence, which would require that we stop asking courts to act as gatekeepers, and to stop asking juries to determine the validity and correctness of expert witness opinions before them. Cheng’s proposal would revert to the nose counting process of Frye and permit consideration of only whether there is an expert witness consensus to support the proffered opinion for any claim or defense.[5] Or in Plato’s allegory of the cave, we need to learn to be content with shadows on the wall rather than striving to know the real thing.

When Cheng’s proposal first surfaced, I wrote briefly about why it was a bad idea.[6] Since his initial publication, a law review symposium was assembled to address and perhaps to celebrate the proposal.[7] The papers from that symposium are now in print.[8] Unsurprisingly, the papers are both largely sympathetic (but not completely) to Cheng’s proposal, and virtually devoid of references to actual experiences of gatekeeping or trials of technical issues.

Cheng contends that the so-called Daubert framework for addressing the admissibility of expert witness opinion is wrong. He does not argue that the existing law, in the form of Federal Rules of Evidence 702 and 703, does not call for an epistemic standard for both admitting opinion testimony, as well for the fact-finders’ assessments. There is no effort to claim that somehow four Supreme Court cases, and thousand of lower courts, have erroneously viewed the whole process. Rather, Cheng simply asserts non-expert judges cannot evaluate the reliability (validity) of expert witness opinions, and that non-expert jurors cannot “reach independent, substantive conclusions about specialized facts.”[9] The law must change to accommodate his judgment.

In his symposium contribution, Cheng expands upon his previous articulation of his proposed “consensus rule.”[10] What is conspicuously absent, however, is any example of failed gatekeeping that excluded valid expert witness opinion. One example, the appellate decision in Rosen v. Ciba-Geigy Corporation,[11] which Cheng does give, is illustrative of Cheng’s project. The expert witness, whose opinion was excluded, was on the faculty of the University of Chicago medical school; Richard Posner, the appellate judge who wrote the opinion that affirmed the expert witness’s exclusion, was on the faculty of that university’s law school. Without any discussion of the reports, depositions, hearings, or briefs, Cheng concludes that “the very idea that a law professor would tell medical school colleagues that their assessments were unreliable seems both breathtakingly arrogant and utterly ridiculous.”[12]

Except, of course, very well qualified scientists and physicians advance invalid and incorrect claims all the time. What strikes me as breathtakingly arrogant and utterly ridiculous is the judgment of a law professor who has little to no experience trying or defending Rule 702 and 703 issues labeling the “very idea” as arrogant and ridiculous. Aside from its being a petitio principia, we could probably add that the reaction is emotive, uninformed, and uninformative, and that it fails to support the author’s suggestion that “Daubert has it all wrong,” and that “[w]e need a different approach.”

Judges and jurors obviously will never fully understand the scientific issues before them. If and when this lack of epistemic competence is problematic, we should honestly acknowledge how we are beyond the realm of the Constitution’s seventh amendment. Since Cheng is fantasizing about what the law should be, why not fantasize about not allowing lay people to decide complex scientific issues? Verdicts from jurors who do not have to give reasons for their decisions, and who are not in any sense peers of the scientists whose work they judge are normatively problematic.

Professor Cheng likens his consensus rule to how the standard of care is decided in medical malpractice litigation. The analogy is interesting, but hardly compelling in that it ignores “two schools of thought” doctrine.[13] In litigation of claims of professional malpractice, the “two schools of thought doctrine” is a complete defense. As explained by the Pennsylvania Supreme Court,[14] physicians may defend against claims that they deviated from the standard of care, or of professional malpractice, by adverting to support for their treatment by a minority of professionals in their field:

“Where competent medical authority is divided, a physician will not be held responsible if in the exercise of his judgment he followed a course of treatment advocated by a considerable number of recognized and respected professionals in his given area of expertise.”[15]

The analogy to medical malpractice litigation seems inapt.

Professor Cheng advertises that he will be giving full-length book treatment to his proposal, and so perhaps my critique is uncharitable in looking at a preliminary, (antic?) law review article. Still, his proposal seems to ignore that “general acceptance” renders consensus, when it truly exists, as relevant to both the court’s gatekeeping decisions, and the fact finders’ determination of the facts and issues in dispute. Indeed, I have never seen a Rule 702 hearing that did not involve, to some extent, the assertion of a consensus, or the lack thereof.

To the extent that we remain committed to trials of scientific claims, we can see that judges and jurors often can detect inconsistencies, cherry picking, unproven assumptions, and other aspects of the patho-epistemology of exert witness opinions. It takes a community of scientists and engineers to build a space rocket, but any Twitter moron can determine when a rocket blows up on launch. Judges in particular have (and certainly should have) the competence to determine deviations from the scientific and statistical standards of care that pertain to litigants’ claims.

Cheng’s proposal also ignores how difficult and contentious it is to ascertain the existence, scope, and actual content of scientific consensus. In some areas of science, such as occupational and environmental epidemiology and medicine, faux consensuses are set up by would-be expert witnesses for both claimants and defendants. A search of the word “consensus” in the PubMed database yields over a quarter of a million hits. The race to the bottom is on. Replacing epistemic validity with sociological and survey navel gazing seems like a fool’s errand.

Perhaps the most disturbing aspect of Cheng’s proposal is what happens in the absence of consensus. Pretty much anything goes, a situation that Cheng finds “interesting,” and I find horrifying:

“if there is no consensus, the legal system’s options become a bit more interesting. If there is actual dissensus, meaning that the community is fractured in substantial numbers, then the non-expert can arguably choose from among the available theories. If the expert community cannot agree, then one cannot possibly expect non-experts to do any better.”[16]

Cheng reports that textbooks and other documents “may be both more accurate and more efficient” evidence of consensus.[17] Maybe; maybe not. Textbooks are typically often dated by the time they arrive on the shelves, and contentious scientists are not beyond manufacturing certainty or doubt in the form of falsely claimed consensus.

Of course, often, if not most of the time, there will be no identifiable, legitimate consensus for a litigant’s claim at trial. What would Professor Cheng do in this default situation? Here Cheng, fully indulging the frolic, tells us that we

“should hypothetically ask what the expert community is likely to conclude, rather than try to reach conclusions on their own.”[18]

So the default situation transforms jurors into tea-leaf readers of what an expert community, unknown to them, will do if and when there is evidence of a quantum and quality to support a consensus, or when that community gets around to articulating what the consensus is. Why not just toss claims that lack consensus support?

[1] Debra Cassens Weiss, “Law Prof Responds After Chief Justice Roberts Disses Legal Scholarship,” Am. Bar Ass’n J. (July 7, 2011).

[2] Richard A. Posner, “Legal Scholarship Today,” 45 Stanford L. Rev. 1647, 1655 (1993), quoted in Walter Olson, “Abolish the Law Reviews!” The Atlantic (July 5, 2012); see also Richard A. Posner, “Against the Law Reviews: Welcome to a world where inexperienced editors make articles about the wrong topics worse,”
Legal Affairs (Nov. 2004).

[3] Brent Newton, “Scholar’s highlight: Law review articles in the eyes of the Justices,” SCOTUS Blog (April 30, 2012); “Fixing Law Reviews,” Inside Higher Education (Nov. 19, 2012).

[4] “More Antic Proposals for Expert Witness Testimony – Including My Own Antic Proposals” (Dec. 30, 2014).

[5] Edward K. Cheng, “The Consensus Rule: A New Approach to Scientific Evidence,” 75 Vanderbilt L. Rev. 407 (2022).

[6] “Cheng’s Proposed Consensus Rule for Expert Witnesses” (Sept. 15, 2022);
“Further Thoughts on Cheng’s Consensus Rule” (Oct. 3, 2022).

[7] Norman J. Shachoy Symposium, The Consensus Rule: A New Approach to the Admissibility of Scientific Evidence (2022), 67 Villanova L. Rev. (2022).

[8] David S. Caudill, “The ‘Crisis of Expertise’ Reaches the Courtroom: An Introduction to the Symposium on, and a Response to, Edward Cheng’s Consensus Rule,” 67 Villanova L. Rev. 837 (2022); Harry Collins, “The Owls: Some Difficulties in Judging Scientific Consensus,” 67 Villanova L. Rev. 877 (2022); Robert Evans, “The Consensus Rule: Judges, Jurors, and Admissibility Hearings,” 67 Villanova L. Rev. 883 (2022); Martin Weinel, “The Adversity of Adversarialism: How the Consensus Rule Reproduces the Expert Paradox,” 67 Villanova L. Rev. 893 (2022); Wendy Wagner, “The Consensus Rule: Lessons from the Regulatory World,” 67 Villanova L. Rev. 907 (2022); Edward K. Cheng, Elodie O. Currier & Payton B. Hampton, “Embracing Deference,” 67 Villanova L. Rev. 855 (2022).

[9] Embracing Deference at 876.

[10] Edward K. Cheng, Elodie O. Currier & Payton B. Hampton, “Embracing Deference,” 67 Villanova L. Rev. 855 (2022) [Embracing Deference]

[11] Rosen v. Ciba-Geigy Corp., 78 F.3d 316 (7th Cir. 1996).

[12] Embracing Deference at 859.

[13] “Two Schools of Thought” (May 25, 2013).

[14] Jones v. Chidester, 531 Pa. 31, 40, 610 A.2d 964 (1992).

[15] Id. at 40. See also Fallon v. Loree, 525 N.Y.S.2d 93, 93 (N.Y. App. Div. 1988) (“one of several acceptable techniques”); Dailey, “The Two Schools of Thought and Informed Consent Doctrine in Pennsylvania,” 98 Dickenson L. Rev. 713 (1994); Douglas Brown, “Panacea or Pandora’ Box: The Two Schools of Medical Thought Doctrine after Jones v. Chidester,” 44 J. Urban & Contemp. Law 223 (1993).

[16] Embracing Deference at 861.

[17] Embracing Deference at 866.

[18] Embracing Deference at 876.

Posted in Evidence, Expert Witnesses, Frye, Rule 702, Rule 703, Scientific Evidence | 1 Comment »

Selikoff Timeline & Asbestos Litigation History (Revised)

February 26th, 2023

The critics and cheerleaders of Dr. Irving John Selikoff agree that he was a charming, charismatic, and courageous man, a compassionate physician, and a zealous advocate for worker safety and health. The consensus falls apart over the merits of Selikoff’s actual research, his credentials, and his advocacy tactics.[1]

Selikoff’s collaborators, protégés, and fellow travelers tend to brand any challenge or criticism as “scurrilous.”[2] They attack the messenger for attacking the messenger, who attacked the messenger, u.s.w.. Certainly in his lifetime, Selikoff attracted harsh and vituperative attacks, some of which were mean-spirited and even anti-semitic. Although I am not a Jew, I am, following Jonathan Miller, “Jew-ish, just not the whole hog.” As such, I can appreciate the ire of some of Selikoff’s defenders over the nature of these attacks.

Selikoff’s legitimate achievements should not be diminished, and his defenders are correct to bemoan the ad hominem attacks on Selikoff, based upon ethnicity and personal characteristics. Some attacks, however, were merited. The time has come to stop evaluating the message by its messenger, and to pay attention to the evidence. Selikoff’s defenders and hagiographers are wrong, therefore, to claim that Selikoff’s training, scientific acumen, advocacy, and false positive claims are somehow off limits. Selikoff advanced his scientific and political agenda by promoting his reputation and work, and he thus put his credentials, work, and methods into issue. Selikoff’s contributions to public health in publicizing the dangers of high exposure, long-term asbestos exposure do not privilege every position he took. Selikoff is a difficult case because he was wrong on many issues, and his reputation, authority and prestige ultimately became much greater than the evidence would ultimately support.

Although Selikoff died in 1992, his legacy lives on in the perpetual litigation machine that is run by the litigation industry and Selikoff’s juniors and imitators, who serve as testifying expert witnesses. One of Selikoff’s great achievements, the federalization of worker safety and health in the Williams-Steiger Occupational Safety and Health Act of 1970,[3] languishes because of inadequate resources for enforcement and frivolous efforts to address non-existent problems, such as the lowering of the crystalline silica permissible exposure limit. Activists have taken to redress the problem by advocating for nugatory “warnings” from remote suppliers, in the face of employer failures to monitor and supervise workers and the workplace, and to provide administrative, engineering, and personal protective controls.

Selikoff diverted regulatory attention from asbestos fiber type, with the result that the OSHA PELs were lowered for both chrysotile and amphibole asbestos, thus leaving the ultra-hazardous crocidolite asbestos in use. Selikoff perpetuated a good deal of mischief and misinformation to keep his myth that all fiber types are the same (and that “asbestos is asbestos is asbestos”). In doing so, he actually hurt many people.

An anonymous snark on Wikipedia noted some of my blog posts about Selikoff, and offered the lame criticism that my writings were not peer reviewed.[4] The snark (Tweedale?) was of course correct on this limited point, but generally in this field, peer review is worth a warm bucket of spit. And there is the matter that the anonymous critic was offering a criticism that was also not peer reviewed.

Selikoffophiles continue to tell tall tales about Selikoff’s work and in particular about how he became involved in asbestos medicine.[5] So here is a timeline of Selikoff’s life and asbestos work, an update of an earlier version. If anyone notes an error or inconsistency in this time line, please let me know, provide better sources, and ask for a correction. If I am wrong, I will readily note the correction and eat my words, but I am sure they will be quite digestible.[6]

1915-01-15. Irving John Selikoff was born as Irving Selecoff in the brain basket of America, Brooklyn, New York, to Abraham and Matilda (Tillie) Selecoff.[7] His father, Abraham, was born on April 6, 1885, in the Kyiv oblast of what is now Ukraine.[8]

1920. According to the 1920 census, the Selekoff family lived at 816 179^th Street, in the Bronx. Irving’s father, Abraham, was self-employed as a hat manufacturer, doing business later as United Headwear Corporation.[9] The family had two children, Irving, and his older sister, Gladys.

1930. The asbestos workers’ journal published a story about the (non-malignant) risks of asbestos exposure.[10]

1935-06. Selikoff was graduated from Columbia University, with a B.S. degree.

1935-12-24. Selikoff arrived in Boston from Yarmouth, Nova Scotia, on the S.S. Yarmouth, on December 24, 1935, apparently en route from Scotland.

1936. Dr. Alice Hamilton, physician and noted labor activist, wrote in a labor union journal to urge more attention to industrial dusts, the knowledge of dangers of which was[11] “still very limited except with regard to silica and asbestos.”

1936-08. Selikoff sat for the university entrance boards in Scotland.

1936-09-27. Selikoff married Lydia Kapilian, in the Bronx.[12]

1936-10-12. Irving Selecoff arrives in Liverpool, from New York, aboard the S.S. Samaria.

1936-10. Selikoff entered Anderson’s College of Medicine, in Glasgow, Scotland.[13]

1936-12-28. Irving John Selikoff is listed in the UK, Medical and Dental Students Registers, 1882-1937, registration date December 28, 1936, in Scotland.

1936. Alice Hamilton published an article on the risks and benefits of industrial asbestos use, in a key labor unionist journal. Alice Hamilton, “Industrial Poisons,” 43 The American Federationist 707-13 (1936).

1937-04-26. Selikoff arrived in New York, from Greenock, Scotland, on the S.S. Carinthia.

1937-10-10. Irving Selecoff arrived in Glasgow, Scotland, from New York City, on board the SS. Cameronia.

1938-07-14. Irving J. Selecoff arrived in Quebec, Canada, from Greenock, Scotland, on the S.S. Duchess Atholl.

1939-06-24. Irving Selecoff arrived in New York, from Liverpool, London, on the S.S. Mauretania. Because of the developing hostilities in Europe, Selikoff apparently did not return to Glasgow, in the fall of 1939.

1939-11. Unable to return to Scotland, Selikoff applied to Melbourne University for coursework to finish his non-degree course of qualification for medication practice in the United Kingdom.[14]

1940-03-04. Selecoff (as his name was then often spelled) arrived in Vancouver from Sydney, on the S.S. Aorangi.

1940-04. Irving Selikoff was living with his parents, and his married sister and her family, in Rye, New York, according to the 1940 census, taken on April 10, 1940.

1940-05-27. Selikoff enrolled in the University of Melbourne as a non-degree student, for coursework to finish his qualification for medical license in Scotland.[15]

1941-03-24. John Selecoff arrived in Los Angeles, California, from Sydney, Australia, on the S.S. Mariposa. According to Bartrip, Selikoff had completed his last course at the University of Melbourne, for his “tailor-made” program, on

1941-02-27. Selikoff never gained entrance to a degree program at Melbourne.[16]

1941-04-21. Irving John Selikoff registered for the draft, in Port Chester, New York.

1941. Selikoff joined the Mount Sinai Hospital as an assistant in Anatomy and Pathology, “immediately following his university training.”[17]

1943-11-01. Selikoff received an M.D., degree from Middlesex University,[18] after two semesters in residence. This school was regarded as “substandard” and not approved by the American Medical Association. The school lost its accreditation in 1946, and closed.[19] After receiving this degree, Selikoff continued his efforts to return to Scotland, to complete his “triple qualification” for medical licensure in Scotland, which would allow him to sit for the licensing examination in one of the United States.

1943 – 1944. Selikoff served as an intern, at the Beth Israel Hospital, in Newark, New Jersey.[20]

1944 – 1946. Selikoff served as a resident, at the Sea View Hospital, in New York City.[21]

1945-04-23. Selikoff was listed in the British Medical Registry, based upon his qualification by the Scottish Conjoint Board for his work at Anderson’s and his non-degree work at the University of Melbourne.[22]

1943-06-02. Irving J Selecoff arrived in Montreal, Quebec, Canada, from Liverpool, aboard the S.S. Axel Johnson.

1945-06-02. Selikoff arrived in Montreal, Quebec, from Liverpool, England, on the S.S. Axel Johnson.

1945-12-21. Selikoff’s mother, Tillie, died.

1946-02. Selikoff married Celia Schiffrin in Manhattan.[23] It was the second marriage for both bride and groom.

1947. After having left Mt. Sinai Hospital, in 1943, for an internship and a residency, Selikoff resumed his association with Mt. Sinai Hospital.[24]

1947-06-30. Selikoff’s father, Abraham Selecoff, married Anna Susser, in Manhattan.[25]

1949. Selikoff opened a medical office at 707 Broadway, Paterson, New Jersey,[26] not far from a factory run by the Union Asbestos and Rubber Company (UNARCO). In the same year, the Selikoffs were living at 965 Fifth Avenue, near 78th Street, in Manhattan.[27]

1950. Selikoff’s medical practice in Paterson, New Jersey, afforded him the opportunity to observe “the incidence of lung disease among workers at the Union Asbestos and Rubber Company (UNARCO),”[28] which operated one of its factories in Paterson.

1950-04-05. Irving J. Selikoff and his wife Celia resided at 93 Broadway, Paterson, New Jersey, USA, according to the 1950 census. By the early 1950s, Selikoff and his wife had moved to 505 Upper Boulevard, Ridgewood, New Jersey.

1951. New Jersey lawyer Carl Gelman retained Dr. Irving Selikoff to examine 17 workers from the Paterson plant of Union Asbestos and Rubber Company (UNARCO). Gelman filed workers’ compensation claims on behalf of the UNARCO workers.[29]

1952. Supported by Selikoff’s report, UNARCO worker Anton Szczesniak settled his worker’s compensation case, involving “intestinal cancer,” for $2,000 in 1952.[30] Selikoff published data on the carcinogenicity of amosite in 1972,[31]a delay of twenty years.[32]

1952. Selikoff and colleagues published the results of a clinical trial of isoniazid for tuberculosis patients.[33]

1952. Selikoff was featured in Life magazine coverage of isoniazid, a chemotherapy for tuberculosis.[34]

1952. Selikoff was an assistant attending physician for thoracic diseases in the department of thoracic diseases at Mt. Sinai Hospital. In this year, Selikoff delivered the monthly Physiological Chemistry Seminar lecture at Mt. Sinai Hospital on: “Antitubercular Hydrazines,” along with Drs. H. H. Fox and Richard J. Schnitzer, of Hoffman-La Roche.

1954. UNARCO closed its Paterson, New Jersey plant, and moved it to Tyler, Texas.[35]

1955. Selikoff received the Albert Lasker Clinical Medical Research Award for his work on the clinical trial of isoniazid to treat tuberculosis, along with Walsh McDermott and Carl Muschenheim, of the Hoffmann-La Roche Research Laboratories, and Edward H. Robitzek, of the Squibb Institute for Medical Research.[36]

1955. Selikoff’s involvement in the isoniazid clinical trials continued to attract media attention. His first television appearance was panned, but he would later develop considerable public speaking skills.[37]

1955. Sir Richard Doll published his epidemiologic study of lung cancer among British asbestos workers.[38] This study was known to Selikoff, who relied upon it in his litigation reports to support the compensation claims of asbestos workers in the 1950s.[39]

1955. In 1955, American labor unions were well aware of the claim that asbestos causes lung cancer. Herbert K. Abrams, union physician and the Medical Director of Local 25 Chicago, Building Service Employees International Union, concluded that asbestos causes cancer in a prominent union journal.[40]

1956. Selikoff became an associate attending physician for thoracic disease at Mt. Sinai Hospital.

1957. For many years, Frederick Legrand had been a pipecoverer and asbestos worker for asbestos contracting firms. In February 1956, Legrand filed a successful claim for worker’s compensation for disability due to asbestosis.[41] Attorney William L. Brach filed perhaps the first civil action (as opposed to worker’s compensation claim), on behalf of LeGrand, against Johns-Manville, for asbestos-related disease, on July 17, 1957. Frederick LeGrande v. Johns-Manville Prods. Corp., No. 741-57 (D.N.J.).[42] Trial commenced on March 4, 1959, before the Hon. Honorable Reynier J. Wortendyke, Jr. In the middle of trial, Johns-Manville (JM) settled the case for $35,000.[43] According to various accounts, JM badly mishandled the defense by falsely asserting that it had no knowledge of potential asbestosis hazards to end-users such as LeGrand. The defense had the dual liability of both being untrue and depriving JM of affirmative defenses of contributory negligence and assumption of risk. JM was apparently able to obtain a stipulation that LeGrand’s condition was not the result of asbestos in JM’s product, which JM used to hide the JM settlement from subsequent claimants. Frederick Legrand died in the fall of 1959.[44]

1957-07. The asbestos insulators’ union’s periodical, distributed to its members, notes that “[t]he problem of hazardous materials was again discussed with the importance of using preventative measures to eliminate inhalation. It is suggested that, when working under dusty conditions, respirators should be used at all times and gloves whenever conditions warrant.”[45]

1957-10. President Sickles, at the International Convention of the Asbestos Heat, Frost and Insulators Union, reported to his union’s delegates that he, “[b]eing well aware of the health hazards in the Asbestos industry, requested authority for the General Executive Board to make a study of the health hazards … that will enable the Board to adopt any policies that will tend to protect the health of our International membership.”[46]

1960. Dr. J. Christopher Wagner published a case series of mesothelioma among persons exposed to crocidolite, in the region of South Africa where crocidolite is mined and milled. After this publication, the causal role of crocidolite became quickly accepted in the scientific and medical community.[47]

1960-1961. Selikoff published two papers on the patho-physiology of asbestosis, based on data from 17 UNARCO workers,[48] obtained from his medico-legal evaluations of the men.[49]

Irving and Celia Selikoff from their 1961 Brazilian visa documents

1961-05. Asbestos insulators’ union discussed collaboration with scientists to discuss lung cancer and other diseases among its membership.[50] Union members, intensely interested in legal redress for compensation, became aware of Selikoff’s research hypothesis in advance of Selikoff’s survey of the members’ smoking habits, which the workers had a motive to under report.

1961-11. The Asbestos insulators’ union’s magazine featured a full page warning of the grim reaper urging insulators to “Wear Your Respirator.”[51] The warning was developed under the guidance of C. V. Krieger of Local No. 28, Safety Superintendent at the Long Beach Naval Shipyard.

1962-07-12. Selikoff visited Asbestos Corporation of America, an intermediary broker of asbestos fibers. In a memorandum Selikoff prepared from his discussions with Wade I. Duym, the general manager of the company, and others, he detailed the widespread use of amphibole asbestos fibers in a variety of products. He noted that amosite was used primarily in the insulation trade, and that it was the asbestos “of choice” for sprayed-on products, high temperature insulating cements and pipecovering (magnesia and calcium silicates). Selikoff described crocidolite, from Africa and Bolivia, as a strong, chemically resistant, relatively inexpensive fiber that was used in asbestos cement products, and in Kent cigarette filters.

1962-09. Selikoff presented to a meeting of the Asbestos Workers, to request their help in conducting his study of insulator mortality and morbidity. Irving Selikoff, “Speech at Asbestos Workers’ Union Annual Meeting,” The Asbestos Worker 8 (Sept. 1962).

1962. Asbestos insulators’ union acknowledged that its leadership has been collaborating with Dr. Irving Selikoff.[52] In September 1962, Selikoff and colleagues began physical examinations of members of the New York and New Jersey locals.[53] `

1962. In a publication for Naval personnel, with virtually no circulation in the general industrial community, the United States government acknowledged that shipyard and on-board exposures greatly exceeded the ACGIH’s then current TLV for asbestos.[54]

1963. Selikoff established the Environmental Sciences Laboratory, later known as the Division of Environmental and Occupational Medicine, in would become the Mt. Sinai School of Medicine’s Department of Community Medicine.

1963-02. The asbestos insulation workers’ union announces that it has begun a large-scale program of examinations for asbestos-related disease in the members of the New York and New Jersey locals.[55]

1964. Selikoff published his first article on cancer in a cohort of union asbestos insulators from New York and New Jersey.[56] Selikoff and his co-authors failed to disclose funding from the union, or the union members’ awareness of the research hypotheses under investigation.

1964. In October 1964, Selikoff organized and co-chaired (with Dr. Jacob Churg) a conference, “The Biological Effects of Asbestos, for the New York Academy of Sciences, in New York City. The conference featured presentations and papers from many international investigators. Several presenters, including Selikoff, documented the prevalent use of amphibole asbestos (both crocidolite and amosite) in the United States.[57]

1965. Papers presented at the 1964 New York Academy of Sciences conference were published in late 1965, in a non-peer reviewed publication, volume 132, of the Annals of the New York Academy of Sciences.

1965. Selikoff testified on behalf of an insulator who claimed that asbestos exposure caused his colorectal cancer.[58] Forty years later, the Institute of Medicine (now the National Academy of Medicine) comprehensively reviewed the extant evidence and announced that the evidence was “suggestive but not sufficient to infer a causal relationship between asbestos exposure and pharyngeal, stomach, and colorectal cancers.”[59] None of Selikoff’s publications, including those on asbestos and colorectal cancer, disclosed his litigation testimonies for claimants.

1966 – 1972. Selikoff continued to testify frequently in civil action and in worker compensation proceedings for claimants who alleged asbestos-related injuries.[60] In 1972, Andrew Haas, President of the asbestos workers’ union thanked Selikoff for his “frequent” expert witness testimony on behalf of union members.[61]

1967-09. In an address to the International Association of Heat and Frost Insulators and Asbestos Workers, Selikoff acknowledged the widespread use of amosite, particularly in shipyards, the absence of lung cancer among non-smoking insulation workers, and the failure of more than 9 out of 10 insulators to wear respirators on dusty jobs. See Irving J. Selikoff, Address to the delegates of the twenty-first convention of the International Association of Heat and Frost Insulators and Asbestos Workers at 8, 9-10, 24 (Chicago, Illinois, Sept. 1967) (“I have yet to see a lung cancer in an asbestos worker who didn’t smoke cigarettes. … “[C]ancer of the lung could be wiped out in your trade if you people wouldn’t smoke cigarettes, period.”).

1968-09. Selikoff “warns” the United States of asbestos hazards that existed and continue to exist in the government’s shipyards.[62] The warning was largely about seeking media attention by Selikoff; the government, and especially the Navy, had long known of asbestos hazards.[63]

1968-05. Selikoff testified that all fibers are equally potent, to Congress in support of a bill that would become the OSH Act.

1968. The Mount Sinai School of Medicine opened in 1968, as part of The City University of New York. The first class in the newly formed medical school had 36 students in the entering class. The school was chartered in 1963. It is now known as the Icahn School of Medicine at Mount Sinai.

1969. Selikoff served as president of the New York Academy of Sciences.

1969-05. Selikoff acknowledged that only four percent of insulators wore a mask despite extensive warnings.[64]

1972. In a published study of variability in the interpretation of chest radiographs, Selikoff was shown consistently to over-read chest radiographs for potential asbestos-related abnormalities, when compared with other pulmonary experts on pneumoconiosis.[65]

1973. Selikoff testified for the government in United States v. Reserve Mining Co., No. 5-72 Civil 19 (D. Minn. Sept. 21, 1973).[66] On September 20, Selikoff testified about the town where Reserve Mining’s taconite mine was located: “I think we ought to have a sign at the entrance to sections of the town ‘Please Close Your Windows Before Driving Through’. I certainly would want to close mine.” When his testimony continued the following day, Selikoff acknowledged that he had been “facetious” in his previous day’s testimony.[67]

1974. After having given “facetious” testimony, Selikoff reduced his testifying activities. Marxist historians Jock McCulloch and Geoffrey Tweedale have falsely suggested that Selikoff “avoided the drama of the courtroom and the role of the expert witness” because of the drain on his time, his desire to avoid antagonizing industry, and his need to prevent discovery of trade union medical files.[68]

1974-05-20. Selikoff’s father, Abraham Selecoff, died in Florida.[69]

1974. Selikoff published a review on asbestos and gastrointestinal cancer, without disclosing his funding from the asbestos insulation union or his receipt of fees for litigation work in which he maintained a causal relationship in advance of any data.[70]

1978-07. The National Cancer Institute (NCI) invited Dr. Hans Weill to co-chair a conference on lung cancer surveillance. Ten days later, the NCI retracted the invitation. When Weill inquired about the reasons for the shoddy treatment, an NCI official (Margaret Sloan) told him that “representatives of organized labor” objected to his participation. Sloan’s superior at NCI stated that Selikoff had raised the question whether the conference’s recommendations would lose credibility if Weill were a co-chair. When asked about his role in this sordid affair, Selikoff equivocated, saying he had “simply” said that “[s]ince Weill was a consultant to the Asbestos Information Center, I didn’t know if this would enhance or detract from hearing all points of view.”[71]

1979-11-05. Barry Castleman, career testifier for the asbestos lawsuit industry, prepared a memorandum to Selikoff to urge him to resist allowing discovery of asbestos worker union members’ knowledge of the hazards of asbestos.[72]

1980-07-29. A Newsday journalist reports that Selikoff is loath to talk about himself, and that he threatens to cut short the interview when asked about his background.[73]

1981. Sir Richard Doll and Professor Richard Peto published a rebuttal to wildly exaggerated asbestos risk assessments based upon Selikoff’s insulator studies.[74]

1984. Selikoff prepared a report on his group’s epidemiologic study of Electric Boat employees, who were engaged in the construction of submarines.[75] The data did not fit the Mt. Sinai Catechism of large increased risks.[76] Selikoff never published these data in a medical journal or a textbook.[77]

1985-03-10. Selikoff retires from Mount Sinai Medical School.[78]

1986-03. Selikoff’s wife, Celia, died.[79]

1987 – 1989. Selikoff’s insulator cohort study data took on an outsize importance in litigation because of plaintiffs’ heavy reliance upon his studies in court cases. When litigants asked for these data, Selikoff consistently refused to share, which necessitated federal court intervention.[80]

1988. Selikoff and William Nicholson prepared a manuscript report of a study of the mortality experience at a New Jersey asbestos product manufacturing plant of Johns Manville.[81] Their report documented the substantial use of crocidolite in various products, and the resulting horrific mesothelioma mortality at this plant. Selikoff never published this crocidolite-exposed cohort, although he tirelessly republished his insulator cohort data repeatedly with the misrepresentation that the insulators were not exposed to crocidolite.

1990-06-07. Selikoff conspired with Ron Motley and others to pervert the course of justice by inviting judges with active asbestos dockets to a one-sided conference on asbestos science, and to pay for their travel and lodging. In his invitation to this ex parte soirée, Selikoff failed to mention that the funding came from plaintiffs’ counsel.[82] Shortly after the Third Circuit spoke on the Mt. Sinai dress rehearsal for the plaintiffs’ asbestos property damage trial case, Judge Jack Weinstein issued a curious mea culpa. Because of a trial in progress, Judge Weinstein did not attend the “Third Wave” conference, but he and a state judge (Justice Helen Freedman) attended an ex parte private luncheon meeting with Dr. Selikoff. Here is how Judge Weinstein described the event:

“But what I did may have been even worse [than Judge Kelly’s conduct that led to his disqualification]. A state judge and I were attempting to settle large numbers of asbestos cases. We had a private meeting with Dr. Irwin [sic] J. Selikoff at his hospital office to discuss the nature of his research. He had never testified and would never testify. Nevertheless, I now think that it was a mistake not to have informed all counsel in advance and, perhaps, to have had a court reporter present and to have put that meeting on the record.”[83]

What is curious is that Judge Weinstein, usually a careful judge and scholar, was so incorrect about Dr. Selikoff’s having never testified. His error could have been avoided by a simple search in the Westlaw or LexisNexis databases. Judge Weinstein’s account points directly to Dr. Selikoff as the source for this falsehood.[84]

1990-10-02. Selikoff wrote to Judge Jack Weinstein and Justice Helen Freedman, presumably after the “regrettable” ex parte luncheon meeting, to hold forth with his views on the health effects of occupational and para-occupational exposure to asbestos.

1992-05-20. Selikoff died several months before the U.S. Court of Appeals for the Third Circuit condemned the Selikoff-Motley conspiracy.[85]

2013. Follow up of the national insulator cohort fails to support multiplicative interaction between smoking and asbestos for lung cancer outcomes in the absence of asbestosis.[86]

[1] Rachel Maines, Asbestos and Fire: Technological Tradeoffs and the Body at Risk 155 (2005) (“charming, courageous, and compassion medical professional with more charisma than credentials”).

[2] Jock McCulloch & Geoffrey Tweedale, “Shooting the messenger: the vilification of Irving J. Selikoff,” 37 Internat’l J. Health Services 619 (2007); “Scientific Prestige, Reputation, Authority & The Creation of Scientific Dogmas” (Oct. 4, 2014); David Egilman, Geoffrey Tweedale, Jock McCulloch, William Kovarik, Barry Castleman, William Longo, Stephen Levin, and Susanna Rankin Bohme, “P.W.J. Bartrip’s Attack on Irving J. Selikoff,” 46 Am. J. Indus. Med. 151, 152 (2004) [Egilman (2004)].

[3] 84 Stat. 1590, et seq., 29 U.S.C. § 651, et seq.

[4] “The Legacy of Irving Selikoff & Wicked Wikipedia” (Mar. 1, 2015). See also “Hagiography of Selikoff” (Sept. 26, 2015); “Historians Should Verify Not Vilify or Abilify – The Difficult Case of Irving Selikoff” (Jan. 4, 2014).

[5] See, e.g., Philip Landrigan, “Stephen Levin, MD, honored with the Collegium Ramazzini’s Irving J. Selikoff Memorial Award in 2009.”

[6] See Wikipedia, “Irving Selikoff” (last visited Dec. 4, 2018).

[7] Kings County Birth Certificate no. 4595 (Jan. 15, 1915). His family later adopted the surname Selikoff. Irving Selikoff’s social security records list his father as Abraham Selikoff and his mother as Tillie Katz.

[8] Abraham Selecoff World War II draft registration, serial no. U1750.

[9] Abraham Selecoff World War II draft registration, serial no. U1750.

[10] See “The Asbestos Menace,” The Asbestos Worker 9-11 (Sept. 1930).

[11] Alice Hamilton, “Industrial Poisons,” American Federationist (1936). This journal was “The Official Magazine of the American Federation of Labor.”

[12] Bronx marriage certificate no. 8246 (1936); Bronx marriage license no. 8652 (1936). Irving’s parents were listed as Abraham Selikoff and Tillie Katz. His residence was at 109 W. 112^th Street. Lydia was listed as the daughter of Mendel Kapilian and Bessie Weller. Irving and Lydia were divorced sometime between 1939 and 1941. The marriage certificates stated Lydia to have been 21 years old. Her Social Security records (SSN 112-052-2143), however, gave her birth date as April 22, 1917, (making her 19), and subsequent marriage names of Quint and Teichner.

[13] This and other details of Selikoff’s medical education come from Peter Bartrip’s exposé. Although Bartrip’s research was attacked for its allegedly gratuitous attacks on Selikoff’s research prowess, Bartrip’s account of Selikoff’s medical education in Scotland, Australia, and the United States has gone largely unrebutted, and must for the present be accepted. Peter W.J. Bartrip, “Irving John Selikoff and the Strange Case of the Missing Medical Degrees,” 58 J. History Med. 8 (2003) [Bartrip 2003]; Peter Bartrip, “Around the World in Nine Years: A Medical Education Revisited,” 59 J. History of Med. 135 (2004). One group of plaintiffs’ expert witnesses took Bartrip to task for not disclosing that he had served as a defense expert witness, but none of the complainants disclosed their substantial testimonial adventures for the litigation industry! While making some interesting points, these critics of Bartrip did not really contest his historical work on Selikoff: “Bartrip’s critiques of Anderson’s College (AC) and Middlesex University School of Medicine (MSUM) may be accurate, but are beside the point.” David Egilman, Geoffrey Tweedale, Jock McCulloch, William Kovarik, Barry Castleman, William Longo, Stephen Levin, and Susanna Rankin Bohme, “P.W.J. Bartrip’s Attack on Irving J. Selikoff,” 46 Am. J. Indus. Med. 151, 152 (2004).

[14] Bartrip 2003, at 15 & n.44-51.

[15] Bartrip 2003, at 17 & n.54-55.

[16] Bartrip 2003 at 18.

[17] William J. Nicholson & Alvin S. Teirstein, “Remembering Irving J. Selikoff,” 61 Mt. Sinai J. Med. 500 (1994) [Nicholson & Teirstein]. This account seems doubtful; Selikoff would not have an M.D. degree until 1943, and then from a school that was about to lose its accreditation.

[18] See Stephen Rushmore, “Middlesex University School of Medicine,” 230 New Engl. J. Med. 217 (1944).

[19] Anthony Seaton, “The Strange Case of Irving Selikoff,” 60 Occup. Med. 53 (2010); Peter W.J. Bartrip, “Irving John Selikoff and the Strange Case of the Missing Medical Degrees,” 58 J. History Med. 3, 27 & n.88-92 (2003) [cited as Bartrip].

[20] Bartrip 2003 at 22.

[21] Bartrip 2003 at 22.

[22] Bartrip 2003 at 21.

[23] New York County marriage license no. 3879 (Feb. 2, 1946). Celia had been married to Nathan Michaels in 1937. Manhattan Marriage License no. 21454 (1937).

[24] Nicholson & Teirstein.

[25] Manhattan Marriage License 21527 (1947).

[26] City Directory of Paterson, New Jersey at p. 218 (1949).

[27] Manhattan Telephone Directory (1949).

[28] George W. Conk, “Deadly Dust: Occupational Health and Safety as a Driving Force in Workers’ Compensation Law and the Development of Tort Doctrine,” 69 Rutgers L. Rev. 1140, 1154 & n. 136 (2017).

[29] Jon L. Gelman, “History of Asbestos and the Law” (Jan. 2, 2001). Carl Gelman was a life-long Paterson resident. His legal practice specialized in workers’ compensation, and he chaired state bar association’s workers’ compensation section for several years. His practice represented claimants from 1936, until his retirement in 1986. In the mid-1970s, with Karl Asch, Gelman’s firm sued asbestos suppliers to Raybestos Manhattan on behalf of multiple employees.Gelman died on February 24, 2009. “Obituary for Carl Gelman,” The Record/Herald News (Mar. 16, 2009); “163 Who Had Jobs at Raybestos Sue,” N.Y. Times (May 7, 1975). The suit for $326 million settled for $15.5 million.

[30] Barry I. Castleman, Asbestos: Medical and Legal Aspects at 142 (1984); Matt Mauney, “Unarco,” Mesothelioma Center (Nov. 2018). Of course, there were no data to support this claim in 1952. Selikoff was publically and positionally committed to his causal hypothesis as a conclusion well in advance of conducting any studies or having any supporting data.

[31] Irving J. Selikoff, E. Cuyler Hammond, and Jacob Churg, “The carcinogenicity of amosite asbestos,” 25 Arch. Envt’l Health 183 (1972). This 1972 publication was the first epidemiologic study on the carcinogenicity of amosite.

[32] David E. Lilienfeld, “The Silence: The Asbestos Industry and Early Occupational Cancer Research – A Case Study,” 81 Am. J. Pub. Health 791 (1991).

[33] Irving J. Selikoff, Edward H. Robitzek, and George G. Ornstein, “Treatment of pulmonary tuberculosis with hydrazine derivatives of isonicotinic acid,” 150 J. Am. Med. Ass’n 973 (1952).

[34] “TB Milestone,” Life (Mar. 3, 1952).

[35] Irving J. Selikoff, “Asbestos in Paterson, New Jersey and Tyler, Texas – A Tale of Two Cities,” Transcript of Lecture (Houston, Texas, Oct. 11, 1979).

[36] See “Isoniazid for treating tuberculosis.”

[37] See “Medical Horizons,” Broadcasting * Telecasting at 14 (Nov. 21, 1955) (describing Selikoff as a plodding presenter). See also “Irving Selikoff – Media Plodder to Media Zealot” (Sept. 9, 2014).

[38] Richard Doll, “Mortality from Lung Cancer in Asbestos Workers,” 12 Br. J. Indus. Med. 81 (1955).

[39] Selikoff letter to Thomas Mancuso (Mar. 30, 1989).

[40] Herbert K. Abrams, “Cancer in Industry,” American Federationist (1955). Dr. Abrams’ article was republished in many union newsletters. See Herbert K. Abrams, “Cancer in Industry,” 69 The Painter & Decorator 15, 16 (Mar. 1955); see also Lester Breslow, LeMar Hoaglin, Gladys Rasmussen & Herbert K. Abrams, “Occupations and Cigarette Smoking as Factors in Lung Cancer,” 44 Am. J. Pub. Health. 171, 171 (1954).

[41] A. C. & S., Inc. v. Asner, 104 Md. App. 608, 633, 657 A.2d 379 (Md. Ct. Spec. App. 1995).

[42] Paul Brodeur, Outrageous Misconduct: The Asbestos Industry on Trial 236-39 (1985). According to Brodeur’s account, Johns-Manville’s defense was clever by halves. By claiming that the company had no knowledge that asbestos could be harmful to applicators such as LeGrand, the company deprived itself of assumption-of-risk and contributory negligence defenses. The company also set itself up to be brutally contradicted by internal documents and communications that showed an awareness of hazards to pipecoverers. By the time J-M understood that the question of responsibility required acknowledging potential hazards that were in the control of the contractors themselves (such as the use of proper respirators and the like), the company filed for bankruptcy.

[43] Greg Gordon, “Health studies drew little action,” Star Tribune (Nov. 9, 2003); Wondie Russell, “Memorandum re Frederick LeGrande v. J-M Products Corp,”(Nov. 3, 1982).

[44] The Freehold Transcript and The Monmouth Inquirer (Freehold, New Jersey) at 17 (Thurs., Oct. 22, 1959).

[45] Asbestos Worker (July 1957).

[46] The Asbestos Worker at 1 (Oct, 1957) (reporting on the Asbestos Workers’ 19th General Convention).

[47] See J. Christopher Wagner, C.A. Sleggs, and Paul Marchand, “Diffuse pleural mesothelioma and asbestos exposure in the North Western Cape Province,” 17 Br. J. Indus. Med. 260 (1960); J. Christopher Wagner, “The discovery of the association between blue asbestos and mesotheliomas and the aftermath,” 48 Br. J. Indus. Med. 399 (1991).

[48] Arthur M. Langer, “Asbestos Studies in the Environmental Sciences Laboratory Mount Sinai School of Medicine 1965 – 1985: Investigations Reflecting State-of-the-Art; Contributions to the Understanding of Asbestos Medicine” (Unpublished MS, Nov. 7, 2008).

[49] Alvin S. Tierstein, A. Gottlieb, Mortimer E. Bader, Richard A. Bader & Irving Selikoff, “Pulmonary mechanics in asbestosis of the lungs,” 8 Clin. Res. 256 (1960); Mortimer E. Bader, Richard A. Bader & Irving Selikoff, “Pulmonary function in asbestosis of the lung; an alveolar-capillary block syndrome, 30 Am. J. Med. 235 (1961).

[50] Asbestos Worker (May 1961) (“The subject matter of Health Hazards was discussed and President Sickles reported on the possibility of an early meeting with people connected with the Medical Association for the purpose of running various tests on certain materials used by our membership in order to determine the extent of their contribution to lung cancer, silicosis, asbestosis, tuberculosis, etc.”).

[51] 15 The Asbestos Worker at 29 (Nov. 1961).

[52] Asbestos Worker (May 1962) (“President Sickles advised the Board as to a meeting which had been held with Vice President Rider and a Dr. Irving Selikoff, of the Paterson Clinic in connection with our issue on Health Hazards and the Committee on Health Hazards with the approval of the Board instructed President Sickles to continue his efforts in this direction.”)

[53] Asbestos Worker at 25 (Feb. 1963).

[54] Capt. H.M. Robbins & William T. Marr, “Asbestosis,” 19 Safety Review 10 (1962) (noting that asbestos dust counts of 200 million particles per cubic foot were not uncommon during insulation ripouts onboard naval vessels).

[55] “Progress Report on Health Hazards,” 16 The Asbestos Worker 25 (Feb. 1963) (the examination were arranged by President Carl Sickles, Vice-President Hugh Mulligan and Vice-President George Rider of the Health Hazards Committee).

[56] Irving J. Selikoff, Jacob Churg, and E. Cuyler Hammond, “Asbestos Exposure and Neoplasia,” 188 J. Am. Med. Ass’n 22 (1964).

[57] Irving J. Selikoff, Jacob Churg, E. Cuyler Hammond, “The Occurrence of Asbestosis among Insulation Workers in the United States,” 132 Ann. N.Y. Acad. Sci. 139, 142 (1965) (“In later specimens so obtained, crocidolite has also been found. Moreover, materials used for ship insulation, while containing the same amounts of asbestos as above, began in 1934 to have significant amounts of amosite in addition to chrysotile, because of the lighter weight of the material.”); Harrington, “Chemical Studies of Asbestos,” 132 Ann. N.Y. Acad. Sci. 31, 41 (1965) (reporting the finding of chrysotile and crocidolite asbestos in equal proportions in specimens of 85% magnesia pipe-covering sections); N.W. Hendry, “The Geology, Occurrences, and Major Uses of Asbestos 132 Annals N.Y. Acad. Sci. 12, 19 (1965) (reporting that, in 1963, the U.S. used 22,000 tons of amosite in manufactured products, and 17,000 tons of crocidolite in acid-resistent filters, packings, insulations, and certain types of lagging. United States Department of Commerce statistics show that for the years 1957 to 1962, more crocidolite was used in the United States than was amosite. In 1962, the use of blue was twice as great as that for brown. 132 Ann. N.Y. Acad. Sci. at 753, Table 17 (1965); see also id. at 762, Table 23 (1965) (South African blue fiber imports exceeded brown fiber imports, starting about 1954). See also “Selikoff and the Mystery of the Disappearing Amphiboles” (Dec. 10, 2010); James R. Millette, Steven Compton, and Christopher DePasquale, “Microscopical Analyses of Asbestos-Cement Pipe and Board,” 66 The Microscope 3 (2018) (reporting analyses of cement formulations with substantial crocidolite).

[58] “Health Hazard Progress Notes,”16 The Asbestos Worker 13 (May 1966) (“A recent decision has widened the range of compensable diseases for insulation workers even further. A member of Local No. 12. Unfortunately died of a cancer of the colon. Dr. Selikoff reported to the compensation court that his research showed that these cancers of the intestine were at least three times as common among the insulation workers as in men of the same age in the general population. Based upon Dr. Selikoff’s testimony, the Referee gave the family a compensation award, holding that the exposure to many dusts during employment was responsible for the cancer. The insurance company appealed this decision. A special panel of the Workman’s Compensation Board reviewed the matter and agreed with the Referee’s judgement and affirmed the compensation award. This was the first case in which a cancer of the colon was established as compensable and it is likely that this case will become an historical precedent.”)

[59] Jonathan Samet, et al., eds., Institute of Medicine Review of Asbestos: Selected Cancers (2006); see also Richard Doll & Julian Peto, Asbestos: Effects on health of exposure to asbestos 8 (1985) (“In particular, there are no grounds for believing that gastrointestinal cancers in general are peculiarly likely to be caused by asbestos exposure.”).

[60] “Selikoff and the Mystery of the Disappearing Testimony” (Dec. 3, 2010); see, e.g., Barros v. United States, 147 F.Supp. 340, 343-44 (E.D.N.Y. 1957) (noting that Dr. Selikoff testified for seaman suing for maintenance and cure as a result of a slip and fall; finding for respondent against libelant); DeRienzo v. Passaic Fire Dept., reported in The News (Paterson, New Jersey) at 27 (Feb. 14, 1957) (Selikoff was a witness for the claimant); Bradshaw v. Twin City Insulation Co. Ltd., Indus. Ct. Indiana, Claim No. O.D.1454 (Oct. 14, 1966); Bradshaw v. Johns-Manville Sales Corp., Civ. Action No. 29433, E. D. Mich. S. Div. (July 6, 1967); Bambrick v. Asten Hill Mfg. Co., Pa. Cmwlth. Ct. 664 (1972); Tomplait v. Combustion Engineering Inc.., E. D. Tex. Civ. Action No. 5402 (March 4, 1968); Babcock & Wilcox, Inc. v. Steiner, 258 Md. 468, 471, 265 A.2d 871 (1970) (affirming workman compensation award for asbestosis); Rogers v. Johns-Manville Products Corp., Cir. Ct. Mo., 16th Jud. Cir., Div. 9, Civ. Action No. 720,071 (Feb. 19, 1971); Utter v. Asten-Hill Mfg. Co., 453 Pa. 401 (1973); Karjala v Johns-Manville Products Corp., D. Minn., Civ. Action Nos. 5–71 Civ. 18, and Civ. 40 (Feb. 8, 1973); Culp Industrial Insulation v. Commonwealth of Pennsylvania Workmen’s Compensation Appeal Board, 57 Pa. Commonwealth Ct. 599, 601-602 (1981).

[61] Andrew Haas, Comments from the General President, 18 Asbestos Worker (Nov. 1972); see also Peter W.J. Bartrip, “Irving John Selikoff and the Strange Case of the Missing Medical Degrees,” 58 J. History Med. 3, 27 & n.88-92 (2003) (citing Haas).

[62] Thomas O’Toole, “U.S. Warned of Asbestos Peril,” Wash. Post. A4 (Dec. 4, 1968).

[63] “The United States Government’s Role in the Asbestos Mess” (Jan. 31, 2012). See also Kara Franke & Dennis Paustenbach, “Government and Navy knowledge regarding health hazards of Asbestos: A state of the science evaluation (1900 to 1970),” 23(S3) Inhalation Toxicology 1 (2011); Capt. H.M. Robbins & W.T. Marr, “Asbestosis,” Safety Review (Oct. 1962); See also Walter Olson, “Asbestos awareness pre-Selikoff,” (Oct. 19, 2007).

[64] “Green Sheet,” The Asbestos Worker (May 1969).

[65] See Charles E. Rossiter, “Initial repeatability trials of the UICC/Cincinnati classification of the radiographic appearances of pneumoconioses,” 29 Brit. J. Indus. Med. 407 (1972) (among physician readers of chest radiographs, Selikoff was at the extreme of least likely to call a film normal (less than half the average of all readers), and the most likely to interpret films to show excess profusion of small irregular linear densities). See “Selikoff and the Mystery of the Disappearing Asbestosis” (Dec. 6, 2010). The unions, of course, interested in maximizing compensation for their members loved Selikoff’s over-reading of chest films. Selikoff’s colleagues (Ruth Lilis) routinely teased Selikoff about not being able to read chest radiographs. Selikoff was rumored to have taken and failed the NIOSH B-Reader examination, a rumor which needs to be resolved by a FOIA request.

[66] United States v. Reserve Mining Co. See United States v. Reserve Mining Co., 56 F.R.D. 408 (D.Minn.1972); Armco Steel Corp. v. United States, 490 F.2d 688 (8th Cir. 1974); United States v. Reserve Mining Co., 380 F.Supp. 11 (D.Minn.1974); Reserve Mining Co. v. United States, 498 F.2d 1073 (8th Cir. 1974); Minnesota v. Reserve Mining Co., 418 U.S. 911 (1974); Minnesota v. Reserve Mining Co., 419 U.S. 802 (1974); United States v. Reserve Mining Co., 394 F.Supp. 233 (D.Minn.1974); Reserve Mining Co. v. Environmental Protection Agency, 514 F.2d 492 (8th Cir. 1975); Minnesota v. Reserve Mining Co., 420 U.S. 1000, 95 S.Ct. 1441, 43 L.Ed.2d 758 (1975); Reserve Mining Co. v. Lord, 529 F.2d 181 (8th Cir. 1976); United States v. Reserve Mining Co., 408 F.Supp. 1212 (D.Minn.1976); United States v. Reserve Mining Co., 412 F.Supp. 705 (D.Minn.1976); United States v. Reserve Mining Co., 417 F.Supp. 789 (D.Minn.1976); United States v. Reserve Mining Co., 417 F.Supp. 791 (D.Minn.1976); 543 F.2d 1210 (1976).

[67] Robert V. Bartlett, The Reserve Mining Controversy: Science, Technology, and Environmental Quality 140-41 (1980) (describing Selikoff’s testimony).

[68] Jock McCulloch & Geoffrey Tweedale, Defending the Indefensible: The Global Asbestos Industry and its Fight for Survival: The Global Asbestos Industry and its Fight for Survival 95 & n.36 (2008). These authors ignored other reasons Selikoff later stood down from the witness chair: his self-serving insistence upon the importance of his own research detracted from the work of previous authors (e.g., Sir Richard Doll, J. Christopher Wagner, et al.) in litigation of personal injury claims of asbestos health effects. Plaintiffs’ counsel needed to push back the dates of first knowledge of asbestos health effects well before Selikoff’s first insulator study in 1964. The litigation industry needed Selikoff to continue to generate publicity, and to stop testifying. Selikoff surely must have had some concerns about how further testifying would eventually lead to questions about his credentials. Furthermore, Selikoff had an entire generation of younger, less politically visible colleagues at Mt. Sinai to fill the ranks of expert witnesses for the litigation industry (Miller, Levin, Nicholson, Lillis, Daum, Anderson, Frank, et al.).

[69] Obituary for Abraham Selecoff, The Miami Herald (May 22, 1974).

[70] Irving J. Selikoff, “Epidemiology of gastrointestinal cancer,” 9 Envt’l Health Persp. 299 (1974) (arguing for causal conclusion between asbestos and all gastrointestinal cancers).

[71] Nicholas Wade, “The Science and Politics of a Disinvitation,” 201 Science 892 (1978) (commenting that the NCI was negligent in failing to evaluate the ad hominem opinions given to it by Selikoff).

[72] See “The Selikoff – Castleman Conspiracy” (Mar. 13, 2011); “What Happens When Historians Have Bad Memories” (Mar. 15, 2014); “Castleman-Selikoff – Can Their Civil Conspiracy Survive Death?” (Dec. 3, 2018). In 2014, Castleman testified that he has no recollection of the memorandum, but he did not deny that had written it.

[73] B.D. Colen, “Knowing When the Chemistry is Right,” Newsday (Suffolk Edition) (Melville, New York) at 85 (Tue., July 29, 1980).

[74] See Richard Doll & Richard Peto, “The causes of cancer: quantitative estimates of avoidable risks of cancer in the United States today,” 66 J. Nat’l Cancer Inst. 1191 (1981).

[75] Irving Selikoff & William Nicholson, “Mortality Experience of 1,918 Employees of the Electric Boat Company, Groton, Connecticut January 1, 1967 – June 30, 1978” (Jan. 27, 1984).

[76] “The Mt. Sinai Catechism” (June 5, 2013).

[77] “Irving Selikoff and the Right to Peaceful Dissembling” (June 5, 2013).

[78] Leo H. Caney, “Noted Cancer Researcher Altering Role,” N.Y. Times (Mar. 10, 1985).

[79] Celia Selikoff Social Security Records, SSN 064-12-6401. Celia was born on Sept. 12, 1908.

[80] A New York state trial court initially sided with Selikoff over this subpoena battle. In re R.J. Reynolds Tobacco Co., 136 Misc.2d 282, 518 N.Y.S.2d 729 (N.Y. Sup. Ct., N.Y. Cty. 1987). The federal court subsequently required Selikoff to honor another litigant’s subpoena. In re American Tobacco Co., 866 F.2d 552 (2d Cir. 1989).

[81] William J. Nicholson & Irving J. Selikoff, “Mortality experience of asbestos factory workers; effect of differing intensities of asbestos exposure”: unpublished manuscript produced in litigation (1988) (“[O]ther asbestos varieties (amosite, crocidolite, anthophyllite) were also used for some products. In general, chrysotile was used for textiles, roofing materials, asbestos cements, brake and friction products, fillers for plastics, etc.; chrysotile with or without amosite for insulation materials; chrysotile and crocidolite for a variety of asbestos cement products.”).

[82] In re School Asbestos Litigation, 977 F.2d 764 (3d Cir. 1992). See Cathleen M. Devlin, “Disqualification of Federal Judges – Third Circuit Orders District Judge James McGirr Kelly to Disqualify Himself So As To Preserve ‘The Appearance of Justice’ Under 28 U.S.C. § 455 – In re School Asbestos Litigation (1992),” 38 Villanova L. Rev. 1219 (1993); Bruce A. Green, “May Judges Attend Privately Funded Educational Programs? Should Judicial Education Be Privatized?: Questions of Judicial Ethics and Policy,” 29 Fordham Urb. L.J. 941, 996-98 (2002).

[83] Jack B. Weinstein, “Learning, Speaking, and Acting: What Are the Limits for Judges?” 77 Judicature 322, 326 (May-June 1994) (emphasis added). Judge Weinstein’s false statement that Selikoff “had never testified” not only reflects an incredible and uncharacteristic naiveté by a distinguished evidence law scholar, but the false statement was in a journal, Judicature, which was widely circulated to state and federal judges. The source of the lie appears to have been Selikoff himself in the ethically dodgy ex parte meeting with judges actively presiding over asbestos personal injury cases.

[84] Jack B. Weinstein, “Learning, Speaking, and Acting: What Are the Limits for Judges?” 77 Judicature 322, 326 (May-June 1994). The point apparently weighed on Judge Weinstein’s conscience. He repeated his mea culpa almost verbatim, along with the false statement about Selikoff’s never having testified, in a law review article in 1994, and then incorporated the misrepresentation into a full-length book. See Jack B. Weinstein, “Limits on Judges’ Learning, Speaking and Acting – Part I- Tentative First Thoughts: How May Judges Learn?” 36 Ariz. L. Rev. 539, 560 (1994) (“He [Selikoff] had never testified and would never testify.”); Jack B. Weinstein, Individual Justice in Mass Tort Litigation: The Effect of Class Actions, Consolidations, and other Multi-Party Devices 117 (1995) (“A court should not coerce independent eminent scientists, such as the late Dr. Irving Selikoff, to testify if, like he, they prefer to publish their results only in scientific journals.”).

[85] Social Security records for Irving John Selikoff, social sec. no. 085-16-1882. See Bruce Lambert, “Irving J. Selikoff Is Dead at 77; TB Researcher Fought Asbestos,” N.Y. Times (May 22, 1992).

[86] Steve Markowitz, Stephen Levin, Albert Miller, and Alfredo Morabia, “Asbestos, Asbestosis, Smoking and Lung Cancer: New Findings from the North American Insulator Cohort,” Am. J. Respir. & Critical Care Med. (2013)).

Posted in Asbestos, Expert Witnesses, State of the Art | 5 Comments »

Reference Manual – Desiderata for 4th Edition – Part IV – Confidence Intervals

February 10th, 2023

Putting aside the idiosyncratic chapter by the late Professor Berger, most of the third edition of the Reference Manual presented guidance on many important issues. To be sure, there are gaps, inconsistencies, and mistakes, but the statistics chapter should be a must-read for federal (and state) judges. On several issues, especially statistical in nature, the fourth edition could benefit from an editor to ensure that the individual chapters, written by different authors, actually agree on key concepts. One such example is the third edition’s treatment of confidence intervals.[1]

The “DNA Identification” chapter noted that the meaning of a confidence interval is subtle,[2] but I doubt that the authors, David Kaye and George Sensabaugh, actually found it subtle or difficult. In the third edition’s chapter on statistics, David Kaye and co-author, the late David A. Freedman, gave a reasonable definition of confidence intervals in their glossary:

“confidence interval. An estimate, expressed as a range, for a parameter. For estimates such as averages or rates computed from large samples, a 95% confidence interval is the range from about two standard errors below to two standard errors above the estimate. Intervals obtained this way cover the true value about 95% of the time, and 95% is the confidence level or the confidence coefficient.”[3]

Intervals, not the interval, which is correct. This chapter made clear that it was the procedure of obtaining multiple samples with intervals that yielded the 95% coverage. In the substance of their chapter, Kaye and Freedman are explicit about how intervals are constructed, and that:

“the confidence level does not give the probability that the unknown parameter lies within the confidence interval.”[4]

Importantly, the authors of the statistics chapter named names; that is, they cited some cases that butchered the concept of the confidence interval.[5] The fourth edition will have a more difficult job because, despite the care taken in the statistics chapter, many more decisions have misstated or misrepresented the meaning of a confidence interval.[6] Citing more cases perhaps will disabuse federal judges of their reliance upon case law for the meaning of statistical concepts.

The third edition’s chapter on multiple regression defined confidence interval in its glossary:

“confidence interval. An interval that contains a true regression parameter with a given degree of confidence.”[7]

The chapter avoided saying anything obviously wrong only by giving a very circular definition. When the chapter substantively described a confidence interval, it ended up giving an erroneous one:

“In general, for any parameter estimate b, the expert can construct an interval around b such that there is a 95% probability that the interval covers the true parameter. This 95% confidence interval is given by: b ± 1.96 (SE of b).”[8]

The formula provided is correct, but the interpretation of a 95% probability that the interval covers the true parameter is unequivocably wrong.[9]

The third edition’s chapter by Shari Seidman Diamond on survey research, on the other hand, gave an anodyne example and a definition:

“A survey expert could properly compute a confidence interval around the 20% estimate obtained from this sample. If the survey were repeated a large number of times, and a 95% confidence interval was computed each time, 95% of the confidence intervals would include the actual percentage of dentists in the entire population who would believe that Goldgate was manufactured by the makers of Colgate.

* * * *

Traditionally, scientists adopt the 95% level of confidence, which means that if 100 samples of the same size were drawn, the confidence interval expected for at least 95 of the samples would be expected to include the true population value.”[10]

Similarly, the third edition’s chapter on epidemiology correctly defined the confidence interval operationally as a process of iterative intervals that collectively cover the true value in 95% of all the intervals:

“A confidence interval provides both the relative risk (or other risk measure) found in the study and a range (interval) within which the risk likely would fall if the study were repeated numerous times.”[11]

Not content to leave it well said, the chapter’s authors returned to the confidence interval and provided another, more problematic definition, a couple of pages later in the text:

“A confidence interval is a range of possible values calculated from the results of a study. If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the same population.”[12]

The first sentence refers to “a study”; that is, one study, one range of values. The second sentence then tells us that “the range” (singular, presumably referring back to the single “a study”), will capture 95% of the results from many resamplings from the same population. Now the definition is not framed with respect to the true population parameter, but the results from many other samples. The authors seem to have given the first sample’s confidence interval the property of including 95% of all future studies, and that is incorrect. From reviewing the case law, courts remarkably have gravitated to the second, incorrect definition.

The glossary to the third edition’s epidemiology chapter clearly, however, runs into the ditch:

“confidence interval. A range of values calculated from the results of a study within which the true value is likely to fall; the width of the interval reflects random error. Thus, if a confidence level of .95 is selected for a study, 95% of similar studies would result in the true relative risk falling within the confidence interval.”[13]

Note that the sentence before the semicolon talked of “a study” with “a range of values,” and that there is a likelihood of that range including the “true value.” This definition thus used the singular to describe the study and to describe the range of values. The definition seemed to be saying, clearly but wrongly, that a single interval from a single study has a likelihood of containing the true value. The second full sentence ascribed a probability, 95%, to the true relative risk’s falling within “the interval.” To point out the obvious, “the interval,” is singular, and refers back to “a study,” also singular. At best, this definition was confusing; at worst, it was wrong.

The Reference Manual has a problem beyond its own inconsistencies, and the refractory resistance of the judiciary to statistical literacy. There are any number of law professors and even scientists who have held out incorrect definitions and interpretations of confidence intervals. It would be helpful for the fourth edition to caution its readers, both bench and bar, to the prevalent misunderstandings.

Here, for instance, is an example of a well-credentialed statistician, who gave a murky definition in a declaration filed in federal court:

“If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the same population.”[14]

The expert witness correctly identifies the repeated sampling, but specifies a 95% probability to “the range,” which leaves unclear whether it is the range of all intervals or “a 95% confidence interval,” which is in the antecedent of the statement.

Much worse was a definition proffered in a recent law review article by well-known, respected authors:

“A 95% conﬁdence interval, in contrast, is a one-sided or two-sided interval from a data sample with 95% probability of bounding a ﬁxed, unknown parameter, for which no nondegenerate probability distribution is conceived, under speciﬁed assumptions about the data distribution.”[15]

The phrase “for which no nondegenerate probability distribution is conceived,” is unclear as to whether the quoted phrase refers to the conﬁdence interval or to the unknown parameter. It seems that the phrase modifies the noun closest to it in the sentence, the “ﬁxed, unknown parameter,” which suggests that these authors were simply trying to emphasize that they were giving a frequentist interpretation and not conceiving of the parameter as a random variable as Bayesians would. The phrase “no nondegenerate” appears to be a triple negative, since a degenerate distribution is one that does not have a variation. The phrase makes the deﬁnition obscure, and raises questions what is being excluded by the phrase.

The more concerning aspect of the quoted footnote is its obfuscation of the important distinction between the procedure of repeatedly calculating conﬁdence intervals (which procedure has a 95% success rate in the long run) and the probability that any given instance of the procedure, in a single confidence interval, contains the parameter. The latter probability is either zero or one.

The deﬁnition’s reference to “a” conﬁdence interval, based upon “a” data sample, actually leaves the reader with no way of understanding the deﬁnition to be referring to the repeated process of sampling, and the set of resulting intervals. The upper and lower interval bounds are themselves random variables that need to be taken into account, but by referencing a single interval from a single data sample, the authors misrepresent the confidence interval and invite a Bayesian interpretation.[16]

Sadly, there is a long tradition of scientists and academics in giving errant definitions and interpretations of the confidence interval.[17] Their error is not harmless because they invite the attribution of a high level of probability to the claim that the “true” population measure is within the reported confidence interval. The error encourages readers to believe that the confidence interval is not conditioned upon the single sample result, and it misleads readers into believing that not only random error, but systematic and data errors are accounted for in the posterior probability.[18]

[1] “Confidence in Intervals and Diffidence in the Courts” (Mar. 4, 2012).

[2] David H. Kaye & George Sensabaugh, “Reference Guide on DNA Identification Evidence” 129, 165 n.76.

[3] David H. Kaye & David A. Freedman, “Reference Guide on Statistics” 211, 284-5 (Glossary).

[4] Id. at 247.

[5] Id. at 247 n.91 & 92 (citing DeLuca v. Merrell Dow Pharms., Inc., 791 F. Supp. 1042, 1046 (D.N.J. 1992), aff’d, 6 F.3d 778 (3d Cir. 1993); SmithKline Beecham Corp. v. Apotex Corp., 247 F. Supp. 2d 1011, 1037 (N.D. Ill. 2003), aff’d on other grounds, 403 F.3d 1331 (Fed. Cir. 2005); In re Silicone Gel Breast Implants Prods. Liab. Litig, 318 F. Supp. 2d 879, 897 (C.D. Cal. 2004) (“a margin of error between 0.5 and 8.0 at the 95% confidence level . . . means that 95 times out of 100 a study of that type would yield a relative risk value somewhere between 0.5 and 8.0.”).

[6] See, e.g., Turpin v. Merrell Dow Pharm., Inc., 959 F.2d 1349, 1353–54 & n.1 (6th Cir. 1992) (erroneously describing a 95% CI of 0.8 to 3.10, to mean that “random repetition of the study should produce, 95 percent of the time, a relative risk somewhere between 0.8 and 3.10”); American Library Ass’n v. United States, 201 F.Supp. 2d 401, 439 & n.11 (E.D.Pa. 2002), rev’d on other grounds, 539 U.S. 194 (2003); Ortho–McNeil Pharm., Inc. v. Kali Labs., Inc., 482 F.Supp. 2d 478, 495 (D.N.J.2007) (“Therefore, a 95 percent confidence interval means that if the inventors’ mice experiment was repeated 100 times, roughly 95 percent of results would fall within the 95 percent confidence interval ranges.”) (apparently relying party’s expert witness’s report), aff’d in part, vacated in part, sub nom. Ortho McNeil Pharm., Inc. v. Teva Pharms Indus., Ltd., 344 Fed.Appx. 595 (Fed. Cir. 2009); Eli Lilly & Co. v. Teva Pharms, USA, 2008 WL 2410420, *24 (S.D. Ind. 2008) (stating incorrectly that “95% percent of the time, the true mean value will be contained within the lower and upper limits of the confidence interval range”); Benavidez v. City of Irving, 638 F.Supp. 2d 709, 720 (N.D. Tex. 2009) (interpreting a 90% CI to mean that “there is a 90% chance that the range surrounding the point estimate contains the truly accurate value.”); Pritchard v. Dow Agro Sci., 705 F. Supp. 2d 471, 481, 488 (W.D. Pa. 2010) (excluding Dr. Bennet Omalu who assigned a 90% probability that an 80% confidence interval excluded relative risk of 1.0), aff’d, 430 F. App’x 102 (3d Cir.), cert. denied, 132 S. Ct. 508 (2011); Estate of George v. Vermont League of Cities and Towns, 993 A.2d 367, 378 n.12 (Vt. 2010) (erroneously describing a confidence interval to be a “range of values within which the results of a study sample would be likely to fall if the study were repeated numerous times”); Garcia v. Tyson Foods, 890 F. Supp. 2d 1273, 1285 (D. Kan. 2012) (quoting expert witness Robert G. Radwin, who testified that a 95% confidence interval in a study means “if I did this study over and over again, 95 out of a hundred times I would expect to get an average between that interval.”); In re Chantix (Varenicline) Prods. Liab. Litig., 889 F. Supp. 2d 1272, 1290n.17 (N.D. Ala. 2012); In re Zoloft Products, 26 F. Supp. 3d 449, 454 (E.D. Pa. 2014) (“A 95% confidence interval means that there is a 95% chance that the ‘‘true’’ ratio value falls within the confidence interval range.”), aff’d, 858 F.3d 787 (3d Cir. 2017); Duran v. U.S. Bank Nat’l Ass’n, 59 Cal. 4th 1, 36, 172 Cal. Rptr. 3d 371, 325 P.3d 916 (2014) (“Statisticians typically calculate margin of error using a 95 percent confidence interval, which is the interval of values above and below the estimate within which one can be 95 percent certain of capturing the ‘true’ result.”); In re Accutane Litig., 451 N.J. Super. 153, 165 A.3d 832, 842 (2017) (correctly quoting an incorrect definition from the third edition at p.580), rev’d on other grounds, 235 N.J. 229, 194 A.3d 503 (2018); In re Testosterone Replacement Therapy Prods. Liab., No. 14 C 1748, MDL No. 2545, 2017 WL 1833173, *4 (N.D. Ill. May 8, 2017) (“A confidence interval consists of a range of values. For a 95% confidence interval, one would expect future studies sampling the same population to produce values within the range 95% of the time.”); Maldonado v. Epsilon Plastics, Inc., 22 Cal. App. 5th 1308, 1330, 232 Cal. Rptr. 3d 461 (2018) (“The 95 percent ‘confidence interval’, as used by statisticians, is the ‘interval of values above and below the estimate within which one can be 95 percent certain of capturing the “true” result’.”); Escheverria v. Johnson & Johnson, 37 Cal. App. 5th 292, 304, 249 Cal. Rptr. 3d 642 (2019) (quoting uncritically and with approval one of plaintiff’s expert witnesses, Jack Siemiatycki, who gave the jury an example of a study with a relative risk of 1.2, with a “95 percent probability that the true estimate is between 1.1 and 1.3.” According to the court, Siemiatycki went on to explain that this was “a pretty tight interval, and we call that a confidence interval. We call it a 95 percent confidence interval when we calculate it in such a way that it covers 95 percent of the underlying relative risks that are compatible with this estimate from this study.”); In re Viagra (Sildenafil Citrate) & Cialis (Tadalafil) Prods. Liab. Litig., 424 F.Supp.3d 781, 787 (N.D. Cal. 2020) (“For example, a given study could calculate a relative risk of 1.4 (a 40 percent increased risk of adverse events), but show a 95 percent “confidence interval” of .8 to 1.9. That confidence interval means there is 95 percent chance that the true value—the actual relative risk—is between .8 and 1.9.”); Rhyne v. United States Steel Corp., 74 F. Supp. 3d 733, 744 (W.D.N.C. 2020) (relying upon, and quoting, one of the more problematic definitions given in the third edition at p.580: “If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the population.”); Wilant v. BNSF Ry., C.A. No. N17C-10-365 CEB, (Del. Super. Ct. May 13, 2020) (citing third edition at p.573, “a confidence interval provides ‘a range (interval) within which the risk likely would fall if the study were repeated numerous times’.”; “[s]o a 95% confidence interval indicates that the range of results achieved in the study would be achieved 95% of the time when the study is replicated from the same population.”); Germaine v. Sec’y Health & Human Servs., No. 18-800V, (U.S. Fed. Ct. Claims July 29, 2021) (giving an incorrect definition directly from the third edition, at p.621; “[a] “confidence interval” is “[a] range of values … within which the true value is likely to fall[.]”).

[7] Daniel Rubinfeld, “Reference Guide on Multiple Regression” 303, 352.

[8] Id. at 342.

[9] See Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman, “Statistical tests, P values, conﬁdence intervals, and power: a guide to misinterpretations,” 31 Eur. J. Epidemiol. 337, 343 (2016).

[10] Shari Seidman Diamond, “Reference Guide on Survey Research” 359, 381.

[11] Michael D. Green, D. Michal Freedman, and Leon Gordis, “Reference Guide on Epidemiology,” 549, 573.

[12] Id. at 580.

[13] Id. at 621.

[14] In re Testosterone Replacement Therapy Prods. Liab. Litig., Declaration of Martin T. Wells, Ph.D., at 2-3 (N.D. Ill., Oct. 30, 2016).

[15] Joseph Sanders, David Faigman, Peter Imrey, and A. Philip Dawid, “Diﬀerential Etiology: Inferring Speciﬁc Causation in the Law from Group Data in Science,” 63 Arizona L. Rev. 851, 898 n.173 (2021).

[16] The authors are well-credentialed lawyers and scientists. Peter Imrey, was trained in, and has taught, mathematical statistics, biostatistics, and epidemiology. He is a professor of medicine in the Cleveland Clinic Lerner College of Medicine. A. Philip Dawid is a distinguished statistician, an Emeritus Professor of Statistics, Cambridge University, Darwin College, and a Fellow of the Royal Society. David Faigman is the Chancellor & Dean, and the John F. Digardi Distinguished Professor of Law at the University of California Hastings College of the Law. Joseph Sanders is the A.A. White Professor, at the University of Houston Law Center. I have previously pointed this problem in these authors’ article. “Differential Etiologies – Part One – Ruling In” (June 19, 2022).

[17] See, e.g., Richard W. Clapp & David Ozonoff, “Environment and Health: Vital Intersection or Contested Territory?” 30 Am. J. L. & Med. 189, 210 (2004) (“Thus, a RR [relative risk] of 1.8 with a confidence interval of 1.3 to 2.9 could very likely represent a true RR of greater than 2.0, and as high as 2.9 in 95 out of 100 repeated trials.”); Erica Beecher-Monas, Evaluating Scientific Evidence: An Interdisciplinary Framework for Intellectual Due Process 60-61 n. 17 (2007) (quoting Clapp and Ozonoff with obvious approval); Déirdre Dwyer, The Judicial Assessment of Expert Evidence 154-55 (Cambridge Univ. Press 2008) (“By convention, scientists require a 95 per cent probability that a finding is not due to chance alone. The risk ratio (e.g. ‘2.2’) represents a mean figure. The actual risk has a 95 per cent probability of lying somewhere between upper and lower limits (e.g. 2.2 ±0.3, which equals a risk somewhere between 1.9 and 2.5) (the ‘confidence interval’).”); Frank C. Woodside, III & Allison G. Davis, “The Bradford Hill Criteria: The Forgotten Predicate,” 35 Thomas Jefferson L. Rev. 103, 110 (2013) (“A confidence interval provides both the relative risk found in the study and a range (interval) within which the risk would likely fall if the study were repeated numerous times.”); Christopher B. Mueller, “Daubert Asks the Right Questions: Now Appellate Courts Should Help Find the Right Answers,” 33 Seton Hall L. Rev. 987, 997 (2003) (describing the 95% confidence interval as “the range of outcomes that would be expected to occur by chance no more than five percent of the time”); Arthur H. Bryant & Alexander A. Reinert, “The Legal System’s Use of Epidemiology,” 87 Judicature 12, 19 (2003) (“The confidence interval is intended to provide a range of values within which, at a specified level of certainty, the magnitude of association lies.”) (incorrectly citing the first edition of Rothman & Greenland, Modern Epidemiology 190 (Philadelphia 1998); John M. Conley & David W. Peterson, “The Science of Gatekeeping: The Federal Judicial Center’s New Reference Manual on Scientific Evidence,” 74 N.C.L.Rev. 1183, 1212 n.172 (1996) (“a 95% confidence interval … means that we can be 95% certain that the true population average lies within that range”).

[18] See Brock v. Merrill Dow Pharm., Inc., 874 F.2d 307, 311–12 (5th Cir. 1989) (incorrectly stating that the court need not resolve questions of bias and confounding because “the studies presented to us incorporate the possibility of these factors by the use of a confidence interval”). Bayesian credible intervals can similarly be misleading when the interval simply reflects sample results and sample variance, but not the myriad other ways the estimate may be wrong.

Posted in Evidence, Expert Witnesses, Reference Manual on Scientific Evidence, statistical evidence | No Comments »

Doctor Moline – Why Can’t You Be True?

December 18th, 2022

Doctor Moline, why can’t you be true?

Oh, Doc Moline, why can’t you be true?

You done started doing the things you used to do.

Mass torts are the product of the lawsuit industry, and since the 1960s, this industry has produced tort claims on a truly industrial scale. The industry now has an economic ally and adjunct in the litigation finance industry, and it has been boosted by the desuetude of laws against champerty and maintenance. The way that mass torts are adjudicated in some places could easily be interpreted as legalized theft.

One governor on the rapaciousness of the lawsuit industry has been the requirement that claims actually be proven in court. Since the Supreme Court’s ruling in Daubert, the defense bar has been able, on notable occasions, to squelch some instances of false claiming. Just as equity often varies with the length of the Chancellor’s foot, gatekeeping of scientific opinion about causation often varies with the scientific acumen of the trial judge. From the decision in Daubert itself, gatekeeping has been under assault form the lawsuit industry and its allies. I have, in these pages, detailed the efforts of the now defunct Project on Scientific Knowledge and Public Policy (SKAPP) to undermine any gatekeeping of scientific opinion testimony for scientific or statistical validity. SKAPP, as well as other organizations, and some academics, in aid of the lawsuit industry, have lobbied for the abandonment of the requirement of proving causation, or for the dilution of the scientific standards for expert opinions of causation.[1] The counter to this advocacy has been, and continues to be, an insistence that the traditional elements of a case, including general and specific causation, be sufficiently proven, with opinion testimony that satisfies the legal knowledge requirement for such testimony.

Alas, expert witness testimony can go awry in other ways besides merely failing to satisfy the validity and relevance requirements of the law of evidence.[2] One way I had not previously contemplated is suing for defamation or “product disparagement.”

We are now half a century since occupational exposures to various asbestos fibers came under general federal regulatory control, with regulatory requirements that employers warn their employees about the hazards involved with asbestos exposure. This federally enforced dissemination of information about asbestos hazards created a significant problem for the asbestos lawsuit industry. Cases of mesothelioma have always occurred among persons non-occupationally exposed to asbestos, but as occupational exposure declined, the relative proportion of mesothelioma cases with no obvious occupational exposures increased. The lawsuit industry could not stand around and let these tragic cases go to waste.

Cosmetic talc variably has some mineral particulate that comes under the category of “elongate mineral particles,” (EMP), which the lawsuit industry could assert is “asbestos.” As a result, this industry has been able to reprise asbestos litigation into a new morality tale against cosmetic talc producers and sellers. LTL Management LLC was formerly known as Johnson & Johnson Consumer Inc. [J&J], a manufacturer and seller of cosmetic talc. J&J became a major target of the lawsuit industry in mesothelioma (and ovarian cancer) cases, based upon claims that EMP/asbestos in cosmetic talc caused their cancers. The lawsuit industry recruited its usual retinue of expert witnesses to support its litigation efforts.

Standing out in this retinue was Dr. Jacqueline Moline. On December 16, J&J did something that rarely happens in the world of mass torts; it sued Dr. Moline for fraud, injurious falsehood and product disparagement, and violations of the Lanham Act (§ 43(a), 15 U.S.C. § 1125(a)).[3] The gravamen of the complaint is that Dr. Moline, in 2020, published a case series of 33 persons who supposedly used cosmetic talc products and later developed malignant mesothelioma. According to her article, the 33 patients had no other exposures to asbestos, which she concluded, showed that cosmetic talc use can cause mesothelioma:

“Objective: To describe 33 cases of malignant mesothelioma among individuals with no known asbestos exposure other than cosmetic talcum powder.

Methods: Cases were referred for medico-legal evaluation, and tissue digestions were performed in some cases. Tissue digestion for the six ases described was done according to standard methodology.

Results: Asbestos of the type found in talcum powder was found in all six cases evaluated. Talcum powder usage was the only source of asbestos for all 33 cases.

Conclusions: Exposure to asbestos-contaminated talcum powders can cause mesothelioma. Clinicians should elicit a history of talcum powder usage in all patients presenting with mesothelioma.”[4]

Jacqueline Moline and Ronald Gordon both gave anemic conflicts disclosures: “Authors J.M. and R.G. have served as expert witnesses in asbestos litigation, including talc litigation for plaintiffs.”[5] Co-author Maya Alexandri was a lawyer at the time of publication; she is now a physician practicing emergency medicine, and also a fabulist. The article does not disclose the nature of Dr. Alexandri’s legal practice.

Dr. Moline is a professor and chair of occupational medicine at the Zucker School of Medicine at Hofstra/Northwell. She received her medical degree from the University of Chicago-Pritzker School of Medicine and a Master of Science degree in community medicine from the Mount Sinai School of Medicine. She completed a residency in internal medicine at Yale New Haven Hospital and an occupational and environmental medicine residency at Mount Sinai Medical Center. Dr. Moline is also a major-league testifier for the lawsuit industry. Over the last quarter century, she has testified from sea to shining sea, for plaintiffs in asbestos, talc, and other litigations.[6]

According to J&J, Dr. Moline was listed as an expert witness for plaintiff, in over 200 talc mesothelioma cases against J&J. There are, of course, other target defendants in this litigation, and the actual case count is likely higher. Moline has testified in 46 talc cases against J&J, and she has testified in 16 of those cases.[7] J&J estimates that she has made millions of dollars in service of the lawsuit industry.[8]

The authors’ own description of the manuscript makes clear the concern over the validity of personal and occupational histories of the 33 cases: “This manuscript is the first to describe mesothelioma among talcum powder consumers. Our case study suggest [sic] that cosmetic talcum powder use may help explain the high prevalence of idiopathic mesothelioma cases, particularly among women, and stresses the need for improved exposure history elicitation among physicians.”[9]

The Complaint alleges that Moline knew that her article, testimony, and public statements about the absence of occupational asbestos exposure in subjects of her case series, were false. After having her testimony either excluded by trial courts, or held on appeal to be legally insufficient,[10] Moline set out to have a peer-reviewed publication that would support her claims. Because mesothelioma is sometimes considered, uncritically, as pathognomonic of amphibole asbestos exposure, Moline was obviously keen to establish the absence of occupational exposure in any of the 33 cases.

Alas, the truth appears to have caught up with Moline because some of the 33 cases were in litigation, in which the detailed histories of each case would be discovered. Defense counsel sought to connect the dots between the details of each of the 33 cases and the details of pending or past lawsuits. The federal district court decision in the case of Bell v. American International Industries blew open the doors of Moline’s alleged fraud.[11] Betty Bell claimed that her use of cosmetic talc had caused her to develop mesothelioma. What Dr. Moline and Bell’s counsel were bound to have known was that Bell had had occupational exposure to asbestos. Before filing a civil action against talc product suppliers, Bell filed workers’ compensation against two textile industry employers.[12] Judge Osteen’s opinion in Bell documents the anxious zeal that plaintiffs’ counsel brought to bear in trying to suppress the true nature of Ms. Bell’s exposure. After Judge Osteen excoriated Moline and plaintiffs’ counsel for their efforts to conceal information about Bell’s occupational asbestos exposures, and about her inclusion in the 33 case series, plaintiffs’ counsel dismissed her case.

Another of the 33 cases was the New Jersey case brought by Stephen Lanzo, for whom Moline testified as an expert witness.[13] In the course of the Lanzo case, the defense developed facts of Mr. Lanzo’s prior asbestos exposure. Crocidolite fibers were found in his body, even though the amphibole crocidolite is not a fiber type found in talc. Crocidolite is orders of magnitude more potent in causing human mesotheliomas than other asbestos fiber types.[14] Despite these facts, Dr. Moline appears to have included Lanzo as one of the 33 cases in her article.

And then there were others, too.

[1] See “Skappology” (May 26, 2020); “SKAPP A LOT” (April 30, 2010); “Manufacturing Certainty” (Oct. 25, 2011); “David Michaels’ Public Relations Problem” (Dec. 2, 2011); “Conflicted Public Interest Groups” (Nov. 3, 2013).

[2] See, e.g., “Legal Remedies for Suspect Medical Science in Products Cases – Part One” (June 2, 2020); “Part Two” (June 3, 2020); “Part Three” (June 5, 2020); “Part 4” (June 7, 2020); “Part 5” (June 8, 2020).

[3] LTL Management LLC v. Dr. Jacqueline Miriam Moline,

Adv. Proc. No. 22- ____, in Chap. 11, Case No. 21-30589, Bankruptcy Ct., D.N.J. (Dec. 16, 2022) [Complaint]

[4] Jacqueline Moline, Kristin Bevilacqua, Maya Alexandri, and Ronald E. Gordon, “Mesothelioma Associated with the Use of Cosmetic Talc,” 62 J. Occup. & Envt’l Med. 11 (Jan. 2020) (emphasis added) [cited as Moline]

[5] Dr. Gordon has had other litigation activities of interest. See William C. Rempel, “Alleged Mob Case May Best Illustrate How Not to Play the Game : Crime: Scheme started in a Texas jail and ended with reputed mobsters charged in $30-million laundering scam,” L.A. Times (July 4, 1993).

[6] See., e.g., Fowler v. Akzo Nobel Chemicals, Inc., 251 N.J. 300, 276 A. 3d 1146 (2022); Lanzo v. Cyprus Amax Minerals Co., 467 N.J. Super. 476, 254 A.3d 691 (App. Div. 2021); Fishbain v. Colgate-Palmolive Co., No. A-1786-15T2 (N.J. App. Div. 2019); Buttitta v. Allied Signal, Inc., N.J. App. Div. (2017); Kaenzig v. Charles B. Chrystal Co., N.J. App. Div. (2015); Anderson v. A.J. Friedman Supply Co., 416 N.J. Super. 46, 3 A.3d 545 (App. Div. 2010); Cioni v. Avon Prods., Inc., 2022 NY Slip Op 33197(U) (2022); Zicklin v. Bergdorf Goodman Inc., 2022 NY Slip Op 32119(U) (N.Y.Sup. N.Y. Cty. 2022); Nemeth v. Brenntag North America, 183 A.D.3d 211, 123 N.Y.S.3d 12 (2020), rev’d, 38 N.Y.3d 336, 345 (2022) (Moline’s testimony insufficient); Olson v. Brenntag North America, Inc., 2020 NY Slip Op 33741(U) (N.Y.Sup. N.Y. Cty. 2020), rev’d, 207 A.D.3d 415, 416 (N.Y. 1st Dep’t 2022) (holding Moline’s testimony on causation insufficient).; Moldow v. A.I. Friedman, L.P., 2019 NY Slip Op 32060(U) (N.Y.Sup. N.Y. Cty. 2019); Zoas v BASF Catalysts, LLC., 2018 NY Slip Op 33009(U) (N.Y.Sup. N.Y. Cty. 2018); Prokocimer v. Avon Prods., Inc., 2018 NY Slip Op 33170(U) (Dec. 11, 2018); Shulman v. Brenntag North America, Inc., 2018 NY Slip Op 32943(U) (N.Y.Sup. N.Y. Cty. 2018); Pistone v. American Biltrite, Inc., 2018 NY Slip Op 30851(U) (2018); Evans v. 3M Co., 2017 NY Slip Op 30756(U) (N.Y.Sup. N.Y. Cty. 2017); Juni v. A.O. Smith Water Prods., 48 Misc.3d 460, 11 N.Y.S.3d 416 (2015), aff’d, 32 N.Y.3d 1116, 116 N.E.3d 75, 91 N.Y.S.3d 784 (2018); Konstantin v. 630 Third Ave. Associates, 121 A.D. 3d 230, 990 N.Y.S. 2d 174 (2014); Lopez v. Gem Gravure Co., 50 A.D.3d 1102, 858 N.Y.S.2d 226 (2008); Lopez v. Superflex, Ltd., 31 A.D. 3d 914, 819 N.Y.S. 2d 165 (2006); DeMeyer v. Advantage Auto, 9 Misc. 3d 306, 797 N.Y.S.2d 743 (2005); Amorgianos v. National RR Passenger Corp., 137 F. Supp. 2d 147 (E.D.N.Y. 2001), aff’d, 303 F. 3d 256 (2d Cir. 2002); Chapp v. Colgate-Palmolive Co., 2019 Wisc. App. 54, 935 N.W.2d 553 (2019); McNeal v. Whittaker, Clark & Daniels, Inc., 80 Cal. App. 853 (2022); Burnett v. American Internat’l Indus., Case No. 3:20-CV-3046 (W.D. Ark. Jan. 27, 2022); McAllister v. McDermott, Inc., Civ. Action No. 18-361-SDD-RLB (M.D.La. Aug. 14, 2020); Hanson v. Colgate-Palmolive Co., 353 F. Supp. 3d 1273 (S.D. Ga. 2018); Norman-Bloodsaw v. Lawrence Berkeley Laboratory, 135 F. 3d 1260 (9th Cir. 1998); Carroll v. Akebono Brake Corp., 514 P. 3d 720 (Wash. App. 2022).

[7] Complaint ¶15.

[8] Complaint ¶19.

[9] Moline at 11.

[10] See, e.g., In re New York City Asbestos Litig. (Juni), 148 A.D.3d 233, 236-37, 239 (N.Y. App. Div. 1st Dep’t 2017), aff’d, 2 N.Y.3d 1116, 1122 (2018); Nemeth v. Brenntag North America, 183 A.D.3d 211, 123 N.Y.S.3d 12 (N.Y. App. Div. 2020), rev’d, 38 N.Y.3d 336, 345 (2022); Olson v. Brenntag North America, Inc., 2020 NY Slip Op 33741(U) (N.Y.Sup. Ct. N.Y. Cty. 2020), rev’d, 207 A.D.3d 415, 416 (N.Y. App. Div. 1st Dep’t 2022).

[11] Bell v. American Internat’l Indus. et al., No. 1:17-CV-00111, 2022 U.S. Dist. LEXIS 199180 (M.D.N.C. Sept. 13, 2022) (William Lindsay Osteen, Jr., J.). See Daniel Fisher, “Key talc/cancer study cited by plaintiffs hid evidence of other exposure, lawyers say” (Dec. 1, 2022).

[12] According to the Complaint against Moline, Bell had filed workers’ compensation claims with the North Carolina Industrial Commission, back in 2015, declaring under oath that she had been exposed to asbestos while working with two textile manufacturing employers, Hoechst Celanese Corporation and Pillowtex Corporation. Complaint at ¶102. As frequently happens in civil actions, the claimant dismisses worker’s compensation without prejudice, to pursue the more lucrative payday in a civil action, without the burden of employers’ liens against the recovery. Complaint at 102.

[13] See “New Jersey Appellate Division Calls for Do-Over in Baby Powder Dust Up” (May 22, 2021).

[14] David H. Garabrant & Susan T. Pastula, “A comparison of asbestos fiber potency and elongate mineral particle (EMP) potency for mesothelioma in humans,” 361 Toxicology & Applied Pharmacol. 127 (2018) (“relative potency of chrysotile:amosite:crocidolite was 1:83:376”). See also D. Wayne Berman & Kenny S. Crump, “Update of Potency Factors for Asbestos-Related Lung Cancer and Mesothelioma,” 38(S1) Critical Reviews in Toxicology 1 (2008).

Posted in Expert Witness Discovery, Expert Witnesses, Rule 702, Scientific Evidence, Scientific Misconduct, Underlying Data | 24 Comments »

Further Thoughts on Cheng’s Consensus Rule

October 3rd, 2022

In “Cheng’s Proposed Consensus Rule for Expert Witnesses,”[1] I discussed a recent law review article by Professor Edward K. Cheng,[2] who has proposed dispensing with expert witness testimony as we know it in favor of having witnesses tell juries what the scientific consensus is on any subject. Cheng’s project is fraught with difficulties and contradictions; and it has clearly anticipatable bad outcomes. Four Supreme Court cases (Daubert, Joiner, Kumho Tire, and Weisgram), and a major revision in Rule 702, ratified by Congress, all embraced the importance of judicial gatekeeping of expert witness opinion testimony to the fact-finding function of trials. Professor Cheng now wants to ditch the entire notion of gatekeeping, as well as the epistemic basis – sufficient facts and data – for expert witnesses’ opinions in favor of reportage of which way the herd is going. Cheng’s proposal is perhaps the most radical attack, in recent times, on the nature of legal factfinding, whether by judges or juries, in the common law world.

Still, there are two claims within his proposal, which although overstated, are worth further discussion and debate. The first is that the gatekeeping role does not sit well with many judges. We see judges ill at ease in their many avoidance tactics, by which they treat serious methodological challenges to expert witness testimony as “merely going to the weight of the conclusion.” The second is that many judges, and especially juries, are completely at sea in the technical knowledge needed to evaluate the scientific issues in many modern day trials.

With respect to the claimed epistemic incompetence, the simpler remedy is to get rid of incompetent judges. We have commercial courts, vaccine courts, and patent courts. Why are litigants disputing a contract or a commercial practice entitled to epistemically competent judges, but litigants in health claim cases are not? Surely, the time has come to have courts with judges that have background and training in the health and statistical sciences. The time for “blue ribbon” juries of properly trained fact finders seems overdue. Somehow we must reconcile the seventh amendment right to a jury with the requirement of “due process” of law. The commitment to jury trials for causes of action known to the common law in 1787, or 1791, is stretched beyond belief for the sorts of technical and complex claims now seen in federal courts and state courts of general jurisdiction.[3]

Several courts have challenged the belief that the seventh amendment right to a jury applies in the face of complex litigation. The United States Court of Appeals explained its understanding of complexity that should remove a case from the province of the seventh amendment:

“A suit is too complex for a jury when circumstances render the jury unable to decide in a proper manner. The law presumes that a jury will find facts and reach a verdict by rational means. It does not contemplate scientific precision but does contemplate a resolution of each issue on the basis of a fair and reasonable assessment of the evidence and a fair and reasonable application of the relevant legal rules. See Schulz v. Pennsylvania RR, 350 U.S. 523, 526 (1956). A suit might be excessively complex as a result of any set of circumstances which singly or in combination render a jury unable to decide in the foregoing rational manner. Examples of such circumstances are an exceptionally long trial period and conceptually difficult factual issues.”[4]

The Circuit’s description of complexity certainly seems to apply to many contemporary claims of health effects.

We should recognize that Professor Cheng’s indictment, and conviction, of judicial gatekeeping and jury decision making as epistemically incompetent directly implies that the judicial process has no epistemic, truth finding function in technical cases of claimed health effects. Cheng’s proposed solution does not substantially ameliorate this implication, because consensus statements are frequently absent, and even when present, are plagued with their own epistemic weaknesses.

Consider for instance, the 1997 pronouncement of the International Agency for Research on Cancer that crystalline silica is a “known” human carcinogen.[5] One of the members of the working group responsible for the pronouncement explained:

“It is hardly surprising that the Working Group had considerable difficulty in reaching a decision, did not do so unanimously and would probably not have done so at all, had it not been explained that we should be concerned with hazard identification, not risk.”[6]

And yet, within months of the IARC pronouncement, state and federal regulatory agencies formed a chorus of assent to the lung cancer “risk” of crystalline silica. Nothing in the scientific record had changed except the permission of the IARC to stop thinking critically about the causation issue. Another consensus group came out, a few years after the IARC pronouncement, with a devastating critical assessment of the IARC review:

“The present authors believe that the results of these studies [cited by IARC] are inconsistent and, when positive, only weakly positive. Other, methodologically strong, negative studies have not been considered, and several studies viewed as providing evidence supporting the carcinogenicity of silica have significant methodological weaknesses. Silica is not directly genotoxic and is a pulmonary carcinogen only in the rat, a species that seems to be inappropriate for assessing particulate carcinogenesis in humans. Data on humans demonstrate a lack of association between lung cancer and exposure to crystalline silica. Exposure-response relationships have generally not been found. Studies in which silicotic patients were not identified from compensation registries and in which enumeration was complete did not support a causal association between silicosis and lung cancer, which further argues against the carcinogenicity of crystalline silica.”[7]

Cheng’s proposal would seem to suppress legitimate courtroom criticism of an apparent consensus statement, which was based upon a narrow majority of a working group, on a controversial dataset, with no examination of the facts and data upon which the putative consensus statement was itself based.

The Avandia litigation tells a cautionary tale of how fragile and ephemeral consensuses can be. A dubious meta-analysis by a well-known author received lead article billing in an issue of the New England Journal of Medicine, in 2007, and litigation claims started to roll in within hours.[8] In face of this meta-analysis, an FDA advisory committee recommended heightened warnings, and a trial court declined to take a careful look at the methodological flaws in the inciting meta-analytic study.[9] Ultimately, a large clinical trial exculpated the medication, but by then the harm had been done, and there was no revisiting of the gatekeeping decision to allow the claims to proceed.[10] The point should be obvious. In 2007, there appeared to be a consensus, with led to an FDA label change, despite the absence of sufficient facts and data to support the litigation claims. Even if plaintiffs’ claims passed through the gate in 2008, they were highly vulnerable to courtroom challenges to the original meta-analysis. Cheng’s proposal, however, would truncate the litigation process into an exploration whether or not there was a “consensus.”

Deviation from Experts’ Standards of Care

The crux of many Rule 702 challenges to an expert witness is that the witness has committed malpractice in his discipline. The challenger must identify a standard of care, and the challenged witness’s deviation(s) from that standard. The identification of the relevant standard of care will, indeed, sometimes involve a consensus, evidenced by texts, articles, professional society statements, or simply implicit in relevant works of scholarship or scientific studies. Consensuses about standards of care are, of course, about methodology. Consensuses about conclusions, however, may also be relevant because if a litigant’s expert witness proffers a conclusion at odds with consensus conclusions, the deviant conclusion implies deviant methodology.

Cheng’s treatment of statistical significance is instructive for how his proposal would create mischief in many different types of adjudications, but especially of claimed health effects. First, Cheng’s misrepresentation of consensus among statisticians is telling for the validity of his project. After all, he holds an advanced degree in statistics, and yet, he is willing write that that:

“[w]hile historically used as a rule of thumb, statisticians have now concluded that using the 0.05 [p-value] threshold is more distortive than helpful.”[11]

Statisticians, without qualification! And as was shown, Cheng is demonstrably wrong in his use of the cited source to support his representation of what certainly seems like a consensus paper. His précis is not even remotely close to the language of the paper, but the consensus paper is hearsay and can only be used by an expert witness in support of an opinion. Presumably, another expert witness might contradict the quoted opinion about what “statisticians” have concluded, but it is unclear whether a court could review the underlying A.S.A. paper, take judicial notice of the incorrectness of the proffered opinion, and then exclude the expert witness opinion.

After the 2016 publication of the A.S.A.’s consensus statement, some statisticians did indeed publish editorials claiming it was time to move beyond statistical significance testing. At least one editorial, by an A.S.A. officer was cited as representing an A.S.A. position, which led the A.S.A. President to appoint a task force to consider the call for an across-the-board rejection of significance testing. In 2021, that task force clearly endorsed significance testing as having a continued role in statistical practice.[12]

Where would this situation leave a gatekeeping court or a factfinding jury? Some obscure psychology journals have abandoned the use of significance testing, but the New England Journal of Medicine has retained the practice, while introducing stronger controls for claims of “significance” when the study at hands has engaged in multiple comparisons.

But Cheng, qua law professor and statistician (and would-be expert witness) claims “statisticians have now concluded that using the 0.05 [p-value] threshold is more distortive than helpful,” and the trial must chase not the validity of the inference of claimed causation but whether there is, or is not, a census about the use of a pre-specified threshold for p-values or confidence intervals. Cheng’s proposal about consensuses would turn trials into disputes about whether consensuses exist, and the scope of the purported agreement, not about truth.

In some instances, there might be a clear consensus, fully supported, on a general causation issue. Consider for instance, the known causal relationship between industrial benzene exposure and acute myelogenous leukemia (AML). This consensus turns out to be rather unhelpful when considering whether minute contamination of carbonated water can cause cancer,[13] or even whether occupational exposure to gasoline, with its low-level benzene (~1%) content, can cause AML.[14]

Frequently, there is also a deep asymmetry in consensus statements. When the evidence for a causal conclusion is very clear, professional societies may weigh in to express their confident conclusions about the existence of causation. Such societies typically do not issue statements that explicitly reject causal claims. The absence of a consensus statement, however, often can be taken to represent a consensus that professional societies do not endorse causal claims, and consider the evidence, at best, equivocal. Those dogs that have not barked can be, and have been, important considerations in gatekeeping.

Contrary to Cheng’s complete dismissal of judges’ epistemic competence, judges can, in many instances, render reasonable gatekeeping decisions by closely considering the absence of consensus statements, or systematic reviews, favoring the litigation claims.[15] At least in this respect, Professor Cheng is right to emphasize the importance of consensus, but he fails to note the importance of its absence, and the ability of litigants and their expert witnesses to inform gatekeeping judges of the relevance of consensus statements or their absence to the epistemic assessment of proferred expert witness opinion testimony.

[1] “Cheng’s Proposed Consensus Rule for Expert Witnesses,” (Sept. 15, 2022).

[2] Edward K. Cheng, “The Consensus Rule: A New Approach to Scientific Evidence,” 75 Vanderbilt L. Rev. 407 (2022) [Consensus Rule]

[3] There is an extensive discussion and debate of viability and the validity of asserting rights to trial by jury for many complex civil actions in the modern era. See, e.g., Stephan Landsman & James F. Holderman, “The Evolution of the Jury Trial in America,” 37 Litigation 32 (2010); Robert A. Clifford, “Deselecting the Jury in a Civil Case,” 30 Litigation 8 (Winter 2004); Hugh H. Bownes, “Should Trial by Jury Be Eliminated in Complex Cases,” 1 Risk 75 (1990); Douglas King, “Complex Civil Litigation and the Seventh Amendment Right to a Jury Trial,” 51 Univ. Chi. L. Rev. 581 (1984); Alvin B. Rubin, “Trial by Jury in Complex Civil Cases: Voice of Liberty or Verdict by Confusion?” 462 Ann. Am. Acad. Political & Social Sci. 87 (1982); William V. Luneburg & Mark A. Nordenberg, “Specially Qualified Juries and Expert Nonjury Tribunals: Alternatives for Coping with the Complexities of Modern Civil Litigation,” 67 Virginia L. Rev. 887 (1981); Richard O. Lempert, “Civil Juries and Complex Cases: Let’s Not Rush to Judgment,” 80 Mich. L. Rev. 68 (1981); Comment, “The Case for Special Juries in Complex Civil Litigation,” 89 Yale L. J. 1155 (1980); James S. Campbell & Nicholas Le Poidevin, “Complex Cases and Jury Trials: A Reply to Professor Arnold,” 128 Univ. Penn. L. Rev. 965 (1980); Barry E. Ungar & Theodore R. Mann, “The Jury and the Complex Civil Case,” 6 Litigation 3 (Spring 1980); Morris S. Arnold, “A Historical Inquiry into the Right to Trial by Jury in Complex Civil Litigation,”128 Univ. Penn. L. Rev. 829 (1980); Daniel H. Margolis & Evan M. Slavitt, “The Case Against Trial by Jury in Complex Civil Litigation,” 7 Litigation 19 (1980); Montgomery Kersten, “Preserving the Right to Jury Trial in Complex Civil Cases,” 32 Stanford L. Rev. 99 (1979); Maralynne Flehner, “Jury Trials in Complex Litigation,” 4 St. John’s Law Rev. 751 (1979); Comment, “The Right to a Jury Trial in Complex Civil Litigation,” 92 Harvard L. Rev. 898 (1979); Kathy E. Davidson, “The Right to Trial by Jury in Complex Litigation,” 20 Wm. & Mary L. Rev. 329 (1978); David L. Shapiro & Daniel R. Coquillette, “The Fetish of Jury Trial in Civil Cases: A Comment on Rachal v. Hill,” 85 Harvard L. Rev. 442 (1971); Comment, “English Judge May Not Order Jury Trial in Civil Case in Absence of Special Circumstances. Sims v. William Howard & Son Ltd. (C. A. 1964),” 78 Harv. L. Rev. 676 (1965); Fleming James, Jr., “Right to a Jury Trial in Civil Actions,” 72 Yale L. J. 655 (1963).

[4] In re Japanese Elec. Prods. Antitrust Litig., 63` F.2d 1069, 1079 (3d Cir 1980). See In re Boise Cascade Sec. Litig., 420 F. Supp. 99, 103 (W.D. Wash. 1976) (“In sum, it appears to this Court that the scope of the problems presented by this case is immense. The factual issues, the complexity of the evidence that will be required to explore those issues, and the time required to do so leads to the conclusion that a jury would not be a rational and capable fact finder.”). See also Ross v. Bernhard, 396 U.S. 532, 538 & n.10, 90 S. Ct. 733 (1970) (discussing the “legal” versus equitable nature of an action that might give rise to a right to trial by jury). Of course, the statistical and scientific complexity of claims was absent from cases tried in common law courts in 1791, at the time of the adoption of the seventh amendment.

[5] IARC Monograph on the Evaluation of Carcinogenic Risks to Humans of Silica, Some Silicates, Coal Dust and para-Aramid Fibrils, vol. 68 (1997).

[6] Corbett McDonald & Nicola Cherry, “Crystalline Silica and Lung Cancer: The Problem of Conflicting Evidence,” 8 Indoor Built Env’t 121, 121 (1999).

[7] Patrick A. Hessel, John F. Gamble, J. Bernard L. Gee, Graham Gibbs, Francis H.Y. Green, W. Keith C. Morgan, and Brooke T. Mossman, “Silica, Silicosis, and Lung Cancer: A Response to a Recent Working Group Report,” 42 J. Occup & Envt’l Med. 704, 704 (2000).

[8] Steven Nissen & K. Wolski, “Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes,” 356 New Engl. J. Med. 2457 (2007); Erratum, 357 New Engl. J. Med. 100 (2007).

[9] In re Avandia Mktg., Sales Practices & Prods. Liab. Litig., 2011 WL 13576 (E.D. Pa. Jan. 4, 2011).

[10] Philip D. Home, Stuart J Pocock, et al., “Rosiglitazone Evaluated for Cardiovascular Outcomes in Oral Agent Combination Therapy for Type 2 Diabetes (RECORD),” 373 Lancet 2125 (2009). The hazard ratios for cardiovascular death was 0.84 (95% C.I., 0·59–1·18), and for myocardial infarction, 1·14 (95% C.I., 0·80–1·63).

[11] Consenus Rule at 424 (emphasis added) (citing Ronald L. Wasserstein & Nicole A. Lazar, “The ASA Statement on p-Values: Context, Process, and Purpose,” 70 Am. Statistician 129, 131 (2016)).

[12] Yoav Benjamini, Richard D. DeVeaux, Bradly Efron, Scott Evans, Mark Glickman, Barry Braubard, Xuming He, Xiao Li Meng, Nancy Reid, Stephen M. Stigler, Stephen B. Vardeman, Christopher K. Wikle, Tommy Wright, Linda J. Young, and Karen Kafadar, “The ASA President’s Task Force Statement on Statistical Significance and Replicability,” 15 Annals of Applied Statistics 1084 (2021); see also “A Proclamation from the Task Force on Statistical Significance” (June 21, 2021).

[13] Sutera v. Perrier Group of America, Inc., 986 F. Supp. 655, 664-65 (D. Mass. 1997).

[14] Burst v. Shell Oil Co., 2015 WL 3755953, at *9 (E.D. La. June 16, 2015), aff’d, 650 F. App’x 170 (5th Cir. 2016). cert. denied. 137 S. Ct. 312 (2016); Henricksen v. ConocoPhillips Co., 605 F. Supp. 2d 1142, 1156 (E.D. Wa. 2009).

[15] In re Mirena Ius Levonorgestrel-Related Prod. Liab. Litig. (No. II), 341 F. Supp. 3d 213 (S.D.N.Y. 2018), aff’d, 982 F.3d 113 (2d Cir. 2020); In re Lipitor (Atorvastatin Calcium) Mktg., Sales Pracs. & Prods. Liab. Litig., 227 F. Supp. 3d 452 (D.S.C. 2017), aff’d, 892 F.3d 624 (4th Cir. 2018); In re: Zoloft (Sertraline Hydrocloride) Prod. Liab. Litig., No. 12-MD-2342, 2015 WL 7776911, at *1 (E.D. Pa. Dec. 2, 2015), aff’d, 858 F.3d 787 (3d Cir. 2017); In re Incretin-Based Therapies Prods. Liab. Litig., 524 F. Supp. 3d. 1007 (S.D. Cal. 2021); In re Viagra (Sildenafil Citrate) & Cialis (Tadalafil) Prod. Liab. Litig., 424 F. Supp. 3d 781, 798–99 (N.D. Cal. 2020).

Posted in Expert Witnesses, Rule 702, Scientific Evidence | 1 Comment »

Cheng’s Proposed Consensus Rule for Expert Witnesses

September 15th, 2022

Edward K. Cheng is the Hess Professor of Law in absentia from Vanderbilt Law School, while serving this fall as a visiting professor at Harvard. Professor Cheng is one of the authors of the multi-volume treatise, Modern Scientific Evidence, and the author of many articles on scientific and statistical evidence. Cheng’s most recent article, “The Consensus Rule: A New Approach to Scientific Evidence,”[1] while thought provoking, follows in the long-standing tradition of law school professors to advocate evidence law reforms, based upon theoretical considerations devoid of practical or real-world support.

Cheng’s argument for a radical restructuring of Rule 702 is based upon his judgment that jurors and judges are epistemically incompetent to evaluate expert witness opinion testimony. The current legal approach has trial judges acting as gatekeepers of expert witness testimony, and jurors acting as judges of factual scientific claims. Cheng would abolish these roles as beyond their ken.[2] Lay persons can, however, determine which party’s position is supported by the relevant expert community, which he presumes (without evidence) possesses the needed epistemic competence. Accordingly, Cheng would rewrite the legal system’s approach to important legal disputes, such as disputes over causal claims, from:

Whether a given substance causes a given disease

Whether the expert community believes that a given substance causes a given disease.

Cheng channels the philosophical understanding of the ancients who realized that one must have expertise to judge whether someone else has used that expertise correctly. And he channels the contemporary understanding that knowledge is a social endeavor, not the unique perspective of an individual in isolation. From these twin premisses, Cheng derives a radical and cynical proposal to reform the law of expert witness testimony. In his vision, experts would come to court not to give their own opinions, and certainly not to try to explain how they arrive at their opinions from the available evidence. For him, the current procedure is too much like playing chess with a monkey. The expert function would consist of telling the jury what the expert witness’s community believes.[3] Jurors would not decide the “actual substantive questions,” but simply decide what they believe the relevant expert witness community accepts as a consensus. This radical restructuring is what Cheng calls the “consensus rule.”

In this proposed “consensus rule,” there is no room for gatekeeping. Parties continue to call expert witnesses, but only as conduits for the “consensus” opinions of their fields. Indeed, Cheng’s proposal would radically limit expert witness to service as pollsters; their testimony would present only their views of what the consensus is in their fields. This polling information is the only evidence that the jury hear from expert witnesses, because this is the only evidence that Cheng believes the jury is epistemically competent to assess.[4]

Under Cheng’s Consensus Rule, when there is no consensus in the realm, the expert witness regime defaults to “anything goes,” without gatekeeping.[5] Judges would continue to exercise some control over who is qualified to testify, but only as far as the proposed experts must be in a position to know what the consensus is in their fields.

Cheng does not explain why, under his proposed “consensus rule,” subject matter experts are needed at all. The parties might call librarians, or sociologists of science, to talk about the relevant evidence of consensus. If a party cannot afford a librarian expert witness, then perhaps lawyers could present directly the results of their PubMed, and other internet searches.

Cheng may be right that his “deferential approach” would eliminate having the inexpert passing judgment on the expert. The “consensus rule” would reduce science to polling, conducted informally, often without documentation or recording, by partisan expert witnesses. This proposal hardly better reflects, as he argues, the “true” nature of science. In Cheng’s vision, science in the courtroom is just a communal opinion, without evidence and without inference. To be sure, this alternative universe is tidier and less disputatious, but it is hardly science or knowledge. We are left with opinions about opinions, without data, without internal or external validity, and without good and sufficient facts and data.

Cheng claims that his proposed Consensus Rule is epistemically superior to Rule 702 gatekeeping. For the intellectual curious and able, his proposal is a counsel of despair. Deference to the herd, he tells us “is not merely optimal—it is the only practical strategy.”[6] In perhaps the most extreme overstatement of his thesis, Cheng tells us that

“deference is arguably not due to any individual at all! Individual experts can be incompetent, biased, error prone, or fickle—their personal judgments are not and have never been the source of reliability. Rather, proper deference is to the community of experts, all of the people who have spent their careers and considerable talents accumulating knowledge in their field.”[7]

Cheng’s hypothesized community of experts, however is worthy of deference only by virtue of the soundness of its judgments. If a community has not severely tested its opinions, then its existence as a community is irrelevant. Cheng’s deference is the sort of phenomenon that helped create Lysenkoism and other intellectual fads that were beyond challenge with actual data.

There is, I fear, some partial truth to Cheng’s judgment of juries and judges as epistemically incompetent, or challenged, to judge science, but his judgment seems greatly overstated. Finding aberrant jury verdicts would be easy, but Cheng provides no meaningful examples of gatekeeping gone wrong. Professor Cheng may have over-generalized in stating that judges are epistemically incompetent to make substantive expert determinations. He surely cannot be suggesting that judges never have sufficient scientific acumen to determine the relevance and reliability of expert witness opinion. If judges can, in some cases, make a reasonable go at gatekeeping, why then is Cheng advocating a general rule that strips all judges of all gatekeeping responsibility with respect to expert witnesses?

Clearly judges lack the technical resources, time, and background training to delve deeply into the methodological issues with which they may be confronted. This situation could be ameliorated by budgeting science advisors and independent expert witnesses, and by creating specialty courts staffed with judges that have scientific training. Cheng acknowledges this response, but he suggests that conflicts with “norms about generalist judges.”[8] This retreat to norms is curious in the face of Cheng’s radical proposals, and the prevalence of using specialist judges for adjudicating commercial and patent disputes.

Although Cheng is correct that assessing validity and reliability of scientific inferences and conclusions often cannot be reduced to a cookbook or checklist approach, not all expertise is as opaque as Cheng suggests. In his view, lawyers are deluded into thinking that they can understand the relevant science, with law professors being even worse offenders.[9] Cross-examining a technical expert witness can be difficult and challenging, but lawyers on both sides of the aisle occasionally demolish the most skilled and knowledgeable expert witnesses, on substantive grounds. And these demolitions happen to expert witnesses who typically, self-servingly claim that they have robust consensuses agreeing with their opinions.

While scolding us that we must get “comfortable with relying on the expertise and authority of others,” Cheng reassures us that deferring to authority is “not laziness or an abdication of our intellectual responsibility.”[10] According to Cheng, the only reason to defer to the opinion of expert is that they are telling us what their community would say.[11] Good reasons, sound evidence, and valid inference need not worry us in Cheng’s world.

Finding Consensus

Cheng tells us that his Consensus Rule would look something like:

“Rule 702A. If the relevant scientific community believes a fact involving specialized knowledge, then that fact is established accordingly.”

Imagine the endless litigation over what the “relevant” community is. For a health effect claim about a drug and heart attacks, is it the community of cardiologists or epidemiologists? Do we accept the pronouncements of the American Heart Association or those of the American College of Cardiology. If there is a clear consensus based upon a clinical trial, which appears to be based upon suspect data, is discovery of underlying data beyond the reach of litigants because the correctness of the allegedly dispositive study is simply not in issue? Would courts have to take judicial notice of the clear consensus and shut down any attempt to get to the truth of the matter?

Cheng acknowledges that cases will involve issues that are controversial or undeveloped, without expert community consensus. Many litigations start after publication of a single study or meta-analysis, which is hardly the basis for any consensus. Cheng appears content, in this expansive area, to revert to anything goes because if the expert community has not coalesced around a unified view, or if the community is divided, then the courts cannot do better than flipping a coin! Cheng’s proposal thus has a loophole the size of the Sun.

Cheng tells us, unhelpfully, that “[d]etermining consensus is difficult in some cases, and less so in others.”[12] Determining consensus may not be straightforward, but no matter. Consensus Rule questions are not epistemically challenging and thus “far more manageable,” because they requires no special expertise. (Again, why even call a subject matter expert witness, as opposed to a science journalist or librarian?) Cheng further advises that consensus is “a bit like the reasonable person standard in negligence,” but this simply conflates normative judgments with the scientific judgments.[13]

Cheng’s Consensus Rule would allow the use of a systematic review or a meta-analysis, not for evidence of the correctness of its conclusions, but only as evidence of a consensus.[14] The thought experiment of how this suggestion plays out in the real world may cause some agita. The litigation over Avandia began within days of the publication of a meta-analysis in the New England Journal of Medicine.[15] So some evidence of consensus; right? But then the letters to the editor within a few weeks of publication showed that the meta-analysis was fatally flawed. Inadmissible! Under the Consensus Rule the correctness or the methodological appropriateness of the meta-analysis is irrelevant. A few months later, another meta-analysis is published, which fails to find the risk that the original meta-analysis claimed. Is the trial now about which meta-analysis represents the community’s consensus, or are we thrown into the game of anything goes, where expert witnesses just say things, without judicial supervision? A few years go by, and now there is a large clinical trial that supersedes all the meta-analyses of small trials.[16] Is a single large clinical trial now admissible as evidence of a new consensus, or are only systematic reviews and meta-analyses relevant evidence?

Cheng’s Consensus Rule will be useless in most determinations of specific causation. It will be a very rare case indeed when a scientific organization issues a consensus statement about plaintiff John Doe. Very few tort cases involve putative causal agents that are thought to cause every instance of some disease in every person exposed to the agent. Even when a scientific community has addressed general causation, it will have rarely resolved all the uncertainty about the causal efficacy of all levels of exposure or the appropriate window of latency. So Cheng’s proposal guarantees to remove specific causation from the control of Rule 702 gatekeeping.

The potential for misrepresenting consensus is even greater than the misrepresentations of actual study results. At least the data are the data, but what will jurors do when they are regaled by testimony about the informal consensus reached in the hotel lobby of the latest scientific conference. Regulatory pronouncements that are based upon precautionary principles will be misrepresented as scientific consensus. Findings by the International Agency for Research on Cancer that a substance is a IIA “probable human carcinogen” will be hawked as a consensus, even though the classification specifically disclaims any quantitative meaning for “probable,” and it directly equates to “insufficient” evidence of carcinogencity in humans.

In some cases, as Cheng notes, organizations such as the National Research Council, or the National Academy of Science, Engineering and Medicine (NASEM), will have weighed in on a controversy that has found its way into court.[17] Any help from such organizations will likely be illusory. Consider the 2006 publication of a comprehensive review of the available studies on non-pulmonary cancers and asbestos exposure by NASEM. The writing group presented its assessment of colorectal cancer as not causally associated with occupational asbestos exposure.[18] By 2007, the following year, expert witnesses for plaintiffs argued that the NASEM publication was no longer a consensus because one or two (truly inconsequential studies) had been published after the report and thus not considered. Under Cheng’s proposal, this dodge would appear to be enough to oust the consensus rule, and default to the “anything goes” rule. The scientific record can change rapidly, and many true consensus statements quickly find their way into the dustbin of scientific history.

Cheng greatly underestimates the difficulty in ascertaining “consensus.” Sometimes, to be sure, professional societies issue consensus statements, but they are often tentative and inconclusive. In many areas of science, there will be overlapping realms of expertise, with different disciplines issuing inconsistent “consensus” statements. Even within a single expert community, there may be two schools of thoughts about a particular issue.

There are instances, perhaps more than a few, when a consensus is epistemically flawed. If, as is the case in many health effect claims, plaintiffs rely upon the so-called linear no-threshold dose-response (LNT) theory of carcinogenesis, plaintiffs will point to regulatory pronouncements that embrace LNT as “the consensus.” When scientists are being honest, they generally recognize LNT as part of a precautionary principle approach, which may make sense as the foundation of “risk assessment.” The widespread assumption of LNT in regulatory agencies, and among scientists who work in such agencies, is understandable, but LNT remains an assumption. Nonetheless, we already see LNT hawked as a consensus, which under Cheng’s Consenus Rule would become the key dispositive issue, while quashing the mountain of evidence that there are, in fact, defense mechanisms to carcinogenesis that result in practical thresholds.

Beyond, regulatory pronouncements, some areas of scientific endeavor have themselves become politicized and extremist. Tobacco smoking surely causes lung cancer, but the studies of environmental tobacco smoking and lung cancer have been oversold. In areas of non-scientific disputes, such as history of alleged corporate malfeasance, juries will be treated to “the consensus” of Marxist labor historians, without having to consider the actual underlying historical documents. Cheng tells us that his Consensus Rule is a “realistic way of treating nonscientific expertise,”[19] which would seem to cover historian expert witness. Yet here, lawyers and lay fact finders are fully capable of exploring the glib historical conclusions of historian witnesses with cross-examination on the underlying documentary facts of the proffered opinions.

The Alleged Warrant for the Consensus Rule

If Professor Cheng is correct that the current judicial system, with decisions by juries and judges, is epistemically incompetent, does his Consensus Rule necessarily follow? Not really. If we are going to engage in radical reforms, then the institutionalization of blue-ribbon juries would make much greater sense. As for Cheng’s claim that knowledge is “social,” the law of evidence already permits the use of true consensus statements as learned treatises, both to impeach expert witnesses who disagree, and (in federal court) to urge the truth of the learned treatise.

The gatekeeping process of Rule 702, which Professor Cheng would throw overboard, has important advantages in that judges ideally will articulate reasons for finding expert witness opinion testimony admissible or not. These reasons can be evaluated, discussed, and debated, with judges, lawyers, and the public involved. This gatekeeping process is rational and socially open.

Some Other Missteps in Cheng’s Argument

Experts on Both Sides are Too Extreme

Cheng’s proposal is based, in part, upon his assessment that the adversarial system causes the parties to choose expert witnesses “at the extremes.” Here again, Cheng provides no empirical evidence for his assessment. There is a mechanical assumption often made by people who do not bother to learn the details of a scientific dispute that the truth must somehow lie in the “middle.” For instance, in MDL 926, the silicone gel breast implant litigation, presiding Judge Sam Pointer complained about the parties’ expert witnesses being too extreme. Judge Pointer believed that MDL judges should not entertain Rule 702 challenges, which were in his view properly heard by the transferor courts. As a result, Judge Robert Jones, and then Judge Jack Weinstein, conducted thorough Rule 702 hearings and found that the plaintiffs’ expert witnesses’ opinions were unreliable and insufficiently supported by the available evidence.[20] Judge Weinstein started the process of selecting court-appointed expert witnesses for the remaining New York cases, which goaded Judge Pointer into taking the process back to the MDL court level. After appointing four, highly qualified expert witnesses, Judge Pointer continued to believe that the parties’ expert witnesses were “extremists,” and that the courts’ own experts would come down somewhere between them. When the court-appointed experts filed their reports, Judge Pointer was shocked that all four of his experts sided with the defense in rejecting the tendentious claims of plaintiffs’ expert witnesses.

Statistical Significance

Along the way, in advocating his radical proposal, Professor Cheng made some other curious announcements. For instance, he tells us that “[w]hile historically used as a rule of thumb, statisticians have now concluded that using the 0.05 [p-value] threshold is more distortive than helpful.”[21] Cheng’s purpose here is unclear, but the source he cited does not remotely support his statement, and certainly not his gross overgeneralization about “statisticians.” If this is the way he envisions experts will report “consensus,” then his program seems broken at its inception. The American Statistical Association’s (ASA) p-value “consensus” statement articulated six principles, the third of which noted that

“[s]cientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”

This is a few light years away from statisticians’ concluding that statistical significance thresholds are more distortive than helpful. The ASA p-value statement further explains that

“[t]he widespread use of ‘statistical significance’ (generally interpreted as ‘p < 0.05’) as a license for making a claim of a scientific finding (or implied truth) leads to considerable distortion of the scientific process.”[22]

In the science of health effects, statistical significance remains extremely important, but it has never been a license for making causal claims. As Sir Austin Bradford Hill noted in his famous after-dinner speech, ruling out chance (and bias) as an explanation for an association was merely a predicate for evaluating the association for causality.[23]

Over-endorsing Animal Studies

Under Professor Cheng’s Consensus Rule, the appropriate consensus might well be one generated solely by animal studies. Cheng tells that “perhaps” scientists do not consider toxicology when the pertinent epidemiology is “clear.” When the epidemiology, however, is unclear, scientists consider toxicology.[24] Well, of course, but the key question is whether a consensus about causation in humans will be based upon non-human animal studies. Cheng seems to answer this question in the affirmative by criticizing courts that have required epidemiologic studies “even though the entire field of toxicology uses tissue and animal studies to make inferences, often in combination with and especially in the absence of epidemiology.”[25] The vitality of the field of toxicology is hardly undermined by its not generally providing sufficient grounds for judgments of human causation.

Relative Risk Greater Than Two

In the midst of his argument for the Consensus Rule, Cheng points critically to what he calls “questionable proxies” for scientific certainty. One such proxy is the judicial requirement of risk ratios in excess of two. His short discussion appears to be focused upon the inference of specific causation in a given case, but it leads to a non-sequitur:

“Some courts have required a relative risk of 2.0 in toxic tort cases, requiring a doubling of the population risk before considering causation.⁷³ But the preponderance standard does not require that the substance more likely than not caused any case of the disease in the population, it requires that the substance more likely than not caused the plaintiff’s case.”[26]

Of course, it is exactly because we are interested in the probability of causation of the plaintiff’s case, that we advert to the risk ratio to give us some sense whether “more likely than not” the exposure caused plaintiff’s case. Unless plaintiff can show he is somehow unique, he is “any case.” In many instances, plaintiff cannot show how he is different from the participants of the study that gave rise to the risk ratio less than two.

[1] Edward K. Cheng, “The Consensus Rule: A New Approach to Scientific Evidence,” 75 Vanderbilt L. Rev. 407 (2022) [Consensus Rule].

[2] Consensus Rule at 410 (“The judge and the jury, lacking in expertise, are not competent to handle the questions that the Daubert framework assigns to them.”)

[3] Consensus Rule at 467 (“Under the Consensus Rule, experts no longer offer their personal opinions on causation or teach the jury how to assess the underlying studies. Instead, their testimony focuses on what the expert community as a whole believes about causation.”)

[4] Consensus Rule at 467.

[5] Consensus Rule at 437.

[6] Consensus Rule at 434.

[7] Consensus Rule at 434.

[8] Consensus Rule at 422.

[9] Consensus Rule at 429.

[10] Consensus Rule at 432-33.

[11] Consensus Rule at 434.

[12] Consensus Rule at 456.

[13] Consensus Rule at 457.

[14] Consensus Rule at 459.

[15] Steven E. Nissen, M.D., and Kathy Wolski, M.P.H., “Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes,” 356 New Engl. J. Med. 2457 (2007).

[16] P.D. Home, et al., “Rosiglitazone Evaluated for Cardiovascular Outcomes in Oral Agent Combination Therapy for Type 2 Diabetes (RECORD), 373 Lancet 2125 (2009).

[17] Consensus Rule at 458.

[18] Jonathan M. Samet, et al., Asbestos: Selected Health Effects (2006).

[19] Consensus Rule at 445.

[20] Hall v. Baxter Healthcare Corp., 947 F. Supp.1387 (D. Or. 1996) (excluding plaintiffs’ expert witnesses’ causation opinions); In re Breast Implant Cases, 942 F. Supp. 958 (E. & S.D.N.Y. 1996) (granting partial summary judgment on claims of systemic disease causation).

[21] Consenus Rule at 424 (citing Ronald L. Wasserstein & Nicole A. Lazar, “The ASA Statement on p-Values: Context, Process, and Purpose,” 70 Am. Statistician 129, 131 (2016)).

[22] Id.

[23] Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965). See Schachtman, “Ruling Out Bias & Confounding is Necessary to Evaluate Expert Witness Causation Opinions” (Oct. 29, 2018); “Woodside & Davis on the Bradford Hill Considerations” (Aug. 23, 2013); Frank C. Woodside, III & Allison G. Davis, “The Bradford Hill Criteria: The Forgotten Predicate,” 35 Thomas Jefferson L. Rev. 103 (2013).

[24] Consensus Rule at 444.

[25] Consensus Rule at 424 & n. 74 (citing to one of multiple court advisory expert witnesses in Hall v. Baxter Healthcare Corp., 947 F. Supp.1387, 1449 (D. Or. 1996), who suggested that toxicology would be appropriate to consider when the epidemiology was not clear). Citing to one outlier advisor is a rather strange move for Cheng considering that the “consensus” was readily discernible to the trial judge in Hall, and to Judge Jack Weinstein, a few months later, in In re Breast Implant Cases, 942 F. Supp. 958 (E. & S.D.N.Y. 1996).

[26] Consensus Rule at 424 & n. 73 (citing Lucinda M. Finley, “Guarding the Gate to the Courthouse: How Trial Judges Are Using Their Evidentiary Screening Role to Remake Tort Causation Rules,” 49 Depaul L. Rev. 335, 348–49 (2000). See Schachtman, “Rhetorical Strategy in Characterizing Scientific Burdens of Proof” (Nov. 15, 2014).

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | 3 Comments »

Amicus Curious – Gelbach’s Foray into Lipitor Litigation

August 25th, 2022

Professor Schauer’s discussion of statistical significance, covered in my last post,[1] is curious for its disclaimer that “there is no claim here that measures of statistical significance map easily onto measures of the burden of proof.” Having made the disclaimer, Schauer proceeds to falls into the transposition fallacy, which contradicts his disclaimer, and, generally speaking, is not a good thing for a law professor eager to advance the understanding of “The Proof,” to do.

Perhaps more curious than Schauer’s error is his citation support for his disclaimer.[2] The cited paper by Jonah B. Gelbach is one of several of Gelbach’s papers that advances the claim that the p-value does indeed map onto posterior probability and the burden of proof. Gelbach’s claim has also been the center piece in his role as an advocate in support of plaintiffs in the Lipitor (atorvastatin) multi-district litigation (MDL) over claims that ingestion of atorvastatin causes diabetes mellitus.

Gelbach’s intervention as plaintiffs’ amicus is peculiar on many fronts. At the time of the Lipitor litigation, Sonal Singh was an epidemiologist and Assistant Professor of Medicine, at the Johns Hopkins University. The MDL trial court initially held that Singh’s proffered testimony was inadmissible because of his failure to consider daily dose.[3] In a second attempt, Singh offered an opinion for 10 mg daily dose of atorvastatin, based largely upon the results of a clinical trial known as ASCOT-LLA.[4]

The ASCOT-LLA trial randomized 19,342 participants with hypertension and at least three other cardiovascular risk factors to two different anti-hypertensive medications. A subgroup with total cholesterol levels less than or equal to 6.5 mmol./l. were randomized to either daily 10 mg. atorvastatin or placebo. The investigators planned to follow up for five years, but they stopped after 3.3 years because of clear benefit on the primary composite end point of non-fatal myocardial infarction and fatal coronary heart disease. At the time of stopping, there were 100 events of the primary pre-specified outcome in the atorvastatin group, compared with 154 events in the placebo group (hazard ratio 0.64 [95% CI 0.50 – 0.83], p = 0.0005).

The atorvastatin component of ASCOT-LLA had, in addition to its primary pre-specified outcome, seven secondary end points, and seven tertiary end points. The emergence of diabetes mellitus in this trial population, which clearly was at high risk of developing diabetes, was one of the tertiary end points. Primary, secondary, and tertiary end points were reported in ASCOT-LLA without adjustment for the obvious multiple comparisons. In the treatment group, 3.0% developed diabetes over the course of the trial, whereas 2.6% developed diabetes in the placebo group. The unadjusted hazard ratio was 1.15 (0.91 – 1.44), p = 0.2493.[5] Given the 15 trial end points, an adjusted p-value for this particular hazard ratio, for diabetes, might well exceed 0.5, and even approach 1.0.

On this record, Dr. Singh honestly acknowledged that statistical significance was important, and that the diabetes finding in ASCOT-LLA might have been the result of low statistical power or of no association at all. Based upon the trial data alone, he testified that “one can neither confirm nor deny that atorvastatin 10 mg is associated with significantly increased risk of type 2 diabetes.”[6] The trial court excluded Dr. Singh’s 10mg/day causal opinion, but admitted his 80mg/day opinion. On appeal, the Fourth Circuit affirmed the MDL district court’s rulings.[7]

Jonah Gelbach is a professor of law at the University of California at Berkeley. He attended Yale Law School, and received his doctorate in economics from MIT.

Professor Gelbach entered the Lipitor fray to present a single issue: whether statistical significance at conventionally demanding levels such as 5 percent is an appropriate basis for excluding expert testimony based on statistical evidence from a single study that did not achieve statistical significance.

Professor Gelbach is no stranger to antic proposals.[8] As amicus curious in the Lipitor litigation, Gelbach asserts that plaintiffs’ expert witness, Dr. Singh, was wrong in his testimony about not being able to confirm the ASCOT-LLA association because he, Gelbach, could confirm the association.[9] Ultimately, the Fourth Circuit did not discuss Gelbach’s contentions, which is not surprising considering that the asserted arguments and alleged factual considerations were not only dehors the record, but in contradiction of the record.

Gelbach’s curious claim is that any time a risk ratio, for an exposure and an outcome of interest, is greater than 1.0, with a p-value < 0.5,[10] the evidence should be not only admissible, but sufficient to support a conclusion of causation. Gelbach states his claim in the context of discussing a single randomized controlled trial (ASCOT-LLA), but his broad pronouncements are carelessly framed such that others may take them to apply to a single observational study, with its greater threats to internal validity.

Contra Kumho Tire

To get to his conclusion, Gelbach attempts to remove the constraints of traditional standards of significance probability. Kumho Tire teaches that expert witnesses must “employ[] in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.”[11] For Gelbach, this “eminently reasonable admonition” does not impose any constraints on statistical inference in the courtroom. Statistical significance at traditional levels (p < 0.05) is for elitist scholarly work, not for the “practical” rent-seeking work of the tort bar. According to Gelbach, the inflation of the significance level ten-fold to p < 0.5 is merely a matter of “weight” and not admissibility of any challenged opinion testimony.

Likelihood Ratios and Posterior Probabilities

Gelbach maintains that any evidence that has a likelihood ratio (LR > 1) greater than one is relevant, and should be admissible under Federal Rule of Evidence 401.[12] This argument ignores the other operative Federal Rules of Evidence, namely 702 and 703, which impose additional criteria of admissibility for expert witness opinion testimony.

With respect to variance and random error, Gelbach tells us that any evidence that generates a LR > 1, should be admitted when “the statistical evidence is statistically significant below the 50 percent level, which will be true when the p-value is less than 0.5.”[13]

At times, Gelbach seems to be discussing the admissibility of the ASCOT-LLA study itself, and not the proffered opinion testimony of Dr. Singh. The study itself would not be admissible, although it is clearly the sort of hearsay an expert witness in the field may consider. If Dr. Singh were to have reframed and recalculated the statistical comparisons, then the Rule 703 requirement of “reasonable reliance” by scientists in the field of interest may not have been satisfied.

Gelbach also generates a posterior probability (0.77), which is based upon his calculations from data in the ASCOT-LLA trial, and not the posterior probability of Dr. Singh’s opinion. The posterior probability, as calculated, is problematic on many fronts.

Gelbach does not present his calculations – for the sake of brevity he says – but he tells us that the ASCOT-LLA data yield a likelihood ratio of roughly 1.9, and a p-value of 0.126.[14] What the clinical trialists reported was a hazard ratio of 1.15, which is a weak association on most researchers’ scales, with a two-sided p-value of 0.25, which is five times higher than the usual 5 percent. Gelbach does not explain how or why his calculated p-value for the likelihood ratio is roughly half the unadjusted, two-sided p-value for the tertiary outcome from ASCOT-LLA.

As noted, the reported diabetes hazard ratio of 1.15 was a tertiary outcome for the ASCOT trial, one of 15 calculated by the trialists, with p-values unadjusted for multiple comparisons. The failure to adjust is perhaps excusable in that some (but certainly not all) of the outcome variables are overlapping or correlated. A sophisticated reader would not be misled; only when someone like Gelbach attempts to manufacture an inflated posterior probability without accounting for the gross underestimate in variance is there an insult to statistical science. Gelbach’s recalculated p-value for his LR, if adjusted for the multiplicity of comparisons in this trial, would likely exceed 0.5, rendering all his arguments nugatory.

Using the statistics as presented by the published ASCOT-LLA trial to generate a posterior probability also ignores the potential biases (systematic errors) in data collection, the unadjusted hazard ratios, the potential for departures from random sampling, errors in administering the participant recruiting and inclusion process, and other errors in measurements, data collection, data cleaning, and reporting.

Gelbach correctly notes that there is nothing methodologically inappropriate in advocating likelihood ratios, but he is less than forthcoming in explaining that such ratios translate into a posterior probability only if he posits a prior probability of 0.5.[15] His pretense to having simply stated “mathematical facts” unravels when we consider his extreme, unrealistic, and unscientific assumptions.

The Problematic Prior

Gelbach’s glibly assumes that the starting point, the prior probability, for his analysis of Dr. Singh’s opinion is 50%. This is an old and common mistake,[16] long since debunked.[17] Gelbach’s assumption is part of an old controversy, which surfaced in early cases concerning disputed paternity. The assumption, however, is wrong legally and philosophically.

The law simply does not hand out 0.5 prior probability to both parties at the beginning of a trial. As Professor Jaffee noted almost 35 years ago:

“In the world of Anglo-American jurisprudence, every defendant, civil and criminal, is presumed not liable. So, every claim (civil or criminal) starts at ground zero (with no legal probability) and depends entirely upon proofs actually adduced.”[18]

Gelbach assumes that assigning “equal prior probability” to two adverse parties is fair, because the fact-finder would not start hearing evidence with any notion of which party’s contentions are correct. The 0.5/0.5 starting point, however, is neither fair nor is it the law.[19] The even odds prior is also not good science.

The defense is entitled to a presumption that it is not liable, and the plaintiff must start at zero. Bayesians understand that this is the death knell of their beautiful model. If the prior probability is zero, then Bayes’ Theorem tells us mathematically that no evidence, no matter how large a likelihood ratio, can move the prior probability of zero towards one. Bayes’ theorem may be a correct statement about inverse probabilities, but still be an inadequate or inaccurate model for how factfinders do, or should, reason in determining the ultimate facts of a case.

We can see how unrealistic and unfair Gelbach’s implied prior probability is if we visualize the proof process as a football field. To win, plaintiffs do not need to score a touchdown; they need only cross the mid-field 50-yard line. Rather than making plaintiffs start at the zero-yard line, however, Gelbach would put them right on the 50-yard line. Since one toe over the mid-field line is victory, the plaintiff is spotted 99.99+% of its burden of having to present evidence to build up 50% probability. Instead, plaintiffs are allowed to scoot from the zero yard line right up claiming success, where even the slightest breeze might give them winning cases. Somehow, in the model, plaintiffs no longer have to present evidence to traverse the first half of the field.

The even odds starting point is completely unrealistic in terms of the events upon which the parties are wagering. The ASCOT-LLA study might have shown a protective association between atorvastatin and diabetes, or it might have shown no association at all, or it might have show a larger hazard ratio than measured in this particular sample. Recall that the confidence interval for hazard ratios for diabetes ran from 0.91 to 1.44. In other words, parameters from 0.91 (protective association) to 1.0 (no association), to 1.44 (harmful association) were all reasonably compatible with the observed statistic, based upon this one study’s data. The potential outcomes are not binary, which makes the even odds starting point inappropriate.[20]

[1] “Schauer’s Long Footnote on Statistical Significance” (Aug. 21, 2022).

[2] Frederick Schauer, The Proof: Uses of Evidence in Law, Politics, and Everything Else 54-55 (2022) (citing Michelle M. Burtis, Jonah B. Gelbach, and Bruce H. Kobayashi, “Error Costs, Legal Standards of Proof, and Statistical Significance,” 25 Supreme Court Economic Rev. 1 (2017).

[3] In re Lipitor Mktg., Sales Practices & Prods. Liab. Litig., MDL No. 2:14–mn–02502–RMG, 2015 WL 6941132, at *1 (D.S.C. Oct. 22, 2015).

[4] Peter S. Sever, et al., “Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac Outcomes Trial Lipid Lowering Arm (ASCOT-LLA): a multicentre randomised controlled trial,” 361 Lancet 1149 (2003). [cited here as ASCOT-LLA]

[5] ASCOT-LLA at 1153 & Table 3.

[6][6] In re Lipitor Mktg., Sales Practices & Prods. Liab. Litig., 174 F.Supp. 3d 911, 921 (D.S.C. 2016) (quoting Dr. Singh’s testimony).

[7] In re Lipitor Mktg., Sales Practices & Prods. Liab. Litig., 892 F.3d 624, 638-39 (2018) (affirming MDL trial court’s exclusion in part of Dr. Singh).

[8] See “Expert Witness Mining – Antic Proposals for Reform” (Nov. 4, 2014).

[9] Brief for Amicus Curiae Jonah B. Gelbach in Support of Plaintiffs-Appellants, In re Lipitor Mktg., Sales Practices & Prods. Liab. Litig., 2017 WL 1628475 (April 28, 2017). [Cited as Gelbach]

[10] Gelbach at *2.

[11] Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152 (1999).

[12] Gelbach at *5.

[13] Gelbach at *2, *6.

[14] Gelbach at *15.

[15] Gelbach at *19-20.

[16] See Richard A. Posner, “An Economic Approach to the Law of Evidence,” 51 Stanford L. Rev. 1477, 1514 (1999) (asserting that the “unbiased fact-finder” should start hearing a case with even odds; “[I]deally we want the trier of fact to work from prior odds of 1 to 1 that the plaintiff or prosecutor has a meritorious case. A substantial departure from this position, in either direction, marks the trier of fact as biased.”).

[17] See, e.g., Richard D. Friedman, “A Presumption of Innocence, Not of Even Odds,” 52 Stan. L. Rev. 874 (2000). [Friedman]

[18] Leonard R. Jaffee, “Prior Probability – A Black Hole in the Mathematician’s View of the Sufficiency and Weight of Evidence,” 9 Cardozo L. Rev. 967, 986 (1988).

[19] Id. at p.994 & n.35.

[20] Friedman at 877.

Posted in Causation, Evidence, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | No Comments »

Schauer’s Long Footnote on Statistical Significance

August 21st, 2022

One of the reasons that, in 2016, the American Statistical Association (ASA) issued, for the first time in its history, a consensus statement on p-values, was the persistent and sometimes deliberate misstatements and misrepresentations about the meaning of the p-value. Indeed, of the six principles articulated by the ASA, several were little more than definitional, designed to clear away misunderstandings. Notably, “Principle Two” addresses one persistent misunderstanding and states:

“P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.

Researchers often wish to turn a p-value into a statement about the truth of a null hypothesis, or about the probability that random chance produced the observed data. The p-value is neither. It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself.”[1]

The ASA consensus statement followed on the heels of an important published article, written by seven important authors in the fields of statistics and epidemiology.[2] One statistician,[3] who frequently shows up as an expert witness for multi-district litigation plaintiffs, described the article’s authors as the “A-Team” of statistics. In any event, the seven prominent thought leaders identified common statistical misunderstandings, including the belief that:

“2. The P value for the null hypothesis is the probability that chance alone produced the observed association; for example, if the P value for the null hypothesis is 0.08, there is an 8% probability that chance alone produced the association. No!”[4]

This is all basic statistics.

Frederick Schauer is the David and Mary Harrison Distinguished Professor of Law at the University of Virginia. Schauer has had contributed prolifically to legal scholarship, and his publications are often well written and thoughtful analyses. Schauer’s recent book, The Proof: Uses of Evidence in Law, Politics, and Everything Else, published by the Harvard University Press is a contribution to the literature of “legal epistemology,” and the foundations of evidence that lie beneath many of our everyday and courtroom approaches to resolving disputes.[5] Schauer’s book might be a useful addition to an undergraduate’s reading list for a course in practical epistemology, or for a law school course on evidence. The language of The Proof is clear and lively, but at times wanders into objectionable and biased political correctness. For example, Schauer channels Naomi Oreskes and her critique of manufacturing industry in his own discussion of “manufactured evidence,”[6] but studiously avoids any number of examples of explicit manufacturing of fraudulent evidence in litigation by the lawsuit industry.[7] Perhaps the most serious omission in this book on evidence is its failure to discuss the relative quality and hierarchy of evidence in science, medicine, and in policy. Readers will not find any mention of the methodology of systematic reviews or meta-analyses in Schauer’s work.

At the end of his chapter on burdens of proof, Schauer adds “A Long Footnote on Statistical Significance,” in which he expresses surprise that the subject of statistical significance is controversial. Schauer might well have brushed up on the statistical concepts he wanted to discuss.

Schauer’s treatment of statistical significance is both distinctly unbalanced, as well as misstated. In an endnote,[8] Schauer cites some of the controversialists who have criticized significance tests, but none of the statisticians who have defended their use.[9]

As for conceptual accuracy, after giving a serviceable definition of the p-value, Schauer immediately goes astray:

“And this likelihood is conventionally described in terms of a p-value, where the p-value is the probability that positive results—rejection of the “null hypothesis” that there is no connection between the examined variables—were produced by chance.”[10]

And again, for emphasis, Schauer tells us:

“A p-value of greater than .05 – a greater than 5 percent probability that the same results would have been the result of chance – has been understood to mean that the results are not statistically significant.”[11]

And then once more for emphasis, in the context of an emotionally laden hypothetical about an experimental drug “cures” a dread, incurable disease, p = 0.20, Schauer tells us that he suspects most people would want to take the medication:

“recognizing that an 80 percent likelihood that the rejection of ineffectiveness was still good enough, at least if there were no other alternatives.”

Schauer wants to connect his discussion of statistical significance to degrees or varying strengths of evidence, but his discursion into statistical significance largely conflates precision with strength. Evidence can be statistically robust but not be very strong. If we imagine a very large randomized clinical trial that found that a medication lowered systolic blood pressure by 1mm of mercury, p < 0.05, we would not consider that finding to constitute strong evidence for therapeutic benefit. If the observation of lowering blood pressure by 1mm came from an observational study, p < 0.05, the finding might not even qualify as evidence in the views of sophisticated cardiovascular physicians and researchers.

Earlier in the chapter, Schauer points to instances in which substantial evidence for a conclusion is downplayed because it is not “conclusive,” or “definitive.” He is obviously keen to emphasize that evidence that is not “conclusive” may still be useful in some circumstances. In this context, Schauer yet again misstates the meaning of significance probability, when he tells us that:

“[j]ust as inconclusive or even weak evidence may still be evidence, and may still be useful evidence for some purposes, so too might conclusions – rejections of the null hypothesis – that are more than 5 percent likely to have been produced by chance still be valuable, depending on what follows from those conclusions.”[12]

And while Schauer is right that weak evidence may still be evidence, he seems loathe to admit that weak evidence may be pretty crummy support for a conclusion. Take, for instance, a fair coin. We have an expected value on ten flips of five heads and five tails. We flip the coin ten times, but we observe six heads and four tails. Do we now have “evidence” that the expected value and the expected outcome are wrong? Not really. The probability of observing the expected outcome on the binomial model that most people would endorse for the thought experiment is 24.6%. The probability of not observing the expected value in ten flips is three times greater. If we look at an epidemiologic study, with a sizable number of participants, the “expected value” of 1.0, embodied in the null hypothesis, is an outcome that we would rarely expect to see, even if the null hypothesis is correct. Schauer seems to have missed this basic lesson of probability and statistics.

Perhaps even more disturbing is that Schauer fails to distinguish the other determinants of study validity and the warrants for inferring a conclusion at any level of certainty. There is a distinct danger that his comments about p-values will be taken to apply to various study designs, descriptive, observational, and experimental. And there is a further danger that incorrect definitions of the p-value and statistical significance probabilities will be used to misrepresent p-values as relating to posterior probabilities. Surely, a distinguished professor of law, at a top law school, in a book published by a prestigious publisher (Belknap Press) can do better. The message for legal practitioners is clear. If you need to define or discuss statistical concepts in a brief, seek out a good textbook on statistics. Do not rely upon other lawyers, even distinguished law professors, or judges, for accurate working definitions.

[1] Ronald L. Wasserstein & Nicole A. Lazar, “The ASA’s Statement on p-Values: Context, Process, and Purpose,” 70 The Am. Statistician 129, 131 (2016).

[2] Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman, “Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations,” 31 European J. Epidemiol. 337 (2016).[cited as “Seven Sachems”]

[3] Martin T. Wells.

[4] Seven Sachems at 340 (emphasis added).

[5] Frederick Schauer, The Proof: Uses of Evidence in Law, Politics, and Everything Else (2022). [Schauer] One nit: Schauer cites a paper by A. Philip Dawid, “Statistical Risk,” 194 Synthese 3445 (2017). The title of the paper is “On individual risk.”

[6] Naomi Oreskes & Erik M. Conway, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Climate Change (2010).

[7] See, e.g., In re Silica Prods. Liab. Litig., 398 F.Supp. 2d 563 (S.D.Tex. 2005); Transcript of Daubert Hearing at 23 (Feb. 17, 2005) (“great red flags of fraud”).

[8] See Schauer endnote 44 to Chapter 3, “The Burden of Proof,” citing Valentin Amrhein, Sander Greenland, and Blake McShane, “Scientists Rise Up against Statistical Significance,” www .nature .com (March 20, 2019), which in turn commented upon Blakey B. McShane, David Gal, Andrew Gelman, Christian Robert, and Jennifer L. Tackett, “Abandon Statistical Significance,” 73 American Statistician 235 (2019).

[9] Yoav Benjamini, Richard D. DeVeaux, Bradly Efron, Scott Evans, Mark Glickman, Barry Braubard, Xuming He, Xiao Li Meng, Nancy Reid, Stephen M. Stigler, Stephen B. Vardeman, Christopher K. Wikle, Tommy Wright, Linda J. Young, and Karen Kafadar, “The ASA President’s Task Force Statement on Statistical Significance and Replicability,” 15 Annals of Applied Statistics 1084 (2021); see also “A Proclamation from the Task Force on Statistical Significance” (June 21, 2021).

[10] Schauer at 55. To be sure, Schauer, in endnote 43 to Chapter 3, disclaims any identification of p-values or measures of statistical significance with posterior probabilities or probabilistic measures of the burden of proof. Nonetheless, in the text, he proceeds to do exactly what he disclaimed in the endnote.

[11] Schauer at 55.

[12] Schauer at 56.

Posted in Evidence, Expert Witnesses, Historian Testimony, statistical evidence | No Comments »

Differential Etiologies – Part One – Ruling In

June 17th, 2022

You put your right foot in

You put your right foot out

You put your right foot in

And you shake it all about

You do the Hokey Pokey and you turn yourself around

That’s what it’s all about!

Ever since the United States Supreme Court decided Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), legal scholars, judges, and lawyers have struggled with the structure and validity of expert opinion on specific causation. Professor David Faigman and others have attempted to articulate the scientific basis (if any) for opinion testimony in health-effects litigation that a give person’s disease has been caused by an exposure or condition.

In 2015, as part of a tribute to the late Judge Jack Weinstein, Professor Faigman offered the remarkable suggestion that in advancing differential etiologies, expert witnesses were inventing wholesale an approach that had no foundation or acceptance in their scientific disciplines:

“Differential etiology is ostensibly a scientific methodology, but one not developed by, or even recognized by, physicians or scientists. As described, it is entirely logical, but has no scientific methods or principles underlying it. It is a legal invention and, as such, has analytical heft, but it is entirely bereft of empirical grounding. Courts and commentators have so far merely described the logic of differential etiology; they have yet to define what that methodology is.”[1]

Faigman is correct that courts often have left unarticulated exactly what the methodology is, but he does not quite make sense when he writes that the method of differential etiology is “entirely logical,” but has no “scientific methods or principles underlying it.” After all, Faigman starts off his essay with a quotation from Thomas Huxley that “science is nothing but trained and organized common sense.”[2] As I have written elsewhere, the form of reasoning involved in differential diagnosis is nothing other than iterative disjunctive syllogism.[3] Either-or reasoning occurs throughout the physical and biological sciences; it is not clear why Faigman declares it un- or extra-scientific.

The strength of Faigman’s claim about the made-up nature of differential etiology appears to be undermined and contradicted by an example that he provides from clinical allergy and immunology:

“Allergists, for example, attempt to identify the etiology of allergic reactions in order to treat them (or to advise the patient to avoid what caused them), though it might still be possible to treat the allergic reactions without knowing their etiology.”

Faigman at 437. Of course, not only allergists try to determine the cause of an individual patient’s disease. Psychiatrists, in the psychoanalytic tradition, certain do so as well. Physicians who use predictive regression models use group data, in multivariate analyses, to predict outcomes, risk, and mortality in individual patients. Faigman’s claim is similarly undermined by the existence of a few diseases (other than infectious diseases) that are defined by the causative exposure. Silicosis and manganism have played a large role in often bogus litigation, but they represent instances in which a differential diagnosis and puzzle may also be an etiological diagnosis and a puzzle. Of course, to the extent that a disease is defined in terms of causative exposures, there may be serious and even intractable problems caused by the lack of specificity and accuracy in the diagnostic criteria for the supposedly pathognomonic disease.

As I noted at the time of Faigman’s 2015 essay, his suggestion that the concept of “differential etiology” was not used in the sciences themselves, was demonstrably flawed and historically inaccurate.[4]

A year earlier, in a more sustained analysis of specific causation, Professor Faigman went astray in a different direction, this time by stating that:

“it is not customary in the ordinary practice of sociology, epidemiology, anthropology, and related fields (for example, cognitive and social psychology) for professionals to make individual diagnostic judgments derived from group-based data.”[5]

Faigman’s invocation of “ordinary practice” of epidemiology was seriously wide of the mark. Medical practitioners and scientists frequently use epidemiologic data, based upon “group-based data” to make individual diagnostic judgments. The inferences from group data to individual range abound in the diagnostic process itself, where the specificity and sensitivity of disease signs and symptoms are measured by group data. Physicians must rely upon group data to make prognoses for individual patients, and they rely upon group data to predict future disease risks for individual patients. Future disease risks, as in the Framingham risk score for hard coronary heart disease, or the Gale model for breast cancer risk, are, of course, based upon “group-based data.” Medical decisions to intervene, surgically, pharmacologically, or by some other method, all involve applying group data to the individual patient.

Faigman’s 2014 law review article was certainly correct, however, in noting that specific causation inferences and conclusions were often left “profoundly underdefined,” with glib identifications of risk with cause.[6] There was thus plenty of room for further elucidation of specific causation decisions, and I welcome Faigman’s most recent effort to nail conceptual jello to the wall, in a law review article that was published last year.[7]

This new article, “Differential Etiology: Inferring Specific Causation in the Law from Group Data in Science,” is the collaborative product of Professor Faigman and three other academics. Joseph Sanders will be immediately recognizable to the legal community as someone who long pondered causation issues, both general and specific, and who has contributed greatly to the law review literature on causation of health outcomes. In addition to the law professors, Peter B. Imrey, a professor of medicine at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, and Philip Dawid, an emeritus professor of statistics in Cambridge University, have joined the effort to make sense of specific causation in the law. The addition of medical and statistical expertise has added greatly to Faigman’s previous efforts, and it has corrected some of his earlier errors and added much nuance to the discussion. The resulting law review article is well worth reading for practitioners. In this post, however, I have not detailed every important insight, but rather I have tried to point out some of the continuing and new errors in the analysis.

The Sanders-Faigman-Imbrey-Dawid analysis begins with a lament that:

“there is no body of science to which experts can turn when addressing this issue. Ultimately, much of the evidence that can be brought to bear on this causal question is the same group-level data employed to prove general causation. Consequently, the expert testimony often feels jerry-rigged, an improvisation designed to get through a tough patch.”[8]

As an assessment of the judicial decisions on specific causation, there can be no dissent or appeal from the judgment of these authors. The authors use of the term “jerry-rigged” is curious. I had first I thought they were straining to avoid using the common phrase “jury rigged” or to avoid inventing a neologism such as “judge rigged.” The American Heritage and Merriam Webster dictionaries, however, describe the phrase “jerry-rigged” as a conflation of “jury-rigged,” a nautical term for a temporary but adequate repair, with “jerry-rigged,” a war-time pejorative term for makeshift devices put together by Germans. So jerry-rigged it is, and the authors are off and running to try to describe, clarify, and justify the process of drawing specific causation inferences by differential etiology. They might have called what passes for judicial decision making in this area as the “hokey pokey.”

The authors begin their analysis of specific causation with a brief acknowledgement that our legal system could abandon any effort to set standards or require rigorous thinking on the matter by simply leaving the matter to the jury.[9] After all, this laissez-faire approach had been the rule of law for centuries. Nevertheless, despite occasional retrograde, recidivist judicial opinions,[10] the authors realize that the law has evolved to a point that some judicial control over specific causation opinions is required. And if judges are going to engage in gatekeeping of specific-causation opinions, they need to explain and justify their decisions in a coherent and cogent fashion.

Having thus dispatched legal nihilism, the authors turn their attention to what they boldly describe as “the first full-scale effort to bring scientific sensibilities – and rigorous statistical thinking – to the legally imperative concept of specific causation.”[11] The claim is remarkable claim given that tort law has been dealing with the issue for decades, but probably correct given how frequently judges have swept the issue under a judicial rug of inpenetrable verbiage and shaggy thinking. The authors also walk back some of Faigman’s earlier claims that there is no science in the assessment of specific causation, although they acknowledge the obvious, that policy issues sometimes play a role in deciding both general and specific causation decisions. The authors also offer the insight, for which they claim novelty, that some of the Bradford Hill guidelines, although stated as part of assessing general causation, have some relevancy to decisions concerning specific causation.[12] Their insight is indeed important, although hardly novel.

Drawing upon some of the clearer judicial decisions, the authors identify three necessary steps to reach a conclusion of specific causation:

“(a) making a proper diagnosis;

(b) supporting (“ruling in”) the plausibility of the alleged cause of the injury on the basis of general evidence and logic; and

(c) particularization, i.e., excluding (‘ruling out’) competing causes in the specific instance under consideration.”[13]

Although this article is ostensibly about specific causation, the authors do not reach a serious discussion of the matter until roughly the 42nd page of a 72 page article. Having described a three-step approach, the authors feel compelled to discuss step one (describing or defining the “diagnosis,” or the outcome of interest), and step two, the “ruling in” process that requires an assessment of general causation.

Although ascertaining general causation is not the focus of this article, the authors give an extensive discourse on it. Indeed, the authors have some useful things to say about steps one and two, and I commend the article to readers for some of its learning. As much as the lawsuit industry might wish to do away with the general causation step, it is not going anywhere soon.[14] The authors also manage to say some things that range from wrong to not even wrong. One example of professoriate wish casting is the following assertion:

“Other things being equal, when the evidence for general causation is strong, and especially when the strength of the exposure–disease relationship as demonstrated in a body of research is substantial, the plaintiff faces a lower threshold in establishing the substance as the cause in a particular case than when the relationship is weaker.”[15]

This assertion appears, sans citation or analysis. The generalization fails in the face of counterexamples. The causal role for estrogen in many breast cancers is extremely strong. The International Agency for Cancer Research classifies estrogen as a Category I, known human carcinogen for breast cancer, even though estrogen is made naturally in the human female, and male, body. In the Women’s Health Initiative clinical trial, researchers reported a hazard ratio of 1.2,[16] but plaintiffs struggled to prevail on specific causation in litigation involving claims of breast cancer caused by post-menopausal hormone therapy. Perhaps the authors meant, by strength of exposure relationship, a high relative risk as well, but that point is taken up when the authors address the “ruling in” step of the three-step approach. In any event, the strength of the case for general causation is quite independent of the specific causation inference, especially in the face of small effect sizes.

On general causation itself, the authors begin their discussion with “threats to validity,” a topic that they characterize as mostly implicit in the Bradford Hill guidelines. But their suggestion that validity is merely implicit in the guidelines is belied by their citation to Dr. Woodside’s helpful article on the “forgotten predicate” to the nine Bradford Hill guidelines.[17] Bradford Hill explicitly noted that the starting point for considering an association to be causal occurred when “[o]ur observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance.”[18] Sir Austin told us in no uncertain terms that there is no need to consider the nine guidelines until random and systematic error have been rejected.[19]

In this article’s discussion of general causation, Professor’s Dawid’s influence can be seen in the unusual care to describe and define the p-value.[20] But the discussion devolves into more wish casting, when the authors state that p-values are not the only way to assess random error in research results.

They double down by stating that “[m]any prominent statisticians and other scientists have questioned it, and the need for change is increasingly accepted.”[21] The source for their statement, the American Statistical Association (ASA) 2016 p-value Statement, did not questioned the utility of the p-value for assessing random error, and this law review provides no other support for other unidentified methods to assess random error. For the most part, the ASA Statement identified misuses and misstatements of p-values, with the caveat that “[s]cientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.” This is hardly questioning the importance or utility of p-values in assessing random error.

When one of the cited authors, Ronald Wasserstein, published an editorial in 2019, proclaiming that it was time to move past the p-value, the then president of the ASA, Professor Karen Kafadar, commissioned a task force on the matter. That task force, consisting of many of the world’s leading statisticians, issued a short, but pointed rejection of Wasserstein’s advocacy, and by implication, the position asserted in this law review.[22] Several of the leading biomedical journals that were lobbied by Wasserstein to abandon statistical significance testing reassessed their statistical guidelines and reaffirmed the use of p-values and tests.[23]

Similarly, this law review’s statements that alternatives to frequentist tests (p-values) such as Bayesian inference are “ascendant” have no supporting citations, and generally are an inaccurate assessment of what most biomedical journals are currently publishing.

Despite the care with which this law review article has defined p-values, the authors run off the road when defining a confidence interval:

“A 95% confidence interval … is a one-sided or two-sided interval from a data sample with 95% probability of bounding a fixed, unknown parameter, for which no nondegenerate probability distribution is conceived, under specified assumptions about the data distribution.”[24]

The emphasis added is to point out that the authors assigned a single confidence interval with the property of bounding the true parameter with 95% probability. That property, however, belongs to the infinite set of confidence intervals based upon repeated sampling of the same size from the same population, and constant variance. There is no probability statement to be made for the true parameter, as either in or not in a given confidence interval.

In an issue that is relevant to general and specific causation, the authors offer some ipse dixit on the issue of “thresholds”:

“with respect to some substance/injury relationships, it is thought that there is no safe threshold. Cancer is the injury for which it is most frequently thought that there is no safe threshold, but even here the mechanism of injury may lead to a different conclusion.”[25]

Here as elsewhere, the authors are repeating dogma, not science, and they ignore the substantial body of scientific evidence that undermines the so-called linear no threshold dose-response curve. The only citation offered is a judicial citation to a case that rejected the no threshold position![26]

So much for “ruling in.” In the next post, I will turn my attention to this law review’s handling of the “ruling out” step of differential etiology.

[1] David L. Faigman & Claire Lesikar, “Organized Common Sense: Some Lessons from Judge Jack Weinstein’s Uncommonly Sensible Approach to Expert Evidence,” 64 DePaul L. Rev. 421, 444 (2015).

[2] Thomas H. Huxley, “On the Education Value of the Natural History Sciences” (1854), in Lay Sermons, Addresses and Reviews 77 (1915).

[3] See, e.g., “Differential Etiology and Other Courtroom Magic” (June 23, 2014) (collecting cases); “Differential Diagnosis in Milward v. Acuity Specialty Products Group” (Sept. 26, 2013).

[4] See David Faigman’s Critique of G2i Inferences at Weinstein Symposium (Sept. 11, 2015); Kløve & D. Doehring, “MMPI in epileptic groups with differential etiology,” 18 J. Clin. Psychol. 149 (1962); Kløve & C. Matthews, “Psychometric and adaptive abilities in epilepsy with differential etiology,” 7 Epilepsia 330 (1966); Teuber & K. Usadel, “Immunosuppression in juvenile diabetes mellitus? Critical viewpoint on the treatment with cyclosporin A with consideration of the differential etiology,” 103 Fortschr. Med. 707 (1985); G.May & W. May, “Detection of serum IgA antibodies to varicella zoster virus (VZV)–differential etiology of peripheral facial paralysis. A case report,” 74 Laryngorhinootologie 553 (1995); Alan Roberts, “Psychiatric Comorbidity in White and African-American Illicity Substance Abusers” Evidence for Differential Etiology,” 20 Clinical Psych. Rev. 667 (2000); Mark E. Mullinsa, Michael H. Leva, Dawid Schellingerhout, Gilberto Gonzalez, and Pamela W. Schaefera, “Intracranial Hemorrhage Complicating Acute Stroke: How Common Is Hemorrhagic Stroke on Initial Head CT Scan and How Often Is Initial Clinical Diagnosis of Acute Stroke Eventually Confirmed?” 26 Am. J. Neuroradiology 2207 (2005);Qiang Fua, et al., “Differential Etiology of Posttraumatic Stress Disorder with Conduct Disorder and Major Depression in Male Veterans,” 62 Biological Psychiatry 1088 (2007); Jesse L. Hawke, et al., “Etiology of reading difficulties as a function of gender and severity,” 20 Reading and Writing 13 (2007); Mastrangelo, “A rare occupation causing mesothelioma: mechanisms and differential etiology,” 105 Med. Lav. 337 (2014).

[5] David L. Faigman, John Monahan & Christopher Slobogin, “Group to Individual (G2i) Inference in Scientific Expert Testimony,” 81 Univ. Chi. L. Rev. 417, 465 (2014).

[6] Id. at 448.

[7] Joseph Sanders, David L. Faigman, Peter B. Imrey, and Philip Dawid, “Differential Etiology: Inferring Specific Causation in the Law from Group Data in Science,” 63 Ariz. L. Rev. 851 (2021) [Differential Etiology]. I am indebted to Kirk Hartley for calling this new publication to my attention.

[8] Id. at 851, 855.

[9] Id. at 855 & n. 8 (citing A. Philip Dawid, David L. Faigman & Stephen E. Fienberg, “Fitting Science into Legal Contexts: Assessing Effects of Causes or Causes of Effects?,” 43 Sociological Methods & Research 359, 363–64 (2014). See also Barbara Pfeffer Billauer, “The Causal Conundrum: Examining the Medical-Legal Disconnect in Toxic Tort Cases from a Cultural Perspective or How the Law Swallowed the Epidemiologist and Grew Long Legs and a Tail,” 51 Creighton L. Rev. 319 (2018) (arguing for a standard-less approach that allows clinicians to offer their ipse dixit opinions on specific causation).

[10] Differential Etiology at 915 & n.231, 919 & n.244 (citing In re Round-Up Prods. Liab. Litig., 358 F. Supp. 3d 956, 960 (N.D. Cal. 2019).

[11] Differential Etiology at 856 (emphasis added).

[12] Differential Etiology at 857.

[13] Differential Etiology at 857 & n.14 (citing Best v. Lowe’s Home Ctrs., Inc., 563 F.3d 171, 180 (6th Cir. 2009)).

[14] See Margaret Berger, “Eliminating General Causation: Notes Toward a New Theory of Justice and Toxic Torts,” 97 Colum L. Rev. 2117 (1997).

[15] Differential Etiology at 864.

[16] Jacques E. Rossouw, et al., “Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the Women’s Health Initiative randomized controlled trial,” 288 J. Am. Med. Ass’n 321 (2002).

[17] Differential Etiology at 884 & n.104, citing Frank Woodside & Allison Davis, “The Bradford Hill Criteria: The Forgotten Predicate,” 35 Thomas Jefferson L. Rev. 103 (2013).

[18] Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965).

[19] Differential Etiology at 865.

[20] Differential Etiology at 869.

[21] Differential Etiology at 872, citing Ronald L. Wasserstein and Nicole A. Lazar, “The ASA Statement on p-Values: Context, Process, and Purpose,” 72 Am. Statistician 129 (2016).

[22] Yoav Benjamini, Richard D. De Veaux, Bradley Efron, Scott Evans, Mark Glickman, Barry I. Graubard, Xuming He, Xiao-Li Meng, Nancy M. Reid, Stephen M. Stigler, Stephen B. Vardeman, Christopher K. Wikle, Tommy Wright, Linda J. Young, and Karen Kafadar, “ASA President’s Task Force Statement on Statistical Significance and Replicability,” 15 Ann. Applied Statistics 1084 (2021), 34 Chance 10 (2021).

[23] See “Statistical Significance at the New England Journal of Medicine” (July 19, 2019); See also Deborah G. Mayo, “The NEJM Issues New Guidelines on Statistical Reporting: Is the ASA P-Value Project Backfiring?” Error Statistics Philosophy (July 19, 2019).

[24] Differential Etiology at 898 n.173 (emphasis added).

[25] Differential Etiology at 890.

[26] Differential Etiology at n.134, citing Chlorine Chemistry Council v. Envt’l Protection Agency, 206 F.3d 1286 (D.C. Cir. 2000), which rejected the agency’s assumption that the carcinogenic effects of chloroform in drinking water lacked a threshold.

Posted in Causation, Expert Witnesses | 1 Comment »