TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Peer Review, Protocols, and QRPs

April 3rd, 2024

In Daubert, the Supreme Court decided a legal question about the proper interpretation of a statute, Rule 702, and then remanded the case to the Ninth Circuit of the Court of Appeals for further proceedings. The Court did, however, weigh in with dicta about some several considerations in admissibility decisions.  In particular, the Court identified four non-dispositive factors: whether the challenged opinion has been empirically tested, published and peer reviewed, and whether the underlying scientific technique or method supporting the opinion has an acceptable rate of error, and has gained general acceptance.[1]

The context in which peer review was discussed in Daubert is of some importance to our understanding its holding peer review out as a consideraton. One of the bases for the defense challenges to some of the plaintiffs’ expert witnesses’ opinions in Daubert was their reliance upon re-analyses of published studies to suggest that there was indeed an increased risk of birth defects if only the publication authors had used some other control group, or taken some other analytical approach. Re-analyses can be important, but these reanalyses of published Bendectin studies were post hoc, litigation driven, and obviously result oriented. The Court’s discussion of peer review reveals that it was not simply creating a box to be checked before a trial court could admit an expert witness’s opinions. Peer review was suggested as a consideration because:

“submission to the scrutiny of the scientific community is a component of “good science,” in part because it increases the likelihood that substantive flaws in methodology will be detected. The fact of publication (or lack thereof) in a peer reviewed journal thus will be a relevant, though not dispositive, consideration in assessing the scientific validity of a particular technique or methodology on which an opinion is premised.”[2]

Peer review, or the lack thereof, for the challenged expert witnesses’ re-analyses was called out because it raised suspicions of lack of validity. Nothing in Daubert, or in later decisions, or more importantly in Rule 702 itself, supports admitting expert witness testimony just because the witness relied upon peer-reviewed studies, especially when the studies are invalid or are based upon questionable research practices. The Court was careful to point out that peer-reviewed publication was “not a sine qua non of admissibility; it does not necessarily correlate with reliability, … .”[3] The Court thus showed that it was well aware that well-ground (and thus admissible) opinions may not have been previously published, and that the existence of peer review was simply a potential aid in answering the essential question, whether the proponent of a proffered opinion has shown “the scientific validity of a particular technique or methodology on which an opinion is premised.”[4]

Since 1993, much has changed in the world of bio-science publishing. The wild proliferation of journals, including predatory and “pay-to-play” journals, has disabused most observers that peer review provides evidence of validity of methods. Along with the exponential growth in publications has come an exponential growth in expressions of concern and out-right retractions of articles, as chronicled and detailed at Retraction Watch.[5] Some journals encourage authors to nominate the peer reviewers for their manuscripts; some journals let authors block some scientists as peer reviewers of their submitted manuscripts. If the Supreme Court were writing today, it might well note that peer review is often a feature of bad science, advanced by scientists who know that peer-reviewed publication is the price of admission to the advocacy arena.

Since the Supreme Court decided Daubert, the Federal Judicial Center and National Academies of Science have provided a Reference Manual for Scientific Evidence, now in its third edition, and with a fourth edition on the horizon, to assist judges and lawyers involved in the litigation of scientific issues. Professor Goodstein, in his chapter “How Science Works,” in the third edition, provides the most extensive discussion of peer review in the Manual, and emphasizes that peer review “works very poorly in catching cheating or fraud.”[6]  Goodstein invokes his own experience as a peer reviewer to note that “peer review referees and editors limit their assessment of submitted articles to such matters as style, plausibility, and defensibility; they do not duplicate experiments from scratch or plow through reams of computer-generated data in order to guarantee accuracy or veracity or certainty.”[7] Indeed, Goodstein’s essay in the Reference Manual characterizes the ability of peer review to warrant study validity as a “myth”:

“Myth: The institution of peer review assures that all published papers are sound and dependable.

Fact: Peer review generally will catch something that is completely out of step with majority thinking at the time, but it is practically useless for catching outright fraud, and it is not very good at dealing with truly novel ideas. … It certainly does not ensure that the work has been fully vetted in terms of the data analysis and the proper application of research methods.”[8]

Goodstein’s experience as a peer reviewer is hardly idiosyncratic. One standard text on the ethical conduct of research reports that peer review is often ineffective or incompetent, and that it may not even catch simple statistical or methodological errors.[9] According to the authors, Shamoo and Resnik:

“[p]eer review is not good at detecting data fabrication or falsification partly because reviewers usually do not have access to the material they would need to detect fraud, such as the original data, protocols, and standard operating procedures.”[10]

Indeed, without access to protocols, statistical analysis plans, and original data, peer review often cannot identify good faith or negligent deviations from the standard of scientific care. There is some evidence to support this negative assessment of peer review from testing of the counter-factual. Reviewers were able to detect questionable, selective reporting when they had access to the study authors’ research protocols.[11]

Study Protocol

The study protocol provides the scientific rationale for a study, clearly defines the research question, the data collection process, defines the key exposure and outcomes, and describes the methods to be applied, before commencing data collection.[12] The protocol also typically pre-specifies the statistical data analysis. The epidemiology chapter of the current edition of the Reference Manual for Scientific Evidence offers blandly only that epidemiologists attempt to minimize bias in observational studies with “data collection protocols.”[13] Epidemiologists and statisticians are much clearer in emphasizing the importance, indeed the necessity, of having a study protocol before commencing data collection. Back in 1988, John Bailar and Frederick Mosteller explained that it was critical in reporting statistical analyses to inform readers about how and when the authors devised the study design, and whether they set the design criteria out in writing before they began to collect data.[14]

The necessity of a study protocol is “self-evident,”[15] and essential to research integrity.[16] The International Society of Pharmacoepidemiology has issued Guidelines for “Good Pharmacoepidemiology Practices,”[17] which calls for every study to have a written protocol. Among the requirements set out in this set of guidelines are descriptions of the research method, study design, operational definitions of exposure and outcome variables, and projected study sample size. The Guidelines provide that a detailed statistical analysis plan may be specified after data collection begins, but before any analysis commences.

Expert witness opinions on health effects are built upon studies, and so it behooves legal counsel to identify the methodological strengths and weaknesses of key studies through questioning whether they have protocols, whether the protocols were methodologically appropriate, and whether the researchers faithfully followed their protocols and their statistical analysis plans. Determining the peer review status of a publication, on the other hand, will often not advance a challenge based upon improvident methodology.

In some instances, a published study will have sufficiently detailed descriptions of methods and data that readers, even lawyers, can evaluate their scientific validity or reliability (vel non). In some cases, however, readers will be no better off than the peer reviewers who were deprived of access to protocols, statistical analysis plans, and original data. When a particular study is crucial support for an adversary’s expert witness, a reasonable litigation goal may well be to obtain the protocol and statistical analysis plan, and if need be, the original underlying data. The decision to undertake such discovery is difficult. Discovery of non-party scientists can be expensive and protracted; it will almost certainly be contentious. When expert witnesses rely upon one or a few studies, which telegraph internal validity, this litigation strategy may provide the strongest evidence against the study’s being reasonably relied upon, or its providing “sufficient facts and data” to support an admissible expert witness opinion.

[1] Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 593-594 (1993).

[2] Id. at 594 (internal citations omitted) (emphasis added).

[3] Id.

[4] Id. at 593-94.

[5] Retraction Watch, at https://retractionwatch.com/.

[6] Reference Manual on Scientific Evidence at 37, 44-45 (3rd ed. 2011) [Manual].

[7] Id. at 44-45 n.11.

[8] Id. at 48 (emphasis added).

[9] Adil E. Shamoo and David B. Resnik, Responsible Conduct of Research 133 (4th ed. 2022).

[10] Id.

[11] An-Wen Chan, Asbjørn Hróbjartsson, Mette T. Haahr, Peter C. Gøtzsche, and David G. Altman, D. G. “Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles,” 291 J. Am. Med. Ass’n 2457 (2004).

[12] Wolfgang Ahrens & Iris Pigeot, eds., Handbook of Epidemiology 477 (2nd ed. 2014).

[13] Michael D. Green, D. Michal Freedman, and Leon Gordis, “Reference Guide on Epidemiology,” in Reference Manual on Scientific Evidence 573 (3rd ed. 2011) 573 (“Study designs are developed before they begin gathering data.”).

[14] John Bailar & Frederick Mosteller, “Guidelines for Statistical Reporting in Articles for Medical Journals,” 108 Ann. Intern. Med. 2266, 268 (1988).

[15] Wolfgang Ahrens & Iris Pigeot, eds., Handbook of Epidemiology 477 (2nd ed. 2014).

[16] Sandra Alba, et al., “Bridging research integrity and global health epidemiology statement: guidelines for good epidemiological practice,” 5 BMJ Global Health e003236, at p.3 & passim (2020).

[17] See “The ISPE Guidelines for Good Pharmacoepidemiology Practices (GPP),” available at <https://www.pharmacoepi.org/resources/policies/guidelines-08027/>.

Posted in Data Sharing, Expert Witness Discovery, Rule 702, Rule 703, statistical evidence, Underlying Data | 14 Comments »

QRPs in Science and in Court

April 2nd, 2024

Lay juries usually function well in assessing the relevance of an expert witness’s credentials, experience, command of the facts, likeability, physical demeanor, confidence, and ability to communicate. Lay juries can understand and respond to arguments about personal bias, which no doubt is why trial lawyers spend so much time and effort to emphasize the size of fees and consulting income, and the propensity to testify only for one side. For procedural and practical reasons, however, lay juries do not function very well in assessing the actual merits of scientific controversies. And with respect to methodological issues that underlie the merits, juries barely function at all. The legal system imposes no educational or experiential qualifications for jurors, and trials are hardly the occasion to teach jurors the methodology, skills, and information needed to resolve methodological issues that underlie a scientific dispute.

Scientific studies, reviews, and meta-analyses are virtually never directly admissible in evidence in courtrooms in the United States. As a result, juries do not have the opportunity to read and ponder the merits of these sources, and assess their strengths and weaknesses. The working assumption of our courts is that juries are not qualified to engage directly with the primary sources of scientific evidence, and so expert witnesses are called upon to deliver opinions based upon a scientific record not directly in evidence. In the litigation of scientific disputes, our courts thus rely upon the testimony of so-called expert witnesses in the form of opinions. Not only must juries, the usual trier of fact in our courts, assess the credibility of expert witnesses, but they must assess whether expert witnesses are accurately describing studies that they cannot read in their entirety.

The convoluted path by which science enters the courtroom supports the liberal and robust gatekeeping process outlined under Rules 702 and 703 of the Federal Rules of Evidence. The court, not the jury, must make a preliminary determination, under Rule 104, that the facts and data of a study are reasonably relied upon by an expert witness (Rule 703). And the court, not the jury, again under Rule 104, must determine that expert witnesses possess appropriate qualifications for relevant expertise, and that these witnesses have proffered opinions sufficiently supported by facts or data, based upon reliable principles and methods, and reliably applied to the facts of the case. (Rule 702). There is no constitutional right to bamboozle juries with inconclusive, biased, and confounded or crummy studies, or selective and incomplete assessments of the available facts and data. Back in the days of “easy admissibility,” opinions could be tested on cross-examination, but limited time and acumen of counsel, court, and juries cry out for meaningful scientific due process along the lines set out in Rules 702 and 703.

The evolutionary development of Rules 702 and 703 has promoted a salutary convergence between science and law. According to one historical overview of systematic reviews in science, the foundational period for such reviews (1970-1989) overlaps with the enactment of Rules 702 and 703, and the institutionalization of such reviews (1990-2000) coincides with the development of these Rules in a way that introduced some methodological rigor into scientific opinions that are admitted into evidence.[1]

The convergence between legal admissibility and scientific validity considerations has had the further result that scientific concerns over the quality and sufficiency of underlying data, over the validity of study design, analysis, reporting, and interpretation, and over the adequacy and validity of data synthesis, interpretation, and conclusions have become integral to the gatekeeping process. This convergence has the welcome potential to keep legal judgments more in line with best scientific evidence and practice.

The science-law convergence also means that courts must be apprised of, and take seriously, the problems of study reproducibility, and more broadly, the problems raised by questionable research practices (QRPs), or what might be called the patho-epistemology of science. The development, in the 1970s, and the subsequent evolution, of the systematic review represented the scientific community’s rejection of the old-school narrative reviews that selected a few of all studies to support a pre-existing conclusion. Similarly, the scientific community’s embarrassment, in the 1980s and 1990s, over the irreproducibility of study results, has in this century grown into an existential crisis over study reproducibility in the biomedical sciences.

In 2005, John Ioannidis published an article that brought the concern over “reproducibility” of scientific findings in bio-medicine to an ebullient boil.[2] Ioannidis pointed to several factors, which alone or in combination rendered most published medical findings likely false. Among the publication practices responsible for this unacceptably high error rate, Ioannidis identified the use of small sample sizes, data-dredging and p-hacking techniques, poor or inadequate statistical analysis, in the context of undue flexibility in research design, conflicts of interest, motivated reasoning, fads, and prejudices, and pressure to publish “positive” results.  The results, often with small putative effect sizes, across an inadequate number of studies, are then hyped by lay and technical media, as well as the public relations offices of universities and advocacy groups, only to be further misused by advocates, and further distorted to serve the goals of policy wonks. Social media then reduces all the nuances of a scientific study to an insipid meme.

Ioannidis’ critique resonated with lawyers. We who practice in health effects litigation are no strangers to dubious research methods, lack of accountability, herd-like behavior, and a culture of generating positive results, often out of political or economic sympathies. Although we must prepare for confronting dodgy methods in front of jury, asking for scientific due process that intervenes and decides the methodological issues with well-reasoned, written opinions in advance of trial does not seem like too much.

The sense that we are awash in false-positive studies was heightened by subsequent papers. In 2011, Uri Simonsohn and others showed that by using simulations of various combinations of QRPs in psychological science, researchers could attain a 61% false-positive rate for research outcomes.[3] The following year saw scientists at Amgen attempt replication of 53 important studies in hematology and oncology. They succeeded in replicated only six.[4] Also in 2012, Dr. Janet Woodcock, director of the Center for Drug Evaluation and Research at the Food and Drug Administration, “estimated that as much as 75 per cent of published biomarker associations are not replicable.”[5] In 2016, the journal Nature reported that over 70% of scientists who responded to a survey had unsuccessfully attempted to replicate another scientist’s experiments, and more than half failed to replicate their own work.[6] Of the respondents, 90% agreed that there was a replication problem. A majority of the 90% believed that the problem was significant.

The scientific community reacted to the perceived replication crisis in a variety of ways, from conceptual clarification of the very notion of reproducibility,[7] to identification of improper uses and interpretations of key statistical concepts,[8] to guidelines for improved conduct and reporting of studies.[9]

Entire books dedicated to identifying the sources of, and the correctives for, undue researcher flexibility in the design, conduct, and analysis of studies, have been published.[10] In some ways, the Rule 702 and 703 case law is like the collected works of the Berenstain Bears, on how not to do studies.

The consequences of the replication crisis are real and serious. Badly conducted and interpreted science leads to research wastage,[11] loss of confidence in scientific expertise,[12] contemptible legal judgments, and distortion of public policy.

The proposed correctives to QRPs deserve the careful study of lawyers and judges who have a role in health effects litigation.[13] Whether as the proponent of an expert witness, or the challenger, several of the recurrent proposals, such as the call for greater data sharing and pre-registration of protocols and statistical analysis plans,[14] have real-world litigation salience. In many instances, they can and should direct lawyers’ efforts at discovery and challenging of the relied upon scientific studies in litigation.

[1] Quan Nha Hong & Pierre Pluye, “Systematic Reviews: A Brief Historical Overview,” 34 Education for Information 261 (2018); Mike Clarke & Iain Chalmers, “Reflections on the history of systematic reviews,” 23 BMJ Evidence-Based Medicine 122 (2018); Cynthia Farquhar & Jane Marjoribanks, “A short history of systematic reviews,” 126 Brit. J. Obstetrics & Gynaecology 961 (2019); Edward Purssell & Niall McCrae, “A Brief History of the Systematic Review,” chap. 2, in Edward Purssell & Niall McCrae, How to Perform a Systematic Literature Review: A Guide for Healthcare Researchers, Practitioners and Students 5 (2020).

[2] John P. A. Ioannidis “Why Most Published Research Findings Are False,” 1 PLoS Med 8 (2005).

[3] Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn, “False-Positive Psychology: UndisclosedFlexibility in Data Collection and Analysis Allows Presenting Anything as Significant,” 22 Psychological Sci. 1359 (2011).

[4] C. Glenn Begley and Lee M. Ellis, “Drug development: Raise standards for preclinical cancer research,” 483 Nature 531 (2012).

[5] Edward R. Dougherty, “Biomarker Development: Prudence, risk, and reproducibility,” 34 Bioessays 277, 279 (2012); Turna Ray, “FDA’s Woodcock says personalized drug development entering ‘long slog’ phase,” Pharmacogenomics Reporter (Oct. 26, 2011).

[6] Monya Baker, “Is there a reproducibility crisis,” 533 Nature 452 (2016).

[7] Steven N. Goodman, Daniele Fanelli, and John P. A. Ioannidis, “What does research reproducibility mean?,” 8 Science Translational Medicine 341 (2016); Felipe Romero, “Philosophy of science and the replicability crisis,” 14 Philosophy Compass e12633 (2019); Fiona Fidler & John Wilcox, “Reproducibility of Scientific Results,” Stanford Encyclopedia of Philosophy (2018), available at https://plato.stanford.edu/entries/scientific-reproducibility/.

[8] Andrew Gelman and Eric Loken, “The Statistical Crisis in Science,” 102 Am. Scientist 460 (2014); Ronald L. Wasserstein & Nicole A. Lazar, “The ASA’s Statement on p-Values: Context, Process, and Purpose,” 70 The Am. Statistician 129 (2016); Yoav Benjamini, Richard D. DeVeaux, Bradly Efron, Scott Evans, Mark Glickman, Barry Braubard, Xuming He, Xiao Li Meng, Nancy Reid, Stephen M. Stigler, Stephen B. Vardeman, Christopher K. Wikle, Tommy Wright, Linda J. Young, and Karen Kafadar, “The ASA President’s Task Force Statement on Statistical Significance and Replicability,” 15 Annals of Applied Statistics 1084 (2021).

[9] The International Society for Pharmacoepidemiology issued its first Guidelines for Good Pharmacoepidemiology Practices in 1996. The most recent revision, the third, was issued in June 2015. See “The ISPE Guidelines for Good Pharmacoepidemiology Practices (GPP),” available at https://www.pharmacoepi.org/resources/policies/guidelines-08027/. See also Erik von Elm, Douglas G. Altman, Matthias Egger, Stuart J. Pocock, Peter C. Gøtzsche, and Jan P. Vandenbroucke, for the STROBE Initiative, “The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement Guidelines for Reporting Observational Studies,” 18 Epidem. 800 (2007); Jan P. Vandenbroucke, Erik von Elm, Douglas G. Altman, Peter C. Gøtzsche, Cynthia D. Mulrow, Stuart J. Pocock, Charles Poole, James J. Schlesselman, and Matthias Egger, for the STROBE initiative, “Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration,” 147 Ann. Intern. Med. W-163 (2007); Shah Ebrahim & Mike Clarke, “STROBE: new standards for reporting observational epidemiology, a chance to improve,” 36 Internat’l J. Epidem. 946 (2007); Matthias Egger, Douglas G. Altman, and Jan P Vandenbroucke of the STROBE group, “Commentary: Strengthening the reporting of observational epidemiology—the STROBE statement,” 36 Internat’l J. Epidem. 948 (2007).

[10] See, e.g., Lee J. Jussim, Jon A. Krosnick, and Sean T. Stevens, eds., Research Integrity: Best Practices for the Social and Behavioral Sciences (2022); Joel Faintuch & Salomão Faintuch, eds., Integrity of Scientific Research: Fraud, Misconduct and Fake News in the Academic, Medical and Social Environment (2022); William O’Donohue, Akihiko Masuda & Scott Lilienfeld, eds., Avoiding Questionable Research Practices in Applied Psychology (2022); Klaas Sijtsma, Never Waste a Good Crisis: Lessons Learned from Data Fraud and Questionable Research Practices (2023).

[11] See, e.g., Iain Chalmers, Michael B Bracken, Ben Djulbegovic, Silvio Garattini, Jonathan Grant, A Metin Gülmezoglu, David W Howells, John P A Ioannidis, and Sandy Oliver, “How to increase value and reduce waste when research priorities are set,” 383 Lancet 156 (2014); John P A Ioannidis, Sander Greenland, Mark A Hlatky, Muin J Khoury, Malcolm R Macleod, David Moher, Kenneth F Schulz, and Robert Tibshirani, “Increasing value and reducing waste in research design, conduct, and analysis,” 383 Lancet 166 (2014).

[12] See, e.g., Friederike Hendriks, Dorothe Kienhues, and Rainer Bromme, “Replication crisis = trust crisis? The effect of successful vs failed replications on laypeople’s trust in researchers and research,” 29 Public Understanding Sci. 270 (2020).

[13] R. Barker Bausell, The Problem with Science: The Reproducibility Crisis and What to Do About It (2021).

[14] See, e.g., Brian A. Noseka, Charles R. Ebersole, Alexander C. DeHavena, and David T. Mellora, “The preregistration revolution,” 115 Proc. Nat’l Acad. Soc. 2600 (2018); Michael B. Bracken, “Preregistration of Epidemiology Protocols: A Commentary in Support,” 22 Epidemiology 135 (2011); Timothy L. Lash & Jan P. Vandenbroucke, “Should Preregistration of Epidemiologic Study Protocols Become Compulsory? Reflections and a Counterproposal,” 23 Epidemiology 184 (2012).

Posted in Expert Witnesses, Rule 702, Rule 703, Scientific Evidence, Scientific Publishing | 52 Comments »

Nullius in verba

March 29th, 2024

The 1975 codification of the law of evidence, in the Federal Rules of Evidence, introduced a subtle, aspirational criterion for expert witness opinion – knowledge. As originally enacted, Rule 702 read:

“If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise.”[1]

In case anyone missed the point, the Advisory Committee Note for the original Rule 702 emphasized that the standard was an epistemic standard:

“An intelligent evaluation of facts is often difficult or impossible without the application of some scientific, technical, or other specialized knowledge. The most common source of this knowledge is the expert witness, although there are other techniques for supplying it.”[2]

Perhaps we should not be too surprised that the epistemic standard was missed by most judges, and even by most lawyers. For a very long time, the common law set out a minimal test for expert witness opinion testimony. The expert witness had to be qualified by training, experience, or education, and the opinion proffered had to be logically and legally relevant to the issues in the case.[3] The enactment of Rule 702, in 1975, barely made a dent in the regime of easy admissibility.

Before the Federal Rules of Evidence, there was, of course, the famous Frye case, which involved an appeal from the excluded expert witness opinion based upon William Marston’s polygraph machine. In 1923, the court in Frye affirmed the exclusion of the expert witness opinion, based upon the lack of general acceptance of the device’s reliability, with its famous twilight zone language:[4]

“Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony deduced from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.”

With the explosion of tort litigation fueled by strict products liability doctrine, lawyers pressed Frye’s requirement of general acceptance into service as a bulwark against unreliable scientific opinions. Many courts, however, limited Frye to novel devices, and in 1993, the Supreme Court, in Daubert,[5] rejected the legal claim that Rule 702 had incorporated the common law “general acceptance” test. Looking to the language of the rule itself, the Supreme Court discerned that the rule laid down an epistemic test, not a call for sociological surveys about the prevalence of beliefs.

Resistance to the spirit and text of Rule 702 has been widespread and deep seated. After Daubert, the Supreme Court decided three more cases to emphasize that the epistemic standard was “exacting” and that it would not go away.[6] Since Daubert was decided in 1993, Rule 702 was amended substantively, in 2000, to incorporate some of the essence of the Supreme Court’s quartet,[7] which required the proponent of expert witness opinion to establish that proffered testimony is based upon sufficient facts or data, is the product of reliable principles and methods, and the result of reliably applying those reliable principles and methods to the facts of the case.

The change in the law of expert witnesses, in the 1990s, left some academic commentators well-nigh apoplectic. One professor of evidence law at a large law school complained that the law was a “conceptual muddle containing within it a threat to liberty and popular participation in government.”[8] Many federal district and intermediate appellate courts responded by ignoring the language of Rule 702, by reverting to pre-Daubert precedent, or by inventing new standards and shifting the burden to the party challenging the expert witness opinion’s admissibility. For many commentators, lawyers, and judges, science had no validity concerns that the law was bound to respect.

The judicial evasion and avoidance of the requirements of Rule 702 did not go unnoticed. Professor David Bernstein and practicing lawyer Eric Lasker wrote a paper in 2015, to call attention to the judicial disregard of the requirements of Rule 702.[9]  Several years of discussion and debate ensued before the Judicial Conference Advisory Committee on Evidence Rules (AdCom), in 2021, acknowledged that “in a fair number of cases, the courts have found expert testimony admissible even though the proponent has not satisfied the Rule 702(b) and (d) requirements by a preponderance of the evidence.”[10] This frank acknowledgement led the AdCom to propose amending Rule 702, “to clarify and emphasize” that gatekeeping requires determining whether the proponent has demonstrated to the court “that it is more likely than not that the proffered testimony meets the admissibility requirements set forth in the rule.”[11]  The Proposed Committee Note written in support of amending Rule 702 observed that “many courts have held that the critical questions of the sufficiency of an expert’s basis, and the application of the expert’s methodology, are questions of weight and not admissibility. These rulings are an incorrect application of Rules 702 and 104(a).”[12]

The proposed new Rule 702 is now law,[13] with its remedial clarification that the proponent of expert witness opinion must show the court that the opinion is sufficiently supported by facts or data,[14] that the opinion is “the product of reliable principles and methods,”[15]  and that the opinion “reflects a reliable application of the principles and methods to the facts of the case.”[16] The Rule prohibits deferring the evaluation of sufficiency of support or reliability of application of method to the trier of fact; there is no statutory support for suggesting that these inquires always or usually go to “weight and not admissibility,” or that there is a presumption of admissibility.

We may not have reached the Age of Aquarius, but the days of “easy admissibility” should be confined to the dustbin of legal history. Rule 702 is quickly approaching its 50th birthday, with the last 30 years witnessing the implementation of the promise and potential of an epistemic standard of trustworthiness for expert witness opinion testimony. Rule 702, in its present form, should go a long way towards putting validity questions squarely before the court under Rule 702. Nullius in verba[17] has been the motto of the Royal Society since 1660; it should now guide expert witness practice in federal court going forward.

[1] Pub. L. 93–595, §1, Jan. 2, 1975, 88 Stat. 1937 (emphasis added).

[2] Notes of Advisory Committee on Proposed Rules (1975) (emphasis added).

[3] See Charles T. McCormick, Handbook of the Law of Evidence 28-29, 363 (1954) (“Any relevant conclusions which are supported by a qualified expert witness should be received unless there are other reasons for exclusion.”)

[4] Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).

[5] Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993).

[6] General Electric Co. v. Joiner, 522 U.S. 136 (1997); Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999); Weisgram v. Marley Co., 528 U.S. 440 (2000).

[7] See notes 5, 6, supra.

[8] John H. Mansfield, “An Embarrassing Episode in the History of the Law of Evidence,” 34 Seton Hall L. Rev. 77, 77 (2003); see also John H. Mansfield, “Scientific Evidence Under Daubert,” 28 St. Mary’s L.J. 1, 23 (1996). Professor Mansfield was the John H. Watson, Jr., Professor of Law, at the Harvard Law School. Many epithets were thrown in the heat of battle to establish meaningful controls over expert witness testimony. See, e.g., Kenneth Chesebro, “Galileo’s Retort: Peter Huber’s Junk Scholarship,” 42 Am. Univ. L. Rev. 1637 (1993). Mr. Chesebro was counsel of record for plaintiffs-appellants in Daubert, well before he became a convicted racketeer in Georgia.

[9] David Bernstein & Eric Lasker, “Defending Daubert: It’s Time to Amend Federal Rules of Evidence 702,” 57 Wm. & Mary L Rev. 1 (2015).

[10] Report of AdCom (May 15, 2021), at https://www.uscourts.gov/rules-policies/archives/committee-reports/advisory-committee-evidence-rules-may-2021. See also AdCom, Minutes of Meeting at 4 (Nov. 13, 2020) (“[F]ederal cases . . . revealed a pervasive problem with courts discussing expert admissibility requirements as matters of weight.”)], at https://www.uscourts.gov/rules-policies/archives/meeting-minutes/advisory-committee-evidence-rules-november-2020.

[11] Proposed Committee Note, Summary of Proposed New and Amended Federal Rules of Procedure (Oct. 19, 2022), at https://www.uscourts.gov/sites/default/files/2022_scotus_package_0.pdf

[12] Id. (emphasis added).

[13] In April 2023, Chief Justice Roberts transmitted the proposed Rule 702, to Congress, under the Rules Enabling Act, and highlighted that the amendment “shall take effect on December 1, 2023, and shall govern in all proceedings thereafter commenced and, insofar as just and practicable all proceedings then pending.” S. Ct. Order, at 3 (Apr. 24, 2023), https://www.supremecourt.gov/orders/courtorders/frev23_5468.pdf; S.Ct. Transmittal Package (Apr. 24, 2023), < https://www.uscourts.gov/sites/default/files/2022_scotus_package_0.pdf>.

[14] Rule 702(b).

[15] Rule 702(c).

[16] Rule 702(d).

[17] Take no one’s word for it.

Posted in Rule 702, Scientific Evidence | 40 Comments »

Dipak Panigrahy – Expert Witness & Putative Plagiarist

March 27th, 2024

Citing an IARC monograph may be in itself questionable, given the IARC’s deviations from good systematic review practice. Taking the language of an IARC, monograph, and passing it off as your own, without citation or attribution, and leaving out the qualifications and limitations stated in the monograph, should be disqualifying for an expert witness.

And in one federal court, it is.

Last week, on March 18, Senior Judge Roy Bale Dalton, Jr., of Orlando, Florida, granted defendant Lockheed Martin’s Rule 702 motion to exclude the proffered testimony of Dr. Dipak Panigrahy.[1] Panigraphy had opined in his Rule 26 report that seven substances[2] present in the Orlando factory cause eight different types of cancer[3] in 22 of the plaintiffs. Lockheed’s motion asserted that Panigrahy copied IARC verbatim, except for its qualifications and limitations. Judge Dalton reportedly found Panigraphy’s conduct so “blatant that it represents deliberate lack of candor” and an “unreliable methodology.” Although Judge Dalton’s opinion is not yet posted on Westlaw or Google Scholar,[4] the report from Legal Newsline quoted the opinion extensively:

“Here, there is no question that Dr. Panigrahy extensively plagiarized his report… .”

“And his deposition made the plagiarism appear deliberate, as he repeatedly outright refused to acknowledge the long swaths of his report that quote other work verbatim without any quotation marks at all – instead stubbornly insisting that he cited over 1,100 references, as if that resolves the attribution issue (it does not).”

“Indeed, the plagiarism is so ubiquitous throughout the report that it is frankly overwhelming to try to make heads or tails of just what is Dr. Panigrahy’s own work – a task that neither he nor Plaintiffs’ counsel even attempts to tackle.”

There is a wide-range of questionable research practices and dubious inferences that lead to the exclusion of expert witnesses under Rule 702, but I would have thought that Panigraphy was the first witness to have been excluded for plagiarism. Judge Dalton did, however, cite cases involving plagiarism by expert witnesses.[5] Although plagiarism might be framed as a credibility issue, the extent of the plagiarism by Panigraphy represented such an egregious lack of candor that it may justify exclusion under Rule 702.

Judge Dalton’s gatekeeping analysis, however, did not stop with the finding of blatant plagiarism from the IARC monograph. Panigraphy’s report was further methodologically marred by reliance upon the IARC, and his confusion of the IARC hazard evaluation with the required determination of causation in the law of torts. Judge Dalton explained that

“the plagiarism here reflects even deeper methodological problems because the report lifts a great deal of its analysis from IARC in particular. As the Court discussed in the interim causation Order, research agencies like IARC are, understandably, focused on protecting public health and recommending protective standards, rather than evaluating causation from an expert standpoint in the litigation context. IARC determines qualitatively whether substances are carcinogenic to humans; its descriptors have “no quantitative significance” such as more likely than not. Troublingly, Dr. Panigrahy did not grasp this crucial distinction between IARC’s classifications and the general causation preponderance standard. Because so much of Dr. Panigrahy’s report is merely a wholesale adoption of IARC’s findings under the guise of his own expertise, and IARC’s findings in and of themselves are insufficient, he fails to reliably establish general causation.”[6]

Dr. Panigraphy was accepted into medical school at the age of 17. His accelerated education may have left him without a firm understanding of the ethical requirements of scholarship.

Earlier this month, Senior Judge Dalton excluded another expert witness’s opinion testimony, from Dr. Donald Mattison, on autism, multiple sclerosis, and Parkinson’s disease, but permitted opinions on the causation of various birth defects.[7] Judge Dalton’s decisions arise from a group of companion cases, brought by more than 60 claimants against Lockheed Martin for various health conditions alleged to have been caused by Lockheed’s supposed contamination of the air, soil, and groundwater, with chemicals from its weapons manufacturing plant.

The unreliability of Panigraphy’s report led to the entry of summary judgment against the 22 plaintiffs, whose cases turned on the Panigraphy report.

The putative plagiarist, Dr. Panigraphy, is an assistant professor of pathology, at Harvard Medical School, in the department of pathology, Beth Israel Deaconess Medical Center, in Boston.  “The Expert Institute” has a profile of Panigraphy, with a compilation of background information and litigation activities. His opinions were excluded in the federal multi-district litigation concerning Zantac/ranitidine.[8]  Very similar opinions were permitted over defense challenges, in a short, perfunctory order, even shorter on reasoning, in the valsartan multi-district litigation.[9]

[1] John O’Brien, “‘A mess’: Expert in Florida toxic tort plagiarizes cancer research of others, tries to submit it to court,” Legal News Line (Mar. 25, 2024).

[2] trichloroethylene, tetrachloroethylene, formaldehyde, arsenic, hexavalent chromium, trichloroethylene, and styrene.

[3] cancers of the kidney, breast, thyroid, pancreas, liver and bile duct, testicles, and anus, as well as Hodgkin’s lymphoma, non-Hodgkin’s lymphoma, and leukemia.

[4] Henderson v. Lockheed Martin Corp., case no. 6:21-cv-1363-RBD-DCI, document 399 (M.D. Fla. Mar. 18, 2024) (Dalton, S.J.).

[5] Henderson Order at 6, citing Moore v. BASF Corp., No. CIV.A. 11-1001, 2012 WL 6002831, at *7 (E.D. La. Nov. 30, 2012) (excluding expert testimony from Bhaskar Kura), aff’d, 547 F. App’x 513 (5th Cir. 2013); Spiral Direct, Inc. v. Basic Sports Apparel, Inc., No. 6:15-cv-641, 2017 WL 11457208, at *2 (M.D. Fla. Apr. 13, 2017); 293 F. Supp. 3d 1334, 1363 n. 20 (2017); Legier & Materne v. Great Plains Software, Inc., No. CIV.A. 03-0278, 2005 WL 2037346, at *4 (E.D. La. Aug. 3, 2005) (denying motion to exclude proffered testimony because expert witness plagiarized a paragraph in his report).

[6] Henderson Order at 8 -10 (internal citations omitted), citing McClain v. Metabolife Internat’l, Inc., 401 F.3d 1233, 1249 (11th Cir. 2005) (distinguishing agency assessment of risk from judicial assessment of causation); Williams v. Mosaic Fertilizer, LLC, 889 F.3d 1239, 1247 (11th Cir. 2018) (identifying “methodological perils” in relying extensively on regulatory agencies’ precautionary standards to determine causation); Allen v. Pennsylvania Eng’g Corp., 102 F.3d 194, 198 (5th Cir. 1996) (noting that IARC’s “threshold of proof is reasonably lower than that appropriate in tort law, which traditionally makes more particularized inquiries into cause and effect and requires a plaintiff to prove that it is more likely than not that another individual has caused him or her harm”); In re Roundup Prods. Liab. Litig., 390 F. Supp. 3d 1102, 1109 (N.D. Cal. 2018) (“IARC classification is insufficient to get the plaintiffs over the general causation hurdle.”), aff’d, 997 F.3d 941 (9th Cir. 2021).

[7] John O’Brien, “Autism plaintiffs rejected from Florida Lockheed Martin toxic tort,” Legal Newsline (Mar. 15, 2024).

[8][8] In re Zantac (ranitidine) Prods. Liab. Litig., MDL NO. 2924 20-MD-2924, 644 F. Supp. 3d 1075, 1100 (S.D. Fla. 2022).

[9] In re Valsartan, Losartan, and Irbesartan Prods. Liab. Litig., Case 1:19-md-02875-RBK-SAK, document 1958 (D.N.J. Mar. 4, 2022).

Posted in Causation, Expert Witness Hall of Shame, Expert Witnesses, Rule 702, Scientific Misconduct | 167 Comments »

A Π-Day Celebration of Irrational Numbers and Other Things – Philadelphia Glyphosate Litigation

March 14th, 2024

Science can often be more complicated and nuanced than we might like. Back in 1897, the Indiana legislature attempted to establish that π was equal to 3.2.[1] Sure, that was simpler and easier to use in calculations, but also wrong. The irreducible fact is that π is an irrational number, and Indiana’s attempt to change that fact was, well, irrational. And to celebrate irrationality, consider the lawsuit’s industry’s jihad against glyphosate, including its efforts to elevate a dodgy IARC evaluation, while suppressing evidence of glyphosate’s scientific exonerations

                                                 

After Bayer lost three consecutive glyphosate cases in Philadelphia last year, observers were scratching their heads over why the company had lost when the scientific evidence strongly supports the defense. The Philadelphia Court of Common Pleas, not to be confused with Common Fleas, can be a rough place for corporate defendants. The local newspapers, to the extent people still read newspapers, are insufferably slanted in their coverage of health claims.

The plaintiffs’ verdicts garnered a good deal of local media coverage in Philadelphia.[2] Defense verdicts generally receive no ink from sensationalist newspapers such as the Philadelphia Inquirer. Regardless, media accounts, both lay and legal, are generally inadequate to tell us what happened, or what went wrong in the court room. The defense losses could be attributable to partial judges or juries, or the difficulty in communicating subtle issues of scientific validity. Plaintiffs’ expert witnesses may seem more sure of themselves than defense experts, or plaintiffs’ counsel may connect better with juries primed by fear-mongering media. Without being in the courtroom, or at least studying trial transcripts, outside observers are challenged to explain fully jury verdicts that go against the scientific evidence. The one thing jury verdicts are not, however, are valid assessments of the strength of scientific evidence, inferences, and conclusions.

Although Philadelphia juries can be rough, they like to see a fight. (Remember Rocky.) It is not a place for genteel manners or delicate and subtle distinctions. Last week, Bayer broke its Philadelphia losing streak, with a win in Kline v. Monsanto Co.[3] Mr. Kline claimed that he developed Non-Hodgkin’s lymphoma (NHL) from his long-term use of Round-Up. The two-week trial, before Judge Ann Butchart, last week went to the jury, which deliberated two hours before returning a unanimous defense verdict. The jury found that the defendants, Monsanto and Nouryon Chemicals LLC, were not negligent, and that the plaintiff’s use of Roundup was not a factual cause of his lymphoma.[4]

Law360 reported that the Kline verdict was the first to follow a ruling on Valentine’s Day, February 14, 2024, which excluded any courtroom reference to the hazard evaluation of Glyphosate by the International Agency for Research on Cancer (IARC). The Law360 article indicated that the IARC found that glyphosate can cause cancer; except of course IARC has never reached such a conclusion.

The IARC working group evaluated the evidence for glyphosate and classified the substance as a category IIA carcinogen, which it labels as “probably” causing human cancer. This label sounds close to what might be useful in a courtroom, except that the IARC declares that “probably,” as used in is IIA classification does not mean what people generally, and lawyers and judges specifically, mean by the word probably.  For IARC, “probable” has no quantitative meaning.  In other words, for IARC, probable, a quantitative concept, which everyone understands to be measured on a scale from 0 to 1, or from 0% to 100%, is not quantitative. An IARC IIA classification could thus represent a posterior probability of 1% in favor of carcinogenicity (and 99% probable not a carcinogen). In other words, on whether glyphosate causes cancer in humans, IARC says maybe in its own made-up epistemic modality.

To find the idiosyncratic definition of “probable,” a diligent reader must go outside the monograph of interest to the so-called Preamble, a separate document, last revised in 2019. The first time the jury will hear of the IARC pronouncement will be in the plaintiff’s case, and if the defense wishes to inform the jury on the special, idiosyncratic meaning of IARC “probable,” they must do it on cross-examination of hostile plaintiffs’ witnesses, or wait longer until they present their own witnesses. Disclosing the IARC IIA classification hurts because the “probable” language lines up with what the trial judges will instruct the juries at the end of the case, when the jurors are told that they need not believe that the plaintiff has eliminated all doubt; they need only find that the plaintiff has shown that each element of his case is “probable,” or more likely than not, in order to prevail. Once the jury has heard “probable,” the defense will have a hard time putting the toothpaste back in the tube. Of course, this is why the lawsuit industry loves IARC evaluations, with its fallacies of semantical distortion.[5]

Although identifying the causes of a jury verdict is more difficult than even determining carcinogenicity, Rosemary Pinto, one of plaintiff Kline’s lawyers, suggested that the exclusion of the IARC evaluation sank her case:

“We’re very disappointed in the jury verdict, which we plan to appeal, based upon adverse rulings in advance of the trial that really kept core components of the evidence out of the case. These included the fact that the EPA safety evaluation of Roundup has been vacated, who IARC (the International Agency for Research on Cancer) is and the relevance of their finding that Roundup is a probable human carcinogen [sic], and also the allowance into evidence of findings by foreign regulatory agencies disguised as foreign scientists. All of those things collectively, we believe, tilted the trial in Monsanto’s favor, and it was inconsistent with the rulings in previous Roundup trials here in Philadelphia and across the country.”[6]

Pinto was involved in the case, and so she may have some insight into why the jury ruled as it did. Still, issuing this pronouncement before interviewing the jurors seems little more than wishcasting. As philosopher Harry Frankfurt explained, “the production of bullshit is stimulated whenever a person’s obligations or opportunities to speak about some topic exceed his knowledge of the facts that are relevant to that topic.”[7] Pinto’s real aim was revealed in her statement that the IARC review was “crucial evidence that juries should be hearing.”[8]  

What is the genesis of Pinto’s complaint about the exclusion of IARC’s conclusions? The Valentine’s Day Order, issued by Judge Joshua H. Roberts, who heads up the Philadelphia County mass tort court, provided that:

AND NOW, this 14th day of February, 2024, upon consideration of Defendants’ Motion to Clarify the Court’s January 4, 2024 Order on Plaintiffs Motion in Limine No. 5 to Exclude Foreign Regulatory Registrations and/or Approvals of Glyphosate, GBHs, and/or Roundup, Plaintiffs’ Response, and after oral argument, it is ORDERED as follows:

  1. The Court’s Order of January 4, 2024, is AMENDED to read as follows: [ … ] it is ORDERED that the Motion is GRANTED without prejudice to a party’s introduction of foreign scientific evidence, provided that the evidence is introduced through an expert witness who has been qualified pursuant to Pa. R. E. 702.

  2. The Court specifically amends its Order of January 4, 2024, to exclude reference to IARC, and any other foreign agency and/or foreign regulatory agency.

  3. The Court reiterates that no party may introduce any testimony or evidence regarding a foreign agency and/or foreign regulatory agency which may result in a mini-trial regarding the protocols, rules, and/or decision making process of the foreign agency and/or foreign regulatory agency. [fn1]

  4. The trial judge shall retain full discretion to make appropriate evidentiary rulings on the issues covered by this Order based on the testimony and evidence elicited at trial, including but not limited to whether a party or witness has “opened the door.”[9]

Now what was not covered in the legal media accounts was the curious irony that the exclusion of the IARC evaluation resulted from plaintiffs’ motion, an own goal of sorts. In previous Philadelphia trials, plaintiffs’ counsel vociferously objected to the defense counsel’s and experts’ references to the determinations by foreign regulators, such as European Union Assessment Group on Glyphosate (2017, 2022), Health Canada (2017), European Food Safety Authority (2017, 2023), Australian Pesticides and Veterinary Medicines Authority (2017), German Federal Institute for Risk Assessment (2019), and others, that rejected the IARC evaluation and reported that glyphosate has not been shown to be carcinogenic.[10]

The gravamen of the plaintiffs’ objection was that such regulatory determinations were hearsay, and that they resulted from various procedures, using various criteria, which would require explanation, and would be subject to litigants’ challenges.[11] In other words, for each regulatory agency’s determination, there would be a “mini-trial,” or a “trial within a trial,” about the validity and accuracy of the foreign agency’s assessment.

In the earlier Philadelphia trials, the plaintiffs’ objections were largely sustained, which created a significant evidentiary bias in the courtrooms. Plaintiffs’ expert witnesses could freely discuss the IARC glyphosate evaluation, but the defense and its experts could not discuss the many determinations of the safety of glyphosate. Jurors were apparently left with the erroneous impression that the IARC evaluation was a consensus view of the entire world’s scientific community.

Now plaintiffs’ objection has a point, even though it seems to prove too much and must ultimately fail. In a trial, each side has expert witnesses who can offer an opinion about the key causal issue, whether glyphosate can cause NHL, and whether it caused this plaintiff’s NHL. Each expert witness will have written a report that identifies the facts and data relied upon, and that explains the inferences drawn and conclusions reached. The adversary can challenge the validity of the data, inferences, and conclusions because the opposing expert witness will be subject to cross-examination.

The facts and data relied upon will, however, be “hearsay,” which will come from published studies not written by the expert witnesses at trial. There will be many aspects of the relied upon studies that will be taken on faith without the testimony of the study participants, their healthcare providers, or the scientists who collected the data, chose how to analyze the data, conducted the statistical and scientific analyses, and wrote up the methods and study findings. Permitting reliance upon any study thus allows for a “mini-trial” or a “trial within a trial,” on each study cited and relied upon by the testifying expert witnesses. This complexity involved in expert witness opinion testimony is one of the foundational reasons for Rule 702’s gatekeeping regime in federal court, and most state courts, but which is usually conspicuously absent in Pennsylvania courtrooms.

Furthermore, the plaintiffs’ objections to foreign regulatory determinations would apply to any review paper, and more important, it would apply to the IARC glyphosate monograph itself. After all, if expert witnesses are supposed to have reviewed the underlying studies themselves, and be competent to do so, and to have arrived at an opinion in some reliable way from the facts and data available, then they would have no need to advert to the IARC’s review on the general causation issue.  If an expert witness were allowed to invoke the IARC conclusion, presumably to bolster his or her own causation opinion, then the jury would need to resolve questions about:

  • who was on the working group;
  • how were working group members selected, or excluded;
  • how the working group arrived at its conclusion;
  • what did the working group rely upon, or not rely upon, and why,
  • what was the group’s method for synthesizing facts and data to reach its conclusion;
  • was the working group faithful to its stated methodology;
  • did the working group commit any errors of statistical or scientific judgment along the way;
  • what potential biases did the working group members have;
  • what is the basis for the IARC’s classificatory scheme; and
  • how are IARC’s key terms such as “sufficient,” “limited,” “probable,” “possible,” etc., defined and used by working groups.

Indeed, a very substantial trial could be had on the bona fides and methods of the IARC, and the glyphosate IARC working group in particular.

The curious irony behind the Valentine’s Day order is that plaintiffs’ counsel were generally winning their objections to the defense’s references to foreign regulatory determinations. But as pigs get fatter, hogs get slaughtered. Last year, plaintiffs’ counsel moved to “exclude foreign regulatory registrations and or approvals of glyphosate.”[12] To be sure, plaintiffs’ counsel were not seeking merely the exclusion of glyphosate registrations, but the scientific evaluations of regulatory agencies and their staff scientists and consulting scientists. Plaintiffs wanted trials in which juries would hear only about IARC, as though it was a scientific consensus. The many scientific regulatory considerations and rejections of the IARC evaluation would be purged from the courtroom.

On January 4, 2024, plaintiffs’ counsel obtained what they sought, an order that memorialized the tilted playing field they had largely been enjoying in Philadelphia courtrooms. Judge Roberts’ order was short and somewhat ambiguous:

“upon consideration of plaintiff’s motion in limine no. 5 to exclude foreign regulatory registrations and/or approvals of glyphosate, GBHs, and/or Roundup, any response thereto, the supplements of the parties, and oral argument, it is ORDERED that the motion is GRANTED without prejudice to a party’s introduction of foreign scientific evidence including, but not limited to, evidence from the International Agency for Research on Cancer (IARC), provided that such introduction does not refer to foreign regulatory agencies.”

The courtroom “real world” outcome after Judge Roberts’ order was an obscene verdict in the McKivison case. Again, there may have been many contributing causes to the McKivison verdict, including Pennsylvania’s murky and retrograde law of expert witness opinion testimony.[13] Mr. McKivison was in remission from NHL and had sustained no economic damages, and yet, on January 26, 2024, a jury in his case returned a punitive compensatory damages award of $250 million, and an even more punitive punitive damage award of $2 billion.[14] It seems at least plausible that the imbalance between admitting the IARC evaluation while excluding foreign regulatory assessments helped create a false narrative that scientists and regulators everywhere had determined glyphosate to be unsafe.

On February 2, 2024, the defense moved for a clarification of Judge Roberts’ January 4, 2024 order that applied globally in the Philadelphia glyphosate litigation. The defendants complained that in their previous trial, after Judge Roberts’ Order of January 4, 2024, they were severely prejudiced by being prohibited from referring to the conclusions and assessments of foreign scientists who worked for regulatory agencies. The complaint seems well founded.  If a hearsay evaluation of glyphosate by an IARC working group is relevant and admissible, the conclusions of foreign scientists about glyphosate are relevant and admissible, whether or not they are employed by foreign regulatory agencies. Indeed, plaintiffs’ counsel routinely complained about Monsanto/Bayer’s “influence” over the United States Environmental Protection Agency, but the suggestion that the European Union’s regulators are in the pockets of Bayer is pretty farfetched. Indeed, the complaint about bias is peculiar coming from plaintifs’ counsel, who command an out-sized influence within the Collegium Ramazzini,[15] which in turn often dominates IARC working groups. Every agency and scientific group, including the IARC, has its “method,” its classificatory schemes, its definitions, and the like. By privileging the IARC conclusion, while excluding all the other many agencies and groups, and allowing plaintiffs’ counsel to argue that there is no real-world debate over glyphosate, Philadelphia courts play a malignant role in helping to generate the huge verdicts seen in glyphosate litigation.

The defense motion for clarification also stressed that the issue whether glyphosate causes NHL or other human cancer is not the probandum for which foreign agency and scientific group statements are relevant.  Pennsylvania has a most peculiar, idiosyncratic law of strict liability, under which such statements may not be relevant to liability questions. Plaintiffs’ counsel, in glyphosate and most tort litigations, however, routinely assert negligence as well as punitive damages claims. Allowing plaintiffs’ counsel to create a false and fraudulent narrative that Monsanto has flouted the consensus of the entire scientific and regulatory community in failing to label Roundup with cancer warnings is a travesty of the rule of law.

What seems clever by halves in the plaintiffs’ litigation approach was that its complaints about foreign regulatory assessments applied equally, if not more so, to the IARC glyphosate hazard evaluation. The glyphosate litigation is not likely as interminable as π, but it is irrational.

*      *     *      *      *     * 

Post Script.  Ten days after the verdict in Kline, and one day after the above post, the Philadelphia Inquirer released a story about the defense verdict. See Nick Vadala, “Monsanto wins first Roundup court case in recent string of Philadelphia lawsuits,” Phila. Inq. (Mar. 15, 2024).

[1] Bill 246, Indiana House of Representatives (1897); Petr Beckmann, A History of π at 174 (1971).

[2] See Robert Moran, “Philadelphia jury awards $175 million after deciding 83-year-old man got cancer from Roundup weed killer,” Phila. Inq. (Oct. 27, 2023); Nick Vadala, “Philadelphia jury awards $2.25 billion to man who claimed Roundup weed killer gave him cancer,” Phila. Inq. (Jan. 29, 2024).

[3] Phila. Ct. C.P. 2022-01641.

[4] George Woolston, “Monsanto Nabs 1st Win In Philly’s Roundup Trial Blitz,” Law360 (Mar. 5, 2024); Nicholas Malfitano, “After three initial losses, Roundup manufacturers get their first win in Philly courtroom,” Pennsylvania Record (Mar. 6, 2024).

[5][5] See David Hackett Fischer, “ Fallacies of Semantical Distortion,” chap. 10, in Historians’ Fallacies: Toward a Logic of Historical Thought (1970); see also “IARC’s Fundamental Distinction Between Hazard and Risk – Lost in the Flood” (Feb. 1, 2024); “The IARC-hy of Evidence – Incoherent & Inconsistent Classification of Carcinogencity” (Sept. 19, 2023).

[6] Malfitano, note 2 (quoting Pinto); see also Law360, note 2 (quoting Pinto).

[7] Harry Frankfurt, On Bullshit at 63 (2005); see “The Philosophy of Bad Expert Witness Opinion Testimony” (Oct. 2, 2010).

[8] See Malifanto, note 2 (quoting Pinto).

[9] In re Roundup Prods. Litig., Phila. Cty. Ct. C.P., May Term 2022-0550, Control No. 24020394 (Feb. 14, 2024) (Roberts, J.). In a footnote, the court explained that “an expert may testify that foreign scientists have concluded that Roundup and· glyphosate can be used safely and they do not cause cancer. In the example provided, there is no specific reference to an agency or regulatory body, and the jury is free to make a credibility determination based on the totality of the expert’s testimony. It is, however, impossible for this Court, in a pre-trial posture, to anticipate every iteration of a question asked or answer provided; it remains within the discretion of the trial judge to determine whether a question or answer is appropriate based on the context and the trial circumstances.”

[10] See National Ass’n of Wheat Growers v. Bonta, 85 F.4th 1263, 1270 (9th Cir. 2023) (“A significant number of . . . organizations disagree with IARC’s conclusion that glyphosate is a probable carcinogen”; … “[g]lobal studies from the European Union, Canada, Australia, New Zealand, Japan, and South Korea have all concluded that glyphosate is unlikely to be carcinogenic to humans.”).

[11] See, e.g., In re Seroquel, 601 F. Supp. 2d 1313, 1318 (M.D. Fla. 2009) (noting that references to foreign regulatory actions or decisions “without providing context concerning the regulatory schemes and decision-making processes involved would strip the jury of any framework within which to evaluate the meaning of that evidence”)

[12] McKivison v. Monsanto Co., Phila. Cty. Ct. C.P., No. 2022- 00337, Plaintiff’s Motion in Limine No. 5 to Exclude Foreign Regulatory Registration and/or Approvals of Glyphosate, GHBs and/or Roundup.

[13] See Sherman Joyce, “New Rule 702 Helps Judges Keep Bad Science Out Of Court,” Law360 (Feb. 13, 2024) (noting Pennsylvania’s outlier status on evidence law that enables dodgy opinion testimony).

[14] P.J. D’Annunzio, “Monsanto Fights $2.25B Verdict After Philly Roundup Trial,” Law360 (Feb. 8, 2024).

[15] “Collegium Ramazzini & Its Fellows – The Lobby” (Nov. 19, 2023).

Posted in Causation, Rule 702, Scientific Evidence | 41 Comments »

Purging Compurgation

March 12th, 2024

“You could file briefs on a napkin right now and get it granted.”

Alan Lange & Tom Dawson, Kings of Torts 87 (2d ed. 2010) (quoting convicted former lawyer, Zach Scruggs)

Back in the 1980s, I started to see expert witnesses stray into the business of psychoanalysis of corporate defendants. Perhaps it took place earlier; it seemed to be a tactic when I first started to try cases. Not only did expert witnesses wish to indict products as causes of plaintiffs’ harms, they wanted to indict the motives and intentions of the manufacturers. Such “motive” testimony should have been cleared from courtrooms by the basic rule of expert witness opinion testimony; namely, the warrant for expert witness testimony is that the subject matter is “beyond the ken” of the jury. Given that the tendentious witnesses had no special skills in divining motives, and that jurors were routinely called upon to infer motives, the offending testimony should have been readily quashed. Almost 100 years ago, Judge Learned Hand, confronted with similar argumentative opinion testimony, held, in his magisterial way, that “[a]rgument is argument whether in the box or at the bar, and its proper place is the last.”[1]

What I found when I started trying cases was that many states had hard rules on expert witnesses, but soft judges. In some litigations, plaintiffs’ counsel offered a witness, such as the late Marc Lappé, not only to assess motives, but also to make ethical pronouncements about defendants’ conduct. More typically, the ethical judgments came from historian witnesses or regulatory expert witnesses. Occasionally, expert witnesses on health effects issues offered psychoanalytic opinions as well. Plaintiffs’ counsel typically argued that Federal Rule of Evidence 704, which declared that “[a]n opinion is not objectionable just because it embraces an ultimate issue,” green lighted their witnesses’ amateur or professional psychoanalysis. Defendants typically argued that the common law requirement that opinions be “beyond the ken” of jurors was carried forward in Rule 702’s requirement of relevant expertise, knowledge, and helpfulness to the trier of fact.  State court analogues to these rules replicated the debate in state courts around the country.

The attempt to deprecate the intentions or motives of a party were not necessarily enhanced when the expert witness compurgator had some semblance of subject-matter expertise. In one case, a frequent statistician testifier for the lawsuit industry, Martin Wells, expressed the opinion that the study at issue in the litigation “was seriously flawed by bad epidemiological practice. The combination of bias and poor epidemiologic practice is so rampant that one can easily conclude the study was intentionally designed to achieve a desired result regardless of the actual findings in the data.”[2]

Wells may have been entitled to his opinion about the quality of the study at issue, and if he had good grounds and a reliable methodology, perhaps he should have been permitted to share that opinion with a jury. The court, however, held that opinions based upon “inferences about the intent or motive of parties … lie outside the bounds of expert testimony, but are instead classic jury questions.”[3] The acceptability of the Wells’ compurgation was not improved or made more admissible by coating it with a patina of expertise about interpreting studies. The trial court found that:

“Dr. Wells’ statements represent his subjective beliefs regarding an alleged bad motive or intent on the part of defendants or others who designed the study. The Court finds that his speculation about the reason for alleged methodological issues in the study are not the product of reliable methods, and will be excluded.”[4]

By 2011, or so, the case law interpreting common law and statutory rules about ethics and motive opinion generally tilted in favor of the defense.[5] Courts routinely excluded expert witness opinions about corporate knowledge, motivations, and intent, as irrelevant and inadmissible under Rule 702.

As though ethicist and historian testimony were not bad enough, imagine an economist offering testimony to deprecate lobbying efforts that are protected first amendment speech. In one multi-district litigation, thinking that they could get away with most anything, plaintiffs’ counsel offered just such an expert witness.

The expert witness at issue was an economist, Glen W. Harrison, of no particular distinction, who sought to serve as a compurgator in litigation. Harrison is an accomplished litigation witness, who was developed and trained by the Motley Rice firm and others in many tobacco cases.[6] What is clear is that he was deployed, in MDL 1535, to lobby the fact-finder inappropriately, without any real expertise in the material science, toxicology, or epidemiology issues in the litigation.

The essence of Glen Harrison’s opinion is that the “manufacturing industry” saw itself as having an “economic incentive” to engage in lobbying. This opinion was either tautologically or trivially true, but plaintiffs sought the opportunity to cast lawful (and constitutionally protected) lobbying as nefarious and tortious. A disinterested observer might have thought that the important issue was whether the lobbying was unlawful and thus inappropriate, but Harrison was not an expert on the law governing stakeholders’ submissions to agencies or to organizations that promulgate standards.

Harrison’s opinion on “inappropriateness” was based upon his inexpert factual review of documents, with occasional inferences or comments about whether the documents were incomplete, or inconsistent with other pieces of evidence. What was remarkable about this bold attempt to subvert the MDL trial process was that Harrison had absolutely no expertise or competence to discuss documents that involved issues of epidemiology, risk assessment, neurology, neuropsychology, toxicology, or exposure measurements. Harrison tried to squeeze out some bare relevance by commenting upon documents with his personal, lay observations that they seemed inaccurate, or that they were incomplete. Of course, a lawyer could equally well argue the point to the jury in summation. Clearly, the goal of proffering Harrison was to have a summation from a witness, with a pleasant Australian accent, in the middle of the plaintiffs’ case in chief. If you listened closely, you could hear a roar of disapproval from the Albany Rural Cemetery.[7]

For some time, the MDL 1535 judge winked at the plaintiffs’ and Harrison’s improper ploy to demonize lawful, appropriate industry conduct, and the MDL resolved before the parties obtained a ruling on Harrison’s proffered testimony. While the issue was before the MDL court, it appeared unmoved by considerations of the First Amendment or of the Noerr-Pennington doctrine,[8] or even the statutory invitation and right to comment upon proposed regulations.[9] Of course, manufacturing and lawsuit industries have a right to participate in notice and comment periods of rulemaking. The courtroom asymmetry threatened by Harrison’s proffered testimony was that plaintiffs’ counsel could comment upon defendants’ lobbying, but defense counsel had no equivalent opportunity to comment upon the lawsuit’s extensive rent seeking.[10]

[1] Nichols v. Universal Pictures Corp., 45 F.2d 119, 123 (2d Cir. 1930) (Hand, J.).

[2] In re Trasylol Prods. Liab. Litig., 08-md-01928, 2010 WL 1489793, at *2 (S.D. Fla. Feb. 24, 2010) (quoting from Rule 26 report of Martin T. Wells ¶ 4, Van Steenburgh Affidavit, Exhibit B, Docket No. 1677).

[3] Id. at *8 (internal quotation marks omitted).

[4] Id. at *2.

[5] See Beck, “Experts Offering Evidence of Corporate Intent, Ethics, And The Like,” Drug & Device Law (May 19, 2011) (collecting cases). See, e.g., Kidder, Peabody & Co., Inc. v. IAG Int’l Acceptance Grp., N.V., 14 F.Supp. 2d 391, 404 (S.D.N.Y.1998); Crown Cork, 2013 WL 978980, at *7 (excluding expert opinions of parties’ knowledge, state of mind, and intent); DePaepe v. General Motors Corp., 141 F.3d 714, 720 (7th Cir. 1998) (disallowing opinion of expert witness, who “lacked any scientific basis for an opinion about … motives,” about defendant’s failure to add safety measure in order to “save money”); In re Diet Drugs Prods. Liab. Litig., 2000 WL 876900, at *9 (E.D.Pa. June 20, 2000) (noting that “question of intent is a classic jury question and not one for experts”); Smith v. Wyeth-Ayerst Laboratories Co., 278 F.Supp.2d 684, 700 (W.D.N.C. 2003) (expert witnesses may not opine about corporate intent and motive) (barring Dr. Moye from giving such testimony); In re Rezulin Products Liab. Litig., 309 F. Supp. 2d 531, 543, 545 n.37 (S.D.N.Y.2004) (excluding opinions on intent and motive, as well as historical narrative gleaned form otherwise admissible documentary evidence); In re Baycol Prods. Liab. Litig., 495 F. Supp. 2d 977, 1001 (D.Minn. 2007) (holding expert witness to have exceeded proper proffer “to the extent that he speculates as to Bayer’s motive, intent, or state of mind”); 532 F. Supp. 2d 1029, 1069 (D. Minn. 2007) (“[A]n expert may not testify as to ethical issues or to his personal views”; “[t]he question of corporate intent is one for the jury, not for an expert”); Reece v. Astrazeneca Pharms., LP, 500 F. Supp. 2d 736, 744-46 (S.D. Ohio 2007) (advisability of tests; warnings needed for particular medical conditions; lack of methodology); In re Guidant Corp. Implantable Defibrillators Prods. Liab. Litig., 2007 WL 1964337, at *8 (D. Minn. June 29, 2007); Singh v. Edwards Lifesciences Corp., 2008 WL 5758387, ¶ELS 6 (Wash. Super. Snohomish Cty. Jan. 31, 2008); In re Fosamax Prods. Liab. Litig., 645 F. Supp. 2d 164, 192 (S.D.N.Y. 2009) (granting defendant’s motion to exclude testimony from Dr. Furberg about purported general ethical standards for conducting clinical trials); In re Xerox Corp. Sec. Litig., 746 F. Supp. 2d 402, 415 (D. Conn. 2010) (“Inferences about the intent or motive of parties or others lie outside the bounds of expert testimony.”) (internal citations omitted); In re Gadolinium-Based Contrast Agents Prods. Liab. Litig., 2010 WL 1796334, at *13 (N.D. Ohio May 4, 2010); In re Levaquin Prods. Liab. Litig., 2010 WL 11470977 (D. Minn. Nov. 10, 2010); Deutsch v. Novartis Pharms. Corp., 768 F. Supp.2d 420, 467 (E.D.N.Y. 2011); In re Heparin Prods. Liab. Litig., 2011 WL 1059660, at *8 (N.D. Ohio March 21, 2011); Lemons v. Novartis Pharms. Corp., 849 F. Supp.2d 608, 615 (W.D.N.C. 2012); Hill v. Novartis Pharms. Corp., 2012 WL 5451809, at *2 (E.D. Cal. Nov. 7, 2012); Georges v. Novartis Pharms. Corp., 2012 WL 9064768, at *13 (C.D. Cal. Nov. 2, 2012); Johnson v. Wyeth LLC, 2012 WL 1204081, at *3 (D. Ariz. Apr. 11, 2012); Pritchett v. I-Flow Corp., 2012 WL 1059948, at *6 (D. Colo. Mar. 28, 2012); Chandler v. Greenstone Ltd., 2012 WL 882756, at *1 (W.D. Wash. Mar. 14, 2012); Winter v. Novartis Pharms. Corp., 2012 WL 827305, at *5 (W.D. Mo. March 8, 2012); Earp v. Novartis Pharms. Corp., 2013 WL 4854488, at *4 (E.D.N.C. Sept. 11, 2013).

[6] See, e.g., Group Health Plan, Inc. v. Philip Morris, Inc., 188 F. Supp. 2d 1122 (D. Minn. 2002); Blue Cross & Blue Shield of N.J. v. Philip Morris, 178 F. Supp. 2d 198 (E.D.N.Y. 2001); Rent-A-Center West Inc.  v. Dept. of Revenue, 418 S.C. 320, 792 S.E.2d 260 (2016).

[7] Where Judge Learned Hand was buried.

[8] See, e.g., Video Int’l Prod., Inc. v. Warner-Amex Cable Comm., Inc., 858 F.2d 1075, 1084 (5th Cir. 1988) (applying Noerr-Pennington doctrine to bar use of evidence of lobbying in tort case); Hamilton v. AccuTek, 935 F. Supp. 1307, 1321 (E.D.N.Y. 1996) (granting summary judgment to gun makers on product liability and fraud claims based upon their efforts to influence federal policies by lawful lobbying); In re Municipal Stormwater Pond, No. 18-cv-3495 (JNE/KMM), 2019 U.S. Dist. LEXIS 227887, at *12 (D. Minn. Dec. 20, 2019) (dismissing claims of fraudulent misrepresentation claims against maker of coal-tar sealant on grounds that the Noerr-Pennington doctrine protected manufacturer’s lobbying before state and local governments); Eiser v. Brown & Williamson Tobacco Corp., Phila. Ct. Com. Pleas LEXIS 43, *20, 2005 WL 1323030 (2005) (invoking Noerr-Pennington doctrine to bar evidence of defendant manufacturer’s lobbying in products liability case). See generally James M. Sabovich, “Petition without Exception: Against the Fraud Exception to Noerr-Pennington Immunity from the Toxic Tort Perspective,” 17 Penn State Envt’l L. Rev. 101 (2008).

[9] See Admin. Procedures Act, 5 U.S.C. § 553; Attorney General’s Manual on the Administrative Procedure Act 31 (1947) (“[t]he objective should be to assure informed administrative action and adequate protection to private interests”).

[10] Lawsuit industry certainly exercises its rent-seeking through legitimate lobbying, and occasionally through illegimate means.  See U.S. v. Scruggs, 691 F.3d 660 (5th Cir. 2012).

Posted in Expert Witnesses, Rule 702 | 4 Comments »

A Citation for Jurs & DeVito’s Unlawful U-Turn

February 27th, 2024

Antic proposals abound in the legal analysis of expert witness opinion evidence. In the courtroom, the standards for admitting or excluding such evidence are found in judicial decisions or in statutes. When legislatures have specified standards for admitting expert witness opinions, courts have a duty to apply the standards to the facts before them. Law professors are, of course, untethered from either precedent or statute, and so we may expect chaos to ensue when they wade into disputes about the proper scope of expert witness gatekeeping.

Andrew Jurs teaches about science and the law at the Drake University Law School, and Scott DeVito is an associate professor of law at the Jacksonville University School of Law. Together, they have recently produced one of the most antic of antic proposals in a fatuous call for the wholesale revision of the law of expert witnesses.[1]

Jurs and DeVito rightly point out that since the Supreme Court, in Daubert,[2] waded into the dispute whether the historical Frye decision survived the enactment of the Federal Rules of Evidence, we have seen lower courts apply the legal standard inconsistently and sometimes incoherently. These authors, however, like many other academics, incorrectly label one or the other standard, Frye or Daubert, as being stricter than the other. Applying the labels of stricter and weaker standards, ignores that they are standards that measure completely different things. Frye advances a sociological standard, and a Frye test challenge can be answered by conducting a survey. Rule 702, as interpreted by Daubert, and as since revised and adopted by the Supreme Court and Congress, is an epistemic standard. Jurs and DeVito, like many other legal academic writers, apply a single adjective to standards that measure two different, incommensurate things. The authors’ repetition of the now 30-plus year-old mistake is a poor start for a law review article that sets out to reform the widespread inconsistency in the application of Rule 702, in federal and in state courts.

In seeking greater adherence to the actual rule, and consistency among decisions, Jurs and DeVito might have urged for judicial education, or blue-ribbon juries, or science courts, or greater use of court-appointed expert witnesses. Instead they have put their marker down on abandoning all meaningful gatekeeping. Jurs and DeVito are intent upon repairing the inconsistency and incoherency in the application of Daubert, by removing the standard altogether.

“To resolve the problem, we propose that the Courts replace the multiple Daubert factors with a single factor—testability—and that once the evidence meets this standard the judge should provide the jury with a proposed jury instruction to guide their analysis of the fact question addressed by the expert evidence.”[3]

In other words, because lower federal courts have routinely ignored the actual statutory language of Rule 702, and Supreme Court precedents, Jurs and DeVito would have courts invent a new standard, that virtually excludes nothing as long as someone can imagine a test for the asserted opinion. Remarkably, although they carry on about the “rule of law,” the authors fail to mention that judges have no authority to ignore the requirements of Rule 702. And perhaps even more stunning is that they have advanced their nihilistic proposal in the face of the remedial changes in Rule 702, designed to address judicial lawlessness in ignoring previously enacted versions of Rule 702. This antic proposal would bootstrap previous judicial “flyblowing” of a Congressional mandate into a prescription for abandoning any meaningful standard. They have articulated the Cole Porter standard: anything goes. Any opinion that “can be tested is science”; end of discussion.  The rest is for the jury to decide as a question of fact, subject to the fact finder’s credibility determinations. This would be a Scott v. Sandford rule[4] for scientific validity; science has no claims of validity that the law is bound to respect.

Jurs and DeVito attempt a cynical trick. They argue that they would fix the problem of “an unpredictable standard” by reverting to what they say is Daubert’s first principle of ensuring the reliability of expert witness testimony, and limiting the evidentiary display at trial to “good science.” Cloaking their nihilism, the authors say that they want to promote “good science,” but advocate the admissibility of any and every opinion, as long as it is theoretically “testable.” In order to achieve this befuddled goal, they simply redefine scientific knowledge as “essentially” equal to testable propositions.[5]

Jurs and DeVito marshal evidence of judicial ignorance of key aspects of scientific method, such as error rate. We can all agree that judges frequently misunderstand key scientific concepts, but their misunderstandings and misapplications do not mean that the concepts are unimportant or unnecessary. Many judges seem unable to deliver an opinion that correctly defines p-value or confidence interval, but their inabilities do not allow us to dispense with the need to assess random error in statistical tests. Our faint-hearted authors never explain why the prevalence of judicial error must be a counsel of despair that drives us to bowdlerize scientific evidence into something it is not. We may simply need better training for judges, or better assistance for them in addressing complex claims. Ultimately, we need better judges.

For those judges who have taken their responsibility seriously, and who have engaged with the complexities of evaluating validity concerns raised in Rule 702 and 703 challenges, the Jurs and DeVito proposal must seem quite patronizing. The “Daubert” factors are simply too complex for you, so we will just give you crayons, or a single, meaningless factor that you cannot screw up.[6]

The authors set out a breezy, selective review of statements by a few scientists and philosophers of science. Rather than supporting the extreme reductionism, Jurs and DeVito’s review reveals that science is much more than identifying a “testable” proposition. Indeed, the article’s discussion of philosophy and practice of science weighs strongly against the authors’ addled proposal.[7]

The authors, for example, note that Sir Isaac Newton emphasized the importance of empirical method.[8] Contrary to the article’s radical reductionism, the authors note that Sir Karl Popper and Albert Einstein stressed that the failure to obtain a predicted experimental result may render a theory “untenable,” which of course requires data and valid tests and inferences to assess. Quite a bit of motivated reasoning has led Jurs and DeVito to confuse a criterion of testability with the whole enterprise of science, and to ignore the various criteria of validity for collecting data, testing hypotheses, and interpreting results.

The authors suggest that their proposal will limit the judicial inquiry to the the legal question of reliability, but this suggestion is mere farce. Reliability means obtaining the same or sufficiently similar results upon repeated testing, but these authors abjure testing itself.  Furthermore, reliability as contemplated by the Supreme Court, in 1993, and by FRE 702 ever since, has meant validity of the actual test that an expert witness argues in support of his or her opinion or claims.

Whimsically, and without evidence, Jurs and DeVito claim that their radical abandonment of gatekeeping will encourage scientists, in “fields that are testable, but not yet tested, to perform real, objective, and detailed research.” Their proposal, however, works to remove any such incentive because untested but testable research becomes freely admissible. Why would the lawsuit industry fund studies, which might not support their litigation claims, when the industry’s witnesses need only imagine a possible test to advance their claims, without the potential embarrassment by facts? The history of modern tort law teaches us that cheap speculation would quickly push out actual scientific studies.

The authors’ proposal would simply open the floodgates of speculation, conjecture, and untested hypothesis, and leave the rest to the vagaries of trials, mostly in front of jurors untrained in evaluating scientific and statistical evidence. Admittedly, some incurious and incompetent gatekeepers and triers of fact will be relieved to know that they will not have to evaluate actual scientific evidence, because it had been eliminated by the Jurs and DeVito proposal to make mere testability the touchstone of admissibility

To be sure, in Aristotelian terms, testability is logical and practically prior to testing, but these relationships do not justify holding out testability as the “essence” of science, and the alpha and omega of science.[9] Of course, one must have an hypothesis to engage in hypothesis testing, but science lies in the clever interrogation of nature, guided by the hypothesis. The scientific process lies in answering the question, not simply in formulating the question.

As for the authors’ professed concern about “rule of law,” readers should note that the Jurs and DeVito article completely ignores the remedial amendment to Rule 702, which went into effect on December 1, 2023, to address the myriad inconsistencies, and failures to engage, in required gatekeeping of expert witness opinion testimony.[10]

The new Rule 702 is now law, with its remedial clarification that the proponent of expert witness opinion must show the court that the opinion is sufficiently supported by facts or data, Rule 702(b), and that the opinion “reflects a reliable application of the principles and methods to the facts of the case,” Rule 702(d). The Rule prohibits deferring the evaluation of sufficiency of support or reliability of application of method to the trier of fact; there is no statutory support for suggesting that these inquires always or usually go to “weight and not admissibility.”

The Jurs and DeVito proposal would indeed be a U-Turn in the law of expert witness opinion testimony. Rather than promote the rule of law, they have issued an open, transparent call for licentiousness in the adjudication of scientific and technical issues.

[1] Andrew Jurs & Scott DeVito, “A Return to Rationality: Restoring the Rule of Law After Daubert’s Disasterous U-Turn,” 164 New Mexico L. Rev. 164 (2024) [cited below as U-Turn]

[2] Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).

[3] U-Turn at 164, Abstract.

[4] 60 U.S. 393 (1857).

[5] U-Turn at 167.

[6] U-Turn at 192.

[7] See, e.g., U-Turn at 193 n.179, citing David C. Gooding, “Experiment,” in W.H. Newton-Smith, ed., A Companion to the Philosophy of Science 117 (2000) (emphasizing the role of actual experimentation, not the possibility of experimentation, in the development of science).

[8] U-Turn at 194.

[9] See U-Turn at 196.

[10] See Supreme Court Order, at 3 (Apr. 24, 2023); Supreme Court Transmittal Package (Apr. 24, 2023).

Posted in Expert Witnesses, Rule 702, Scientific Evidence | 269 Comments »

Grumpy Old Men

February 11th, 2024

This blog is not about politics, although sometimes I have wandered into the political thicket when the events of the day involved scientific and statistical issues.[1] Our current events today do not involve statistical evidence so much as political, moral, and practical judgment. At some point, however, I cannot avoid a sense of responsibility to speak on the issues. On the one side, we have a grumpy old man who has evidenced some senior moments. On the other side, we have a grumpy old man who has evidenced senior moments as well as a vile character and temperament, generally devoid of decency, historical knowledge, and practical judgment.

Some have argued that the first grumpy old man is propped up by Marxists, the ultra-woke, and “haters” of the USA. Others have argued that the second grumpy old man is a creature of a personality cult, and that he is a documented rapist, fraudster, liar, hustler, and worse.[2]

Well, I know I am not going to firm up the political quagmire on which we now stand. But I have a humble, and I hope constructive, suggestion. Supporters of both grumpy old men would, I think, agree that they care about strength, intelligence, tactical and strategic ability, mental acuity and stamina, and a desire to win.  My suggestion is instead of having debates between the main protagonists, which many would agree to be a waste of time and money, we should instead require the grumpy old men to face off in a chess match. It need not be long; it could be, say, best of three games. Their moves would be observable by all, including their ability to conform to the rules of game, which require, among other things, refraining from talking and making noise. The American public, and the world, would see whether one or the other, or both, or neither, of the grumpy old men have the cognitive and intellectual ability and a true “killer” instinct, to be a national leader in these troubled times. Professional psychologists could analyze the intelligence of the players in real time.

Perhaps the rules of the game might be modified to include shock collars to discipline either player who spoke during the game, other than to offer a draw, or to resign.

Perhaps I am grumpy old man myself.

[1] See “Pernicious Probabilities in the Supreme Court” (Nov. 21, 2020); “A Trumpence for Your Thoughts” (Nov. 21, 2020). And I have supported sound deployment of statistical evidence. See, e.g., Brief of 27 Election Law, Scientific Evidence, and Empirical Legal Scholars as Amici Curiae in Support of Appellees, in Rucho v. Common Cause, No. 18-422, Supreme Court of the United States (Mar. 9, 2019); Brief of 44 Election Law, Scientific Evidence, and Empirical Legal Scholars as Amici Curiae in Support of Appellees, in Gill v. Whitford, No. 16-1161, Supreme Court of the United States (Sept. 1, 2017).

[2] The supporters of the second grumpy old man have criticized those who criticize his supporters, on grounds that there are substantive issues at stake in their support.  See Julian Adorney, “Stop Dunking on Trump Supporters: No one is beyond reach—unless everyone around them refuses to reach out,” Quilette (Feb. 7, 2024). This of course misses the point that the supporters have had plenty of other viable candidates to support who would have actually been articulate voices on their behalf.  The supporters have shown, consistently over eight years, that they do not care about immorality or sycophancy to dictators, and that they prefer a sexual abuser and fraudster.

Posted in Uncategorized | 35 Comments »

IARC’S Fundamental Distinction Between Hazard and Risk – Lost in the Flood

February 1st, 2024

Socrates viewed philosophy as beginning in wonder,[1] but Socrates and his philosophic heirs recognized that philosophy does not get down to business until it starts to clarify the terms of discussion. By the middle of the last century, failure to understand the logic of language replaced wonder as the beginning of philosophy.[2] Even if philosophy could not cure all conceptual pathology, most writers came to see that clarifying terms, concepts, and usage was an essential starting point in thinking clearly about a subject.[3]

Hazard versus Risk

Precision in scientific exposition often follows from the use of measurements, using agreed upon quantitative units, and accepted, accurate, reliable procedures for measurements. When scientists substitute qualitative measures for what are inherently quantitative measures, they frequently lapse into error. For example, beware of rodent studies that proclaim harms at “low doses,” which turn out to low in comparison to other rodent studies, but orders of magnitude greater exposure than experienced by human beings.

Risk is a quantitative term meaning a rate of some specified outcome. A Dictionary of Epidemiology, for instance, defines risk as:

“The probability of an adverse or beneficial event in a defined population over a specified time interval. In epidemiology and in clinical research it is commonly measured through the cumulative incidence and the incidence proportion.”[4]

An increased risk thus requires a measurement of a rate or probability of an outcome greater than expected in the absence of the exposure of interest. We might be uncertain of the precise measure of the risk, or of an increased risk, but conceptually a risk connotes a rate or a probability that is, at least in theory, measurable.

Hazard is a categorical concept; something is, or is not, a hazard without regard to the rate or incidence of harm. The definition of “hazard,” in the Dictionary of Epidemiology provides a definition that captures the non-quantitative, categorical natural of some exposure’s being a hazard:

“The inherent capability of a natural or human-made agent or process to adversely affect human life, health, property, or activity, with the potential to cause a disease.”[5]

The International Agency for Research on Cancer (IARC) purports to set out a classification scheme, for human cancer hazards. As used by IARC, its classification scheme involves a set of epistemic modal terms: “known,” “probably,” “possibly,” and “indeterminate.” These epistemic modalities characterize the strength of the evidence that an agent is carcinogenic, and not the magnitude of quantitative risk of cancer from exposure at a given level. The IARC Preamble, which attempts to describe the Agency’s methodology, explains that the distinction between hazard and risk is “fundamental”:

“A cancer hazard is an agent that is capable of causing cancer, whereas a cancer risk is an estimate of the probability that cancer will occur given some level of exposure to a cancer hazard. The Monographs assess the strength of evidence that an agent is a cancer hazard. The distinction between hazard and risk is fundamental. The Monographs identify cancer hazards even when risks appear to be low in some exposure scenarios. This is because the exposure may be widespread at low levels, and because exposure levels in many populations are not known or documented.”[6]

This attempted explanation reveals an important problem in IARC’s project, as stated in the Preamble. First, there is an unproven assumption that there will be cancer hazards regardless of the exposure levels. The IARC contemplates that there may circumstances of low levels of risk from low levels of exposure, but it elides the important issue of thresholds of exposure, below which there is no risk. The Preamble suggests that IARC does not attempt to provide evidence for or against meaningful thresholds of hazardousness, but this failure greatly undermines the project.  Exposure circumstances may be such that there is no hazard at all, and so the risk is zero.

The purported distinction between hazard and risk, supposedly fundamental, is often blurred by the Agency, in the monographs produced by working groups on specific exposure circumstances. Consider for instance how a working group characterized the “hazard” of inhalation of respirable crystalline silica:

“ln making the overalI evaluation, the Working Group noted that carcinogenicity in humans was not detected in all industrial circumstances studied. Carcinogenicity may be dependent on inherent characteristics of the crystalline silica or on external factors affecting its biological activity or distribution of its polymorphs.

Crystalline silica inhaled in the form of quartz or cristobalite from occupational sources is carcinogenic to humans (Group 1)”[7]

So some IARC classifications actually do specify that exposure to a substance is not a hazard in all circumstances, a qualification that implies that the same exposure in some exposure circumstances is not a hazard, and so the risk is zero.

We know something about the deliberations of the crystalline silica working group. The members were deadlocked for some time, and the switch of one vote ultimately gave a bare majority to reclassifying crystalline silica as a Group I exposure. Here is how the working group member, Corbett McDonald described the situation:

“The IARC Working Group, in 1997, had considerable difficulty in reaching a decision and might well not have done so had it not been made clear that it was concerned with hazard identification, not risk.”[8]

It was indeed Professor McDonald who changed his vote based upon this linguistic distinction between hazard and risk. His own description of the dataset, however, suggests that the elderly McDonald was railroaded by more younger, more strident members of the group:

“Of the many studies reviewed by the Working Group … nine were identified as providing the least confounded evidence. Four studies which we considered positive included two of refractory brick workers, one in the diatomite industry and our own in pottery workers; the five which seemed negative or equivocal included studies of South Dakota gold miners, Danish stone workers, US stone workers and US granite workers. This further example that the truth is seldom pure and never simple underlines the difficulty of establishing a rational control policy for some carcinogenic materials.”[9]

In defense of his vote, McDonald meekly offered that

“[s]ome equally expert panel of scientists presented with the same information on another occasion could of course have reached a different verdict. The evidence was conflicting and difficult to assess and such judgments are essentially subjective. Of course, when the evidence is conflicting, it cannot be said to be sufficient. Not only was the epidemiologic evidence conflicting, but so was the whole animal toxicology, which found a risk of tumors in rats, but not in mice or hamsters.”

Aside from endorsing a Group I classification for crystalline silica, the working group ignored the purportedly fundamental distinction between hazard and risk, by noting that not all exposure circumstances posed a hazard of cancer. The same working group did even greater violence to the supposed distinction between risk and hazard in its evaluation of coal dust exposure and human cancer. Coal miners have been studied extensively for cancer risk, and the working group reviewed and evaluated the nature of their exposures and their cancer outcomes. Coal dust virtually always contains crystalline silica, often making up a sizable percentage of the total inhalational exposures (40% or so) of coal dust.[10] And yet, when the group studied the cancer rates among coal miners, and in animals, it concluded that there was “inadequate evidence in humans, and “in experimental animals,” for carcinogenicity. The same working group that agreed, on a divided vote, to place crystalline silica in Group 1, voted that “[c]oal dust cannot be classified as to its carcinogenicity to humans (Group 3).”[11]

The conceptual confusion between hazard and risk is compounded by the IARC’s use of epistemic modalities – known, probably, possibly, and indeterminate – to characterize the existence of a hazard. The Preamble, in Table 4, summarizes the categories and “the stream of evidence” needed to place any particular exposure in a one epistemic modal class or another. What is inexplicable is how and why a single substance such as crystalline silica goes from a known cancer hazard in some unspecified occupational setting but then its putative carcinogenicity becomes indeterminate when it makes up 40% of the inhaled dust in a coal mine.

 

The conceptual difficulty created by IARC’s fundamental distinction between hazard and risk is that risk might well vary across exposure circumstances, but there is no basis for varying the epistemic modality for the hazard assessment simply because coal dust is only say 40% crystalline silica. Some of the exposure circumstances evaluated for the Group I silica hazard classification actually were lower than the silica content of coal.  Granite quarrying, for example, involves exposure to rock dust that is roughly 30% crystalline silica.

The conceptual and epistemic confusion caused by IARC’s treatment of the same substance in different exposure circumstances is hardly unique to its treatment of crystalline silica and coal dust. Benzene has long been classified as a Group I human carcinogen, for its ability to cause a specific form of leukemia.[12] Gasoline contains, on average, about one percent benzene, and so gasoline exposure inevitably involves benzene exposure. And yet, benzene exposure in the form of inhaling gasoline fumes is only a “possible” human carcinogen, Group 2B.[13]

Similarly, in 2018, the IARC classified the evidence for the human carcinogenicity of coffee as “indeterminate,” Group 3.[14] And yet every drop of coffee inevitably contains acrylamide, which is, according to IARC, a Group 2A “probable human carcinogen.”[15] Rent-seeking groups, such as the Council for Education and Research on Toxics (founded by Carl Cranor and Martyn Smith) have tried shamelessly to weaponize the IARC 2A classification for acrylaminde by claiming a bounty against coffee sellers such as Star-Bucks in California Proposition 65 litigation.[16]

Similarly confusing, IARC designates acetaldehyde on its own a “possible” human carcinogen, group 2B, even though acetaldehyde is invariably associated with the metabolism of ethyl alcohol, which itself is a Group I human carcinogen.[17] There may well be other instances of such confusions, and I would welcome examples from readers.

These disparate conclusions strain credulity, and undermine confidence that the hazard-risk distinction does any work at all. Hazard and risk do have different meanings, and I would not want to be viewed as anti-semantic. IARC’s use of the hazard-risk distinction, however, lends itself to the interpretation that hazard is simply risk without the quantification. This usage actually is worse than having no distinction at all, because it ignores the existence of thresholds below which exposure carries no risk, as well as ignoring different routes of exposure and exposure circumstances that carry no risk at all. The vague and unquantified categorical determination that a substance is a hazard allows fear mongers to substitute subjective, emotive, and unscientific judgments for scientific assessment of carcinogenicity under realistic conditions of use and exposure.

[1] Plato, Theaetetus 155d (Fowler transl. 1921).

[2] Ludwig Wittgenstein, Philosophical Investigations (1953).

[3] See, e.g., Richard M. Rorty, ed., The Linguistic Turn: Essays in Philosophical Method (1992); Nicholas Rescher, Concept Audits: A Philosophical Method (2016); Timothy Williamson, Philosophical Method: A Very Short Introduction 32 (2020) (discussing the need to clarify terms).

[4] Miquel Porta, Sander Greenland, Miguel Hernán, Isabel dos Santos Silva, John M. Last, and Andrea Burón, A Dictionary of Epidemiology 250 (6th ed. 2014).

[5] Id. at 128.

[6] IARC Monographs on the Identification of Carcinogenic Hazards to Humans – Preamble (2019) (emphasis added).

[7] IARC Monograph on the Evaluation of Carcinogenic Risks to Humans: Volume 68, Silica, Some Silicates, Coal Dust, and para-Aramid Fibrils 210-211 (1997).

[8] Corbett McDonald & Nicola Cherry, “Crystalline Silica and Lung Cancer: The Problem of Conflicting Evidence,” 8 Indoor Built Environment 121, 121 (1999).

[9] Id.

[10] IARC Monograph on the Evaluation of Carcinogenic Risks to Humans: Volume 68, Silica, Some Silicates, Coal Dust, and para-Aramid Fibrils 340 (1997).

[11] Id. at 393.

[12] IARC Monograph, Volume 120: Benzene (2018).

[13] IARC Monographs on the Evaluation of Carcinogenic Risks to Humans: Volume 45, Occupational Exposures in Petroleum Refining; Crude Oil and Major Petroleum Fuels 194 (1989).

[14] IARC Monograph No. 116, Drinking Coffee, Mate, and Very Hot Beverages (2018).

[15] IARC Monograph no. 60, Some Industrial Chemicals (1994).

[16] See “Coffee with Cream, Sugar & a Dash of Acrylamide” (June 9, 2018); “The Council for Education and Research on Toxics” (July 9, 2013).

[17] IARC Monographs on the Evaluation of Carcinogenic Risks to Humans Volume 96 1278 (2010).

Posted in Causation, Scientific Evidence | 3 Comments »

The Maestro and Mesothelioma – Wikipedia & False Claims

January 21st, 2024

The Maestro is a biographical film of the late Leonard Bernstein. The film, starring Bradley Cooper as Bernstein, had a limited release before streaming on Netflix. As a work of biography, the film is peculiar in its focus on Bernstein’s sexuality and filandering, while paying virtually no attention to his radical chic politics, or his engagement with teaching music appreciation.

In any event, the film sent me to Wikipedia to fact check some details of Bernstein’s life, and I was surprised to see that Wikipedia described Bernstein’s cause of death as involving mesothelioma:

“Bernstein announced his retirement from conducting on October 9, 1990.[174] He died five days later at the age of 72, in his New York apartment at The Dakota, of a heart attack brought on by mesothelioma.[175][2]”

Bernstein certainly did not have occupational exposure to amphibole asbestos, but he did smoke cigarettes, several packs a day, for decades. Mesothelioma seemed unlikely, unless perhaps he smoked Kent cigarettes in the 1950s, when they had crocidolite filters. As you can see from the above quote, the Wikipedia article cites two sources, a newspaper account and a book. Footnote number 2 is an obituary was written by Donal Henahan, and printed in the New York Times.[1] The Times reported that:

“Leonard Bernstein, one of the most prodigally talented and successful musicians in American history, died yesterday evening at his apartment at the Dakota on the Upper West Side of Manhattan. He was 72 years old.

*   *   *   *   *   *   *

Mr. Bernstein’s spokeswoman, Margaret Carson, said he died of a heart attack caused by progressive lung failure.”

There is no mention of mesothelioma in the Times article, and the citation provided does not support the assertion that mesothelioma was involved in the cause of Bernstein’s death. The obituary cited was published the day following Bernstein’s death the night before, which suggests that there was no information from an autopsy, which would have been important in ascertaining any tissue pathology for an accurate and complete cause of death. In 1990, the diagnosis of malignant mesothelioma was often uncertain, even with extensive tissue available post-mortem.

The other citation provided by the Wikipedia article was even less impressive. Footnote 175 pointed to a book of short articles on musicians, with an entry for Bernstein.[2] The book tells us that

“Bernstein is most remembered, perhaps, for his flamboyant conducting style. *** Leonard Bernstein died at his home from cardiac arrest brought on by mesothelioma.”

The blurb on Bernstein provides no support for the statement that cardiac arrest was brought on by mesothelioma, and the narrative struck me as odd in leaving out the progressive lung failure caused by non-malignant smoking-induced lung disease.

I set out to find what else may have been written about the causes of Bernstein’s death. I was surprised to find other references to mesothelioma, but all without any support. One online article seemed promising, but offered a glib conclusion without any source:

“Leonard Bernstein, a towering figure in American music, met his end on October 14, 1990, just five days after retiring from his illustrious career as a conductor. Found in his New York apartment, the cause of his death was a heart attack induced by mesothelioma, a consequence of a lifetime of smoking.”[3]

The lack of any foot- or end-notes disqualifies this source, and others, for establishing a diagnosis of mesothelioma. Other internet articles, inspired by the Cooper production of Maestro, made very similar statements, all without citing any source.[4] Some of the internet articles likely plagiarized others, but I was unable to find who first gave rise to the conclusion that Bernstein died of complications of “mesothelioma” caused by smoking.

Whence came the Wikipedia’s pronouncement that Bernstein died of, or with, mesothelioma? Two “mainstream” print newspapers provided some real information and insight. An article in the Washington Post elaborated on Bernstein’s final illness and the cause of his death:

“Leonard Bernstein, 72, a giant in the American musical community who was simultaneously one of this nation’s most respected and versatile composers and preeminent conductors, died yesterday at his Manhattan apartment. He died in the presence of his physician, who said the cause of death was sudden cardiac arrest caused by progressive lung failure.

On the advice of the doctor, Kevin M. Cahill, Bernstein had announced through a spokeswoman Tuesday that he would retire from conducting. Cahill said progressive emphysema complicated by a pleural tumor and a series of lung infections had left Bernstein too weak to continue working.”[5]

Ah a pleural tumor, but no report or representation that it was malignant mesothelioma.

The Los Angeles Times, with the benefit of an extra three hours to prepare its obituary for a west coast audience, provided similar, detailed information about Bernstein’s death:

“Bernstein, known and beloved by the world as ‘Lenny’, died at 6:15 p.m. in the presence of his son, Alexander, and physician, Kevin M. Cahill, who said the cause of death was complications of progressive lung failure. On Cahill’s advice, the conductor had announced Tuesday that he would retire. Cahill said progressive emphysema complicated by a pleural tumor and a series of lung infections had left Bernstein too weak to continue working.”[6]

Now a pleural tumor can be benign or malignant. And if the tumor were malignant, it may or may not be a primary tumor of the pleura. Metastatic lesions of the pleura, or in the lung parenchyma adjacent to the pleura are common enough that the physician’s statement about tumor of the pleura cannot be transformed into a conclusion about mesothelioma.[7]

Feeling good about having sorted a confusion, I thought I could add to the font of all knowledge, Wikipedia, by editing its unsupported statement about mesothelioma to “pleural tumor.” I made the edit, but within a few days, someone had changed the text back to mesothelioma, without adding any support. The strength of any statement is, of course, based entirely upon its support and the strength of its inferences. Wikipedia certainly can be a reasonable starting place to look for information, but it has no ability to support a claim, whether historical, scientific, or medical. Perhaps I should have added the citation to the Washington Post obituary when I made my edit. Still, it was clear that nothing in article’s footnotes supported the text, and someone felt justified in returning the mention of mesothelioma based upon two completely unsupportive sources. Not only is the Bernstein article in Wikipedia suspect, but there is actually an entry in Wikipedia for “Deaths from Mesothelioma,” which lists Bernstein as well. The article has but one sentence: “This is a list of individuals who have died as a result of mesothelioma, which is usually caused by exposure to asbestos.” And then follows a list of 67 persons, of varying degree of noteworthiness, who supposedly died of mesothelioma. I wonder how many of the entries are false.

[1] Donal  Henahan, “Leonard Bernstein, 72, Music’s Monarch, Dies,” New York Times (October 15, 1990).

[2] Scott Stanton, The Tombstone Tourist: Musicians at 29 (2003).

[3] Soumyadeep Ganguly, “Leonard Bernstein’s cause of death explored: How does Bradley Cooper Maestro end? Movie ending explored,” SK POP (modified Dec 25, 2023).

[4] See, e.g., Gargi Chatterjee, “How did Leonard Bernstein die?” pinkvilla (Dec 23, 2023).

[5] Bart Barnes, “Conductor Leonard Bernstein Dies at 72,” Wash. Post (Oct. 15, 1990) (emphasis added).

[6] Myrna Oliver, “Leonard Bernstein Dies; Conductor, Composer. Renaissance man of his art was 72. The longtime leader of the N.Y. Philharmonic carved a niche in history with ‘West Side Story’,” Los Angeles Times (Oct. 15, 1990) (emphasis added).

[7] See, e.g., Julie Desimpel, Filip M. Vanhoenacker, Laurens Carp, and Annemiek Snoeckx, “Tumor and tumorlike conditions of the pleura and juxtapleural region: review of imaging findings,” 12 Insights Imaging 97 (2021).

Posted in Asbestos, Peer Review | 140 Comments »

« Older Entries
Newer Entries »

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.