TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Reference Manual on Scientific Evidence v4.0

February 28th, 2021

The need for revisions to the third edition of the Reference Manual on Scientific Evidence (RMSE) has been apparent since its publication in 2011. A decade has passed, and the federal agencies involved in the third edition, the Federal Judicial Center (FJC) and the National Academies of Science Engineering and Medicine (NASEM), are assembling staff to prepare the long-needed revisions.

The first sign of life for this new edition came back on November 24, 2020, when the NASEM held a short, closed door virtual meeting to discuss planning for a fourth edition.[1] The meeting was billed by the NASEM as “the first meeting of the Committee on Emerging Areas of Science, Engineering, and Medicine for the Courts: Identifying Chapters for a Fourth Edition of The Reference Manual on Scientific Evidence.” The Committee members heard from John S. Cooke (FJC Director), and Alan Tomkins and Reggie Sheehan, both of the National Science Foundation (NSF). The stated purpose of the meeting was to review the third edition of the RMSE to identify “identify areas of science, technology, and medicine that may be candidates for new or updated chapters in a proposed new (fourth) edition of the manual.” The only public pronouncement from the first meeting was that the committee would sponsor a workshop on the topic of new chapters for the RMSE, in early 2021.

The Committee’s second meeting took place a week later, again in closed session.[2] The stated purpose of the Committee’s second meeting was to review the third edition of the RMSE, and to discuss candidate areas for inclusion as new and updated chapters for a fourth edition.

Last week saw the Committee’s third, public meeting. The meeting spanned two days (Feb. 24 and 25, 2021), and was open to the public. The meeting was sponsored by NASEM, FJC, along with the NSF, and was co-chaired by Thomas D. Albright, Professor and Conrad T. Prebys Chair at the Salk Institute for Biological Studies, and the Hon. Kathleen McDonald O’Malley, who sits on the United States Court of Appeals for the Federal Circuit. Identified members of the committee include:

Steven M. Bellovin, professor in the Computer Science department at Columbia University;

Karen Kafadar, Departmental Chair and Commonwealth Professor of Statistics at the University of Virginia, and former president of the American Statistical Association;

Andrew Maynard, professor, and director of the Risk Innovation Lab at the School for the Future of Innovation in Society, at Arizona State University;

Venkatachalam Ramaswamy, Director of the Geophysical Fluid Dynamics Laboratory of the National Oceanic and Atmospheric Administration (NOAA) Office of Oceanic and Atmospheric Research (OAR), studying climate modeling and climate change;

Thomas Schroeder, Chief Judge for the U.S. District Court for the Middle District of North Carolina;

David S. Tatel, United States Court of Appeals for the District of Columbia Circuit; and

Steven R. Kendall, Staff Officer

The meeting comprised five panel presentations, made up of remarkably accomplished and talented speakers. Each panel’s presentations were followed by discussion among the panelists, and the committee members. Some panels answered questions submitted from the public audience. Judge O’Malley opened the meeting with introductory remarks about the purpose and scope of the RMSE, and of the inquiry into additional possible chapters.

  1. Challenges in Evaluating Scientific Evidence in Court

The first panel consisted entirely of judges, who held forth on their approaches to judicial gatekeeping of expert witnesses, and their approach to scientific and technical issues. Chief Judge Schroeder moderated the presentations of panelists:

Barbara Parker Hervey, Texas Court of Criminal Appeals;

Patti B. Saris, Chief Judge of the United States District Court for the District of Massachusetts,  member of President’s Council of Advisors on Science and Technology (PCAST);

Leonard P. Stark, U.S. District Court for the District of Delaware; and

Sarah S. Vance, Judge (former Chief Judge) of the U.S. District Court for the Eastern District of Louisiana, chair of the Judicial Panel on Multidistrict Litigation.

  1. Emerging Issues in the Climate and Environmental Sciences

Paul Hanle, of the Environmental Law Institute moderated presenters:

Joellen L. Russell, the Thomas R. Brown Distinguished Chair of Integrative Science and Professor at the University of Arizona in the Department of Geosciences;

Veerabhadran Ramanathan, Edward A. Frieman Endowed Presidential Chair in Climate Sustainability at the Scripps Institution of Oceanography at the University of California, San Diego;

Benjamin D. Santer, atmospheric scientist at Lawrence Livermore National Laboratory; and

Donald J. Wuebbles, the Harry E. Preble Professor of Atmospheric Science at the University of Illinois.

  1. Emerging Issues in Computer Science and Information Technology

Josh Goldfoot, Principal Deputy Chief, Computer Crime & Intellectual Property Section, at U.S. Department of Justice, moderated panelists:

Jeremy J. Epstein, Deputy Division Director of Computer and Information Science and Engineering (CISE) and Computer and Network Systems (CNS) at the National Science Foundation;

Russ Housley, founder of Vigil Security, LLC;

Subbarao Kambhampati, professor of computer science at Arizona State University; and

Alice Xiang, Senior Research Scientist at Sony AI.

  1. Emerging Issues in the Biological Sciences

Panel four was moderated by Professor Ellen Wright Clayton, the Craig-Weaver Professor of Pediatrics, and Professor of Law and of Health Policy at Vanderbilt Law School, at Vanderbilt University. Her panelists were:

Dana Carroll, distinguished professor in the Department of Biochemistry at the University of Utah School of Medicine;

Yaniv Erlich, Chief Executive Officer of Eleven Therapeutics, Chief Science Officer of MyHeritage;

Steven E. Hyman, director of the Stanley Center for Psychiatric Research at Broad Institute of MIT and Harvard; and

Philip Sabes, Professor Emeritus in Physiology at the University of California, San Francisco (UCSF).

  1. Emerging areas in Psychology, Data, and Statistical Sciences

Gary Marchant, Lincoln Professor of Emerging Technologies, Law and Ethics, at Arizona State University’s Sandra Day O’Connor College of Law, moderated panelists:

Xiao-Li Meng, the Whipple V. N. Jones Professor of Statistics, Harvard University, and the Founding Editor-in-Chief of Harvard Data Science Review;

Rebecca Doerge, Glen de Vries Dean of the Mellon College of Science at Carnegie Mellon University, member of the Dietrich College of Humanities and Social Sciences’ Department of Statistics and Data Science, and of the Mellon College of Science’s Department of Biological Sciences;

Daniel Kahneman, Professor of Psychology and Public Affairs Emeritus at the Princeton School of Public and International Affairs, the Eugene Higgins Professor of Psychology Emeritus at Princeton University, and a fellow of the Center for Rationality at the Hebrew University in Jerusalem; and

Goodwin Liu, Associate Justice of the California Supreme Court.

The Proceedings of this two day meeting were recorded and will be published. The website materials are unclear whether the verbatim remarks will be included, but regardless, the proceedings should warrant careful reading.

Judge O’Malley, in her introductory remarks, emphasized that the RMSE must be a neutral, disinterested source of information for federal judges, an aspirational judgment from which there can be no dissent. More controversial will be Her Honor’s assessment that epidemiologic studies can “take forever,” and other judges’ suggestion that plaintiffs lack financial resources to put forward credible, reliable expert witnesses. Judge Vance corrected the course of the discussion by pointing out that MDL plaintiffs were not disadvantaged, but no one pointed out that plaintiffs’ counsel were among the wealthiest individuals in the United States, and that they have been known to sponsor epidemiologic and other studies that wind up as evidence in court.

Panel One was perhaps the most discomforting experience, as it involved revelations about how sausage is made in the gatekeeping process. The panel was remarkable for including a state court judge from Texas, Judge Barbara Parker Hervey, of the Texas Court of Criminal Appeals. Judge Hervey remarked that [in her experience] if we judges “can’t understand it, we won’t read it.” Her dictum raises interesting issues. No doubt, in some instances, the judicial failure of comprehension is the fault of the lawyers. What happens when the judges “can’t understand it”? Do they ask for further briefing? Or do they ask for a hearing with viva voce testimony from expert witnesses? The point was not followed up.

Leonard P. Stark’s insights were interesting in that his docket in the District of Delaware is flooded with patent and Hatch-Waxman Act litigation. Judge Stark’s extensive educational training is in politics and political science. The docket volume Judge Stark described, however, raised issues about how much attention he could give to any one case.

When the panel was asked how they dealt with scientific issues, Judge Saris discussed her presiding over In re Neurontin, which was a “big challenge for me to understand,” with no randomized trials or objective assessments by the litigants.[3] Judge Vance discussed her experience of presiding in a low-level benzene exposure case, in which plaintiff claimed that his acute myelogenous leukemia was caused by gasoline.[4]

Perhaps the key difference in approach to Rule 702 emerged when the judges were asked whether they read the underlying studies. Judge Saris did not answer directly, but stated she reads the reports. Judge Vance, on the other hand, noted that she reads the relied upon studies. In her gasoline-leukemia case, she read the relied-upon epidemiologic studies, which she described as a “hodge podge,” and which were misrepresented by the expert witnesses and counsel. She emphasized the distortions of the adversarial system and the need to moderate its excesses by validating what exactly the expert witnesses had relied upon.

This division in judicial approach was seen again when Professor Karen Kafadar asked how the judges dealt with peer review. Judge Saris seemed to suggest that the peer-reviewed published article was prima facie reliable. Others disagreed and noted that peer reviewed articles can have findings that are overstated, and wrong. One speaker noted that Jerome Kassirer had downplayed the significance of, and the validation provided by, peer review, in the RMSE (3rd ed 2011).

Curiously, there was no discussion of Rule 703, either in Judge O’Malley’s opening remarks on the RMSE, or in the first panel discussion. When someone from the audience submitted a question about the role of Rule 703 in the gatekeeping process, the moderator did not read it.

Panel Two. The climate change panel was a tour de force of the case for anthropogenic climate change. To some, the presentations may have seemed like a reprise of The Day After Tomorrow. Indeed, the science was presented so confidently, if not stridently, that one of the committee members asked whether there could be any reasonable disagreement. The panelists responded essentially by pointing out that there could be no good faith opposition. The panelists were much less convincing on the issue of attributability. None of the speakers addressed the appropriateness vel non of climate change litigation, when the federal and state governments encouraged, licensed, and regulated the exploitation and use of fossil fuel reserves.

Panel Four. Dr. Clayton’s panel was fascinating and likely to lead to new chapters. Professor Hyman presented on heritability, a subject that did not receive much attention in the RMSE third edition. With the advent of genetic claims of susceptibility and defenses of mutation-induced disease, courts will likely need some good advice on navigating the science. Dana Carroll presented on human genome editing (CRISPR). Philip Sabes presented on brain-computer interfaces, which have progressed well beyond the level of sci-fi thrillers, such as The Brain That Wouldn’t Die (“Jan in the Pan”).

In addition to the therapeutic applications, Sabes discussed some of potential forensic uses, such as lie detectors, pain quantification, and the like. Yaniv Erlich, of MyHeritage, discussed advances in forensic genetic genealogy, which have made a dramatic entrance to the common imagination through the apprehension of Joseph James DeAngelo, the Golden State killer. The technique of triangulating DNA matches from consumer DNA databases has other applications, of course, such as identifying lost heirs, and resolving paternity issues.

Panel Five. Professor Marchant’s panel may well have identified some of the most salient needs for the next edition of the RMSE. Nobel Laureate Daniel Kahneman presented some of the highlights from his forthcoming book about “noise” in human judgment.[5] Kahneman’s expansion upon his previous thinking about the sources of error in human – and scientific – judgment are a much needed addition to the RMSE. Along the same lines, Professor Xiao Li Meng, presented on selection bias, and how it pervades scientific work, and detracts from the strength of evidence in the form of:

  1. cherry picking
  2. subgroup analyses
  3. unprincipled handling of outliers
  4. selection in methodologies (different tests)
  5. selection in due diligence (check only when you don’t like results)
  6. publication bias that results from publishing only impressive or statistically significant results
  7. selection in reporting, not reporting limitations all analyses
  8. selection in understanding

Professor Meng’s insights are sorely lacking in the third edition of the RMSE, and among judicial gatekeepers generally.  All too often, undue selectivity in methodologies and in relied-upon data is treated by judges as an issue that “goes to the weight, not the admissibility” of expert witness opinion testimony. In actuality, the selection biases, and other systematic and cognitive biases, are as important as, if not more important than, random error assessments. Indeed a close look at the RMSE third edition reveals a close embrace of the amorphous, anything-goes “weight of the evidence” approach in the epidemiology chapter.  That chapter marginalizes meta-analyses and fails to mention systematic review techiniques altogether. The chapter on clinical medicine, however, takes a divergent approach, emphasizing the hierarchy of evidence inherent in different study types, and the need for principled and systematic reviews of the available evidence.[6]

The Committee co-chairs and panel moderators did a wonderful job to identify important new trends in genetics, data science, error assessment, and computer science, and they should be congratulated for their efforts. Judge O’Malley is certainly correct in saying that the RMSE must be a neutral source of information on statistical and scientific methodologies, and it needs to be revised and updated to address errors and omissions in the previous editions. The legal community should look for, and study, the published proceedings when they become available.

——————————————————————————————————

[1]  See “Emerging Areas of Science, Engineering, and Medicine for the Courts: Identifying Chapters for a Fourth Edition of The Reference Manual on Scientific Evidence – Committee Meeting” (Nov. 24, 2020).

[2]  See “Emerging Areas of Science, Engineering, and Medicine for the Courts: Identifying Chapters for a Fourth Edition of The Reference Manual on Scientific Evidence – Committee Meeting 2 (Virtual)” (Dec. 1, 2020).

[3]  In re Neurontin Marketing, Sales Practices & Prods. Liab. Litig., 612 F. Supp. 2d 116 (D. Mass. 2009) (Saris, J.).

[4]  Burst v. Shell Oil Co., 104 F.Supp.3d 773 (E.D.La. 2015) (Vance, J.), aff’d, ___ Fed. App’x ___, 2016 WL 2989261 (5th Cir. May 23, 2016), cert. denied, 137 S.Ct. 312 (2016). See “The One Percent Non-solution – Infante Fuels His Own Exclusion in Gasoline Leukemia Case” (June 25, 2015).

[5]  Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, Noise: A Flaw in Human Judgment (anticipated May 2021).

[6]  See John B. Wong, Lawrence O. Gostin, and Oscar A. Cabrera, “Reference Guide on Medical Testimony,” Reference Manual on Scientific Evidence 723-24 (3ed ed. 2011) (discussing hierarchy of medical evidence, with systematic reviews at the apex).

Posted in Evidence, Expert Witnesses, Peer Review, Reference Manual on Scientific Evidence, Rule 702, Rule 703, Scientific Evidence | 139 Comments »

Falsehood Flies – The ASA 2016 Statement on Statistical Significance

February 26th, 2021

Under the heading of “falsehood flies,” we have the attempt by the American Statistical Association (ASA) to correct misinterpretations and misrepresentations of “statistical significance,” in a 2016 consensus statement.[1] Almost before the ink was dried, lawsuit industry lawyers seized upon the ASA statement to proclaim a new freedom from having to exclude random error.[2] Those misrepresentations were easily enough defeated by the actual text of the ASA statement, as long as lawyers bothered to read it carefully.

In 2019, Ronald Wasserstein, the ASA executive director, along with two other authors wrote an editorial, which explicitly called for the abandonment of using “statistical significance.” Although the piece, published in the American Statistician, was labeled “editorial,”[3] I predicted that Wasserstein’s official title, which appears in the editorial, and the absence of a disclaimer that the piece was not an ex cathedra pronouncement, would lead to widespread confusion, abuse, and further misrepresentations of the ASA’s views.[4]

Some people pooh-poohed the danger of confusion, but I was doubtful, given the experience with what happened with the anodyne 2016 ASA statement. What I did not realize until recently was that the Wasserstein editorial was misunderstood to be an official policy statement by the ASA’s own publication, Significance!

Significance is a bimonthly magazine on statistics for educated laypeople, published jointly the ASA and the Royal Statistical Society. In August 2019, the editor of Significance, Brian Turran, published an editorial that clearly reflected that Turran interpreted the Wasserstein editorial as an official ASA pronouncement.[5] Indeed, Turran cited the Wasserstein 2019 editorial as the ASA “recommendation.”

Donald Macnaughton, President of MatStat Research Consulting Inc., in Toronto, wrote a letter to point out Turran’s error.[6] Macnaughton noted that Wasserstein had disclaimed an official imprimatur for his ideas in various oral presentations, and that the editors of the New England Journal of Medicine had explicitly rejected the editorial’s call for abandoning statistical significance.[7]

In reply, Tarran graciously acknowledge the mistake, and pointed to an ASA press release that had led him astray:

“Thank you for this clarification. Our mistake was to give too much weight to the headline of a press release, ‘ASA Calls Time on “Statistically Significant” in Science Research’ (bit.ly/2UBWKNq).”

Inquiring minds might wonder why the ASA allowed such a press release to go out.

In 2019, then President of the ASA, Karen Kafadar, wrote on multiple occasions, in AmStat News, to correct any confusion or misimpression created by Wasserstein’s editorial:

“One final challenge, which I hope to address in my final month as ASA president, concerns issues of significance, multiplicity, and reproducibility. In 2016, the ASA published a statement that simply reiterated what p-values are and are not. It did not recommend specific approaches, other than ‘good statistical practice … principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, interpretation of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean’.

The guest editors of the March 2019 supplement to The American Statistician went further, writing: ‘The ASA Statement on P-Values and Statistical Significance stopped just short of recommending that declarations of “statistical significance” be abandoned. We take that step here. … [I]t is time to stop using the term “statistically significant” entirely’.

Many of you have written of instances in which authors and journal editors – and even some ASA members – have mistakenly assumed this editorial represented ASA policy. The mistake is understandable: The editorial was coauthored by an official of the ASA. In fact, the ASA does not endorse any article, by any author, in any journal – even an article written by a member of its own staff in a journal the ASA publishes.”[8]

Kafadar did not address the hyperactivity of the ASA public relations’ office, but her careful statement of the issues should put the matter to bed. There are now citable sources to correct the incorrect claim that the ASA has recommended the complete abandonment of significance testing.

——————————————————————————————————————–

[1]  Ronald L. Wasserstein & Nicole A. Lazar, “The ASA’s Statement on p-Values: Context, Process, and Purpose,” 70 The Am. Statistician 129 (2016); see “The American Statistical Association’s Statement on and of Significance” (March 17, 2016).

[2]  “The American Statistical Association Statement on Significance Testing Goes to Court – Part I” (Nov. 13, 2018); “The American Statistical Association Statement on Significance Testing Goes to Court – Part 2” (Mar. 7, 2019).

[3]  Ronald L. Wasserstein, Allen L. Schirm, and Nicole A. Lazar, “Editorial: Moving to a World Beyond ‘p < 0.05’,” 73 Am. Statistician S1, S2 (2019).

[4]  “Has the American Statistical Association Gone Post-Modern?” (Mar. 24, 2019); “American Statistical Association – Consensus versus Personal Opinion” (Dec. 13, 2019). See also Deborah G. Mayo, “The 2019 ASA Guide to P-values and Statistical Significance: Don’t Say What You Don’t Mean,” Error Statistics Philosophy (June 17, 2019); B. Haig, “The ASA’s 2019 update on P-values and significance,” Error Statistics Philosophy  (July 12, 2019).

[5]  Brian Tarran, “THE S WORD … and what to do about it,” Significance (Aug. 2019).

[6]  Donald Macnaughton, “Who Said What,” Significance 47 (Oct. 2019).

[7]  See “Statistical Significance at the New England Journal of Medicine” (July 19, 2019); See also Deborah G. Mayo, “The NEJM Issues New Guidelines on Statistical Reporting: Is the ASA P-Value Project Backfiring?” Error Statistics Philosophy  (July 19, 2019).

[8]  Karen Kafadar, “The Year in Review … And More to Come,” AmStat News 3 (Dec. 2019) (emphasis added); see Kafadar, “Statistics & Unintended Consequences,” AmStat News 3,4 (June 2019).

Posted in statistical evidence | No Comments »

Larding Up the Literature

February 20th, 2021

Another bio-medical journal?

In October 2019, The Journal of Scientific Practice and Integrity published its inaugural volume one, number one issue, online. This journal purports to cover scientific integrity issues, which may well not be adequately covered in the major biomedical journals. There are reasons to believe, however, that this journal may be more of a threat to scientific integrity than a defender.

Thenew journal describes itself as:

“an interdisciplinary, peer-reviewed journal that publishes scholarly debate and original research on scientific practices that impact human and environmental health.”

The editorial board reads like a Who’s Who’s list of “political scientists” who testify a LOT for claimants, and who, when not working for the lawsuit industry, practice occupational and environmental medicine for the redistribution of wealth.

David Egilman, contemnor and frequent plaintiffs’ expert witness in personal injury litigation is editor in chief. Tess Bird, an Egilman protégé, is managing editor. Another Egilman protégé, Susana Rankin Bohme, an associate Director of Research at Corporate Accountability International, also sits on the editorial board. You may be forgiven for believing that this journal will be an Egilman vanity press. The editorial board also includes some high-volume testifying plaintiffs expert witnesses:

Peter Infante, of Peter F. Infante Consulting, LLC, Virginia

Adriane Fugh-Berman, of PharmedOut

Barry Castleman,

William E. Longo, President, MAS, LLC

David Madigan,

Michael R. Harbut,

David Rosner, and

Gerald Markowitz

The journal identifies the Collegium Ramazzini as one of its “partners.” Cue the “Интернационал”!

The first issue of this new journal features a letter[1] from the chief and managing editors, Egilman and Bird, which states wonderfully aspirational goals. The trick will be whether the journal can apply its ethical microscope to all actors in the world of scientific publishing, or whether this new journal is just not another lawsuit industry propaganda outlet.

Egilman’s previous editorial perch was at the International Journal of Occupational and Environmental Health, which was published by Maney Publishing. In 2015, the British company, the Taylor & Francis Group, acquired the IJOEH, with Maney’s other journals, and installed a new editor-in-chief, Andrew Maier. Egilman was cast out; hence the new journal.

Egilman’s new journal will feature among other types of articles, “reviews of legal testimony,” as a scholarly subject. It will be interesting to see whether such reviews assess the testimony of lawsuit industry witnesses, as well as manufacturing industry witnesses.

The new journal requires the use of the International Committee of Medical Journal Editors (ICMJE) conflict-of-interest and funding disclosure rules, and the use of the ICMJE form. Accordingly, authors “should” report all conflicts, including:

“[a]ny financial contributions, payments, or funding for the present work;

relevant financial activities outside of the submitted work;

any patents or copyrights broadly relevant to the work; and

any relationships that readers could perceive to influence the submitted work.”

There have been only two issues of Egilman’s new journal so far, but I decided to spot check compliance. The first article[2] I saw was by Colin Soskolne, who has testified for the lawsuit industry in a diacetyl case.[3] Oops; no disclosure.

Does Soskolne’s bias show? In the spot-checked article, authors Sokolne and Baur reprise a publication previously part of a 2018 Collegium Ramazzini convocation entitled “Corporate Influence Threatens the Public Health.” The aim of the convocation speakers was to press their claims that [manufacturing] corporate influence undermines scientific integrity through discernible methods, all by “those in the pay of industry”:

  • infiltrating journal editorial boards by scientists, with the resulting publication of poorly designed, biased research that foments doubt;
  • interfering with “the independent activities of IARC” and similar agencies;
  • blocking “much needed” regulation of “hazardous agents,” such as pesticides and polyfluoroalkyl substances (PFAS); and
  • promulgating causal criteria, which are baseless and which “block workers’ access to legal remedies for occupational illness and premature death.”[4]

There can be little doubt that Soskolne is not interested in messing with “those in the pay of the lawsuit industry.” Soskolne’s biases are fairly clear, clear enough for us to complain that he has not disclosed that he has been compensated by the lawsuit industry, and that he has deep positional conflicts as well. Ironically, he is writing in a journal that itself appears to lack “balance.” The editorial board of the journal for which Soskolne was writing is composed of many of “those in the pay of the lawsuit industry.”

Soskolne is keen to preserve the independence of IARC, but that perceived independence has become a sad, sick joke, with the exclusion of most anyone who has had any working relationship with manufacturing industry, while engaging many with deep ties to the lawsuit industry. Soskolne’s assessment of “much needed” regulation ultimately must be evaluated on the facts and data of each putative toxic substance. If the claim of harmful effects is correct, then regulation may well be “much needed.” If the claim is not correct, then regulation will be much “unneeded.” As for promulgating causal criteria, there is no doubt that the Soskolne, along with the editorial board of this new journal, would like to see the abrogation of causal criteria, so that workers have legal remedies ad libitum.

Soskolne and Baur provide their hit list of the methods of obfuscation or of techniques used to undermine science and policy.[5] There is precious little in their list, however, that is not common place among all journals that publish occupational and environmental epidemiology, including the journals that have been captured by the lawsuit industry’s scientists. Soskolne and Baur also provide a catalogue of how lawsuit industry scientists would subvert science and lock in their biased and selective interpretation of data:

  • elevate biological plausibility into sufficient basis for causal inference
  • conflate species and ignore species differences in order to allow animal studies to suffice for causal inference for humans
  • ignore substantial, relevant biological differences in even slight structural differences among various molecules to enable assertions of harm based upon similar molecular structure of a putative toxic substance

Soskolne ends with a quote from the “pink panthers,” two radical, labor historians, both editorial board members of this new journal, and who both have testified many times for the lawsuit industry:

“[A]s a society, we cannot entrust those with self-interest to be the judge and jury of what is and what is not a danger[;] … that can only lead to compromised science, a questionable decisionmaking process, and a potentially polluted world.”[6]

The pink panthers are, of course, correct, but we must understand that self-interest and conflict of interest can be, and are, both ideological, positional, as well as economic.

[1]  Tess Bird & David Egilman, “Letter from the Editors: An Introduction to the Journal of Scientific Practice and Integrity,” 1 J. Sci. Practice & Integrity 1 (2019).

[2]  Colin Soskolne & Xaver Baur, “How Corporate Influence Continues to Undermine the Public’s Health,” 1 J. Sci. Practice & Integrity 1 (2019), available at DOI: 10.35122/jospi.2019.878137 [cited as Soskolne & Baur]

[3]  See Watson v. Dillon Companies, 797 F. Supp. 2d 1138 (D. Colo. 2011) (addressing Soskolne’s testimony).
[4]  Soskolne & Baur at 1-2.

[5]  Soskolne & Baur at 3.

[6]  Soskolne & Baur at 4, quoting from Gerald Markowitz & David Rosner, “Monsanto, PCBs, and the creation of a ‘world-wide ecological problem’,” 39 J. Pub. Health Policy 463 (2018).

Posted in Cognitive Biases, Conflicts of Interest, Expert Witnesses, Scientific Publishing | No Comments »

Dr. Harry Shubin – Asbestos Litigation Hall of Shame

February 19th, 2021

Many physicians took and failed the so-called B-reader examination for proficiency in using the International Labor Office’s grading schema of chest radiographs for pneumoconiosis. Famously, Irving Selikoff was someone who took but failed this examination, and he stopped serving as a clinical expert witness in asbestos cases. No one can say for sure whether there was cause and effect. In Philadelphia, South Jersey pulmonary physician, Dr. Joseph Sokolowski, was a frequent testifier for plaintiffs, despite his having failed the B-Reader examination three times. Sokolowski was conditionally fined and disqualified from testifying in Philadelphia because of his refusal to comply with compulsory process for his B-Reader test results.[1]

The only physician I encountered who lied outright about his B-reader examination results was the late Harry Shubin. Some people may know of Dr. Shubin only because of the “Harry Shubin, M.D. Statesman in Healthcare Administration Award,” presented by the American Academy of Medical Administrators. Indeed, Shubin had a long career in hospital administration. He was graduated from Temple School of Medicine in 1937, and went into family practice, at the age of 22. Later, he specialized in the treatment of tuberculosis and pulmonary disease.[2] Over his long career, he was a patron of the arts and of other charities in Philadelphia.

In 1955, Dr Shubin became the president of the American Academy of Tuberculosis Physicians.[3] In 1958, Shubin was elected head of staff at Pennsylvania General Hospital.[4] He later became the medical director of the Northern Division of Philadelphia General Hospital.

In 1962, along with 18 other physicians, Shubin purchased a 104-bed hospital, known as the Broad Street Hospital, at 739 South Broad Street, for which he would become medical director.[5] When the Jefferson Medical College sold its Barton Memorial Hospital for “diseases of the chest,” including silicosis and coal workers’ pneumoconiosis, at Broad and Fitzwater Streets, the South Broad Street Medical Center, Inc., bought it.  Shubin was the medical director.[6]

Shubin’s career as a medical director suffered in the ensuing decades.[7] In 1970, after a hearing, Dr. Shubin was disqualified by the U.S. Food & Drug Administration from receiving investigational products.[8] And then in 1981, his Center City Hospital, at 1829 Pine Street, closed, under pressure from both Blue Cross and the federal Health Systems Agency.[9]

Perhaps those setbacks explain Dr. Shubin’s foray into the lucrative world of expert witnessing in the asbestos litigation. Shubin started turning up as an expert witness for plaintiffs in hundreds of cases, in 1985. Dr. Peter Theodos, a prominent Philadelphia pulmonary physician had been designated to testify in these cases, but he died while the cases languished due to the huge backlog of asbestos cases in Philadelphia.

Shubin’s qualifications to replace Theodos were thin, Aside from an insignificant paper in a Pennsylvania Medicine,[10] Shubin had not written anything on asbestos or asbestos-related disease. Shubin was, however, a media hound, and in 1985, he was turning up at union halls and town hall meetings across Pennsylvania.[11]

Despite his lack of significant experience and expertise in asbestos-related disease, Shubin could put on a great show for uncritical Philadelphia juries. He was charming and persuasive. His years of public speaking on behalf of his beleaguered hospitals gave him confidence and skills of evasion in the witness box. When confronted with statements from Selikoff’s book or articles, Shubin would chuckle, smile knowingly, and say that he taught Selikoff everything about asbestos. His demeanor was avuncular and cherubic, which made cross-examination even more difficult.

Initially, Shubin testified that he was a B-reader, but after the defense verified his absence from the NIOSH list of certified B-readers, he “modified” his next trial’s testimony to state that he had started, but had not finished, the examination because of an eye problem.

After inquiring at NIOSH and learning that Shubin had finished the examination, defense counsel confronted Shubin yet again, only to have his tale embellished by a claim that he had withdrawn from the examination after finishing because of the eye problem, and wrote to NIOSH to ask that his examination not be evaluated.

In 1986, the leading asbestos plaintiffs’ firm was pushing one of its many cases to trial, with a Shubin report to support its claims.[12] Although the defense expert witnesses had given this plaintiff a clean bill of health, we had sufficient concerned about Shubin’s testimonial skills, that I decided to document Shubin’s perjury on the subject of his B-Reader status.

In 1986, the records’ custodian of B-Reader test results for NIOSH was Mitizie Martin, in Morgantown, West Virginia. After a good deal of procedural wrangling, I was able to obtain a court order that required plaintiffs’ counsel’s appearance for a deposition of Ms. Martin, in Morgantown. I noticed her deposition in all Philadelphia asbestos cases and gave all plaintiffs’ firms notice of the event. And so, in the first week of September, 1986, plaintiffs’ counsel, John DiDonato, and I were on board a small airplane for a bumpy ride to Morgantown, for Ms. Martin’s deposition.[13]

Martin was a wonderful witness. In 1986, she was Chief of the X-ray Receiving Center Section, Division of X-Ray Disease Studies, for NIOSH. She had been with NIOSH or its predecessor for 20 years.[14] Martin explained NIOSH’s role in teaching the A-Reader course, and in administering the B-Reader examination, along with the American College of Radiology, and its record retention policies.

Martin described the B-reader examination, which at the time required a showing of proficiency in interpreting 125 chest films for the presence and absence of various pneumoconiosis, according to the ILO scale.[15]

Finally, and most important, Martin authenticated the Shubin file, and laid a foundation for its admissibility at all future asbestos trials. Shubin had sought to become a B-Reader by taking the required test, in March 1982, in Orlando, Florida.[16] He completed but failed that examination in 1982, after which he wrote to Ms. Martin, to express his desire to take the test again. He made no mention of an eye problem; nor did he request that the 1982 results be disregarded. Martin wrote back to explain that he would have to wait a year before taking the examination again.[17]

Shubin waited three years, but he eventually signed up for, and took, the B-Reader examination a second time, in March 1985, on Kiawah Island, South Carolina. This second attempt resulted in a second failure, and a notification to Shubin that he had received a “failing grade, deficient in most categories.”[18] Shubin had thus taken the B-Reader examination twice, and failed both times. He had never abandoned the test; nor had he written to NIOSH to request a withdrawal from the examination, or to notify the agency of an eye-sight problem.[19]

My then colleague, Terri Keeley, was set to try the next-schedule case in which Dr. Shubin was supposed to have testified, in September 1986. Unsurprisingly, the case resolved. As soon as I received the transcript, I provided a copy gratis to every asbestos plaintiffs’ firm in Philadelphia. The result was that we never saw Dr. Shubin in an asbestos case again. Occasionally, we would receive a Dr. Shubin report, but a quick reminder of the procedural history behind Ms. Mitzie Martin’s deposition testimony sealed his effectual exclusion.

Remarkably, despite the notoriety he had gained in the Philadelphia asbestos litigation, Shubin showed up in the high-profile Paoli Railroad PCB case, as an expert witness on causation.[20] In the Paoli case, Shubin gave rather unscientific testimony that conflated general and specific causation, and assumed that PCBs caused individual plaintiffs’ disease because PCBs can cause disease.[21] As far I can determine, he was never confronted with his perjuries from the Philadelphia asbestos cases.

————————————————————————————————————————

[1]  Downing v. Johns-Manville Corp., Philadelphia Cty. Ct. C.P. 1981- 11- 3762, Order sur Motion to Compel Dr. Sokolowski to answer interrogatories and produce documents related to his failure of the B-Reader examination (Mar. 31, 1988).

[2]  “Harry Shubin Obituary,” Phila. Inq. at 25 (Mar. 7, 1997) (Dr. Shbuin died March 2, 1997).

[3]  “News from the Field,” 45 Am. J. Pub. Health 1399 (1955).

[4]  “Pennsylvania General Hospital,” Phila. Inq. at 29 (Mar. 25, 1958).

[5]  “Doctor-Owned Hospital Opens in South Philadelphia,” Phila. Daily News at p.19 (Feb. 5, 1962).

[6]  “Medical Center Sells Building,” Phila. Daily News at 57 (May 5, 1961).

[7]  Frank Dougherty, “11 Hospitals Cut from Blue Cross,” Phila. Daily News at p.3 (Nov. 14, 1972) (Broad Street Hospital cut).

[8]  Office of Regulatory Affairs Compliance References: Bioresearch Monitoring (Mar. 16, 1970).

[9]  Phila. Inq. at 5 (Mar. 18, 1981) (Shubin, medical director).

[10]  Harry Shubin, “Asbestosis: inhalation disease of the lungs,” 88 Pa. Med. 54, 56, 58 (1985).

[11]  See, e.g., “Pat Purcell, Angry Man,” Pottsville Republican (Pottsville, PA) at p.1 (Mar. 14, 1987); Frank Scholz, “Help Coming for Asbestos Victims Living Here,” The Times-Tribune (Scranton, PA Sept. 15, 1985).

[12]  Ove Nyman v. Johns-Mansville Corp., Phila. Cty. Ct. C.P. (April Term, 1981). Years later, I met Mr. Nyman at my sister-in-law’s wedding, where he and his wife were dancing vigorously.

[13]  Deposition of Mitzie Martin, in Ove Nyman v. Johns-Mansville Corp., Phila. Cty. Ct. C.P. (April Term, 1981), taken on Sept. 5, 1986.

[14]  Id. at 3.

[15]  Id. at 7.

[16]  Id. at 12-13.

[17]  Id. at 14.

[18]  Id. at 14-15.

[19]  Id. at 16-19.

[20] In re Paoli R.R. Yard PCB Litig., 706 F. Supp. 358, 364-65 (E.D. Pa. 1988), rev’d, 916 F.2d 829, 840 (3d Cir. 1990), cert. denied, 111 S. Ct. 1584 (1991).

[21]  Susan R. Poulter, “Science and Toxic Torts: Is There a Rational Solution to the Problem of Causation?” 7 High Technology L.J. 189, 238 (1993).

Posted in Asbestos, Expert Witness Hall of Shame, Expert Witnesses | No Comments »

Carl Cranor’s Inference to the Best Explanation

February 12th, 2021

Carl Cranor pays me the dubious honor of quoting my assessment of weight of the evidence (WOE) pseudo-methodology as used by lawsuit industry expert witnesses, in one of his recent publications:

“Take all the evidence, throw it into the hopper, close your eyes, open your heart, and guess the weight. You could be a lucky winner! The weight of the evidence suggests that the weight-of-the-evidence (WOE) method is little more than subjective opinion, but why care if it helps you to get to a verdict!”[1]

Cranor’s intent was to deride my comments, but they hold up fairly well. I have always maintained that if were wrong, I would eat my words, but that they will be quite digestible. Nothing to eat here, though.

In his essay in the Public Affairs Quarterly, Cranor attempts to explain and support his advocacy of WOE in the notorious case, Milward, in which Cranor, along with his friend and business partner, Martyn Smith, served as partisan, paid expert witnesss.[2] Not disclosed in this article is that after the trial court excluded the opinions of Cranor and Smith under Federal Rule of Evidence 702, and plaintiff appealed, the lawsuit industry, acting through The Council for Education and Research on Toxics (CERT) filed an amicus brief to persuade the Court of Appeals to reverse the exclusion. The plaintiffs’ counsel, Cranor and Smith, and CERT failed to disclose that CERT was founded by the two witnesses, Cranor and Smith, whose exclusion was at issue.[3] Many of the lawsuit industry’s regular testifiers were signatories, and none raised any ethical qualms about the obvious conflict of interest, or the conspiracy to pervert the course of justice.[4]

Cranor equates WOE to “inference to the best explanation,” which reductively strips science of its predictive and reproducible nature. Readers may get the sense he is operating in the realm of narrative, not science, and they would be correct. Cranor goes on to conflate WOE methodology with “diagnostic induction,” and “differential diagnosis.”[5] The latter term is well understood in both medicine and in law to involve the assessment of an individual patient’s condition, based upon what is already known upon good and sufficient bases. The term has no accepted or justifiable meaning for assessing general causation. Cranor’s approach would pretermit the determination of general causation by making the disputed cause a differential.

Cranor offers several considerations in support of his WOE-ful methodology. First, he notes that the arguments for causal claims are not deductive. True, but indifferent as to his advocacy for WOE and inference to the best explanation.

Second, Cranor describes a search for relevant evidence once the scientific issue (hypothesis?) is formulated. Again, there is nothing unique about this described step, but Cranor intentionally leaves out considerations of validity, as in extrapolations between high and low dose, or between species. Similarly, he leaves out considerations of validity of study designs (such as whether any weight would be given to case studies, cross-sectional, or ecological studies) or of validity of individual studies.

Cranor’s third step is the formulation of a “sufficiently complete range of reasonable and plausible explanations to account for the evidence.” Again, nothing unique here about WOE, except that Cranor’s WOE abridges the process by ignoring the very real possibility that we do not have the correct plausible explanation available.

Fourth, according to Cranor, scientists rank, explicitly or implicitly, the putative “explanations” by plausibility and persuasiveness, based upon the evidence at hand, in view of general toxicological and background knowledge.[6] Note the absence of consideration of the predictive abilities of the competing explanations, or any felt need to assess the quality of evidence or the validity of study design.

For Cranor, the fifth consideration is to use the initial plausibility assessments, made on incomplete understanding of the phenomena, and on incomplete evidence, to direct “additionally relevant /available evidence to separate founded explanations from less well-founded ones.” Obviously missing from Cranor’s scheme is the idea of trying to challenge or test hypotheses severely to see whether withstand such challenges.

Sixth, Cranor suggests that “all scientifically relevant information” should be considered in moving to the “best supported” explanation. Because “best” is determined based upon what is available, regardless of the quality of the data, or the validity of the inference, Cranor rigs his WOE-ful methodology in favor of eliminating “indeterminate” as a possible conclusion.

In a seventh step, Cranor points to the need to “integrate, synthesize, and assess or evaluate,” all lines of “available relevant evidence.” There is nothing truly remarkable about this step, which clearly requires judgment. Cranor notes that there can be convergence of disparate lines of evidence, or divergence, and that some selection of “lines” of evidence may be endorsed as supporting the “more persuasive conclusion” of causality.[7] In other words, a grand gemish.

Cranor’s WOE-ful approach leaves out any consideration of random error, or systematic bias, or data quality, or study design. The words “bias” and “confounding” do not appear in Cranor’s essay, and he erroneously discusses “error” and “error rates,” only to disparage them as the machinations of defense lawyers in litigation. Similarly, Cranor omits any serious mention of reproducibility, or of the need to formulate predictions that have the ability to falsify tentative conclusions.

Quite stridently, Cranor insists that there is no room for any actual weighting of study types or designs. In apparent earnest, Cranor writes that:

“this conclusion is in accordance with a National Cancer Institute (NCI) recommendation that ‘there should be no hierarchy [among different types of scientific methods to determine cancer causation]. Epidemiology, animal, tissue culture and molecular pathology should be seen as integrating evidences in the determination of human carcinogenicity.”[8]

There is much whining and special pleading about the difficulty, expense, and lack of statistical power of epidemiologic studies, even though the last point is a curious backdoor endorsement of statistical significance. The first two points ignore the availability of large administrative databases from which large cohorts can be identified and studied, with tremendous statistical power. Case-control studies can in some instances be assembled quickly as studies nested in existing cohorts.

As I have noted elsewhere,[9] Cranor’s attempt to level all types of evidence starkly misrepresents the cited “NCI” source, which is not at all an NCI recommendation, but rather a “meeting report” of a workshop of non-epidemiologists.[10] The cited source is not an official pronouncement of the NCI, the authors were not NCI scientists, and the NCI did not sponsored the meeting. The meeting report appeared in the journal Cancer Research as a paid advertisement, not in the NCI’s Journal of the National Cancer Institute as a scholarly article:

“The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.”[11]

Tellingly, Cranor’s deception was relied upon and cited by the First Circuit, in its Milward, decision.[12] The scholarly fraud hit its mark. As a result of Cranor’s own dubious actions, the Milward decision has both both ethical and scholarship black clouds hovering over it.  The First Circuit should withdraw the decision as improvidently decided.

The article ends with Cranor’s triumphant view of Milward,[13] which he published previously, along with the plaintiffs’ lawyer who hired him.[14] What Cranor leaves out is that the First Circuit’s holding is now suspect because of the court’s uncritical acceptance of Cranor’s own misrepresentations and CERT’s omissions of conflict-of-interest disclosures, as well as the subsequent procedural history of the case. After the Circuit reversed the Rule 702 exclusions, and the Supreme Court denied the petition for a writ of certiorari, the case returned to the federal district court, where the defense lodged a Rule 702 challenge to expert witness opinion that attributed plaintiff’s acute promyelocytic leukemia to benzene exposure. This specific causation issue was not previously addressed in the earlier proceedings. The trial court sustained the challenge, which left the plaintiff unable to show specific causation. The result was summary judgment for the defense, which the First Circuit affirmed on appeal.[15] The upshot of the subsequent proceedings, with their dispositive ruling in favor of the defense on specific causation, is that the earlier ruling on general causation is no longer necessary to the final judgment, and not the holding of the case when all the proceedings are considered.

In the end, Cranor’s WOE leaves us with a misdirected search for an “explanation of causation,” rather than a testable, tested, reproducible, and valid “inference of causation.” Cranor’s attempt to invoke the liberalization of the Federal Rules of Evidence ignores the true meaning of “liberal” in being free from dogma and authority. Evidence does not equal eminence, and expert witnesses in court must show their data and defend their inferences, whatever their explanations may be.

——————————————————————————————————–

[1]  Carl F. Cranor, “How Courts’ Reviews of Science in Toxic Tort Cases Have Changed and Why That’s a Good Thing,” 31 Public Affairs Q. 280 (2017), quoting from Schachtman, “WOE-fully Inadequate Methodology – An Ipse Dixit by Another Name” (May 1, 2012).

[2]  Milward v. Acuity Specialty Products Group, Inc., 639 F. 3d 11 (1st Cir. 2011), cert. denied, 132 S.Ct. 1002 (2012).

[3]  See “The Council for Education and Research on Toxics” (July 9, 2013).

[4] Among the signatures were Nachman Brautbar, David C. Christiani, Richard W. Clapp, James Dahlgren, Arthur L. Frank, Peter F. Infante, Philip J. Landrigan, Barry S. Levy, David Ozonoff, David Rosner, Allan H. Smith, and Daniel Thau Teitelbaum.

[5]  Cranor at 286-87.

[6]  Cranor at 287.

[7]  Cranor at 287-88.

[8]  Cranor at 290.

[9]  “Cranor’s Defense of Milward at the CPR’s Celebration” (May 12, 2013).

[10]  Michelle Carbone, Jack Gruber, and May Wong, “Modern criteria to establish human cancer etiology,” 14 Semin. Cancer Biol. 397 (2004).

[11]  Michele Carbone, George Klein, Jack Gruber and May Wong, “Modern Criteria to Establish Human Cancer Etiology,” 64 Cancer Research 5518 (2004).

[12]  Milward v. Acuity Specialty Products Group, Inc., 639 F. 3d 11, 17 (1st Cir. 2011) (“when a group from the National Cancer Institute was asked to rank the different types of evidence, it concluded that ‘[t]here should be no such hierarchy’.”), cert. denied, 132 S.Ct. 1002 (2012).

[13]  Cranor at 292.

[14]  See “Wake Forest Publishes the Litigation Industry’s Views on Milward” (April 20, 2013).

[15]  Milward v. Acuity Specialty Products Group, Inc., 969 F. Supp. 2d 101 (D. Mass. 2013), aff’d sub nom. Milward v. Rust-Oleum Corp., 820 F.3d 469 (1st Cir. 2016).

Posted in Causation, Conflicts of Interest, Public Nuisances, Scientific Evidence, Scientific Misconduct, Systemic Bias | 1 Comment »

On Praising Judicial Decisions – In re Viagra

February 8th, 2021

We live in strange times. A virulent form of tribal stupidity gave us Trumpism, a personality cult in which it impossible to function in the Republican party and criticize der Führe. Even a diehard right-winger such as Liz Cheney, who dared to criticize Trump is censured, for nothing more than being disloyal to a cretin who fomented an insurrection that resulted in the murder of a Capital police officer and the deaths of several other people.[1]

Unfortunately, a similar, even if less extreme, tribal chauvinism affects legal commentary, from both sides of the courtroom. When Judge Richard Seeborg issued an opinion, early in 2020), in the melanoma – phosphodiesterase type 5 inhibitor (PDE5i) litigation,[2] I praised the decision for not shirking the gatekeeping responsibility even when the causal claim was based upon multiple, consistent statistically significant observational studies that showed an association between PDE5i medications and melanoma.[3] Although many of the plaintiffs’ relied-upon studies reported statistically significant associations between PDE5i use and melanoma occurrence, they also found similar size associations with non-melanoma skin cancers. Because skin carcinomas were not part of the hypothesized causal mechanism, the study findings strongly suggested a common, unmeasured confounding variable such as skin damage from ultraviolet light. The plaintiffs’ expert witnesses’ failure to account for confounding was fatal under Rule 702, and Judge Seeborg’s recognition of this defect, and his willingness to go beyond multiple, consistent, statistically significant associations was what made the decision important.

There were, however, problems and even a blatant error in the decision that required attention. Although the error was harmless in that its correction would not have required, or even suggested, a different result, Judge Seeborg, like many other judges and lawyers, tripped up over the proper interpretation of a confidence interval:

“When reviewing the results of a study it is important to consider the confidence interval, which, in simple terms, is the ‘margin of error’. For example, a given study could calculate a relative risk of 1.4 (a 40 percent increased risk of adverse events), but show a 95 percent ‘confidence interval’ of .8 to 1.9. That confidence interval means there is 95 percent chance that the true value—the actual relative risk—is between .8 and 1.9.”[4]

This statement about the true value is simply wrong. The provenance of this error is old, but the mistake was unfortunately amplified in the Third Edition of the Reference Manual on Scientific Evidence,[5] in its chapter on epidemiology.[6] The chapter, which is often cited, twice misstates the meaning of a confidence interval:

“A confidence interval provides both the relative risk (or other risk measure) found in the study and a range (interval) within which the risk likely would fall if the study were repeated numerous times.”[7]

and

“A confidence interval is a range of possible values calculated from the results of a study. If a 95% confidence interval is specified, the range encompasses the results we would expect 95% of the time if samples for new studies were repeatedly drawn from the same population. Thus, the width of the interval reflects random error.”[8]

The 95% confidence interval does represent random error, 1.96 standard errors above and below the point estimate from the sample date. The confidence interval is not the range of possible values, which could well be anything, but the range of reasonable compatible estimates with this one, particular study sample statistic.[9] Intervals have lower and upper bounds, which are themselves random variables, with approximately normal (or some other specified) distributions. The essence of the interval is that no value within the interval would be rejected as a null hypothesis based upon the data collected for the particular sample. Although the chapter on statistics in the Reference Manual accurately describes confidence intervals, judges and many lawyers are misled by the misstatements in the epidemiology chapter.[10]

Given the misdirection created by the Federal Judicial Center’s manual, Judge Seeborg’s erroneous definition of a confidence interval is understandable, but it should be noted in the context of praising the important gatekeeping decision in In re Viagra. Certainly our litigation tribalism should not “allow us to believe” impossible things.[11] The time to revise the Reference Manual is long overdue.

_____________________________________________________________________

[1]  John Ruwitch, “Wyoming GOP Censures Liz Cheney For Voting To Impeach Trump,” Nat’l Pub. Radio (Feb. 6, 2021).

[2]  In re Viagra (Sildenafil Citrate) and Cialis (Tadalafil) Prods. Liab. Litig., 424 F. Supp. 3d 781 (N.D. Cal. 2020) [Viagra].

[3]  See “Judicial Gatekeeping Cures Claims That Viagra Can Cause Melonoma” (Jan. 24, 2020).

[4]  Id. at 787.

[5]  Federal Judicial Center, Reference Manual on Scientific Evidence (3rd ed. 2011).

[6]  Michael D. Green, D. Michal Freedman, & Leon Gordis, “Reference Guide on Epidemiology,” in Federal Judicial Center, Reference Manual on Scientific Evidence 549 (3rd ed. 2011).

[7]  Id. at 573.

[8]  Id. at 580.

[9] Michael O. Finkelstein & Bruce Levin, Statistics for Lawyers 171, 173-74 (3rd ed. 2015). See also Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman, “Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations,” 31 Eur. J. Epidem. 337 (2016).

[10]  See, e.g., Derek C. Smith, Jeremy S. Goldkind, and William R. Andrichik, “Statistically Significant Association: Preventing the Misuse of the Bradford Hill Criteria to Prove Causation in Toxic Tort Cases,” 86 Defense Counsel J. 1 (2020) (mischaracterizing the meaning of confidence intervals based upon the epidemiology chapter in the Reference Manual).

[11]  See, e.g., James Beck, “Tort Pandemic Countermeasures? The Ten Best Prescription Drug/Medical Device Decisions of 2020,” Drug and Device Law Blog (Dec. 30, 2020) (suggesting that Judge Seeborg’s decision represented the rejection of plausibility and a single “association” as insufficient); Steven Boranian, “General Causation Experts Excluded In Viagra/Cialis MDL,” (Jan. 23, 2020).

Posted in Confounding, Reference Manual on Scientific Evidence, Rule 702, statistical evidence | No Comments »

American Law Institute – Medical Monitoring vs. Medical Mongering

February 5th, 2021

One of the key activities of the American Law Institute (ALI) has been the researching, writing, and publication of Restatements.[1] According to the ALI’s website, the basic idea was that the ALI “should address uncertainty in the law through a restatement of basic legal subjects that would tell judges and lawyers what the law was.” This self-appointed task has a huge influence on the development of the law in the United States, and indeed around the world, mostly for the better. Restatements can and often do reduce uncertainty and eliminate unnecessary complexity and obfuscation. The ALI also holds itself out “promote those changes which will tend better to adapt the laws to the needs of life.” The ALI has thus characterized its Restatements as having a “critical and constructive” goal as well as a clarifying and simplifying function.

This ambiguity in its mission statement makes ALI Restatement provisions occasionally controversial on occasion. Controversy can arise from the ALI’s addressing factual situations “not yet discussed by courts or dealt with by legislatures….” Perhaps more disconcerting, however, are situations that have been addressed by courts and legislatures at length, but where the ALI attempts to impose its policy judgments in place of those that carry the imprimatur of a majority of jurisdictions in the United States.

Take the ALI’s recently proposed “restatement” [sic] of medical monitoring law in the Restatement (Third) of Torts: Concluding Provisions:[2]

“A person can recover for medical monitoring expenses, even absent present bodily harm, if:

(a) an actor’s tortious conduct has exposed a person to a significant risk of serious future bodily harm;

(b) the exposure makes medical monitoring reasonable and necessary in order to prevent or mitigate the future bodily harm;

(c) the person has incurred the monitoring expense, will incur the monitoring expense, or would incur the monitoring expense if he or she could afford it; and;

(d) the actor’s liability is not indeterminate.”

Remarkably, the ALI has “restated” the law of medical monitoring to dispense with the requirement of present injury. The ALI Council Draft claims a slim majority of courts favor the abandonment of a present injury predicate, but the scholarship for this claim is shaky at best, as shown in an important recent article by Mark Behrens and Christopher Appel.[3]

Each subpart of the ALI’s proposed rule is problematic. Subpart (a) creates a requirement of “significant risk of serious future bodily harm.” Although there might be general agreement on what is serious bodily harm, the predicate of “significant risk” is incredibly vague. Risk is undefined, and by all rights should represent a cause ex ante, but the lawsuit industry often uses “risk” to connote a possible cause. At the very least, the ALI should clarify that the risk of subpart (a) is one that would pass muster as a general cause under the relevant substantive and evidentiary law. Furthermore, what might appear to be a “significant” risk, may be trivial. Consider an exposure to amphibole asbestos that doubled the risk of mesothelioma. If the baseline annual risk of mesothelioma were one in a million, the exposed claimant would consequently have a two in a million risk. A doubling of risk sounds significant and ominous, but the absolute annual attributable risk would be one in a million. The rule leaves open whether the significance of risk should be evaluated by subjective reactions of the jury, or some quantitative measure. If the latter, the rule leaves open whether the measure is relative or absolute risk.

Subpart (b) specifies that “the exposure makes medical monitoring reasonable and necessary in order to prevent or mitigate the future bodily harm.” Some medical monitoring regimens can allow for earlier interventions that are potentially curative, but others simply increase the time between diagnosis and death by moving up the diagnosis date. This subsection leaves open exactly what counts as mitigation of the future harm.

Subpart (c) ignores that many medical monitoring regimes are already a potential claimant’s personal responsibility, and they involve procedures that are covered by health insurance contracts. Given that any “significant risk” would not change the pre-existing need for such screening procedures as mammography in women, prostate-specific antigen in men, lung cancer screening in adults who have engaged in habitual tobacco self-abuse, and the like, this subpart leaves unclear why and how the burden paying for such routine screening should be shifted to the alleged tortfeasor.

Subpart (d) specifies that the rule would not apply if the defendant’s liability is indeterminate, presumably because of the number of potential claimants. Comment g to the Council’s draft provision attempts to explain:

“defendant whose conduct exposes a vast number of people to risk-creating agents or behaviors is not subject to liability for medical monitoring if the defendant is able to show that liability would be highly unpredictable and virtually unlimited.”[4]

Comment g goes on to suggest that the proposed rule would not apply if the liability it creates were:

“likely to exceed the defendant’s resources and insurance coverage and thereby meaningfully reduce monies available to those exposed persons who ultimately develop bodily harm.”[5]

What subpart (d) leaves unclear is how the defendant is suppose to satisfy this burden to prove “indeterminancy.” If the first case in court is one individual’s claim for medical monitoring costs, how does the defendant show that the aggregate liability will become indeterminate at some later date? Is the defendant suppose to present evidence of its insurance coverage, and perhaps interplead its carrier who had denied coverage? Comment g suggests a completely unworkable proof scheme for trial.

One of the key limitations on strict liability was its requirement of “physical harm.” As the Restatement (Second) of Torts made clear:

“harm implies a loss or detriment to a person, and not a mere change or alteration in some physical person, object, or thing. Physical changes or alterations may be either beneficial, detrimental, or of no consequence to a person. In so far as physical changes have a detrimental effect on a person that person suffers harm.”[6]

For this reason, juries are permitted to return no cause or no damage verdicts in cases brought by asbestos claimants with pleural plaques or pleural thickening without pulmonary impairment.[7]

Although not universal, the better rule, which the ALI should adopt, was articulated by the United States Supreme Court, in Metro-North Commuter Railroad v. Buckley,[8] when it rejected medical monitoring claims for asymptomatic asbestos claimants. In Buckley, the Court acknowledged[9] the obvious without relegating it to an affirmative defense: unlimited and unpredictable liability is sufficient reason in the abstract to going down the road of imposing medical monitoring liability.

—————————————————————————————————————-

[1]  See Am. L. Inst., Capturing the Voice of The American Law Institute: A Handbook for ALI Reporters and Those Who Review Their Work 3 (rev. ed. 2015).

[2]  Restatement (Third) of Torts: Concluding Provisions, Council Draft No.

1, Medical Monitoring (Aug. 24, 2020); Restatement (Third) of Torts: Concluding Provisions, Prelim. Draft No. 1, Medical Monitoring (Feb. 3, 2020).

[3]  Mark A. Behrens & Christopher E. Appel, “American Law Institute Proposes Controversial Medical Monitoring Rule in Final Part of Torts Restatement,” 87 Defense Counsel J. 1 (2021).

[4]  Id. at cmt. g.

[5]  Id.

[6]  Restatement (Second) of Torts § 7 (1965).

[7]  See, e.g., Caterinicchio v. Pittsburgh Corning Corp., 127 N.J. 428, 605 A.2d 1092 (1992) (rejecting claim that pleural thickening is an injury as a matter of law that requires the assessment of damages). See also James Beck & Mark Herrmann, “No Injury Cheat Sheet,” Drug & Device Law Blog (July 3, 2008).

[8]  Metro-North Commuter R.R. v. Buckley, 521 U.S. 424, 433 (1997).

[9]  Id. at 433 (quoting Consolidated Rail Corp. v. Gottshall, 512 U. S. 532, 557 (1994)). See also Herbert L. Zarov, Sheila Finnegan, Craig A. Woods, and Stephen J. Kane, “A Medical Monitoring Claim for Asymptomatic Plaintiffs: Should Illinois Take the Plunge?” 12 DePaul J. Health Care L. 1 (2009).

Posted in Medical monitoring, Risk and Risk Factor | 163 Comments »

Lawsuit Industry Advertising Indirectly Stimulates Adverse Event Reporting

February 4th, 2021

The lawsuit industry spends many millions of dollars each year to persuade people that they are ill from the medications they use, and that lawsuit industry lawyers will enrich them for their woes. But does the lawyer advertising stimulate the reporting of adverse events by consumers’ filing of MedWatch reports in the Federal Adverse Event Reporting System (FAERS)?

The question is of some significance. Adverse event reporting is a recognized, important component of pharmacovigilence. Regulatory agencies around the world look to an increased rate of reporting of a specific adverse event as a potential signal that there may be an underlying association between medication use and the reported harm. In the last two decades, pharmacoepidemiologists have developed techniques for mining databases of adverse event reports for evidence of a disproportionate level of reporting for a particular medication – adverse event pair. Such studies can help identify “signals” of potential issues for further study with properly controlled epidemiologic studies.[1]

One of the vexing misuses of pharmacovigilance techniques in the pharmaceutical products litigation is the use of adverse events reporting, either as case reports or in the form of disproportionality analyses to claim causal inference. In some litigations, lawsuit industry lawyers have argued that case reports, in the FAERS, standing alone support their claims of causation.[2] Desperate to make their case through anecdotes, plaintiffs’ counsel will sometimes retreat to the claim that they want to introduce the MedWatch reports in support of a lesser claim that the reports put the defendant on “notice.” Typically, the notice argument leaves open exactly what the content of the notice is, but the clear intent is to argue notice that (1) there is an increased risk, and (2) the defendant was aware of the increased risk.[3]

Standard textbooks on pharmacovigilance and pharmacoepidemiology, as well as regulatory agency guidance, emphatically reject the use of FAERS anecdotes or their transmogrification into disportionality analyses (DPAs) to support causal claims. The U.S. FDA’s official guidance on good pharmacovigilance practices, for example, elaborates on DPAs as an example of data mining, and instructs us that:

“[d]ata mining is not a tool for establishing causal attributions between products and adverse events.”[4]

The FDA specifically cautions that the signals detected by data mining techniques should be acknowledged to be “inherently exploratory or hypothesis generating.”[5] The agency exercises caution when making its own comparisons of adverse events between products in the same class because of the low quality of the data themselves, and uncontrollable and unpredictable biases in how the data are collected.[6] Because of the uncertainties in DPAs, the FDA urges “extreme causation” in comparing reporting rates, and generally considers DPA and similar analyses as “exploratory or hypothesis-generating.”[7]

The European Medicines Agency offers similar advice and caution:

“Therefore, the concept of SDR [Signal of Disproportionate Reporting] is applied in this guideline to describe a ‘statistical signal’ that has originated from a statistical method. The underlying principle of this method is that a drug–event pair is reported more often than expected relative to an independence model, based on the frequency of ICSRs on the reported drug and the frequency of ICSRs of a specific adverse event. This statistical association does not imply any kind of causal relationship between the administration of the drug and the occurrence of the adverse event.”[8]

Because the lawsuit industry frequently relies upon and over-endorses DPAs in its pharmaceutical litigations, inquiring minds may want to know whether the industry itself is stimulating reporting of adverse events through its media advertising.

Recently, two investigators published a study that attempted to look at whether lawsuit industry advertising was associated with stimulation of adverse event reporting in the FAERS.[9] Tippett and Chen conducted a multivariate regression analysis of FAERS reporting with independent variables of Google searches, attorney advertising, and FDA actions that would affect reporting over the course of a single calendar year (mid-2015 to mid-2016). The authors analyzed 412,901 adverse event reports to FAERS, involving 28 groups of drugs that were the subject of solicitous advertising.

The authors reported that they found associations (statistically significant, p < 0.05) for regression coefficients for FDA safety actions and Google searches, but not for attorney advertising. Using lag periods of one, two, three, and four weeks, or one or two months, between FAERS reporting and the variables did not result in statistically significant coefficients for lawyer advertising.

The authors variably described their finding as “preliminarily” supporting a claim that FAERS reporting is not stimulated by “direct attorney submission or drug injury advertising,” or as failing to find “a statistically significant relationship between drug injury advertising and adverse event reports.”[10] The authors claim that their analyses show that litigation advertisements “do not appear to have spurred patients, providers, attorneys, or other individuals to file a FAERS report, as shown in our regression and graphical results.”[11]

There are substantial problems with this study. For most of the 28 drugs and drug groups studied, attorneys made up a very small proportion of all submitters of adverse event reports. The authors present no pre-study power analysis for this aspect of their study. The authors do not tell us how many analyses they have done before the one presented in this journal article, but they do acknowledge having done “exploratory analyses.” Contrary to the 2016 guidance of the American Statistical Association,[12] they present no actual p-values, and they provide no confidence or prediction intervals for their coefficients. The study did not include local television advertising, and so the reported statistical non-significance of attorney advertising must be qualified to show the limitations of the authors’ data.

Perhaps the most serious problem with this observational study of attorney advertising and stimulated reporting is the way in which the authors framed their hypothesis. Advertising stimulates people to call the toll-free number to learn more how they too may hit the litigation jackpot. The point of attorney advertising is designed to persuade people to become legal clients, not to file MedWatch forms. In the following weeks and months that follow, paralegals interview the callers, collect information, and only then FAERs happen. Lag times of one to four weeks are generally irrelevant, as is the hypothesis studied and reported upon in this article.

After decades of working in this area, I have never seen an advertisement that encourages filing a MedWatch report, and the authors do not suggest otherwise. Advertising is only the initial part of a client intake mechanism that would result in the viewers’ making a telephone call, with a subsequent interview by lawfirm personnel, a review of the putative claim, and the viewers’ obtaining and signing retainer agreements and authorizations to obtain medical records. The scope of the study, which looked at FAERS filings and attorney advertisements after short lag periods could not detect an association given how long the recruitment takes.

The authors speculate, without evidence, that the lawsuit industry may discourage their clients from filing MedWatch reports and that the industry lawyers may hesitate to file the reports to avoid serving as a fact witness in their client’s case.[13] Indeed, the authors themselves adduce compelling evidence to the contrary, in the context of the multidistrict litigation over claimed harms from the use of testosterone therapies.

In their aggregate analysis of the 28 drugs and drug groups, the authors found that the lawsuit industry submitted only six percent of MedWatch reports. This low percentage would have been much lower yet but for the very high proportion (68%) of lawyer-submitted reports concerning the use of testosterone. The litigation-driven filings lagged the relevant attorney advertising by about six months, which should have caused the authors to re-evaluate their conclusions and their observational design that looked for correlations within one or two months. The testosterone data shows rather clearly that attorney advertising leads to recruitment of clients, which in turn leads to the filing of litigation-driven adverse event reports.

As the authors explain, attorney advertising and trolling for clients occurred in the summer of 2015, but FAERS reporting did not increase until an extreme burst of filings took place several months later. The authors’ graph tells the story even better:

So the correct conclusion is that attorney advertising stimulates client recruitment, which results in mass filings of MedWatch reports.

___________________________________________________________________

[1]  Sean Hennessy, “Disproportionality analyses of spontaneous reports,” 13 Pharmacoepidemiology & Drug Safety 503, 503 (2004). See also “Disproportionality Analyses Misused by Lawsuit Industry” (Apr. 20, 2020).

[2]  See, e.g., Fred S. Longer, “The Federal Judiciary’s Super Magnet,” 45 Trial 18, 18 (July 2009) (arguing that “adverse events . . . established a causal association between Piccolomal and liver disease at statistically significant levels”).

[3]  See, e.g., Paul D. Rheingold, “Drug Products Liability and Malpractice Cases,” 17 Am. Jur. 1, Trials, Cumulative Supplement (1970 & Supp. 2019) (“Adverse event reports (AERs) created by manufacturers when users of their over-the-counter pain reliever experienced adverse events or problems, were admissible to show notice” of the elevated risk.).

[4]  FDA, “Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment Guidance for Industry” at 8 (2005) (emphasis added).

[5]  Id. at 9.

[6]  Id.

[7]  Id. at 11.

[8] EUDRAVigilance Expert Working Group, European Medicines Agency, “Guideline on the Use of Statistical Signal Detection Methods in the EUDRAVigilance Data Analysis System,” at 3 (2006) (emphasis added). See also Gerald J. Dal Pan, Marie Lindquist & Kate Gelperin, “Postmarketing Spontaneous Pharmacovigilance Reporting Systems,” in Brian L. Strom & Stephen E. Kimmel and Sean Hennessy, Pharmacoepidemiology at 185 (6th ed. 2020).

[9]  Elizabeth C. Tippett & Brian K. Chen, “Does Attorney Advertising Stimulate Adverse Event Reporting?” 74 Food & Drug Law J. 501 (2020) [Tippett].

[10]  Id. at 502.

[11]  Id.

[12]  Ronald L. Wasserstein & Nicole A. Lazar, “The ASA’s Statement on p-Values: Context, Process, and Purpose,” 70 The Am. Statistician 129 (2016).

[13]  Tippett at 591.

Posted in Causation, Confounding | 321 Comments »

Center for Truth in Science

February 2nd, 2021

The Center for Truth in Science

Well, now I have had the complete 2020 experience, trailing into 2021. CoVid-19, a.k.a. Trump flu happened. The worst for me is now mostly over, and I can see a light at the end of the tunnel. Fortunately it is not the hypoxemic end-of-life light at the end of the tunnel.

Kurt Gödel famously noted that the “the meaning of world is the separation of wish and fact.” The work of science in fields that touch on religion, politics, and other dogmas requires nothing less than separating wish from fact. Sadly, most people are cut off from the world of science by ignorance, lack of education, and social media that blur the distinction between wish and fact, and ultimately replace the latter with the former.

It should go without saying that truth is science and science is truth, but our current crises show that truth and science are both victims of the same forces that blur wish with fact. We might think that a center established for “truth in science” is as otiose as a center for justice in the law, but all the social forces at work to blur wish and fact make such a center an imperative for our time.

The Center for Truth in Science was established last year, and has already weighed in on important issues and scientific controversies that occupy American courtrooms and legislatures. Championing “fact-based” science, the Center has begun to tackle some of the difficult contemporary scientific issues that loom large on the judicial scene – talc, glyphosate, per- and polyfluoroalkyl substances, and other – as well as methodological and conceptual problems that underlie these issues. (Of course, there is no other kind of science than fact-based, but there are many pseudo-, non-fact based knock offs out there.) The Center has already produced helpful papers on various topics, with many more important papers in progress. The Center’s website is a welcomed resource for news and insights on science that matters for current policy decisions.

The Center is an important and exciting development, and its work promises to provide the tools to help us separate wish from fact. Nothing less than the meaning of the world is at stake.

Posted in Evidence, Scientific Evidence, Scientific Publishing, Uncategorized | Comments Off on Center for Truth in Science