TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

April Fool – Zambelli-Weiner Must Disclose

April 2nd, 2020

Back in the summer of 2019, Judge Saylor, the MDL judge presiding over the Zofran birth defect cases, ordered epidemiologist, Dr. Zambelli-Weiner to produce documents relating to an epidemiologic study of Zofran,[1] as well as her claimed confidential consulting relationship with plaintiffs’ counsel.[2]

This previous round of motion practice and discovery established that Zambelli-Weiner was a paid consultant in advance of litigation, that her Zofran study was funded by plaintiffs’ counsel, and that she presented at a Las Vegas conference, for plaintiffs’ counsel only, on [sic] how to make mass torts perfect. Furthermore, she had made false statements to the court about her activities.[3]

Zambelli-Weiner ultimately responded to the discovery requests but she and plaintiffs’ counsel withheld several documents as confidential, pursuant to the MDL’s procedure for protective orders. Yesterday, April 1, 2020, Judge Saylor entered granted GlaxoSmithKline’s motion to de-designate four documents that plaintiffs claimed to be confidential.[4]

Zambelli-Weiner sought to resist GSK’s motion to compel disclosure of the documents on a claim that GSK was seeking the documents to advance its own litigation strategy. Judge Saylor acknowledged that Zambelli-Weiner’s psycho-analysis might be correct, but that GSK’s motive was not the critical issue. According to Judge Saylor, the proper inquiry was whether the claim of confidentiality was proper in the first place, and whether removing the cloak of secrecy was appropriate under the facts and circumstances of the case. Indeed, the court found “persuasive public-interest reasons” to support disclosure, including providing the FDA and the EMA a complete, unvarnished view of Zambelli-Weiner’s research.[5] Of course, the plaintiffs’ counsel, in close concert with Zambelli-Weiner, had created GSK’s need for the documents.

This discovery battle has no doubt been fought because plaintiffs and their testifying expert witnesses rely heavily upon the Zambelli-Weiner study to support their claim that Zofran causes birth defects. The present issue is whether four of the documents produced by Dr. Zambelli-Weiner pursuant to subpoena should continue to enjoy confidential status under the court’s protective order. GSK argued that the documents were never properly designated as confidential, and alternatively, the court should de-designate the documents because, among other things, the documents would disclose information important to medical researchers and regulators.

Judge Saylor’s Order considered GSK’s objections to plaintiffs’ and Zambelli-Weiner’s withholding four documents:

(1) Zambelli-Weiner’s Zofran study protocol;

(2) Undisclosed, hidden analyses that compared birth defects rates for children born to mothers who used Zofran with the rates seen with the use of other anti-emetic medications;

(3) An earlier draft Zambelli-Weiner’s Zofran study, which she had prepared to submit to the New England Journal of Medicine; and

(4) Zambelli-Weiner’s advocacy document, a “Causation Briefing Document,” which she prepared for plaintiffs’ lawyers.

Judge Saylor noted that none of the withheld documents would typically be viewed as confidential. None contained “sensitive personal, financial, or medical information.”[6]  The court dismissed Zambelli-Weiner’s contention that the documents all contained “business and proprietary information,” as conclusory and meritless. Neither she nor plaintiffs’ counsel explained how the requested documents implicated proprietary information when Zambelli-Weiner’s only business at issue is to assist in making lawsuits. The court observed that she is not “engaged in the business of conducting research to develop a pharmaceutical drug or other proprietary medical product or device,” and is related solely to her paid consultancy to plaintiffs’ lawyers. Neither she nor the plaintiffs’ lawyers showed how public disclosure would hurt her proprietary or business interests. Of course, if Zambelli-Weiner had been dishonest in carrying out the Zofran study, as reflected in study deviations from its protocol, her professional credibility and her business of conducting such studies might well suffer. Zambelli-Weiner, however, was not prepared to affirm the antecedent of that hypothetical. In any event, the court found that whatever right Zambelli-Weiner might have enjoyed to avoid discovery evaporated with her previous dishonest representations to the MDL court.[7]

The Zofran Study Protocol

GSK sought production of the Zofran study protocol, which in theory contained the research plan for the Zofran study and the analyses the researchers intended to conduct. Zambelli-Weiner attempted to resist production on the specious theory that she had not published the protocol, but the court found this “non-publication” irrelevant to the claim of confidentiality. Most professional organizations, such as the International Society of Pharmacoepidemiology (“ISPE”), which ultimately published Zambelli-Weiner’s study, encourage the publication and sharing of study protocols.[8] Disclosure of protocols helps ensure the integrity of studies by allowing readers to assess whether the researchers have adhered to their study plan, or have engaged in ad hoc data dredging in search for a desired result.[9]

The Secret, Undisclosed Analyses

Perhaps even more egregious than withholding the study protocol was the refusal to disclose unpublished analyses comparing the rate of birth defects among children born to mothers who used Zofran with the birth defect rates of children with in utero exposure to other anti-emetic medications.  In ruling that Zambelli-Weiner must produce the unpublished analyses, the court expressed its skepticism over whether these analyses could ever have been confidential. Under ISPE guidelines, researchers must report findings that significantly affect public health, and the relative safety of Zofran is essential to its evaluation by regulators and prescribing physicians.

Not only was Zambelli-Weiner’s failure to include these analyses in her published article ethically problematic, but she apparently hid these analyses from the Pharmacovigilance Risk Assessment Committee (PRAC) of the European Medicines Agency, which specifically inquired of Zambelli-Weiner whether she had performed such analyses. As a result, the PRAC recommended a label change based upon Zambelli-Weiner’s failure to disclosure material information. Furthermore, the plaintiffs’ counsel represented they intended to oppose GSK’s citizen petition to the FDA, based upon the Zambelli-Weiner study. The apparently fraudulent non-disclosure of relevant analyses could not have been more fraught for public health significance. The MDL court found that the public health need trumped any (doubtful) claim to confidentiality.[10] Against the obvious public interest, Zambelli-Weiner offered no “compelling countervailing interest” in keeping her secret analyses confidential.

There were other aspects to the data-dredging rationale not discussed in the court’s order. Without seeing the secret analyses of other anti-emetics, readers were deprive of an important opportunity to assess actual and potential confounding in her study. Perhaps even more important, the statistical tools that Zambelli-Weiner used, including any measurements of p-values and confidence intervals, and any declarations of “statistical significance,” were rendered meaningless by her secret, undisclosed, multiple testing. As noted by the American Statistical Association (ASA) in its 2016 position statement, “4. Proper inference requires full reporting and transparency.”

The ASA explains that the proper inference from a p-value can be completely undermined by “multiple analyses” of study data, with selective reporting of sample statistics that have attractively low p-values, or cherry picking of suggestive study findings. The ASA points out that common practices of selective reporting compromises valid interpretation. Hence the correlative recommendation:

“Researchers should disclose the number of hypotheses explored during the study, all data collection decisions, all statistical analyses conducted and all p-values computed. Valid scientific conclusions based on p-values and related statistics cannot be drawn without at least knowing how many and which analyses were conducted, and how those analyses (including p-values) were selected for reporting.”[11]

The Draft Manuscript for the New England Journal of Medicine

The MDL court wasted little time and ink in dispatching Zambelli-Weiner’s claim of confidentiality for her draft New England Journal of Medicine manuscript. The court found that she failed to explain how any differences in content between this manuscript and the published version constituted “proprietary business information,” or how disclosure would cause her any actual prejudice.

Zambelli-Weiner’s Litigation Road Map

In a world where social justice warriors complain about organizations such as Exponent, for its litigation support of defense efforts, the revelation that Zambelli-Weiner was helping to quarterback the plaintiffs’ offense deserves greater recognition. Zambelli-Weiner’s litigation road map was clearly created to help Grant & Eisenhofer, P.A., the plaintiffs’ lawyers,, create a causation strategy (to which she would add her Zofran study). Such a document from a consulting expert witness is typically the sort of document that enjoys confidentiality and protection from litigation discovery. The MDL court, however, looked beyond Zambelli-Weiner’s role as a “consulting witness” to her involvement in designing and conducting research. The broader extent of her involvement in producing studies and communicating with regulators made her litigation “strategery” “almost certainly relevant to scientists and regulatory authorities” charged with evaluating her study.”[12]

Despite Zambelli-Weiner’s protestations that she had made a disclosure of conflict of interest, the MDL court found her disclosure anemic and the public interest in knowing the full extent of her involvement in advising plaintiffs’ counsel, long before the study was conducted, great.[13]

The legal media has been uncommonly quiet about the rulings on April Zambelli-Weiner, in the Zofran litigation. From the Union of Concerned Scientists, and other industry scolds such as David Egilman, David Michaels, and Carl Cranor – crickets. Meanwhile, while the appeal over the admissibility of her testimony is pending before the Pennsylvania Supreme Court,[14] Zambelli-Weiner continues to create an unenviable record in Zofran, Accutane,[15] Mirena,[16] and other litigations.

[1]  April Zambelli‐Weiner, Christina Via, Matt Yuen, Daniel Weiner, and Russell S. Kirby, “First Trimester Pregnancy Exposure to Ondansetron and Risk of Structural Birth Defects,” 83 Reproductive Toxicology 14 (2019).

[2]  See In re Zofran (Ondansetron) Prod. Liab. Litig., 392 F. Supp. 3d 179, 182-84 (D. Mass. 2019) (MDL 2657) [cited as In re Zofran].

[3]  “Litigation Science – In re Zambelli-Weiner” (April 8, 2019); “Mass Torts Made Less Bad – The Zambelli-Weiner Affair in the Zofran MDL” (July 30, 2019). See also Nate Raymond, “GSK accuses Zofran plaintiffs’ law firms of funding academic study,” Reuters (Mar. 5, 2019).

[4]  In re Zofran Prods. Liab. Litig., MDL No. 1:15-md-2657-FDS, Order on Defendant’s Motion to De-Designate Certain Documents as Confidential Under the Protective Order (D.Mass. Apr. 1, 2020) [Order].

[5]  Order at n.3

[6]  Order at 3.

[7]  See In re Zofran, 392 F. Supp. 3d at 186.

[8]  Order at 4. See also Xavier Kurz, Susana Perez-Gutthann, the ENCePP Steering Group, “Strengthening standards, transparency, and collaboration to support medicine evaluation: Ten years of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP),” 27 Pharmacoepidemiology & Drug Safety 245 (2018).

[9]  Order at note 2 (citing Charles J. Walsh & Marc S. Klein, “From Dog Food to Prescription Drug Advertising: Litigating False Scientific Establishment Claims Under the Lanham Act,” 22 Seton Hall L. Rev. 389, 431 (1992) (noting that adherence to study protocol “is essential to avoid ‘data dredging’—looking through results without a predetermined plan until one finds data to support a claim”).

[10]  Order at 5, citing Anderson v. Cryovac, Inc., 805 F.2d 1, 8 (1st Cir. 1986) (describing public-health concerns as “compelling justification” for requiring disclosing of confidential information).

[11]  Ronald L. Wasserstein & Nicole A. Lazar, “The ASA’s Statement on p-Values: Context, Process, and Purpose,” 70 The American Statistician 129 (2016)

See also “The American Statistical Association’s Statement on and of Significance” (March 17, 2016).“Courts Can and Must Acknowledge Multiple Comparisons in Statistical Analyses” (Oct. 14, 2014).

[12]  Order at 6.

[13]  Cf. Elizabeth J. Cabraser, Fabrice Vincent & Alexandra Foote, “Ethics and Admissibility: Failure to Disclose Conflicts of Interest in and/or Funding of Scientific Studies and/or Data May Warrant Evidentiary Exclusions,” Mealey’s Emerging Drugs Reporter (Dec. 2002) (arguing that failure to disclose conflicts of interest and study funding should result in evidentiary exclusions).

[14]  Walsh v. BASF Corp., GD #10-018588 (Oct. 5, 2016, Pa. Ct. C.P. Allegheny Cty., Pa.) (finding that Zambelli-Weiner’s and Nachman Brautbar’s opinions that pesticides generally cause acute myelogenous leukemia, that even the smallest exposure to benzene increases the risk of leukemia offended generally accepted scientific methodology), rev’d, 2018 Pa. Super. 174, 191 A.3d 838, 842-43 (Pa. Super. 2018), appeal granted, 203 A.3d 976 (Pa. 2019).

[15]  In re Accutane Litig., No. A-4952-16T1, (Jan. 17, 2020 N.J. App. Div.) (affirming exclusion of Zambelli-Weiner as an expert witness).

[16]  In re Mirena IUD Prods. Liab. Litig., 169 F. Supp. 3d 396 (S.D.N.Y. 2016) (excluding Zambelli-Weiner in part).

Posted in Conflicts of Interest, Data Sharing, Expert Witness Discovery, Expert Witness Hall of Shame, Expert Witnesses, Scientific Misconduct, Scientific Publishing, statistical evidence, Trial Procedure, Underlying Data | Comments Off on April Fool – Zambelli-Weiner Must Disclose

Practical Solutions for the Irreproducibility Crisis

March 3rd, 2020

I have previously praised the efforts of the National Association of Scholars (NAS) for its efforts to sponsor a conference on “Fixing Science: Practical Solutions for the Irreproducibility Crisis.” The conference was a remarkable event, with a good deal of diverse view points, civil discussion and debate, and collegiality.

The NAS has now posted a follow up to its conference, with a link to slide presentations, and to a You Tube page with videos of the presentations. The NAS, along with The Independent Institute, should be commended for their organizational efforts, and their transparency in making the conference contents available now to a wider audience.

The conference took place on February 7th and 8th, and I had the privilege of starting the event with my presentation, “Not Just an Academic Dispute: Irreproducible Scientific Evidence Renders Legal Judgments Unsafe”.

Some, but not all, of the interesting presentations that followed:

Tim Edgell, “Stylistic Bias, Selective Reporting, and Climate Science” (Feb. 7, 2020)

Patrick J. Michaels, “Biased Climate Science” (Feb. 7, 2020)

Daniele Fanelli, “Reproducibility Reforms if there is no Irreproducibility Crisis” (Feb. 8, 2020)

On Saturday, I had the additional privilege of moderating a panel on “Group Think” in science, and its potential for skewing research focus and publication:

Lee Jussim, “Intellectual Diversity Limits Groupthink in Scientific Psychology” (Feb. 8, 2020)

Mark Regnerus, “Groupthink in Sociology” (Feb. 8, 2020)

Michael Shermer, “Giving the Devil His Due” (Feb. 8, 2020)

Later on Saturday, the presenters turned to methodological issues, many of which are key to understanding ongoing scientific and legal controversies:

Stanley Young, “Prevention and Management of Acute and Late Toxicities in Radiation Oncology”

James E. Enstrom, “Reproducibility is Essential to Combating Environmental Lysenkoism”

Deborah Mayo, “P-Value ‘Reforms’: Fixing Science or Threats to Replication and Falsification?” (Feb. 8, 2020)

Ronald L. Wasserstein, “What Professional Organizations Can Do To Fix The Irreproducibility Crisis” (Feb. 8, 2020)

Louis Anthony Cox, Jr., “Causality, Reproducibility, and Scientific Generalization in Public Health” (Feb. 8, 2020)

David Trafimow, “What Journals Can Do To Fix The Irreproducibility Crisis” (Feb. 8, 2020)

David Randall, “Regulatory Science and the Irreproducibility Crisis” (Feb. 8, 2020)

Posted in Scientific Evidence, Scientific Misconduct, Scientific Publishing, statistical evidence | Comments Off on Practical Solutions for the Irreproducibility Crisis

Science Journalism – UnDark Noir

February 23rd, 2020

Critics of the National Association of Scholars’ conference on Fixing Science pointed readers to an article in Undark, an on-line popular science site for lay audiences, and they touted the site for its science journalism. My review of the particular article left me unimpressed and suspicious of Undark’s darker side. When I saw that the site featured an article on the history of the Supreme Court’s Daubert decision, I decided to give the site another try. For one thing, I am sympathetic to the task science journalists take on: it is important and difficult. In many ways, lawyers must commit to perform the same task. Sadly, most journalists and lawyers, with some notable exceptions, lack the scientific acumen and English communication skills to meet the needs of this task.

The Undark article that caught my attention was a history of the Daubert decision and the Bendectin litigation that gave rise to the Supreme Court case.[1] The author, Peter Andrey Smith, is a freelance reporter, who often covers science issues. In his Undark piece, Smith covered some of the oft-told history of the Daubert case, which has been told before, better and in more detail in many legal sources. Smith gets some credit for giving the correct pronunciation of the plaintiff’s name – “DAW-burt,” and for recounting how both sides declared victory after the Supreme Court’s ruling. The explanation Smith gives of the opinion by Associate Justice Harry Blackmun is reasonably accurate, and he correctly notes that a partial dissenting opinion by Chief Justice Rehnquist complained that the majority’s decision would have trial judges become “amateur scientists.” Nowhere in the article will you find, however, the counter to the dissent: an honest assessment of the institutional and individual competence of juries to decide complex scientific issues.

The author’s biases eventually, however, become obvious. He recounts his interviews with Jason Daubert and his mother, Joyce Daubert. He earnestly reports how Joyce Daubert remembered having taken Bendectin during her pregnancy with Jason, and in the moment of that recall, “she felt she’d finally identified the teratogen that harmed Jason.” Really? Is that how teratogens are identified? Might it have been useful and relevant for a scientific journalist to explain that there are four million live births every year in the United States and that 3% of children born each year have major congenital malformations? And that most malformations have no known cause? Smith ingenuously relays that Jason Daubert had genetic testing, but omits that genetic testing in the early 1990s was fairly primitive and limited. In any event, how were any expert witnesses supposed to rule out base-line risk of birth defects, especially given weak to non-existent epidemiologic support for the Daubert’s claims? Smith does answer these questions; he does not even acknowledge the questions.

Smith later quotes Joyce Daubert as describing the litigation she signed up for as “the hill I’ll die on. You only go to war when you think you can win.” Without comment or analysis, Smith gives Joyce Daubert an opportunity to rant against the “injustice” of how her lawsuit turned out. Smith tells us that the Dauberts found the “legal system remains profoundly disillusioning.” Joyce Daubert told Smith that “it makes me feel stupid that I was so naïve to think that, after we’d invested so much in the case, that we would get justice.”  When called for jury duty, she introduces herself as

“I’m Daubert of Daubert versus Merrell Dow … ; I don’t want to sit on this jury and pretend that I can pass judgment on somebody when there is no justice. Please allow me to be excused.”

But didn’t she really get all the justice she deserved? Given her zealotry, doesn’t she deserve to have her name on the decision that serves to rein in expert witnesses who outrun their scientific headlights? Smith is coy and does not say, but in presenting Mrs. Daubert’s rant, without presenting the other side, he is using his journalistic tools in a fairly blatant attempt to mislead. At this point, I begin to get the feeling that Smith is preaching to a like-minded choir over there at Undark.

The reader is not treated to any interviews with anyone from the company that made Bendectin, any of its scientists, or any of the scientists who published actual studies on whether Bendectin was associated with the particular birth defects Jason Daubert had, or for that matter, with any birth defects at all. The plaintiffs’ expert witnesses quoted and cited never published anything at all on the subject. The readers are left to their imagination about how the people who developed Bendectin felt about the litigation strategies and tactics of the lawsuit industry.

The journalistic ruse is continued with Smith’s treatment of the other actors in the Daubert passion play. Smith describes the Bendectin plaintiffs’ lawyer Barry Nace in hagiographic terms, but omits his bar disciplinary proceedings.[2] Smith tells us that Nace had an impressive background in chemistry, and quotes him in an interview in which he described the evidentiary rules on scientific witness testimony as “scientific evidence crap.”

Smith never describes the Daubert’s actual affirmative evidence in any detail, which one might expect in a sophisticated journalistic outlet. Instead, he described some of their expert witnesses, Shanna Swan, a reproductive epidemiologist, and Alan K. Done, “a former pediatrician from Wayne State University.” Smith is secretive about why Done was done in at Wayne State; and we learn nothing about the serious accusations of perjury on credentials by Done. Instead, Smith regales us with Done’s tsumish theory, which takes inconclusive bits of evidence, throws them together, and then declares causation that somehow eludes the rest of the scientific establishment.

Smith tells us that Swan was a rebuttal witness, who gave an opinion that the data did not rule out “the possibility Bendectin caused defects.” Legally and scientifically, Smith is derelict in failing to explain that the burden was on the party claiming causation, and that Swan’s efforts to manufacture doubt were beside the point. Merrell Dow did not have to rule out any possibility of causation; the plaintiffs had to establish causation. Nor does Smith delve into how Swan sought to reprise her performance in the silicone gel breast implant litigation, only to be booted by several judges as an expert witness. And then for a convincer, Smith sympathetically repeats plaintiffs’ lawyer Barry Nace’s hyperbolic claim that Bendectin manufacturer, Merrell Dow had been “financing scientific articles to get their way,” adding by way of emphasis, in his own voice:

“In some ways, here was the fake news of its time: If you lacked any compelling scientific support for your case, one way to undermine the credibility of your opponents was by calling their evidence ‘junk science’.”

Against Nace’s scatalogical Jackson Pollack approach, Smith is silent about another plaintiffs’ expert witness, William McBride, who was found guilty of scientific fraud.[3] Smith reports interviews of several well-known, well-respected evidence scholars. He dutifully report Professor Edward Cheng’s view that “the courts were right to dismiss the [Bendectin] plaintiffs’ claims.” Smith quotes Professor D. Michael Risinger that claims from both sides in Bendectin cases were exaggerated, and that the 1970s and 1980s saw an “unbridled expansion of self-anointed experts,” with “causation in toxic torts had been allowed to become extremely lax.” So a critical reader might wonder why someone like Professor Cheng, who has a doctorate in statistics, a law degree from Harvard, and teaches at Vanderbilt Law School, would vindicate the manufacturers’ position in the Bendectin litigation. Smith never attempts to reconcile his interviews of the law professors with the emotive comments of Barry Nace and Joyce Daubert.

Smith acknowledges that a reformulated version of Bendectin, known as  Diclegis, was approved by the Food and Drug Administration in the United States, in 2013, for treatment of  nausea and vomiting during pregnancy. Smith tells us that Joyce is not convinced the drug should be back on the market,” but really why would any reasonable person care about her view of the matter? The challenge by Nav Persaud, a Toronto physician, is cited, but Persaud’s challenge is to the claim of efficacy, not to the safety of the medication. Smith tells us that Jason Daubert “briefly mulled reopening his case when Diclegis, the updated version of Bendectin, was re-approved.” But how would the approval of Diclegis, on the strength of a full new drug application, somehow support his claim anew? And how would he “reopen” a claim that had been fully litigated in the 1990s, and well past any statute of limitations?

Is this straight reporting? I think not. It is manipulative and misleading.

Smith notes, without attribution, that some scholars condemn litigation, such as the cases involving Bendectin, as an illegitimate form of regulation of medications. In opposition, he appears to rely upon Elizabeth Chamblee Burch, a professor at the University of Georgia School of Law for the view that because the initial pivotal clinical trials for regulatory approvals take place in limited populations, litigation “serves as a stopgap for identifying rare adverse outcomes that could crop up when several hundreds of millions of people are exposed to those products over longer periods of time.” The problem with this view is that Smith ignores the whole process of pharmacovigilance, post-registration trials, and pharmaco-epidemiologic studies conducted after the licensing of a new medication. The suggested necessity of reliance upon the litigation system as an adjunct to regulatory approval is at best misplaced and tenuous.

Smith correctly explains that the Daubert standard is still resisted in criminal cases, where it could much improve the gatekeeping of forensic expert witness opinion. But while the author gets his knickers in a knot over wrongful convictions, he seems quite indifferent to wrongful judgments in civil action.

Perhaps the one positive aspect of this journalistic account of the Daubert case was that Jason Daubert, unlike his mother, was open minded about his role in transforming the law of scientific evidence. According to Smith, Jason Daubert did not see the case as having “not ruined his life.” Indeed, Jason seemed to approve the basic principle of the Daubert case, and the subsequent legislation that refined the admissibility standard: “Good science should be all that gets into the courts.”

[1] Peter Andrey Smith, “Where Science Enters the Courtroom, the Daubert Name Looms Large: Decades ago, two parents sued a drug company over their newborn’s deformity – and changed courtroom science forever,” Undark (Feb. 17, 2020).

[2]  Lawyer Disciplinary Board v. Nace, 753 S.E.2d 618, 621–22 (W. Va.) (per curiam), cert. denied, 134 S. Ct. 474 (2013).

[3] Neil Genzlinger, “William McBride, Who Warned About Thalidomide, Dies at 91,” N.Y. Times (July 15, 2018); Leigh Dayton, “Thalidomide hero found guilty of scientific fraud,” New Scientist (Feb. 27, 1993); G.F. Humphrey, “Scientific fraud: the McBride case,” 32 Med. Sci. Law 199 (1992); Andrew Skolnick, “Key Witness Against Morning Sickness Drug Faces Scientific Fraud Charges,” 263 J. Am. Med. Ass’n 1468 (1990).

Posted in Causation, Evidence, Expert Witness Hall of Shame, Expert Witnesses, Rule 702, Scientific Evidence, Scientific Misconduct | Comments Off on Science Journalism – UnDark Noir

Counter Cancel Culture – The NAS Conference on Irreproducibility

February 9th, 2020

“The meaning of the world is the separation of wish and fact.”  Kurt Gödel

Back in October 2019, David Randall, the Director of Research, of the National Association of Scholars, contacted me to ask whether I would be interested in presenting at a conference, to be titled “Fixing Science: Practical Solutions for the Irreproducibility Crisis.” David explained that the conference would be aimed at a high level consideration of whether such a crisis existed, and if so, what salutary reforms might be implemented.

As for the character and commitments of the sponsoring organizations, David was candid and forthcoming. I will quote him, without his permission, and ask his forgiveness later:

“The National Association of Scholars is taken to be conservative by many scholars; the Independent Institute is (broadly speaking) in the libertarian camp. The NAS is open to but currently agnostic about the degree of human involvement in climate change. The Independent Institute I take to be institutionally skeptical of consensus climate change theory–e.g., they recently hosted Willie Soon for lecture. A certain number of speakers prefer not to participate in events hosted by institutions with these commitments.”

To me, the ask was for a presentation on how the so-called replication crisis, or the irreproducibility crisis, affected the law. This issue was certainly one I have had much occasion to consider. Although I am aware of the “adjacency” arguments made by some that people should be mindful of whom they align with, I felt that nothing in my participation would compromise my own views or unduly accredit institutional positions of the sponsors.

I was flattered by the invitation, but I did some due diligence on the sponsoring organizations. I vaguely recalled the Independent Institute from my more libertarian days, but the National Association of Scholars (NAS, not to be confused with Nathan A. Schachtman) was relatively unknown to me. A little bit of research showed that the NAS had a legitimate interest in the irreproducibility crisis. David Randall had written a monograph for the organization, which was a nice summary of some of the key problems. The Irreproducibility Crisis of Modern Science: Causes, Consequences,and the Road to Reform (2018).

On other issues, the NAS seemed to live up to its description as “an organization of scholars committed to higher education as the catalyst of American freedom.” I listened to some of the group’s podcasts, Curriculum Vitae, and browsed through its publications. I found myself agreeing with many positions articulated by or through the NAS, and disagreeing with a few positions very strongly.

In looking over the list of other invited speakers, I saw great diversity of view points and approaches, One distinguished speaker, Daniele Fanelli, had criticized the very notion that there was a reproducibility crisis. In the world of statistics, there were strong defenders of statistical tests, and vociferous critics. I decided to accept the invitation, not because I was flattered, but because the replication issue was important, and I believed that I could add something to the discussion before an audience of professional scientists, statisticians, and educated lay persons. In writing to David Randall to accept the invitation, I told him that with respect to the climate change issues, I was not at all put off by healthy skepticism in the face all dogmas. Every dogma will have its day.

I did not give any further consideration to the political aspect of the conference until early January, when I received an email from a scientist, Lenny Teytelman, Ph.D., the C.E.O. of a company protocols.io, which addresses reproducibility issues. Dr Teytelman’s interest in improving reproducibility seemed quite genuine, but he wrote to express his deep concern about the conference and the organizations that were sponsoring it.

Perhaps a bit pedantically, he cautioned me that the NAS was not the National Academy of Sciences, a confusion that never occurred to me because the National Academies has been known as the National Academies of Science, Engineering and Medicine for several years now. Dr. Teytelman’s real concern seemed to be that the NAS is a “‘politically conservative advocacy group’.” (The internal scare quotes were Teytelman’s, but I was not afraid.) According to Dr. Teytelman, the NAS sought to undermine climate science and environmental protection by advancing a call for more reproducible science. He pointed me to what he characterized as an exposé on NAS, in Undark,1 and he cautioned me that the National Association of Scholars’ work is “dangerous.” Finally, Dr. Teytelman urged me to reconsider my decision to participate in the conference.

I did reconsider my decision, but reaffirmed it in an email I sent back to Dr. Teytelman. I realized that I could be wrong, in which case, I would eat my words, confident that they would be most digestible:

Dear Dr Teytelman,

Thank you for your note. I was aware of the piece on Undark’s website, as well as the difference between the NAS and the NASEM. I don’t believe anyone involved in science education would likely to be confused between the two organizations. A couple of years ago, I wrote a teaching module on biomedical causation for the National Academies. This is my first presentation at the request of the NAS, and frankly I am honored by the organization’s request that I present at its conference.

I have read other materials that have been critical of the NAS and its publications on climate change and other issues. I know that there are views of the organization from which I would dissent, but I do not see my disagreement on some issues as a reason not to attend, and present at a conference on an issue of great importance to the legal system.

I am hardly an expert on climate change issues, and that is my failing. Most of my professional work involves health effects regulation and litigation. If the NAS has advanced sophistical arguments against a scientific claim, then the proper antidote will be to demonstrate its fallacious reasoning and misleading marshaling of evidence. I should think, however, as someone interested in improving the reproducibility of scientific research, you will agree that there is much common ground for discussion and reform of scientific practice, on a broader arrange [sic] of issues than climate change.

As for the political ‘conservatism’, of the organization, I am not sure why that is a reason to eschew participation in a conference that should be of great importance to people of all political views. My own politics probably owe much to the influence of Michael Oakeshott, which puts me in perhaps the smallest political tribe of any in the United States. If conservatism means antipathy to post-modernism, identity politics, political orthodoxies, and assaults on Enlightenment values and the Rule of Law, then count me in.

In any event, thanks for your solicitude. I think I can participate and return with my soul intact.

All the best.

Nathan

To his credit, Dr. Teytelman tenaciously continued. He acknowledged that the political leanings of the organizers were not a reason to boycott, but he politely pressed his case. We were now on a first name basis:

Dear Nathan,

I very much applaud all efforts to improve the rigour of our science. The problem here is that this NAS organization has a specific goal – undermining the environmental protection and denying climate change. This is why 7 out of the 21 speakers at the event are climate change deniers. [https://docs.google.com/spreadsheets/d/136FNLtJzACc6_JbbOxjy2urbkDK7GefRZ/edit?usp=sharing] And this isn’t some small fringe effort to be ignored. Efforts of this organization and others like them have now gotten us to the brink of a regulatory change at the United States Environmental Protection Agency which can gut the entire EPA (see a recent editorial against this I co-authored). This conference is not a genuine effort to talk about reproducibility. The reproducibility part is a clever disguise for pushing a climate change denialism agenda.

Best,

Lenny

I looked more carefully at Lenny’s spreadsheet, and considered the issue afresh. We were both pretty stubborn:

Dear Lenny,

Thank you for this information. I will review with interest.

I do not see that the conference is primarily or even secondarily about climate change vel non. There are two scientists, Trafimow and Wasserstein with whom I have some disagreements about statistical methodology. Tony Cox and Stan Young, whatever their political commitments or views on climate change may be, are both very capable statisticians, from whom I have learned a great deal. The conference should be a lively conversation about reproducibility, not about climate change. Given your interests and background, you should go.

I believe that your efforts here are really quite illiberal, although they are in line with the ‘cancel culture’, so popular on campuses these days.

Forty three years ago, I entered a Roman Catholic Church to marry the woman I love. There were no lightning bolts or temblors, even though I was then and I am now an atheist. Yes, I am still married to my first wife. Although I share the late Christopher Hitchins’ low view of the Catholic Church, somehow I managed to overcome my antipathy to being married in what some would call a house of ill repute. I even manage to agree with some Papist opinions, although not for the superstitious ‘reasons’ Papists embrace.

If I could tolerate the RC Church’s dogma for a morning, perhaps you could put aside the dichotomous ‘us and them’ view of the world and participate in what promises to be an interesting conference on reproducibility?

All the best.

Nathan

Lenny kindly acknowledged my having considered his issues, and wrote back a nice note, which I will quote again in full without permission, but with the hope that he will forgive me and even acknowledge that I have given his views an airing in this forum.

Hi Nathan,

We’ll have to agree to disagree. I don’t want to give a veneer of legitimacy to an organization whose goal is not improving reproducibility but derailing EPA and climate science.

Warmly,

Lenny

The business of psychoanalyzing motives and disparaging speakers and conference organizers is a dangerous business for several reasons. First motives can be inscrutable. Second, they can be misinterpreted. And third, they can be mixed. When speaking of organizations, there is the further complication of discerning a corporate motive among the constituent members.

The conference was an exciting, intellectually challenging event, which took place in Oakland, California, on February 7 and 8. I can report back to Lenny that his characterizations of and fears about the conference were unwarranted. While there were some assertions of climate change skepticism made with little or no evidence, the evidence-based presentations essentially affirmed climate change and sought to understand its causes and future course in a scientific way. But climate change was not why I went to this conference. On the more general issue of reform of scientific procedures and methods, we had open debates, some agreement on important principles, and robust and reasoned disagreement.

Lenny, you were correct that the NAS should not be ignored, but you should have gone to the meeting and participated in the conversation.

1 Michael Schulson, “A Remedy for Broken Science, Or an Attempt to Undercut It?” Undark (April 18, 2018).

Posted in Cognitive Biases, Conflicts of Interest, Scientific Evidence, Scientific Misconduct, statistical evidence | Comments Off on Counter Cancel Culture – The NAS Conference on Irreproducibility

Mass Torts Made Less Bad – The Zambelli-Weiner Affair in the Zofran MDL

July 30th, 2019

Judge Saylor, who presides over the Zofran MDL, handed down his opinion on the Zambelli-Weiner affair, on July 25, 2019.[1] As discussed on these pages back in April of this year,[2] GlaxoSmithKline (GSK), the defendant in the Zofran birth defects litigation, sought documents from plaintiffs and Dr Zambelli-Weiner (ZW) about her published study on Zofran and birth defects.[3] Plaintiffs refused to respond to the discovery on grounds of attorney work product,[4] and of consulting expert witness confidential communications.[5] After an abstract of ZW’s study appeared in print, GSK subpoenaed ZW and her co-author, Dr. Russell Kirby, for a deposition and for production of documents.

Plaintiffs’ counsel sought a protective order. Their opposition relied upon a characterization of ZW as a research scientist; they conveniently ommitted their retention of her as a paid expert witness. In December 2018, the MDL court denied plaintiffs’ motion for a protective order, and allowed the deposition to go forward to explore the financial relationship between counsel and ZW.

In January 2019, when GSK served ZW with its subpoena duces tecum, ZW through her own counsel moved for a protective order, supported by ZW’s affidavit with factual assertions to support her claim to be not subject to the deposition. The MDL court quickly denied her motion, and in short order, her lawyer notified the court that ZW’s affidavit contained “factual misrepresentations,” which she refused to correct, and he sought leave to withdraw.

According to the MDL court, the ZW affidavit contained three falsehoods. She claimed not to have been retained by any party when she had been a paid consultant to plaintiffs at times over the previous five years, since December 2014. ZW claimed that she had no factual information about the litigation, when in fact she had participated in a Las Vegas plaintiffs’ lawyers’ conference, “Mass Torts Made Perfect,” in October 2015. Furthermore, ZW falsely claimed that monies received from plaintiffs’ law firms did not go to fund the Zofran study, but went to her company, Translational Technologies International Health Research & Economics, for unrelated work. ZW received in excess of $200,000 for her work on the Zofran study.

After ZW obtained new counsel, she gave deposition testimony in February 2019, when she acknowledged the receipt of money for the study, and the lengthy relationship with plaintiffs’ counsel. Armed with this information, GSK moved for full responses to its document requests. Again, plaintiffs’ counsel and ZW resisted on grounds of confidentiality and privilege.

Judge Saylor reviewed the requested documents in camera, and held last week that they were not protected by consulting expert witness privilege or by attorney work product confidentiality. ZW’s materials and communications in connection with the Las Vegas plaintiffs’ conference never had the protection of privilege or confidentiality. ZW presented at a “quasi-public” conference attended by lawyers who had no connection to the Zofran litigation.[6]

With respect to work product claims, Judge Saylor found that GSK had shown “exceptional circumstances” and “substantial need” for the requested materials given that the plaintiffs’ testifying expert witnesses had relied upon the ZW study, which had been covertly financially supported by plaintiffs’ lawyers.[7] With respect to whatever was thinly claimed to be privileged and confidential, Judge Saylor found the whole arrangement to fail the smell test:[8]

“It is troublesome, to say the least, for a party to engage a consulting, non-testifying expert; pay for that individual to conduct and publish a study, or otherwise affect or influence the study; engage a testifying expert who relies upon the study; and then cloak the details of the arrangement with the consulting expert in the confidentiality protections of Rule 26(b) in order to conceal it from a party opponent and the Court. The Court can see no valid reason to permit such an arrangement to avoid the light of discovery and the adversarial process. Under the circumstances, GSK has made a showing of substantial need and an inability to obtain these documents by other means without undue hardship.

Furthermore, in this case, the consulting expert made false statements to the Court as to the nature of her relationship with plaintiffs’ counsel. The Court would not have been made aware of those falsehoods but for the fact that her attorney became aware of the issue and sought to withdraw. Certainly plaintiffs’ counsel did nothing at the time to correct the false impressions created by the affidavit. At a minimum, the submission of those falsehoods effectively waived whatever protections might otherwise apply. The need to discover the truth and correct the record surely outweighs any countervailing policy in favor of secrecy, particularly where plaintiffs’ testifying experts have relied heavily on Dr. Zambelli-Weiner’s study as a basis for their causation opinions. In order to effectively cross-examine plaintiffs’ experts about those opinions at trial, GSK is entitled to review the documents. At a minimum, the documents shed additional light on the nature of the relationship between Dr. Zambelli-Weiner and plaintiffs’ counsel, and go directly to the credibility of Dr. Zambelli-Weiner and the reliability of her study results.”

It remains to be seen whether Judge Saylor will refer the matter of ZW’s false statements in her affidavit to the U.S. Attorney’s office, or the lawyers’ complicity in perpetuating these falsehoods to disciplinary boards.

Mass torts will never be perfect, or even very good. Judge Saylor, however, has managed to make the Zofran litigation a little less bad.

[1]  Memorandum and order on In Camera Production of Documents Concerning Dr. April Zambelli-Weiner, In re Zofran Prods. Liab. Litig., MDL 2657, D.Mass. (July 25, 2019) [cited as Mem.].

[2]  NAS, “Litigation Science – In re Zambelli-Weiner” (April 8, 2019).

[3]  April Zambelli-Weiner, et al., “First Trimester Ondansetron Exposure and Risk of Structual Birth Defects,” 83 Reproductive Toxicol. 14 (2019).

[4]  Fed. R. Civ. P. 26(b)(3).

[5]  Fed. R. Civ. P. 26(b)(4)(D).

[6]  Mem. at 7-9.

[7]  Mem. at 9.

[8]  Mem. at 9-10.

Posted in Conflicts of Interest, Data Sharing, Expert Witness Discovery, Scientific Misconduct | Comments Off on Mass Torts Made Less Bad – The Zambelli-Weiner Affair in the Zofran MDL

Litigation Science – In re Zambelli-Weiner

April 8th, 2019

Back in 2001, in the aftermath of the silicone gel breast implant litigation, I participated in a Federal Judicial Center (FJC) television production of “Science in the Courtroom, program 6” (2001). Program six was a round-table discussion among the directors (past, present, and future) of the FJC, all of whom were sitting federal judges, with two lawyers in private practice, Elizabeth Cabraser and me.1 One of the more exasperating moments in our conversation came when Ms. Cabraser, who represented plaintiffs in the silicone litigation, complained that Daubert was unfair because corporate defendants were able to order up supporting scientific studies, whereas poor plaintiffs counsel did not have the means to gin up studies that confirmed what they knew to be true.2 Refraining from talking over her required all the self-restraint I could muster, but I did eventually respond by denying her glib generalization and offering the silicone litigation as one in which plaintiffs, plaintiffs’ counsel, and plaintiffs’ support groups were all involved in funding and directing some of the sketchiest studies, most of which managed to find homes in so-called peer-reviewed journals of some sort, even if not the best.

The litigation connections of the plaintiff-sponsored studies in the silicone litigation were not apparent on the face of the published articles. The partisan funding and provenance of the studies were mostly undisclosed and required persistent discovery and subpoenas. Cabraser’s propaganda reinforced the recognition of what so-called mass tort litigation had taught me about all scientific studies: “trust but verify.” Verification is especially important for studies that are sponsored by litigation-industry actors who have no reputation at stake in the world of healthcare.

Verification is not a straightforward task, however. Peer-review publication usually provides some basic information about “methods and materials,” but rarely if ever do published articles provide sufficient data and detail about methodology to replicate the reported analysis. In legal proceedings, verification of studies conducted and relied upon by testifying expert witnesses is facilitated by the rules of expert witness discovery. In federal court, expert witnesses must specify all opinions and all bases for their opinions. When such witnesses rely upon their own studies, and thus have had privileged access to the complete data and all analyses, courts have generally permitted full inquiry into the underlying materials of relied-upon studies. On the other, when the author of a relied-upon study is a “stranger to the litigation,” neither a party nor a retained expert witness, courts have permitted generally more limited discovery of the study’s full data set and analyses. Regardless of the author’s status, the question remains how litigants are to challenge an adversary’s expert witness’s trusted reliance upon a study, which cannot be “verified.”

Most lawyers would prefer, of course, to call an expert witness who has actually conducted studies pertinent to the issues in the case. The price, however, of allowing the other side to discover the underlying data and materials of the author expert witness’s studies may be too high. The relied-upon studies may well end up discredited, as well as the professional reputation of the expert witness. The litigation industry has adapted to these rules of discovery by avoiding, in most instances, calling testifying expert witnesses who have published studies that might be vulnerable.3

One work-around to the discovery rules lies in the use of “consulting, non-testifying expert witnesses.” The law permits the use of such expert witnesses to some extent to facilitate candid consultations with expert witnesses, usually without concerns that communications will be shared with the adversary party and witnesses. The hope is that such candid communications will permit realistic assessment of partisan positions, as well as allowing scientists and scholars to participate in an advisory capacity without the burden of depositions, formal report writing, and appearances at judicial hearings and trials. The confidentiality of consulting expert witnesses is open to abuse by counsel who would engage the consultants to conduct and publish studies, which can then be relied upon by the testifying expert witnesses. The upshot is that legal counsel can manipulate the published literature in a favorable way, without having to disclose their financial sponsorship or influence of the published studies used by their testifying expert witnesses.

This game of hiding study data and sponsorship through the litigation industry’s use of confidential consulting expert witnesses pervades so-called mass tort litigation, which provides ample financial incentives for study sponsorship and control. Defendants will almost always be unable to play the game, without detection. A simple interrogatory or other discovery request about funding of studies will reveal the attempt to pass off a party-sponsored study as having been conducted by disinterested scientists. Furthermore, most scientists will feel obligated to reveal corporate funding as a potential conflict of interest, in their submission of manuscripts for publication.

Revealing litigation-industry (plaintiffs’) funding of studies is more complicated. First, the funding may be through one firm, which is not the legal counsel in the case for which discovery is being conducted. In such instances, the plaintiff’s lawyers can truthfully declare that they lack personal knowledge of any financial support for studies relied upon by their testifying expert witnesses. Second, the plaintiffs’ lawyer firm is not a party is not itself subject to discovery. Even if the plaintiffs’ lawyers funded a study, they can claim, with plausible deniability, that they funded the study in connection with another client’s case, not the client who is plaintiff in the case in which discovery is sought. Third, the plaintiffs’ firm may take the position, however dubious it might be, that the funding of the relied-upon study was simply a confidential consultation with the authors of that study, and not subject to discovery.

The now pending litigation against ondansetron (Zofran) provides the most recent example of the dubious use of consulting expert witnesses to hide party sponsorship of an epidemiologic study. The plaintiffs, who are claiming that Zofran causes birth defects in this multi-district litigation assigned to Judge F. Dennis Saylor, have designated Dr. Carol Luik as their sole testifying expert witness on epidemiology. Dr. Luik, in turn, has relied substantially upon a study conducted by Dr. April Zambelli-Weiner.4

According to motion papers filed by defendants,5 the plaintiffs’ counsel initially claimed that they had no knowledge of any financial support or conflicts for Dr Zambelli-Weiner. The conflict-of-interest disclosure in Zambelli-Weiner’s paper was, to say the least, suspicious:

“The authors declare that there was no outside involvement in study design; in the collection, analysis and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication.”

As an organization TTi reports receiving funds from plaintiff law firms involved in ondansetron litigation and a manufacturer of ondansetron.”

According to its website, TTi

“is an economically disadvantaged woman-owned small business headquartered in Westminster, Maryland. We are focused on the development, evaluation, and implementation of technologies and solutions that advance the transformation of data into actionable knowledge. TTi serves a diverse clientele, including all stakeholders in the health space (governments, payors, providers, pharmaceutical and device companies, and foundations) who have a vested interest in advancing research to improve patient outcomes, population health, and access to care while reducing costs and eliminating health disparities.”

According to defendants’ briefing, and contrary to plaintiffs’ initial claims and Zambelli-Weiner’s anemic conflicts disclosure, plaintiffs’ counsel eventually admitted that “Plaintiffs’ Leadership Attorneys paid $210,000 as financial support relating to” Zambelli-Weiner’s epidemiologic study. The women at TTi are apparently less economically disadvantaged than advertised.

The Zofran defendants served subpoenas duces tecum and ad testificandum on two of the study authors, Drs. April Zambelli-Weiner and Russell Kirby. Curiously, the plaintiffs (who would seem to have no interest in defending the third-party subpoenas) sought a protective order by arguing that defendants were harassing “third-party scientists.” Their motion for protection conveniently and disingenuously omitted, that Zambelli-Weiner had been a paid consultant to the Zofran plaintiffs.

Judge Saylor refused to quash the subpoenas, and Zambelli-Weiner appeared herself, through counsel, to seek a protective order. Her supporting affidavit averred that she had not been retained as an expert witness, and that she had no documents “concerning any data analyses or results that were not reported in the [published study].” Zambelli-Weiner’s attempt to evade discovery was embarrassed by her having presented a “Zofran Litigation Update” with Plaintiffs’ counsel Robert Jenner and Elizabeth Graham at a national conference for plaintiffs’ attorneys. Judge Saylor was not persuaded, and the MDL court refused Dr. Zambelli-Weiner’s motion. The law and the public has a right to every man’s, and every woman’s, (even if economically disadvantaged) evidence.6

Tellingly, in the aftermath of the motions to quash, Zambelli-Weiner’s counsel, Scott Marder, abandoned his client by filing an emergency motion to withdraw, because “certain of the factual assertions in Dr. Zambelli-Weiner’s Motion for Protective Order and Affidavit were inaccurate.” Mr. Marder also honorably notified defense counsel that he could no longer represent that Zambelli-Weiner’s document production was complete.

Early this year, on January 29, 2019, Zambelli-Weiner submitted, through new counsel, a “Supplemental Affidavit,” wherein she admitted she had been a “consulting expert” witness for the law firm of Grant & Eisenhofer on the claimed teratogenicity of Zofran.7 Zambelli-Weiner also produced a few extensively redacted documents. On February 1, 2019, Zambelli-Weiner testified at deposition that the moneys she received from Grant & Eisenhofer were not to fund her Zofran study, but for other, “unrelated work.” Her testimony was at odds with the plaintiffs’ counsel’s confession that the $210,000 related to her Zofran study.

Zambelli-Weiner’s etiolated document production was confounded by the several hundred of pages of documents produced by fellow author, Dr. Russell Kirby. When confronted with documents from Kirby’s production, Zambelli-Weiner’s lawyer unilaterally suspended the deposition.

Deja Vu All Over Again

Federal courts have seen the Zambelli maneuver before. In litigation over claimed welding fume health effects, plaintiffs’ counsel Richard (Dickie) Scruggs and colleagues funded some neurological researchers to travel to Alabama and Mississippi to “screen” plaintiffs and potential plaintiffs in litigation for over claims of neurological injury and disease from welding fume exposure, with a novel videotaping methodology. The plaintiffs’ lawyers rounded up the research subjects (a.k.a. clients and potential clients), talked to them before the medical evaluations, and administered the study questionnaires. The study subjects were clearly aware of Mr. Scruggs’ “research” hypothesis, and had already promised him 40% of any recovery.8

After their sojourn, at Scruggs’ expense to Alabama and Mississippi, the researchers wrote up their results, with little or no detail of the circumstances of how they had acquired their research “participants,” or those participants’ motives to give accurate or inaccurate medical and employment history information.9

Defense counsel served subpoenas upon both Dr. Racette and his institution, Washington University St. Louis, for the study protocol, underlying data, data codes, and all statistical analyses. Racette and Washington University resisted sharing their data and materials with every page in the Directory of Non-Transparent Research. They claimed that the subpoenas sought production of testimony, information and documents in violation of:

“(1) the Federal Regulations set forth in the Department of Health and Human Services Policy for Protection of Human Research Subjects,

(2) the Federal regulations set forth in the HIPPA Regulations,

(3) the physician/patient privilege,

(4) the research scholar’s privilege,

(5) the trade secret/confidential research privilege and

(6) the scope of discovery as codified by the Federal Rules of Civil Procedure and the Missouri Rules of Civil Procedure.”

After a long discovery fight, the MDL court largely enforced the subpoenas.10 The welding MDL court ordered Racette to produce

“a ‘limited data set’ which links the specific categories requested by defendants: diagnosis, occupation, and age. This information may be produced as a ‘deidentified’ data set, such that the categories would be linked to each particular patient, without using any individual patient identifiers. This data set should: (1) allow matching of each study participant’s occupational status and age with his or her neurological condition, as diagnosed by the study’s researchers; and (2) to the greatest extent possible (except for necessary de-identification), show original coding and any code-keys.”

After the defense had the opportunity to obtain and analyze the underlying data in the Scruggs-Racette study, the welding plaintiffs retreated from their epidemiologic case. Various defense expert witnesses analyzed the underlying data produced by Racette, and prepared devastating rebuttal reports. These reports were served upon plaintiffs’ counsel, whose expert witnesses never attempted any response. Reliance upon Racette’s study was withdrawn or abandoned. After the underlying data were shared with the parties to MDL 1535, no scientist appeared to defend the results in the published papers.11 The Racette Alabama study faded into the background of the subsequent welding-fume cases and trials.

The motion battle in the welding MDL revealed interesting contradictions, similar to those seen in the Zambelli-Weiner affair. For example, Racette claimed he had no relationship whatsoever with plaintiffs’ counsel, other than showing up by happenstance in Alabama at places where Scruggs’ clients also just happened to show up. Racette claimed that the men and women he screened were his patients, but he had no license to practice in Alabama, where the screenings took place. Plaintiffs’ counsel disclaimed that Racette was a treating physician, which acknowledgment would have made the individual’s screening results discoverable in their individual cases. And more interestingly, plaintiffs’ counsel claimed that both Dr. Racette and Washington University were “non-testifying, consulting experts utilized to advise and assist Plaintiffs’ counsel with respect to evaluating and assessing each of their client’s potential lawsuit or claim (or not).”12

Over the last decade or so, best practices and codes of conduct for the relationship between pharmacoepidemiologists and study funders have been published.13 These standards apply with equal force to public agencies, private industry, and regulatory authories. Perhaps it is time for them to specify that the apply to the litigation industry as well.

1 See Smith v. Wyeth-Ayerst Labs. Co., 278 F. Supp. 2d 684, 710 & n. 56 (W.D.N.C. 2003).

2 Ironically, Ms. Cabraser has published her opinion that failure to disclose conflicts of interest and study funding should result in evidentiary exclusions, a view which would have simplified and greatly shortened the silicone gel breast implant litigation. See Elizabeth J. Cabraser, Fabrice Vincent & Alexandra Foote, “Ethics and Admissibility: Failure to Disclose Conflicts of Interest in and/or Funding of Scientific Studies and/or Data May Warrant Evidentiary Exclusions,” Mealey’s Emerging Drugs Reporter (Dec. 2002).

3 Litigation concerning Viagra is one notable example where plaintiffs’ counsel called an expert witness who was the author of the very study that supposedly supported their causal claim. It did not go well for the plaintiffs or the expert witness. See Lori B. Leskin & Bert L. Slonim, “A Primer on Challenging Peer-Reviewed Scientific Literature in Mass Tort and Product Liability Actions,” 25 Toxics L. Rptr. 651 (Jul. 1, 2010).

4 April Zambelli‐Weiner, Christina Via, Matt Yuen, Daniel Weiner, and Russell S. Kirby, “First Trimester Pregnancy Exposure to Ondansetron and Risk of Structural Birth Defects,” 83 Reproductive Toxicology 14 (2019).

5 Nate Raymond, “GSK accuses Zofran plaintiffs’ law firms of funding academic study,” Reuters (Mar. 5, 2019).

6 See Branzburg v. Hayes, 408 U.S. 665, 674 (1972).

7 Affidavit of April Zambelli-Weiner, dated January 9, 2019 (Doc. No. 1272).

8 The plaintiffs’ lawyers’ motive and opportunity to poison the study by coaching their “clients” was palpable. See David B. Resnik & David J. McCann, “Deception by Research Participants,” 373 New Engl. J. Med. 1192 (2015).

9 See Brad A. Racette, S.D. Tabbal, D. Jennings, L. Good, J.S. Perlmutter, and Brad Evanoff, “Prevalence of parkinsonism and relationship to exposure in a large sample of Alabama welders,” 64 Neurology 230 (2005); Brad A. Racette, et al., “A rapid method for mass screening for parkinsonism,” 27 Neurotoxicology 357 (2006) (a largely duplicative report of the Alabama welders study).

10 See, e.g., In re Welding Fume Prods. Liab. Litig., MDL 1535, 2005 WL 5417815 (N.D. Ohio Oct. 18, 2005) (upholding defendants’ subpoena for protocol, data, data codes, statistical analyses, and other things from Dr. Racette’s Alabama study on welding and parkinsonism).

11 Racette sought and obtained a protective order for the data produced, and thus I still cannot share the materials he provided asking that any reviewer sign the court-mandated protective order. Revealingly, Racette was concerned about who had seen his underlying data, and he obtained a requirement in the court’s non-disclosure affidavit that any one who reviews the underlying data will not sit on peer review of his publications or his grant applications. See Motion to Compel List of Defendants’ Reviewers of Data Produced by Brad A. Racette, M.D., and Washington University Pursuant to Protective Order, in In re Welding Fume Products Liab. Litig., MDL No. 1535, Case 1:03-cv-17000-KMO, Document 1642-1 (N.D. Ohio Feb. 14, 2006). Curiously, Racette never moved to compel a list of Plaintiffs’ Reviewers!

12 Plaintiffs’ Motion for Protective Order, Motion to Reconsider Order Requiring Disclovery from Dr. Racette, and Request for In Camera Inspection as to Any Responses or Information Provided by Dr. Racette, filed in Solis v. Lincoln Elec. Co., case No. 1:03-CV-17000, MDL 1535 (N.D. Ohio May 8, 2006).

13 See, e.g., Xavier Kurz, Susana Perez‐Gutthann, and the ENCePP Steering Group, “Strengthening standards, transparency, and collaboration to support medicine evaluation: Ten years of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP),” 27 Pharmacoepidem. & Drug Safety 245 (2018).

Posted in Conflicts of Interest, Data Sharing, Expert Witness Discovery, Scientific Misconduct, Underlying Data | Comments Off on Litigation Science – In re Zambelli-Weiner

Two Stanford Researchers Are Anti-Semantic

August 4th, 2018

Two Stanford University communications researchers have shown that fraudulent publications and authors’ linguistic obfuscation are correlated, p < 0.05. David M. Markowitz & Jeffrey T. Hancock, “Linguistic Obfuscation in Fraudulent Science,” 35 J. Language & Social Psych. 435 (2016); Bjorn Carey, “Stanford researchers uncover patterns in how scientists lie about their data,” Stanford Report (Nov. 16, 2015) [Carey, below]

Stanford Professor of Communication, Jeff Hancock, and graduate student David Markowitz observed that there are repeating patterns of expression in the language used by scientific fraudfeasors. They hypothesized that scientific fraudfeasors would signal their duplicity in their linguistic expressions as well. These authors created a linguistic obfuscation index based upon the prevalence of jargon, abstraction, positive emotion terms, and readability. They then compared the obfuscation index scores of 253 papers retracted for fraudulent data with 253 unretracted papers, and 63 papers retracted for reasons other than fraud. Not surprisingly, Hancock and Markowitz found differences, with fraudulent papers having higher obfuscation scores, and generally more jargon.

As Markowitz explained:

“We believe the underlying idea behind obfuscation is to muddle the truth. *** Scientists faking data know that they are committing a misconduct and do not want to get caught. Therefore, one strategy to evade this may be to obscure parts of the paper. We suggest that language can be one of many variables to differentiate between fraudulent and genuine science.”

Carey. Professor Hancock acknowledged that there remained a high error rate in their obfuscation analysis, which needed to be lowered before automatic linguistic analyses could be useful for detecting fraud. Hancock also acknowledged that such the use of such a computerized linguistic tool might undermine the trust upon which science is based. Id.

Well detecting fraud might undermine trust, but look where trust has gotten us in science.

Trust but verify.

I cannot wait until I proffer the first expert witness rebuttal report in litigation, to show that my adversary’s expert witness has crossed the obfuscation line.

Posted in Scientific Misconduct | Comments Off on Two Stanford Researchers Are Anti-Semantic

Infante-lizing the IARC

May 13th, 2018

Peter Infante, a frequently partisan, paid expert witness for the Lawsuit Industry, recently published a “commentary” in the red journal, the American Journal of Industrial Medicine, about the evils of scientists with economic interests commenting upon the cancer causation pronouncements of the International Agency for Research on Cancer (IARC). Peter F. Infante, Ronald Melnick, James Huff & Harri Vainio, “Commentary: IARC Monographs Program and public health under siege by corporate interests,” 61 Am. J. Indus. Med. 277 (2018). Infante’s rant goes beyond calling out scientists with economic interests on IARC working groups; Infante would silence all criticism of IARC pronouncements by anyone, including scientists, who has an economic interest in the outcome of a scientific debate. Infante points to manufacturing industry’s efforts to “discredit” recent IARC pronouncements on glyphosate and red meat, by which he means that there were scientists who had the temerity to question IARC’s processes and conclusions.

Apparently, Infante did not think his bias was showing or would be detected. He and his co-authors invoke militaristic metaphors to claim that the IARC’s monograph program, and indeed all of public health, is “under siege by corporate interests.” A farcical commentary at that, coming from such stalwarts of the Lawsuit Industry. Infante lists his contact information as “Peter F. Infante Consulting, LLC, Falls Church, Virginia,” and co-author Ronald Melnick can be found at “Ronald Melnick Consulting, LLC, North Logan, Utah.” A search on Peter Infante in Westlaw yields 141 hits, all on the plaintiffs’ side of health effects disputes; he is clearly no stranger to the world of litigation. Melnick is, to be sure, harder to find, but he does show up as a signatory on Raphael Metzger’s supposed amicus briefs, filed by Metzger’s litigation front organization, Council for Education and Research on Toxics.1

Of the commentary’s authors, only James Huff, of “James Huff Consulting, Durham, North Carolina,” disclosed a connection with litigation, as a consultant to plaintiffs on animal toxicology of glyphosate. Huff’s apparent transparency clouded up when it came to disclosing how much he has been compensated for his consulting activities for claimants in glyphosate litigation. In the very next breath, in unison, the authors announce unabashedly that “[a]ll authors report no conflicts of interest.” Infante at 280.

Of course, reporting “no conflicts of interest” does not mean that the authors have no conflicts of interest, financial, positional, and idealogical. Their statement simply means that they have not reported any conflicts, through inadvertence, willfulness, or blindness. The authors, and the journal, are obviously content to mislead their readership by not-so-clever dodges.

The clumsiness of the authors’ inability to appreciate their very own conflicts infects their substantive claims in this commentary. These “consultants” tell us solemnly that IARC “[m]eetings are openly transparent and members are vetted for conflicts of interest.” Infante at 277. Working group members, however, are vetted but only for manufacturing industry conflicts, not for litigation industry conflicts or for motivational conflicts, such as advancing their own research agendas. Not many scientists have a research agenda to show that chemicals do not cause cancer.

At the end of this charade, the journal provides additional disclosures [sic]. As for “Ethics and Approval Consent,” we are met with a bold “Not Applicable.” Indeed; ethics need not apply. Perhaps, the American Journal of Industrial Medicine is beyond good and evil. The journal’s “Editor of Record,” Steven B. Markowitz “declares that he has no conflict of interest in the review and publication decision regarding this article.” This is, of course, the same Markowitz who testifies frequently for the Lawsuit Industry, without typically disclosing this conflict on his own publications.

This commentary is yet another brushback pitch, which tries to chill manufacturing industry and scientists from criticizing the work of agencies, such as IARC, captured by lawsuit industry consultants. No one should be fooled other than Mother Jones.

1See, e.g., Ramos v. Brenntag Specialties, Inc., 372 P.3d 200 (Calif. 2016) (where plaintiff was represented by Metzger, and where CERT filed an amicus brief by the usual suspects, plaintiffs’ expert witnesses, including Melnick).

Posted in Conflicts of Interest, Scientific Misconduct | Comments Off on Infante-lizing the IARC

Statistical Deontology

March 2nd, 2018

In courtrooms across America, there has been a lot of buzzing and palavering about the American Statistical Association’s Statement on Statistical Significance Testing,1 but very little discussion of the Society’s Ethical Guidelines, which were updated and promulgated in the same year, 2016. Statisticians and statistics, like lawyers and the law, receive their fair share of calumny over their professional activities, but the statistician’s principal North American professional organization is trying to do something about members’ transgressions.

The American Statistical Society (ASA) has promulgated ethical guidelines for statisticians, as has the Royal Statistical Society,2 even if these organizations lack the means and procedures to enforce their codes. The ASA’s guidelines3 are rich with implications for statistical analyses put forward in all contexts, including in litigation and regulatory rule making. As such, the guidelines are well worth studying by lawyers.

The ASA Guidelines were prepared by the Committee on Professional Ethics, and approved by the ASA’s Board in April 2016. There are lots of “thou shall” and “thou shall nots,” but I will focus on the issues that are more likely to arise in litigation. What is remarkable about the Guidelines is that if followed, they probably are more likely to eliminate unsound statistical practices in the courtroom than the ASA State on P-values.

Defining Good Statistical Practice

“Good statistical practice is fundamentally based on transparent assumptions, reproducible results, and valid interpretations.” Guidelines at 1. The Guidelines thus incorporate something akin to the Kumho Tire standard that an expert witness ‘‘employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.’’ Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152 (1999).

A statistician engaged in expert witness testimony should provide “only expert testimony, written work, and oral presentations that he/she would be willing to have peer reviewed.” Guidelines at 2. “The ethical statistician uses methodology and data that are relevant and appropriate, without favoritism or prejudice, and in a manner intended to produce valid, interpretable, and reproducible results.” Id. Similarly, the statistician, if ethical, will identify and mitigate biases, and use analyses “appropriate and valid for the specific question to be addressed, so that results extend beyond the sample to a population relevant to the objectives with minimal error under reasonable assumptions.” Id. If the Guidelines were followed, a lot of spurious analyses would drop off the litigation landscape, regardless whether they used p-values or confidence intervals, or a Bayesian approach.

Integrity of Data and Methods

The ASA’s Guidelines also have a good deal to say about data integrity and statistical methods. In particular, the Guidelines call for candor about limitations in the statistical methods or the integrity of the underlying data:

“The ethical statistician is candid about any known or suspected limitations, defects, or biases in the data that may impact the integrity or reliability of the statistical analysis. Objective and valid interpretation of the results requires that the underlying analysis recognizes and acknowledges the degree of reliability and integrity of the data.”

Guidelines at 3.

The statistical analyst openly acknowledges the limits of statistical inference, the potential sources of error, as well as the statistical and substantive assumptions made in the execution and interpretation of any analysis,” including data editing and imputation. Id. The Guidelines urge analysts to address potential confounding not assessed by the study design. Id. at 3, 10. How often do we see these acknowledgments in litigation-driven analyses, or in peer-reviewed papers, for that matter?

Affirmative Actions Prescribed

In the aid of promoting data and methodological integrity, the Guidelines also urge analysts to share data when appropriate without revealing the identities of study participants. Statistical analysts should publicly correct any disseminated data and analyses in their own work, as well as working to “expose incompetent or corrupt statistical practice.” Of course, the Lawsuit Industry will call this ethical duty “attacking the messenger,” but maybe that’s a rhetorical strategy based upon an assessment of risks versus benefits to the Lawsuit Industry.

Multiplicity

The ASA Guidelines address the impropriety of substantive statistical errors, such as:

“[r]unning multiple tests on the same data set at the same stage of an analysis increases the chance of obtaining at least one invalid result. Selecting the one “significant” result from a multiplicity of parallel tests poses a grave risk of an incorrect conclusion. Failure to disclose the full extent of tests and their results in such a case would be highly misleading.”

Guidelines at 9.

There are some Lawsuit Industrialists who have taken comfort in the pronouncements of Kenneth Rothman on corrections for multiple comparisons. Rothman’s views on multiple comparisons are, however, much broader and more nuanced than the Industry’s sound bites.4 Given that Rothman opposes anything like strict statistical significance testing, it follows that he is relatively unmoved for the need for adjustments to alpha or the coefficient of confidence. Rothman, however, has never deprecated the need to consider the multiplicity of testing, and the need for researchers to be forthright in disclosing the the scope of comparisons originally planned and actually done.

2 Royal Statistical Society – Code of Conduct (2014); Steven Piantadosi, Clinical Trials: A Methodologic Perspective 609 (2d ed. 2005).

3 Shelley Hurwitz & John S. Gardenier, “Ethical Guidelines for Statistical Practice: The First 60 Years and Beyond,” 66 Am. Statistician 99 (2012) (describing the history and evolution of the Guidelines).

4 Kenneth J. Rothman, “Six Persistent Research Misconceptions,” 29 J. Gen. Intern. Med. 1060, 1063 (2014).

Posted in Data Sharing, Expert Witness Discovery, Scientific Misconduct, statistical evidence, Underlying Data | Comments Off on Statistical Deontology

Wrong Words Beget Causal Confusion

February 12th, 2018

In clinical medical and epidemiologic journals, most articles that report about associations will conclude with a discussion section in which the authors hold forth about

(1) how they have found that exposure to X “increases the risk” of Y, and

(2) how their finding makes sense because of some plausible (even if unproven) mechanism.

In an opinion piece in Significance,1 Dalmeet Singh Chawla cites to a study that suggests the “because” language frequently confuses readers into believing that a causal claim is being made. The study abstract explains:

“Most researchers do not deliberately claim causal results in an observational study. But do we lead our readers to draw a causal conclusion unintentionally by explaining why significant correlations and relationships may exist? Here we perform a randomized study in a data analysis massive online open course to test the hypothesis that explaining an analysis will lead readers to interpret an inferential analysis as causal. We show that adding an explanation to the description of an inferential analysis leads to a 15.2% increase in readers interpreting the analysis as causal (95% CI 12.8% – 17.5%). We then replicate this finding in a second large scale massive online open course. Nearly every scientific study, regardless of the study design, includes explanation for observed effects. Our results suggest that these explanations may be misleading to the audience of these data analyses.”

Leslie Myint, Jeffrey T. Leek, and Leah R. Jager, “Explanation implies causation?” (Nov. 2017) (on line manuscript).

Invoking the principle of charity, these authors suggest that most researchers are not deliberately claiming causal results. Indeed, the language of biomedical science itself is biased in favor of causal interpretation. The term “statistical significance” suggests causality to naive readers, as does stats talk about “effect size,” and “fixed effect models,” for data sets that come no where near establishing causality.

Common epidemiologic publication practice tolerates if not encourages authors to state that their study shows (finds, demonstrates, etc.) that exposure to X “increases the risk” of Y in the studies’ samples. This language is deliberately causal, even if the study cannot support a causal conclusion alone or even with other studies. After all, a risk is the antecedent of a cause, and in the stochastic model of causation involved in much of biomedical research, causation will manifest in a change of a base rate to a higher or lower post-exposure rate. Given that mechanism is often unknown and not required, then showing an increased risk is the whole point. Eliminating chance, bias, confounding, and study design often is lost in the irrational exuberance of declaring the “increased risk.”

Tighter editorial control might have researchers qualify their findings by explaining that they found a higher rate in association with exposure, under the circumstances of the study, followed by an explanation that much more is needed to establish causation. But where is the fun and profit in that?

Journalists, lawyers, and advocacy scientists often use the word “link,” to avoid having to endorse associations that they know, or should know, have not been shown to be causal.2 Using “link” as a noun or a verb clearly implies a causal chain metaphor, which probably is often deliberately implied. Perhaps publishers would defend the use of “link” by noting that it is so much shorter than “association,” and thus saves typesetting costs.

More attention is needed to word choice, even and especially when statisticians and scientists are using their technical terms and jargon.3 If, for the sake of argument, we accept the sincerity of scientists who work as expert witnesses in litigation in which causal claims are overstated, we can see that poor word choices confuse scientists as well as lay people. Or you can just read the materials and methods and the results of published study papers; skip the introduction and discussion sections, as well as the newspaper headlines.

1 Dalmeet Singh Chawla, “Mind your language,” Significance 6 (Feb. 2018).

2 See, e.g., Perri Klass, M.D., “https://www.nytimes.com/2017/12/04/well/family/does-an-adhd-link-mean-tylenol-is-unsafe-in-pregnancy.html,” N.Y. Times (Dec. 4, 2017); Nicholas Bakalar, “Body Chemistry: Lower Testosterone Linked to Higher Death Risk,” N.Y. Times (Aug. 15, 2006).

3 Fang Xuelan & Graeme Kennedy, “Expressing Causation in Written English,” 23 RELC J. 62 (1992); Bengt Altenberg, “Causal Linking in Spoken and Written English,” 38 Studia Linguistica 20 (1984).

Posted in Causation, Scientific Misconduct | Comments Off on Wrong Words Beget Causal Confusion

« Older Entries
Newer Entries »