Epistemic Crimes
In U.S. v. Harkonen, the government prosecuted a physician, company CEO, for issuing a press release that stated a clinical trial “demonstrated” benefit when the government believed that the clinical trial was inconclusive. No doubt the government was intent upon punishing what it thought was off-label promotion in the same press release, but the jury acquitted on the charge of misbranding, and convicted on the wire fraud count. The trial court denied post-trial motions, and recently, the United States Court of Appeals, for the Ninth Circuit, affirmed, in an unpublished per curiam opinion. United States v. Harkonen, No. 11-10209, No. 11-10242, 2013 WL 782354, 2013 U.S. App. LEXIS 4472 (9th Cir. March 4, 2013).
A Gedanken Experiment
An expert witness writes a report that X, a drug therapy, causes Y, a benefit in survival, for a disease, Z.
The expert witness sent his report by email, and regular mail, to counsel, who then served it upon his adversary. The report set out some of the support for the opinion, as follows.
The expert witness relied upon a randomized clinical trial, conducted with one primary and nine secondary endpoints. The multiple endpoints were chosen because of uncertainty of how the anticipated benefit would manifest. Mortality (survival), although obviously a very important endpoint, was not made primary endpoint because the scientists who conducted the trial did not anticipate sufficient deaths over the course of the trials to see a statistically significant benefit.
This clinical trial had surprising results. Although the trial did not show a difference on the primary endpoint, a composite defined in terms of various pulmonary functional changes or death, the trial did “demonstrate,” according to the witness, a survival benefit. Indeed, the survival benefit was clinically significant. Patients randomized to therapy experienced a 40% decrease in mortality, compared to those randomized to placebo. (p = 0.084). The expert witness pointed out, in his report, that the survival benefit was even stronger in a subgroup of the clinical trial, which consisted of the patients who had mild- to moderate-disease at the time of randomization. For this subgroup, the decrease in mortality was even more dramatic, 70%, p = 0.004. The witness’s report did not clearly label this subgroup as “post-hoc,” although a discerning reader might well have assessed it as such.
The expert witness was not relying upon only one clinical trial. His report identified an earlier trial, published in a leading clinical medical journal, which reported benefit from the drug, p < 0.001. This trial was extended, with continuing strong evidence of differential survival. In terms of survival at five years, the earlier trial showed survival in the therapy group at 77.8%, compared to 16.7% in the control groups, p = 0.009.
The expert witness’s report did not explicitly reference clinical experience, or the in vitro and in vivo mechanistic evidence that the therapy, X, plays a role in inhibiting processes that are clearly involved in producing the disease, Z. The expert witness could have written a stronger expert witness report with these references, but did not expect that this level of completeness was required. The expert witness did note that he would marshal the data in more detail at a later time. The expert witness further relied upon the assessment of the principal investigator of the later clinical, who had written that the benefit against mortality of X was “compelling,” and that the finding was “a major breakthrough.” The principal investigator of the trial noted that X was “the first treatment ever to show any meaningful clinical impact in this disease in rigorous clinical trials, and these results would indicate that [X] should be used early in the course of this disease in order to realize the most favorable long-term survival benefit.” The report went on to note, accurately, that there are no FDA-approved therapies for Z.
Adversary counsel, receiving this report, moved pursuant to Federal Rules of Evidence 702 and 703, to exclude the expert witness’s report and his opinions. The motion to exclude was made in advance of the deposition, and without a preliminary motion for more detail about the supporting data. In particular, the motion to exclude claimed that the expert witness was unjustified in concluding that a benefit had been “demonstrated,” as opposed to being merely suggested.
What would be the challenger’s chances of success on the Rule 702 motion? The outcome, Y, was not “statistically significant” at the conventional two-tail 5% (but would have been on a one-tail test). The subgroup that sported a p-value of 0.004 was not clearly marked as a post-hoc subgroup, although the challenger could discern that it was likely exploratory, and challenged it as uncorrected for multiple testing. The challenger, however, did not attempt to offer a modified p-value that took into account of multiple testing. The essence of this challenge was that the expert witness’s statement that a benefit had been “demonstrated” was not supported by sufficient evidence, and that the low p-value of 0.004 was not truly “significant” because the result emerged from an analysis that was not pre-planned.
My hunch, based upon published judicial opinions on both state and federal Rule 702 motions, is that many judges would allow the challenged expert witness to testify. There would be the usual judicial hand waving about the challenge’s going to the weight not the admissibility of the expert witness’s opinion. Perhaps an occasional judge might order additional discovery. I believe that most judges would not find that this expert witness had engaged in pathologically bad science such that the party proponent should be denied its expert witness.
Transmuting Disputed Causal Inferences Into Criminal Fraud
Instead of moving to exclude the expert witness’s opinion, why not turn the report over to the U.S. Attorney’s office to prosecute for wire or mail fraud? Even if a trial court were to brand the opinion “inadmissible,” that outcome would hardly suggest that the opinion was the kind of speech that could qualify as fraudulent under federal wire or mail fraud statutes. Branding a scientist as a fraudfeasor, however, was exactly the result reached in U.S. v. Harkonen, where the Ninth Circuit upheld a wire fraud conviction of a physician whose written statements would likely have been admissible in most federal courtrooms, under Federal Rule 702. As much as I would like to see more stringent gatekeeping of expert witness opinions, there is something unseemly about the government’s efforts here to criminalize scientific opinions with which it disagrees.
Dr. Harkonen has petitioned the Ninth Circuit for reconsideration, in a brief filed by attorneys, Mark Haddad and colleagues, of Sidley Austin. Petition for Rehearing En Banc (filed 29, 2013). The case raises important First Amendment and due process issues, which were addressed by the party and amici briefs before the Panel.
The case also raises the specter of prosecutions of scientists for speech in various contexts, including grant applications and reports, under the False Claims Act, for witness perjury for testimony in judicial, administrative, or legislative proceedings, or for wire or mail fraud for manuscript submissions to journals. On April 8th, Professor Robert Makuch, of Yale University, Professor Timothy Lash, of Emory University, and I filed an amicus brief, which addresses the government’s controversial branding statements “false as a matter of statistics.” The government has gone from one extreme of painting, broad brush, that statistical significance is not important or necessary (in Matrixx Initiatives Inc. v. Siracusano), to the other extreme that statistical significance is so important that a scientist who his opinion on causality on evidence the government believes is not statistically significant has committed fraud (in Harkonen). Both extreme positions are untenable.
