The Solicitor General’s office has obtained yet another extension in which to file its opposition to Dr. Harkonen’s petition for a writ of certiorari. The new due date is November 8, 2013.
This week, Nature published a news article on the Harkonen case. See Ewen Callaway, “Uncertainty on trial,” 502 Nature 17 (2013). Mr. Calloway’s story accurately recounts that Thomas Fleming, a biostatistician at the University of Washington, chaired the data safety monitoring board for the InterMune trial, and that he had told Dr. Harkonen and others at InterMune that he, Fleming, believed that the press release was misleading. But this “fact” simply represents that Fleming disagreed with the causal inference of efficacy. His opinion might well have been correct, but it did not make Dr. Harkonen’s press release “demonstrably” false. Overstating confidence in a conclusion may be the occasion for disputing the evidentiary warrant for the conclusion, but it does not make the speaker a liar.
Calloway also reports that the government believed that documents suggested that there had been off-label promotion of interferon γ-1b, but of course, the jury acquitted Dr. Harkonen of mislabeling. Calloway’s recitation of these discredited allegations, however, provide important context for why the federal government continues to overreach by pressing its opposition to Dr. Harkonen’s appeal on the conviction for criminal wire fraud.
Mr. Calloway notes that there were “statisticians, clinical researchers and legal scholars” who criticized the judgment of conviction on grounds that it rested upon misinterpretations and misunderstandings of statistics, and that it could criminalize much expert witness testimony, grant applications, and article submissions. But Mr. Calloway’s presentation is subtly biased. He fails to identify those “statisticians, clinical researchers and legal scholars,” other than a few whom he then impugns as having been “compensated” by the defense.
He quotes Stanford Professor Steven Goodman as filing a brief stating that:
“You don’t want to have on the books a conviction for a practice that many scientists do, and in fact think is critical to medical research… .”
Calloway errs in suggesting that Professor Goodman was a brief writer rather than an affiant. Dr. Zibrak, a pulmonary physician is quoted, with the note that he was compensated to tell other physicians about his clinical experience with interferon γ-1b in patients with idiopathic pulmonary fibrosis. By playing the “compensation card,” Calloway tries to diminish the force of Goodman’s and Zibrak’s substantive arguments. This sly attempt, however, is blunted by the significant number of legal scholars and scientists who filed amicus briefs without compensation. More important, the attempt is irrelevant to the issues in the case.
MISLEADING REPORTING
Calloway described the trial as showing that only “slightly fewer” patients had died on interferon γ-1b than on placebo, but that the difference was not statistically significant “because the probability that it was not due to the drug was greater than 5%, a widely accepted statistical threshold.” Well, the p-value was 0.08 on the intent-to-treat analysis, and 0.055 on the per-protocol analysis. When the investigators published a more sophisticated time-to-event analysis in the New England Journal of Medicine, their reported “hazard ratio for death in the interferon gamma-1b group, as compared with the placebo group, was 0.3 (95 percent confidence interval, 0.1 to 0.9).” Raghu et al. N. Engl. J. Med. 350, 125–133; 2004 (for the entire trial, not the “controversial” subgroup). Calloway notes the publication of the results, but fails to inform the Nature readers of this hazard ratio or the confidence interval. Some might say that Mr. Calloway misled readers by inaptly describing this hazard ratio as “slightly fewer” deaths on interferon γ-1b than placebo, and by failing to provide all the pertinent information.
Oh; wait. Is failure to present all the facts, fraud???
TRANSPOSITION FALLACY
Perhaps more ironic was that Mr. Calloway’s interpretation of statistical significance is wrong. The p-value is not the “probability that it was not due to the drug” or the probability that the null hypothesis is true. Good thing that Mr. Calloway does not live in the United States where statistical errors of this sort can be a criminal offense.
The Nature news story quoted Gordon Guyatt, from McMaster University, who thinks that Dr. Harkonen skewed the findings:
“This guy gave a very unbalanced presentation; whether it is sufficiently unbalanced that you should send him to jail, I don’t know… .”
But the data were all accurately presented; it was the use of the verb “demonstrate,” which triggered the prosecution. And it was hardly a “presentation”; it was a press release, which clearly communicated that a presentation was forthcoming within a couple of weeks at a scientific conference.
The story also cites Patricia Zettler, a former FDA attorney, who now teaches at Stanford Law School, for her doubts that the case will matter to most scientists. See also Zettle, “U.S. v. Harkonen: Should Scientists Worry About Being Prosecuted for How They Interpret Their Research Results?” (Oct. 7, 2013). If her prediction is correct, then this is a sad commentary on the scientific community. Ms. Zettler suggests that the Supreme Court is likely to deny the petition, and leave Dr. Harkonen’s conviction in place, and that this denial will not seriously affect scientific discourse. If this suggestion is true, then courts will have acquiesced in a very selective prosecution, given the widespread prevalence of the statistical reporting practices that were on trial here.
As much as everyone would like to see editors, scientists, governments, companies, and universities held to higher standards in science reporting, criminalizing the commonplace because the speaker is an unpopular scientist who has a commercial, as well as a scientific, interest is profoundly disturbing. Ultimately, all scientists, from private or public sectors, from academic or non-academic institutions, have financial or reputational interests to be advanced in their communication of scientific results.
The irony is that many federal judges would not exclude an expert witness who would testify under oath to a conclusion based upon much weaker evidence than Dr. Harkonen presented in a press release, and which announced a much fuller discussion at an upcoming scientific conference in a couple of weeks. If Ms. Zettler is correct, it will be much more difficult for federal and state trial judges to reject challenges to expert witness testimony based upon statistically “non-significant” results, with the old “goes to the weight, not the admissibility” excuse.
