The history of statistics, epidemiology, and products liability are intertwined in ways that call for greater attention.  The 1950s and 1960s witnessed increasingly sophisticated statistical approaches to epidemiologic evidence. Starting in 1950, and continuing throughout the 1950s, Sir Richard Doll and Sir Austin Bradford Hill began their epidemiologic exploration of lung cancer among smokers. See, e.g., Richard Doll & A. Bradford Hill, “Smoking and Carcinoma of the Lung,” 2 Br. Med. J. 739 (1950); Richard Doll & A. Bradford Hill, “The mortality of doctors in relation to their smoking habits; a preliminary report,” 1 Br. Med. J. 1451 (1954).  In 1955, Sir Richard Doll published his important paper that suggested an association between asbestosis and lung cancer.  Richard Doll, “Mortality from Lung Cancer in Asbestos Workers,”  12 Br. J. Indus. Med. 81 (1955).  No disparity between observed and expected rates of lung cancer was observed among workers without asbestosis. Measures of p-values were used to assess the strength of the evidence against a null hypothesis of no association. As important an advance as was Doll’s paper, and as careful an investigator as he was, it is remarkable that Doll neglected to consider the potential role of smoking in producing the excess lung cancer rates among the factory workers with asbestosis. 

Starting in the 1960s, Dr. Irving Selikoff began publishing his epidemiologic studies of American asbestos insulators. See, e.g., Irving J. Selikoff , Jacob Churg,  and E. Cuyler Hammond, “Asbestos exposure and neoplasia,” 188 J. Am. Med. Ass’n 22 (1964). Selikoff neglected to stratify his observational data by the presence or absence of clinical asbestosis (although his later studies suggested that there was a very high prevalence of asbestosis after 20 years from first employment).  In addition, these insulator studies used crude measures of smoking, which lumped the very rare non-smoking insulators in with those who “never smoked regularly.”

In 1965, Sir Austin Bradford Hill published his lecture to the Royal Society of Medicine, in which he gave a spirited defense of inferring causality from observational epidemiologic studies. Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295 (1965). Hill was justly proud of the success of observational epidemiology, at least for very large effect sizes that made residual confounding and bias unlikely.

The same years as Hill’s lecture, the American Law Institute published the Restatement (Second) of Torts, with its controversial Section 402A. Before 1965, employee-plaintiff lawsuits against remote suppliers of raw materials and products to employers were a rarity in American law.  Bradford Hill’s lecture on causal assessments of “clear-cut” statistical associations came just as epidemiologic, statistical evidence was working its way into tort cases involving smoking and lung cancer.  Not surprisingly, some of the first uses of epidemiologic evidence occurred in cases involving claims that tobacco caused lung cancer. See, e.g., Lartigue v. R.J. Reynolds Tobacco Co., 317 F.2d 19 (1963) (affirming defense verdict in case noted for plaintiffs’ use of epidemiologic evidence) (“The plaintiff contends that the jury’s verdict was contrary to the manifest weight of the evidence. … The jury had the benefit of chemical studies, epidemiological studies, reports of animal experiments, pathological evidence, reports of clinical observations, and the testimony of renowned doctors. The plaintiff made a convincing case, in general, for the causal connection between tobacco and cancer and, in particular, for the causal connection between Lartigue’s smoking and his cancer. The defendants made a convincing case for the lack of any causal connection.”), cert. denied, 375 U.S. 865 (1963), and cert. denied, 379 U.S. 869 (1964).

Epidemiologic and statistical evidence in tort cases has become a commonplace, even when it is distorted and abused by litigants and judges. Recent decisions involving claims that benzene caused various cancers are illustrative.  See, e.g., Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009) (granting motion to exclude opinions that substantially distorted epidemiologic evidence under the vague rubric of “weight of the evidence”), rev’d, 639 F.3d 11 (1st Cir. 2011) (closing off scrutiny of expert witness’s abuse of epidemiologic evidence in one of the most controversial, reactionary decisions involving federal gatekeeping decisions of recent years), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). See also David E. Bernstein, “The Misbegotten Judicial Resistance to the Daubert Revolution,” 89 Notre Dame L. Rev. 27 (2013); “WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name” (Sept. 2, 2011); “Milward — Unhinging the Courthouse Door to Dubious Scientific Evidence” (Sept. 2, 2011)

Given the problematic Milward decision, we might wonder what challenges to benzene-leukemia cases looked like just before Hill’s defense of inferring causality from observational studies began to infuse the witches’ brew of Rule 402A. What did statistical evidence look like before Hill’s paper?  In court cases, typically, statistical evidence was presented crudely or not at all.

In 1960, there was little opportunity to challenge causation opinions on admissibility grounds; rather sufficiency of the evidence to support a verdict or judgment was the primary means to gain review of an adverse decision.  Case reports of leukemia in workers very heavily exposed to benzene appeared in the 1920s, but it was not until the 1960s that analytical epidemiologic evidence (case-control and cohort studies) of association between leukemia and benzene were published. See generally Deborah Glass, Christopher Gray, Damien Jolley, Carl Gibbons, and Malcolm Sim, “The health watch case – control study of leukemia and benzene: the story so far,” 1076 Ann. N.Y. Acad. Sci. 80-89 (2006).  Thus, when the New York Court of Appeals decided a case involving a claim of benzene-induced leukemia, in 1960, the judicial decision was driven largely by the absence of specific quantification of risk of leukemia among workers occupationally exposed to benzene.[1]  Miller v. National Cabinet Co., 8 N.Y.2d 277, 281, 168 N.E.2d 811, 813, 204 N.Y.S.2d 129, 132, modified on other grounds, 8 N.Y.2d 1025, 70 N.E.2d 214, 206 N.Y.S.2d 795 (1960). The New York high court wrestled with the formalistic aspects of the expert witnesses’ testimony, including whether they expressed themselves in terms of “possibilities” or “probabilities.” Miller, 8 N.Y.2d at 284, 168 N.E.2d  at 814-15, 204 N.Y.S.2d at 134.

Focusing in one of the more knowledgeable of plaintiffs’ expert witnesses, Dr. Reznikoff, the Miller court was impressed that this witness disclaimed any intent to support an inference, from statistical analyses, to the plaintiff’s decedent.  Id. at 283. Furthermore, the court suggested that Reznikoff’s evidence might have been sufficient were it not for concession:

“I am sorry I can’t give you any statistics, but we don’t have them.”

Id. at 283.  The court appeared also to be concerned that Reznikoff’s approach provided no mechanistic insight or understanding into why many cases of  leukemia followed benzene exposure. Id.  

Reznikoff’s qualifications to speak to the subject were not dispositive of the question; the court was looking for data that were not available in the 1950s, when the case was tried:

“Not every supposition of a witness concerning what might be has the force of evidence, even though he has been licensed to practice medicine. If the witness is unfamiliar with any statistical data in the medical literature or in his own practice to give an inkling either to himself or to the court or board of how high the incidence of these cases is in situations of this kind, then the doctor’s assumption that it is ‘quite high’ is without significance. The lack of any kind of statistical data, which in the absence of scientific understanding is all that there would be to go on, is the more inexplicable if the claim is well founded in view of the large number of persons who die of leukemia and of workers in industry who are exposed to benzol. If there were any observed correlation between the two, it is certain that a physician of Dr. Reznikoff’s standing would be in possession of the information.”

Id. at 283-84. The court did not excuse the claimant’s evidentiary display with the indulgent, “this was the best evidence available,” when the evidence was inadequate.  Nor did the court engage in soothsaying that causality would someday be demonstrated.  We might feel some frustration today, looking back, that the court missed this opportunity, but case reports, and even case reports and epidemiologic studies, have generated many false-positive associations.  Clearly, more is required, and the New York Court of Appeals recognized the necessity for more.

Traumatic Cancers Distinguished

In 1960, the courts still indulged the proto-scientific opinion that traumatic injury caused cancer.[2]  Some medical writers supported this opinion, but by 1960, the opinion was already falling out of favor due to an improved understanding of carcinogenesis.

There is much irony, therefore, in the Miller court drawing’s an invidious distinction between Reznikoff’s proto-epidemiologic evidence and traumatic cancer cases that were still prevalent in the 1960s.  Id. at 285-86. In the traumatic cancer cases, in the 1960s, and even in the 1970s, courts sustained verdicts for cancer claimants who had shown that their cancers were diagnosed very shortly after a traumatic blow to the precise portion of the body where cancer manifested. The Miller court referred to these traumatic cancer cases as presenting the kind of causal inferences that could be understood and made by judges and juries.  Today, 40 years later, we see those causal inferences as mostly rubbish, based upon incorrect, inadequate, and discarded theories of carcinogenesis.

The prospect of cancer cases sustained by epidemiologic (statistical) evidence clearly troubled the New York court:

“The courts have been confronted before with cancer cases, and this is not likely to be the last. This is not an isolated situation. Questions of causation are common to actions based on warranty, tort or workmen’s compensation proceedings. Would, for example, evidence that there are 4 to 11 times as many cases of lung cancer among cigarette smokers as among nonsmokers be sufficient to establish a cause of action for breach of warranty in the sale of cigarettes? … There appear to be no decisions upholding causation in so complex a variety of the disease as leukemia. The cancer decisions in the courts where recovery has been allowed have dealt almost entirely with trauma, and there only in instances where the trauma occurred in the spot in the body where the pre-existing cancer was and the symptoms of its aggravation were immediately apparent … . In all of those cases the immediacy of the symptoms of aggravation of the cancer by a traumatic injury suffered in the area where the cancer was located was accepted as a substitute for scientific evidence or understanding of cause and effect. Absent that, damage claims of this nature have been dismissed on the law for lack of evidence of causation.”

Id. at 285-86 (internal citations omitted). The Miller court went on to note that New York law required that the cancer must develop at the exact location of the injury.  Furthermore, latency between the traumatic blow and the clinical recognition of the cancer was fatal to the claim, even in the face of opinion testimony that a plaintiff’s cancer was a “very slow growing” tumor.  Today, we understand that latency between first-exposure and clinical manifestation is necessitated by the length of induction periods and the doubling time of solid tumor cancers.  As a result of the Miller court’s reliance upon some dodgy notions of cancer causation, it held that Mr. Miller’s latency period disqualified the case from the immediate impact rule of traumatic cancer cases. Id. at 287-88 (distinguishing Hagy v. Allied Chemical & Dye Corp., 122 Cal. App. 2d 361, 265 P.2d 86 (1953), which involved a diagnosis of laryngeal cancer following immediately upon exposure to sulfuric acid mists).

The majority in Miller further expressed its concern that the understanding of cancer causation was marked by such uncertainty that the mere possibilities of chemical carcinogenesis should not be tolerated in this and similar cases:

“… [F]or so long as the causes of a disease — like cancer — are unknown to science, everyone contracting the disease could secure medical testimony that it is ‘possible’ that the disease is contracted from a wide variety of causes, choosing in each instance the particular possibility having the greatest promise of holding liable some responsible defendant. Any cancer expert could readily state that cancer could be caused by virus infection or by exposure to automobile exhaust fumes, sunlight, radiation, smog, smoking, hormone imbalance or according to any other theory which has been entertained by researchers or specialists as a possibility. Is a malpractice suit pending against some doctor who has given cortisone or ACTH as medicine? Then appears a medical witness who testifies that possibly cancer is caused by hormone imbalance induced thereby. Is it an action for breach of an implied warranty in the sale of cigarettes? Then the medical witness will testify that cigarettes could be a cause of lung cancer. Is it X ray or working in a garage where there have been exhaust fumes? Then the ‘possibility’ doctrine is adapted to creating questions of fact in those fields — and the same with benzol exposure and leukemia. Such a doctrine would overturn the rule that the burden is on the party asserting that a disease is based on actionable facts to prove causation. It would mean that, wherever such a cause is possible, the burden rests on the opposite party to prove that the disease resulted from something else. Consequently, for so long as the causes of the disease are unknown to medical science, the claimant or plaintiff can always recover — if the trier of the fact is favorably disposed — since no one can prove that the disease had other causes. This is a perversion of the normal rule that the disease must have resulted from the occupation and that the burden of proving causation is upon the party asserting it. The law does not intend that the less that is known about a disease the greater shall be the opportunity of recovery in court.”

Id. at 289.

The Miller decision provoked a dissent, mostly on formalistic grounds.  Id. at 290. The dissenting judges asserted, without much analysis, that there was substantial evidence to support causality. Given that qualified expert witnesses showed up for the claimant seemed sufficient on this score for the dissenters.  To the extent that the claimant’s expert witnesses expressed themselves in terms of possibilities, the dissenters opined that possibilities are sufficient, especially in the context of workman’s compensation cases, in which the burden of proof standards are lower than in common law civil liability cases.

The majority opinion stands as an eloquent expression of concern about the need for quantitative evidence of statistical risk in chemical exposure cancer cases. The court also presciently saw what would become a plague of litigation involving claims of cancer causation.  In 1960, for benzene and leukemia, the evidence was clearly, even by the standards of the day, inadequate, and the claimant’s expert witnesses were appropriately modest about what inferences could be drawn both with respect to general and specific causation.  The 1970s would witness a growing immodesty among available expert witnesses, as well as an explosive growth in the techniques and applications of analytical epidemiology to many problems, including the relationship between benzene and leukemia.

On August 5, 2013, Dr. Scott Harkonen filed his petition for a writ of certiorari with the United States Supreme Court. As noted in some previous posts, Dr. Harkonen was acquitted of misbranding, but convicted of wire fraud, for his role in issuing a press release about the results of a clinical trial of interferon gamma 1b, in patients with idiopathic pulmonary fibrosis.  (See Multiplicity versus Duplicity – The Harkonen Conviction; The Matrixx Motion in U.S. v. Harkonen; The (Clinical) Trial by Franz Kafka).

Dr. Harkonen’s petition presents two questions:

“1. Whether a conclusion about the meaning of scientific data, one on which scientists may reasonably disagree, satisfies the element of a “false or fraudulent” statement under the wire fraud statute, 18 U.S.C. § 1343?

2. Whether applying 18 U.S.C. § 1343 to scientific conclusions drawn from accurate data violates the First Amendment’s proscription against viewpoint discrimination, or renders the statute, as applied , unconstitutionally vague.”

Both questions are important given that the government has conceded that Dr. Harkonen’s press release accurately presented the raw data and calculated p-values.  The crime, if crime it be, lay in Dr. Harkonen’s drawing a causal inference from a subgroup, p = 0.004, which was not prespecified, in a specified secondary endpoint of survival (p = 0.08), when the subgroup was clearly based upon the goals of the trials, and there was other corroborative evidence in the form of two previous trials, clinical practice, and strong mechanistic evidence.

The government argued that NO inferences could be drawn from a trial that “failed” on its primary endpoint.  The government’s embrace of this statistical orthodoxy greatly misrepresented scientific practice to the courts below.  The only “failed” trial is one that is not conducted.

There are many who would go to great lengths to distort the facts of the Harkonen case in order to demonize the pharmaceutical industry, or to arm the Justice Department with a weapon that can shut down scientific speech about pharmaceutical interventions.  The expansion of the Wire Fraud Act, seen in the Harkonen case, to achieve these political goals will not only affect pharmaceutical company scientists, but also government and academic scientists.  The standard for falsity, drawn from an out-dated, tendentious, and overly rigid conception of hypothesis testing will apply equally to non-industry scientists in False Claim Act cases.  Perhaps in future posts, I can provide some good examples, on condition that any qui tam relators share their bounty with me.

Back in May, Aaron Kesselheim presented (by video) a paper, written with Michelle Mello, of the Harvard School of Public Health, on “The Prospect of Continued FDA Regulation of Manufacturer Promotion in an Era of Expanding Commercial Speech.”  Kesselheim went out of his way to misrepresent the facts of the Harkonen case, as part of his brief against off-market promotion.

By way of background, Aaron S. Kesselheim is a physician and a lawyer, and an Assistant Professor of Medicine at Harvard Medical School.  He is also a faculty member in the Division of Pharmacoepidemiology and Pharmacoeconomics in the Department of Medicine at Brigham and Women’s Hospital.   Given his position and his training in two professions, as well as the extraordinary stakes involved in allowing the government to prosecute scientists for drawing allegedly false conclusions about facts that the government concedes are accurate, Dr. Kesselheim should have exercised much greater care in checking his own assertions more closely.

Dr. Kesselheim focused primarily on the Second Circuit’s recent decision in United States v. Caronia, 703 F.3d 149 (2d Cir. 2012) , which reversed a judgment of conviction for off-label promotion, on First Amendment grounds.  About nine minutes into his presentation, Kesselheim turned to alternative strategies for the government to use to squelch off-label promotion.  One of his suggestions was to follow the model of the Harkonen prosecution, and to prosecute off-label promotion as false and misleading speech.

In his discussion of his suggested strategy, Kesselheim suggested that Dr. Harkonen had made misleading “conclusory, unsubstantiated claims for efficacy,“ and “without reference to supporting evidence.”  It is Kesselheim, however, who seriously mislead his listeners and readers by stating that Dr. Harkonen had made “conclusory, unsubstantiated claims for efficacy.”  The Press Release that was the subject of the government’s indictment set out accurately actual count data and calculated p-values.  No data were fabricated or falsified.  Within the limited space and the informal context of a Press Release, Dr. Harkonen had provided a substantial account of the data from InterMune’s clinical trial, as well as citing a previous, independent clinical trial and its extension, clinical experience, and mechanism research on the action of interferon γ-1b.  Unfortunately, it is Kesselheim who is speaking in conclusory sound bites when he ignores the context and content of the actual Press Release at issue.

Kesselheim went on to suggest that Harkonen’s statement was refuted by a “company-sponsored clinical trial showing that the drug was not effective.” This statement is not only false, but shows a flagrant disregard for statistical analysis and the data in the Harkonen case.  Kesselheim implies that a clinical trial that fails to show treatment efficacy thereby shows that the treatment was not effective.  His statement commits the fundamental error of equating a failure to reject the null hypothesis at a specified level of attained significance with acceptance of the null hypothesis.  This reasoning is fallacious and fundamentally flawed.

To be sure, the prespecified secondary survival endpoint in InterMune’s clinical trial did not meet the 0.05 cutoff (it was 0.08), although the per-protocol analysis for this endpoint came up at 0.055, on a preliminary analysis of the data. When the clinical trial was fully analyzed and written up for publication in the New England Journal of Medicine, the treatment-adherent analysis for survival in the entire clinical trial was 0.02, with a statistically significant hazard ratio for survival, favoring the therapy:

“Analysis of the treatment-adherent cohort of patients showed an absolute reduction in the risk of death of 9 percent in the interferon gamma-1b group, as compared with the placebo group, and a relative reduction in the risk of 66 percent (5 percent of 126 patients in the interferon gamma-1b group and 14 percent of 143 patients in the placebo group died, P=0.02). The hazard ratio for death in the interferon gamma-1b group, as compared with the placebo group, was 0.3 (95 percent confidence interval, 0.1 to 0.9).”

Ganesh Raghu, Kevin K. Brown, Williamson Z. Bradford, Karen Starko, Paul W. Noble, David A. Schwartz, and Talmadge E. King, Jr., for the Idiopathic Pulmonary Fibrosis Study Group, “A Placebo-Controlled Trial of Interferon Gamma-1b in Patients with Idiopathic Pulmonary Fibrosis,” 350 New Engl. Med. J. 125, 129-30 (2004).

Dr. Harkonen, in his Press Release, did focus on what seems like an eminently sensible subgroup, within the survival secondary endpoint, of mild- and moderate-cases, which, a priori, were believed to be the patients mostly likely to benefit from the interferon γ-1b therapy.  (What was not known before the trial was at what point in disease progression might patients no longer respond with greater survival, and hence the difficulty in setting the boundary between moderate and severe cases.)  Kesselheim might argue that the interferon γ-1b clinical trial, standing alone, was inconclusive, but he certainly cannot argue truthfully that the trial showed that the biological product to be ineffective.  Clinical trials do not neatly divide the world of possible results into demonstrations of efficacy and demonstrations of inefficacy.  Not only does the evidence come in degrees, but there is a range of “inconclusiveness” in between the two extremes. Given his background, training, and experience, Kesselheim certainly should know this, and he should apologize for his inaccurate statements.

Kesselheim might well have stopped there, but he went on to acknowledge that the company-sponsored clinical trial at issue did find, in post-hoc analyses, a non-significant trend of benefit in a subset of patients.  Talk of misleading speech!  The p-value at issue was 0.004, uncorrected for multiple comparisons, but no one, not Kesslheim, not the government or anyone else, has offered any appropriate adjustment for multiple comparisons that would inflate that 0.004 to over 0.05.  Kesselheim has no warrant for branding the subgroup finding “non-significant,” until he shows that the p = 0.004, when appropriately modified (if it can be), exceeds 0.05.

Kesselheim mangles other, less technical facts.  He claims that the company saw a ten-fold increase in sales of interferon γ-1b for idiopathic pulmonary fibrosis.  No such fact was ever, or could ever, be established in the Harkonen case.  Kesselheim claims that Dr. Harkonen admitted, in emails, that he did not really believe that the trial “demonstrated” benefit; no such emails were ever adduced at trial, and this seems to be part of a fictional narrative that Dr. Kesselheim has manufactured.  Finally, Kesselheim harrumphs that FDA declined to approve drug.  The company never filed a new drug application for the idiopathic pulmonary fibrosis indication; there was no application to reject.  Perhaps more important is that the Press Release was issued before InterMune had made any formal submission of data to the FDA, an event that did not take place until the following year.

Kesselheim sighs that the Harkonen prosecution will be a difficult act to follow because it requires a case-by-case showing of falsity, with the necessity of expert testimony, and heavy cognitive demands on lay jurors. How ironic that Kesselheim, a lawyer and a physician, and a Harvard Medical School faculty member, buckled under the cognitive demands of his topic. Indeed, Kesselheim’s confusion is a strong argument for why the Supreme Court should put a stop to the practice of asking jurors to second guess whether a scientist has incorrectly inferred causation from accurately presented facts.

Let’s hope Dr. Harkonen gets a fair hearing in the Supreme Court.