TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Goodman v Viljoen – Statistical Fallacies from Both Sides

June 8th, 2014

There was a deep irony to the Goodman[1] case.  If a drug company, in 1995, marketed antenatal corticosteroid (ACS) for the prevention of cerebral palsy (CP) in the United States, the government might well have prosecuted the company for misbranding.  The company might also be subject to a False Claims Act case as well. No clinical trial had found ACS efficacious for the prevention of CP at the significance level typically required by the FDA; no meta-analysis had found ACS statistically significantly better than placebo for this purpose.  In the Goodman case, however, failure to order a full course of ACS was malpractice with respect to the claimed causation of CP in the Goodman twins.

The Goodman case also occasioned a well-worn debate over the difference between scientific and legal evidence, inference, and standards of “proof.” The plaintiffs’ case rested upon a Cochrane review of ACS with respect to various outcomes. For CP, the Cochrane meta-analyzed only clinical trial data, and reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”[2]

The defendant, Dr. Viljoen, appeared to argue that the Cochrane meta-analysis must be disregarded because it did not provide a showing of efficacy for ACS in preventing CP, at a significance probability less than 5 percent.  Here is the trial court’s characterization of Dr. Viljoen’s argument:

“[192] The argument that the Cochrane data concerning the effects of ACS on CP must be ignored because it fails to reach statistical significance rests on the flawed premise that legal causation requires the same standard of proof as medical/scientific causation. This is of course not the case; the two standards are in fact quite different. The law is clear that scientific certainty is not required to prove causation to the legal standard of proof on a balance of probabilities (See: Snell v. Farrell, [1990] 2 S.C.R. 311, at para. 34). Accordingly, the defendant’s argument in this regard must fail and for the purposes of this court, I accept the finding of the Cochrane analysis that ACS reduces the instance [sic] of CP by 40%.”

“Disregard” seems extreme for a meta-analysis that showed a 40% reduction in risk of a serious central nervous system disorder, with p = 0.065.  Perhaps Dr. Viljoen might have tempered his challenge some by arguing that the Cochrane analysis was insufficient.  One problem with Dr. Viljoen’s strident argument about statistical significance was that it overshadowed the more difficult, qualitative arguments about threats to validity in the Cochrane finding from loss to follow up in the aggregated trial data. These threats were probably stronger arguments against accepting the Cochrane “trend” as a causal conclusion. Indeed, the validity and the individual studies and the meta-analyses, along with questions about the accuracy of data, were not reflected in Bayesian analysis.

Another problem is that Dr. Viljoen’s strident assertion that p < 0.05 was absolutely necessary fed plaintiffs’ argument that the defendant was attempting to change the burden of proof for plaintiffs from greater than 50% to 95% or greater.  Given the defendant’s position, great care was required to prevent the trial court from committing the transposition fallacy.

Justice Walters rejected the suggestion that a meta-analysis with a p-value of 6.5% should be disregarded, but the court’s discussion skirts the question whether and how the Cochrane data can be sufficient to support a conclusion of ACS efficacy. Aside from citing a legal case, however, Justice Walters provided no basis for suggesting that the scientific standard of proof was different from the legal standard. From the trial court’s opinion, the parties or their expert witnesses appeared to conflate “confidence,” a technical term when used to describe intervals or random error around sample statistics, with “level of certainty” in the obtained result.

Justice Walters is certainly not the first judge to fall prey to the fallacious argument that the scientific burden of proof is 95%.[3]  The 95% is, of course, the coefficient of confidence for the confidence interval that is based upon a p-value of 5%. No other explanation for why 95% is a “scientific” standard of proof was offered in Goodman; nor is it likely that anyone could point to an authoritative source for the claim that scientists actually adjudge facts and theories by this 95 percent probability level.

Justice Walters’ confusion was led by the transposition fallacy, which confuses posterior and significance probabilities.  Here is a sampling from Her Honor’s opinion, first from Dr. Jon Barrett, one of the plaintiffs’ expert witnesses, an obstetrician and fetal maternal medicine specialist at Sunnybrook Hospital, in Toronto, Ontario:

“[85] Dr. Barrett’s opinion was not undermined during his lengthy cross-examination. He acknowledged that the scientific standard demands 95% certainty. He is, however, prepared to accept a lower degree of certainty. To him, 85 % is not merely a chance outcome.

                                                                                        * * *

[87] He acknowledged that scientific evidence in support of the use of corticosteroids has never shown statistical significance with respect to CP. However, he explained it is very close at 93.5%. He cautioned that if you use a black and white outlook and ignore the obvious trends, you will falsely come to the conclusion that there is no effect.”

Dr. Jon (Yoseph) Barrett is a well-respected physician, who specializes in high-risk pregnancies, but his characterization of a black-white outlook on significance testing as leading to a false conclusion of no effect was statistically doubtful.[4]  Dr. Barrett may have to make divinely inspired choices in surgery, but in a courtroom, expert witnesses are permitted to say that they just do not know. Failure to achieve statistical significance, with p < 0.05, does support a conclusion that there is no effect.

Professor Andrew Willan was plaintiffs’ testifying expert witness on statistics.  Here is how Justice Walters summarized Willan’s testimony:

“[125] Dr. Willan described different statistical approaches and in particular, the frequentist or classical approach and the Bayesian approach which differ in their respective definitions of probability. Simply, the classical approach allows you to test the hypothesis that there is no difference between the treatment and a placebo. Assuming that there is no difference, allows one to make statements about the probability that the results are not due to chance alone.

To reach statistical significance, a standard of 95% is required. A new treatment will not be adopted into practice unless there is less than a 5% chance that the results are due to chance alone (rather than due to true treatment effect).

[127] * * * The P value represents the frequentist term of probability. For the CP analysis [from the Cochrane meta-analysis], the P value is 0.065. From a statistical perspective, that means that there is a 6.5% chance that the differences that are being observed between the treatment arm versus the non-treatment arm are due to chance rather than the treatment, or conversely, a 93.5% chance that they are not.”

Justice Walters did not provide transcript references for these statements, but they are clear examples of the transposition fallacy. The court’s summary may have been unfair to Professor Willan, who seems to have taken care to avoid the transposition fallacy in his testimony:

“And I just want to draw your attention to the thing in parenthesis where it says, “P = 0.065.” So, basically that is the probability of observing data this extremely, this much in favor of ACS given, if, if in fact the no [sic, null] hypothesis was true. So, if, if the no hypothesis was true, that is there was no difference, then the probability of observing this data is only 6.5 percent.”

Notes of Testimony of Andrew Willan at 26 (April , 2010). In this quote, Professor Willan might have been more careful to point out that the significance probability of 6.5%  is a cumulative probability by describing the data observed “this extremely” and more. Nevertheless, Willan certainly made clear that the probability measure was based upon assuming the correctness of the null hypothesis. The trial court, alas, erred in stating the relevant statistical concepts.

And then there was the bizarre description by Justice Walters, of the Cochrane data, as embodying a near-uniform distribution represented by the Cochrane data:

“[190] * * * The Cochrane analysis found that ACS reduced the risk of CP (in its entirety) by 40%, 93.5% of the time.”

The trial court did not give the basis for this erroneous description of the Cochrane ACS/CP data.[5] To be sure, if the Cochrane result were true, then 40% reduction might be the expected value for all trials, but it would be a remarkable occurrence for 93.5% of the trials to obtain the same risk ratio as the one observed in the meta-analysis.

The defendant’s expert witness on statistical issues, Prof. Robert Platt, similarly testified that the significance probability reported by the Cochrane was dependent upon an assumption of the null hypothesis of no association:

“What statistical significance tells us, and I mentioned at the beginning that it refers to the probability of a chance finding could occur under the null-hypothesis of no effect. Essentially, it provides evidence in favour of there being an effect.  It doesn’t tell us anything about the magnitude of that effect.”

Notes of Testimony of Robert Platt at 11 (April 19, 2010)

Perhaps part of the confusion resulted from Prof. Willan’s sponsored Bayesian analysis, which led him to opine that the Cochrane data permitted him to state that there was a 91 to 97 percent probability of an effect, which might have appeared to the trial court to be saying the same thing as interpretation of the Cochrane’s p-value of 6.5%.  Indeed, Justice Walters may have had some assistance in this confusion from the defense statistical expert witness, Prof. Platt, who testified:

“From the inference perspective the p-value of 0.065 that we observe in the Cochrane review versus a 91 to 97 percent probability that there is an effect, those amount to the same thing.”

Notes of Testimony of Robert Platt at 50 (April 19, 2010).  Now the complement of the p-value, 93.5%, may have fallen within the range of posterior probabilities asserted by Professor Willan, but these probabilities are decidedly not the same thing.

Perhaps Prof. Platt was referring only to the numerical equivalence, but his language, “the same thing,” certainly could have bred misunderstanding.  The defense apparently attacked the reliability of the Bayesian analysis before trial, only to abandon the challenge by the time of trial.  At trial, defense expert witness Prof. Platt testified that he did not challenge Willan’s Bayesian analysis, or the computation of posterior probabilities.  Platt’s acquiescence in Willan’s Bayesian analysis is unfortunate because the parties never developed testimony exactly as to how Willan arrived at his posterior probabilities, and especially as to what prior probability he employed.

Professor Platt went on to qualify his understanding of Willan’s Bayesian analysis as providing a posterior probability that there is an effect, or in other words, that the “effect size” is greater than 1.0.  At trial, the parties spent a good deal of time showing that the Cochrane risk ratio of 0.6 represented the decreased risk for CP of administering a full course of ACS, and that this statistic could be presented as an increased CP risk ratio of 1.7, for not having administered a full course of ACS.  Platt and Willan appeared to agree that the posterior probability described the cumulative posterior probabilities for increased risks above 1.0.

“[T]he 91% is a probability that the effect is greater than 1.0, not that it is 1.7 relative risk.”

Notes of Testimony of Robert Platt at 51 (April 19, 2010); see also Notes of Testimony of Andrew Willan at 34 (April 9, 2010) (concluding that ACS reduces risk of CP, with a probability of 91 to 97 percent, depending upon whether random effects or fixed effect models are used).[6]

One point on which the parties’ expert witnesses did not agree was whether the failure of the Cochrane’s meta-analysis to achieve statistical significance was due solely to the sparse data aggregated from the randomized trials. Plaintiffs’ witnesses appeared to have testified that had the Cochrane been able to aggregate additional clinical trial data, the “effect size” would have remained constant, and the p-value would have shrunk, ultimately to below the level of 5 percent.  Prof. Platt, testifying for the defense, appropriately criticized this hand-waving excuse:

“Q. and the probability factor, the P value, was 0.065, which the previous witness had suggested is an increase in probability of our reliability on the underlying data.  Is it reasonable to assume that this data that a further increase in the sample size will achieve statistical significance?

A. No, that’s not a reasonable assumption….”

Notes of Testimony of Robert Platt at 29 (April 19, 2010).

Positions on Appeal

Dr. Viljoen continued to assert the need for significance on appeal. As appellant, he challenged the trial court’s finding that the Cochrane review concluded that there was a 40% risk reduction. See Goodman v. Viljoen, 2011 ONSC 821, at ¶192 (CanLII) (“I accept the finding of the Cochrane analysis that ACS reduces the instance of CP by 40%”). Dr. Viljoen correctly pointed out that the Cochrane review never reached such a conclusion. Appellant’s Factum, 2012 CCLTFactum 20936, ¶64.  It was the plaintiffs’ expert witnesses, not the Cochrane reviewers, who reached the conclusion of causality from the Cochrane data.

On appeal, Dr. Viljoen pressed the point that his expert witnesses described statistical significance in the Cochrane analysis would have been “a basic and universally accepted standard” for showing that ACS was efficacious in preventing CP or PVL. Id. at ¶40. The appellant’s brief then commits to the very error that Dr. Barrett complained would follow from a finding that did not have statistical significance; Dr. Viljoen maintained that the “trend” of reduced CP reduced CD rates from ACS administration “is the same as a chance occurrence.” Defendant (Appellant), 2012 CCLTFactum 20936, at ¶40; see also id. at ¶14(e) (arguing that the Cochrane result for ACS/CP “should be treated as pure chance given it was not a statistically significant difference”).

Relying upon the Daubert decision from the United States, as well as Canadian cases, Dr. Viljoen framed one of his appellate issues as whether the trial court had “erred in relying upon scientific evidence that had not satisfied the benchmark of statistical significance”:

“101. Where a scientific effect is not shown to a level of statistical significance, it is not proven. No study has demonstrated a reduction in cerebral palsy with antenatal corticosteroids at a level of statistical significance.

102. The Trial Judge erred in law in accepting that antenatal corticosteroids reduce the risk of cerebral palsy based on Dr. Willan’s unpublished Bayesian probability analysis of the 48 cases of cerebral palsy reviewed by Cochrane—an analysis prepared for the specific purpose of overcoming the statistical limitations faced by the Plaintiffs on causation.”

Defendant (Appellant), 2012 CCLTFactum 20936. The use of the verb “proven” is problematic because it suggests a mathematical demonstration, which is never available for empirical propositions about the world, and especially not for the biological world.  The use of a mathematical standard begs the question whether the Cochrane data were sufficient to establish a scientific conclusion of the efficacy of ACS in preventing CP.

In opposing Dr. Viljoen’s appeal, the plaintiffs capitalized upon his assertion that science requires a very high level of posterior probability for establishing a causal claim, by simply agreeing with it. See Plaintiffs’ (Respondents’) Factum,  2012 CCLTFactum 20937, at ¶31 (“The scientific method requires statistical significance at a 95% level.”).  By accepting the idealized notion that science somehow requires 95% certainty (as opposed to 95% confidence levels as a test for assessing random error), the plaintiffs made the defendant’s legal position untenable.

In order to keep the appellate court thinking that the defendant was imposing an extra-legal, higher burden of proof upon plaintiffs, the plaintiffs went so far as to misrepresent the testimony of their own expert witness, Professor Willan, as having committed the transposition fallacy:

“49. Dr. Willan provided the frequentist explanation of the Cochrane analysis on CP:

a. The risk ratio (RR) is .060 which means that there is a 40% risk reduction in cerebral palsy where there has been administration of antenatal corticosteroids;

b. The upper limit of the confidence interval (CI) barely crosses 1 so it just barely fails to meet the rigid test of statistical significance;

c. The p value represents the frequentist term of probability;

d. In this case the p value is .065;

e. From a statistical perspective that means that there is a 6.5% chance that the difference observed in CP rates is due to chance alone;

f. Conversely there is a 93.5% chance that the result (the 40% reduction in CP) is due to a true treatment effect of ACS.”

2012 CCLTFactum 20937, at ¶49 (citing Evidence of Dr. Willan, Respondents’ Compendium, Tab 4, pgs. 43-52).

Although Justice Doherty dissented from the affirmance of the trial court’s judgment, he succumbed to the parties’ misrepresentations about scientific certainty, and their prevalent commission of the transposition fallacy. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36 (“Scientists will draw a cause and effect relationship only when a result follows at least 95 per cent of the time. The results reported in the Cochrane analysis fell just below that standard.”), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

The statistical errors on both sides redounded to the benefit of the plaintiffs.


[1] Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[2] Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews, at 8, Issue 3. Art. No. CD004454 (2006).

[3] See, e.g., In re Ephedra Prods. Liab. Litig., 393 F.Supp. 2d 181, 191, 193 (S.D.N.Y. 2005) (fallaciously arguing that the use of a critical value of less than 5% of significance probability increased the “more likely than not” burden of proof upon a civil litigant.  Id. at 188, 193.  See also Michael O. Finkelstein, Basic Concepts of Probability and Statistics in the Law 65 (2009) (criticizing the Ephedra decision for confusing posterior probability with significance probability).

[4] I do not have the complete transcript of Dr. Barrett’s testimony, but the following excerpt from April 9, 2010, at page 100, suggests that he helped lead Justice Walters into error: “When you say statistical significance, if you say that something is statistically significance, it means you’re, for the scientific notation, 95 percent sure. That’s the standard we use, 95 percent sure that that result could not have happened by chance. There’s still a 5 percent chance it could. It doesn’t mean for sure, but 95 percent you’re sure that the result you’ve got didn’t happen by chance.”

[5] On appeal, the dissenting judge erroneously accepted Justice Walters’ description of the Cochrane review as having supposedly reported a 40% reduction in CP incidence, 93.5% of the time, from use of ACS. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36, leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[6] The Bayesian analysis did not cure the attributability problem with respect to specific causation.

 

Goodman v Viljoen – Subterfuge to Circumvent Relative Risks Less Than 2

June 6th, 2014

Back in March, I wrote about a “Black Swan” case, in which litigants advanced a Bayesian analysis to support their claims. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

Goodman was a complex medical practice case in which Mrs. Goodman alleged that her obstetrician, Dr. Johan Viljoen, deviated from the standard of care by failing to prescribe antenatal corticosteroids (ACS) sufficiently in advance of delivery to reduce the risks attendant early delivery for her twin boys, of early delivery. Both boys developed cerebral palsy (CP). The parties and their experts agreed that the administration of ACS reduced the risks of respiratory distress and other complications of pre-term birth, but they disputed the efficacy of ACS to avoid or diminish the risk of CP.

According to the plaintiffs, ACS would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition.  Dr. Viljoen disputed both general and specific causation. Evidence of general causation came from both randomized clinical trials (RCTs) and observational studies.

Limitations Issue

There were many peculiar aspects to the Goodman case, not the least of which was that the twins sued Dr. Viljoen over a decade after they were born.  Dr. Viljoen had moved his practice in the passage of time, and he was unable to produce crucial records that supported his account of how his staff responded to Mrs. Goodman’s telephone call about signs and symptoms of labor. The prejudice to Dr. Viljoen illustrates the harshness of broad tolling statutes, the unfairness of which could be reduced by requiring infant plaintiffs to give notice of their intent to sue, even if they wait until the age of majority before filing their complaints.

State of the Art Issue

Dr. Viljoen suffered perhaps a more serious prejudice in the form of hindsight bias that resulted from the evaluation of his professional conduct by evidence that was unavailable when the twins were born in 1995. The following roughly contemporaneous statement from the New England Journal of Medicine is typical of serious thinking at the time of the alleged malpractice:

“Antenatal glucocorticoid therapy decreases the incidence of several complications among very premature infants. However, its effect on the occurrence of cystic periventricular leukomalacia, a major cause of cerebral palsy, remains unknown.”

Olivier Baud, Laurence Laurence Foix l’Hélias, et al., “Antenatal Glucocorticoid- Treatment and Cystic Periventricular Leukomalacia in Very Premature Infants,” 341 New Engl. J. Med. 1190, 1190 (1999) (emphasis added). The findings of this observational study illustrate some of the difficulties with the claim that Dr. Viljoen failed to prevent an avoidable consequence of pre-term delivery:

“Our results suggest that exposure to betamethasone but not dexamethasone is associated with a decreased risk of cystic periventricular leukomalacia.”

Id. at 1194. Results varied among various corticosteroids, among doses, among timing regimens.  There hardly seemed enough data in 1995 to dictate a standard of care.

Meta-Analysis Issues

Over ten years after the Goodman twins were born, the Cochrane collaboration published a meta-analysis that was primarily concerned with the efficacy of ACS for lung maturation. Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews Issue 3. Art. No. CD004454 (2006). The trials included mostly post-dated the birth of the twins, and the alleged malpractice. The relevance of the trials to address the causation of CP in infants who experienced periventricular leukomalacia (PVL) was hotly disputed, but for now, I will gloss over the external validity problem of the Cochrane meta-analysis.

The Cochrane Collaboration usually limits its meta-analyses to the highest quality evidence, or RCTs, but in this instance, the RCTs did not include CP in its primary pre-specified outcomes. Furthermore, the trials were generally designed to ascertain short-term benefits from ACS, and the data in the trials were uncertain with respect to longer-term outcomes, which may have been ascertained differentially. Furthermore, the trials were generally small and were plagued by sparse data.  None of the individual trials was itself statistically significant at the 5 percent level.  The meta-analysis did not show a statistically significant decrease in CP from ACS treatment.  The authors reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”

 Id. at 8 (emphasis added).

The Cochrane authors were appropriately cautious in interpreting the sparse data:

“Results suggest that antenatal corticosteroids result in less neurodevelopmental delay and possibly less cerebral palsy in childhood.”

Id. at 13-14 (emphasis added).

The quality of the trials included in the Cochrane meta-analysis varied, as did the trial methodologies.  Despite the strong clinical heterogeneity, the Cochrane authors performed their meta-analysis with a fixed-effect model. The confidence interval, which included 1.0, reflected a p-value of 0.065, but that p-value would have certainly increased if a more appropriate random-effects model had been used.

Furthermore, the RCTs were often no better than observational studies on the CP outcome. The RCTs here perhaps should not have been relied upon to the apparent exclusion of observational epidemiology.

Relative Risk Less Than Two

There is much to be said about the handling of statistical significance, the Bayesian analysis, the arguments about causal inference, but for now, let us look at one of the clearest errors in the case:  the inference of specific causation from a relative risk less than two.  To be sure, the Cochrane meta-analysis reported a non-statistically significant 40% decrease, but if we were to look at this outcome in terms of the increase in risk of CP from the physician’s failure to administer ACS timely, then the risk ratio would be 1.67, or a 67% increase.  On either interpretation, fewer than half the cases of CP can be attributed to the failure to administer ACS fully and timely in the case.

The parties tried their case before Justice Walters, in St. Catherines, Ontario. Goodman v. Viljoen, 2011 ONSC 821 (CanLII).  Justice Walters recognized that specific causation was essential and at the heart of the parties’ disagreement:

“[47] In order to succeed, the plaintiffs must establish that the failure to receive a full course of ACS materially affected the twins’ outcome. That is, they must establish that “but for” the failureto receive a full course of ACS, the twins would not have suffered from the conditions they now do, or that the severity of these afflictions would have been materially reduced.

[48] Not surprisingly, this was the most contentious issue at trial and the court heard a good deal of evidence with respect to the issue of causation.”

One of the defendant’s expert witnesses, Robert Platt, a professor of statistics at McGill University School of Medicine, testified, according to Justice Walters:

“[144] Dr. Platt also stated that the absolute risk in and of itself does not tell us anything about what might have happened in a specific case absent clinical and mechanistic explanations for that specific case.”

The plaintiffs’ expert witnesses apparently conceded the point.  Professor Andrew Willan, a statistician, testifying for the plaintiffs, attempted to brush Platt’s point aside by suggesting it would render clinical research useless, but that was hardly the point.  Platt embraced clinical research for what it could show about the “averages” in a sample of the population, even if we cannot discern causal efficacy retrospectively in a specific patient:

“[133] Dr. Willan also responded to Dr. Platt’s criticism that it was impossible to determine the distribution of the effect across the population. Professor Willan felt this issue was a red herring, and if it were valid, it would render most clinical research useless. There is really no way of knowing who will benefit from a treatment and who will not. Unless there are reasons to believe otherwise, it is best to apply the population average effect to each person.”

Although Willan labeled Platt’s point as cold-blooded and fishy, he ultimately concurred that the population average effect should be applied to each person in the absence of evidence of risk being sequestered in a subgroup.

A closer look at Willan’s testimony at trial is instructive. Willan acknowledged, on direct examination, that the plaintiffs were at increased risk, even if their mother had received a full course of ACS.  All he would commit to, on behalf of the plaintiffs, was that their risk would have been less had the ACS been given earlier:

“All we can say is that there’s a high probability that that risk would be reduced and that this is probably the best estimate of the excess risk for not being treated and I would say that puts that in the 70 percent range of excess risk and I would say the probability that the risk would have been reduced is into the 90 percentage points.”

Notes of Testimony of Andrew Willan at 62 (April 6, 2010).  The 90 percentage points reference here was Willan’s posterior probability that the claimed effect was real.

On cross-examination, the defense pressed the point:

Q. What you did not do in this, in this report, is provide any quantification for the reduction in the risk, true?

A. That’s correct.

Notes of Testimony of Andrew Willan at 35 (April 9, 2010)

Q. And you stated that there is no evidence that the benefits of steroids is restricted to any particular subgroup of patients?

A. I wasn’t given any. I haven’t seen any evidence of that.

Id. at 43.

Q. And what you’re suggesting with that statement, is that the statistics should be generally, should be considered by the court to be generally applicable, true?

A. That’s correct.

Id. at 44.

Q. But given your report, you can’t offer assistance on the clinical application to the statistics, true?

A. That’s true.

Id. at 46.

With these concessions in hand, defense counsel elicited the ultimate concession relevant to the “but for” standard of causation:

Q. And to do that by looking at an increase in risk, the risk ratio from the data must achieve 2 in order for there to be a 50 percent change in the underlying data, true?

A. Yeah, to double the risk, the risk ratio would have to be 2, to double the risk.

Id. at 63.

* * *

Q. So, none of this data achieves the threshold of a 50 percent change in the underlying data, whether you look at it as an increase in risk or …

A. Sure.

Q …. a decrease in risk …

A. Yeah.

Id. at 66.

Leaping Inferences

The legal standard for causation in Canada is the same counterfactual requirement that applies in most jurisdictions in the United States.  Goodman v. Viljoen, 2011 ONSC 821 (CanLII), at ¶14, 47. The trial court well understood that the plaintiffs’ evidence left them short of showing that their CP would not have occurred but for the delay in administering ACS. Remarkably, the court permitted the plaintiffs to use non-existing evidence to bridge the gap.

According to Dr. Max Perlman, plaintiffs’ expert witness on neonatology and pediatrics, CP is not a dichotomous condition, but a spectrum that is manifested on a continuum of signs and symptoms.  The RCTs relied upon had criteria for ascertaining CP and including it as an outcome.  The result of these criteria was that CP was analyzed as a binary outcome.  Dr. Perlman, however, held forth that “common sense and clinical experience” told him that CP is not a condition that is either present or not, but rather presented on a continuum. Id. at [74].

Without any evidence, Perlman testified that when CP is not avoided by ACS, “it is likely that it is less severe for those who do go on to develop it.” Id. [75].  Indeed, Perlman made the absence of evidence a claimed virtue; with all his experience and common sense, he “could not think of a single treatment which affects a basic biological process that has a yes or no effect; they are all on a continuum.” Id. From here, Perlman soared to his pre-specified conclusion that “that it is more likely than not that the twins would have seen a material advantage had they received the optimal course of steroids.” Id. at [76].

Perlman’s testimony is remarkable for inventing a non-existing feature of biological evidence:  everything is a continuum. Justice Walters could not resist this seductive testimony:

“[195] The statistical information is but one piece of the puzzle; one way of assessing the impact of ACS on CP. Notably, the 40% reduction in CP attributable to ACS represents an all or nothing proposal. In other words, 93.5% of the time, CP is reduced in its entirety by 40%. It was the evidence of Dr. Perlman, which I accept, that CP is not a black and white condition, and, like all biological processes, it can be scaled on a continuum of severity. It therefore follows that in those cases where CP is not reduced in its entirety, it is likely to be less severe for those who go on to develop it. Such cases are not reflected in the Cochrane figure.

[196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical  conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.

[197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP.  Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.”

Doubly unfortunate, because Perlman’s argument was premised on a counterfactual assumption.  Many biological phenomena are dichotomous.  Pregnancy, for instance, does not admit of degrees.  Disease states are frequently dichotomous, and no evidence was presented that CP was not dichotomous. Threshold effects abound in living organisms. Perlman’s argument further falls apart when we consider that the non-experimental arm of the RCTs would also have had additional “less-severe” CP cases, with no evidence that they occurred disproportionately in the control arms of these RCTs. Furthermore, high-quality observational studies might have greater validity than post-hoc RCTs in this area, and there have been, and likely will continue to be, such studies to attempt better understanding of the efficacy of ACS, as well as differing effects among the various corticosteroids, doses, and patterns of administration.

On appeal, the Justice Walters’ verdict for plaintiffs was affirmed, but over a careful, thoughtful dissent. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) (Doherty, J., dissenting). Justice Doherty caught the ultimate futility of Dr. Perlman’s opinion based upon non-existent evidence: even if there were additional sub-CP cases in the treatment arms of the RCTs, and if they occurred disporportionately more often in the treatment than in the placebo arms, we are still left guessing about the quantitative adjustment to make to the 40% decrease, doubtful as it was, which came from the Cochrane review.

Recrudescence of Traumatic Cancer Claims

June 4th, 2014

In 1991, Peter Huber, discussing traumatic cancer claims, wrote:

“After years of floundering in the junk science morass of traumatic cancer, judges slowly abandoned sequence-of-events logic, turned away from the sympathetic speculations of family doctors, and struggled on to the higher and firmer ground of epidemiology and medical science.  Eventually, the change of heart among appellate judges was communicated back down to trial judges and worker’s compensation boards, and traumatic cancer went into almost complete remission.”

Peter W. Huber, Galileo’s Revenge: Junk Science in the Courtroom 55-56 (1991).

With the advent of Daubert and meaningful gatekeeping of expert witness opinion testimony, the traumatic cancer claims did recede. For a while. Plaintiffs’ counsel, and stalwart opponent of epistemic standards for scientific claims in court, Kenneth Chesebro attacked Huber’s précis of the traumatic cancer law and science. Kenneth J. Chesebro, “Galileo’s Retort: Peter Huber’s Junk Scholarship,” 42 Am. Univ. L. Rev. 1637 (1993). Defenses of the dubious science continue to appear, although mostly in non-peer-reviewed publications.[1]

One of the more disturbing implications of the West Virginia Supreme Court’s decision in Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013), was the Court’s reliance upon its own, recent approval of traumatic cancer claims.  The Harris Court cited, with approval, a 2002 traumatic cancer case, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  The Wiseman case involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation.  The West Virginia Supreme Court blinked at the challenge to the physician expert witness who advanced the causal claim in Wiseman; and in Harris, the Court made clear that blinking is what trial courts should do when confronted with methodological challenges to far-fetched causal opinions.

A couple of years ago, the New York Times ran an article about traumatic cancer. C. Claiborne Ray, “Injury and Insult” (Nov. 5, 2012), responding to the question “Is it possible for cancer to develop as a result of an injury?” Here is how Times science reporter responded:

A.It’s a common myth that injuries can cause cancer,” the American Cancer Society says on its Web site. Until the 1920s, some doctors believed trauma did cause cancer, “despite the failure of injury to cause cancer in experimental animals.” But most medical authorities, including the cancer society and the National Cancer Institute, see no such link. The more likely explanation, the society suggests, is that a visit to the doctor for an injury could lead to finding an existing cancer.

Other possibilities are that scar tissue from an old trauma could look like a cancerous lesion and that an injured breast or limb would be more closely watched for cancer to develop.

Ms. Ray went on to note a published study, in which would-be myth-busters presented observational data purportedly showing a relationship between physical injury and subsequent breast cancer.  The paper cited by Ms. Ray was a report on a small case-control study done by investigators at the Department of Geography, Lancaster University. See Jan Rigby, et al., “Can physical trauma cause breast cancer?” 11 Eur. J. Cancer. Prev. 307 (2002). The study consisted of 67 breast cancer cases and 134 controls, matched on age, family history, age of menarche, parity, age at first birth, and menopausal status.

Not surprisingly, considering its small size, the Rigby study reported no statistically significant differences for several factors known to be associated with breast cancer: social class, education, residence, smoking and alcohol consumption.  Although lacking power to detect differences of known risk factors, this study turned up a large, statistically significant association between physical trauma and breast cancer:

“Women with breast carcinoma were more likely to report physical trauma to the breast in the previous 5 years than were the controls (odds ratio (OR) 3.3, 95% confidence interval (CI) 1.3-10.8, P < 0.0001).”

* * * * *

“More likely to [self-]report” hardly implies causation, but the authors jumped not only to a causal explanation but to a causal conclusion:

* * * * *

“In conclusion, recall bias is an unlikely explanation for these results in view of the nature and severity of physical trauma. Models of epithelial cell generation indicate that a causal link between physical trauma and cancer is plausible. A latent interval between cancer onset and presentation of under 5 years is also plausible. The most likely explanation of the findings is that physical trauma can cause breast cancer.”

Rigby at 307.

The Rigby study is a valuable demonstration of how malleable researchers can be in discovering plausible explanations for their data.  The authors fail to discuss the natural history of breast carcinoma, such as tumor doubling time, which would make their five-year window decidedly implausible.  The Rigby paper also demonstrates how strident researchers can be in claiming that they have produced a study that has eliminated bias in observational research, when they have barely scratched the surface of bias or confounding. Magical thinking is not the exclusive domain of lawyers.

Until reading the Harris and Wiseman cases, I had thought that the legal system had graduated from the “mythology” of traumatic cancer cases.[2]  To be sure, in the past, any number of physicians have supported traumatic cancer claims, in print and in the courtroom.[3] Some authors attempted to put some rational limits on the extent of the traumatic cancer claims.[4] By 1947, at least, the trauma theory was criticized in leading texts.[5]  In 1974, the Mayo Clinic published a review that emphasized the lack of experimental evidence to support the claim that uncomplicated trauma causes cancer.[6] The law review literature attempted to make sense of the compensation-frenzied courts, without much success.[7]

Many cases from most jurisdictions have approved traumatic cancer claims.  Some are set out below. Some courts heroically resisted the pro-compensation Zeitgeist, usually on case-specific evidentiary issues.[8]

In New York, judges seem to be well aware that post hoc ergo propter hoc is a fallacy.  Cassano v. Hagstrom, 5 N.Y.2d 643, 159 N.E.2d 348, 187 N.Y.S.2d 1 (1959) (affirming dismissal of case based because of plaintiffs’ attempt to use fallacious reasoning in the form of  “post hoc ergo propter hoc”); Holzberg v. Flower & Fifth Ave. Hosps., 39 AD 2d 526 (N.Y. 1st Dep’t 1972). Still, the New York courts struggled with traumatic cancer claims, and appeared to oscillate wildly without clear guidance on whether or to what extent the courts could reject specious claiming supported by speculative or unreliable expert witness opinion testimony.[9] Given the current hostility to gatekeeping of expert witness opinion, a recrudescence of traumatic cancer claims is likely.

Opinions Approving Causation in Traumatic Cancer Cases

California

Santa Ana Sugar Co. v. Industrial Accid. Comm’n, 170 P. 630, 630 (Cal. Dist. Ct. App. 1917)

Colorado

Canon Reliance Coal Co. v. Indus. Comm’n, 72 Colo. 477, 211 P. 868, 869-70 (1922) (cancer caused by being hit on cheek with a lump of coal)

Georgia

National Dairy Prods. Corp. v. Durham, 154 S.E.2d 752, 753-54 (Ga. Ct. App. 1967)

Kentucky

Louisville Ry v. Steubing’s Adm’r, 136 S.W. 634, 634 (Ky. Ct. App. 1911)

Louisiana

Reed v. Mullin Wood Co., 274 So. 2d 845, 846-47 (La. Ct. App. 1972), cert. denied, 275 So. 2d 729, 791 (La. 1973);

Thompson v. New Orleans Ry. & Light Co., 83 So. 19, 20 (La. 1919)

Michigan

Wilson v. Doehler-Jarvis Div. of Nat’l Lead Co., 353 Mich. 363, 91 N.W.2d 538, 539-40 (1958) (blow to lip caused cancer)

Mooney v. Copper Range RR, 27 N.W.2d 603, 604 (Mich. 1947)

Minnesota

Daly v. Bergstedt, 267 Minn. 244, 126 N.W.2d 242, 247–48 (1964) (affirming jury finding of causation between traumatic leg fracture and breast cancer; six physicians testified against causation; one stated cancer “could” result from trauma; imagining that scientific and legal standards of causation differ)

Pittman v. Pillsbury Flour Mills, Inc., 48 N.W.2d 735, 736 (Minn. 1951)

Hertz v. Watab Pulp & Paper Co., 237 N.W. 610, 611 (Minn. 1931)

Austin v. Red Wing Sewer Pipe Co., 163 Minn. 397, 204 N.W. 323, 323-24 (Minn. 1925) (cancer developed one year after worker was hit in the face with coal)

Gaetz v. City of Melrose, 193 N.W. 691, 692 (Minn. 1923)

Missouri

Vitale v. Duerbeck, 338 Mo. 536, 92 S.W.2d 691, 695 (1936)

New Hampshire

Jewell v. Grand Trunk Ry, 55 N.H. 84 (1874) (reversing traumatic cancer verdict on other grounds)

New Mexico

White v. Valley Land Co., P.2d 707, 708-10 (N.M. 1957)

Ohio

Hanna v. Aetna Ins., 24 Ohio Misc. 27, 52 Ohio Op. 2d 316, 259 N.E.2d 177, 177-79 (Ohio Mun. Ct. Dayton 1970)(breast lump found three months after car accident)

Glenn v. National Supply, 129 N.E.2d 189, 190-91 (Ohio Ct. App. 1954)

Oregon

Devine v. Southern Pacific Co., 207 Or. 261, 295 P.2d 201 (1956) (holding that physician’s testimony as to “probable” causation between shoulder fracture and lung cancer was sufficient; jury verdict for plaintiff reversed on other grounds).

Pennsylvania

Baker v. DeRosa, 413 Pa. 164, 196 A.2d 387, 389–90 (Pa. 1964)

Menarde v. Philadelphia Transp. Co., 376 Pa. 497, 103 A.2d 681, 684(1954) (the fact that breast cancer was found in the same place as the injury-caused bruise helped establish causation);

Southern S.S. Co. v. Norton, 41 F. Supp. 103 (E.D. Pa. 1940) (trauma to skull and lower back held to have caused lung cancer)

Tennessee

Koehring-Southern & Am. Mut. Ins. Co. v. Burnette, 464 S.W.2d 820, 821 (Tenn. 1970)

Boyd v. Young, 193 Tenn. 272, 246 S.W.2d 10, 10 (Tenn. 1951)

Rhode Island

Valente v. Bourne Mills, 77 R.I. 274, 278-79, 75 A.2d 191, 193-94 (1950) (adopting house of cards position in which any rational inference suffices even if not supported by expert medical opinion)

Emma v. A.D. Julliard & Co., 75 R.I. 94, 63 A.2d 786, 787-89 (R.I. 1949)(plaintiff had malignant tumor removed from her breast seven weeks after being hit with a can of juice)

Texas

Traders & General Insur. Co. v. Turner, 149 S.W.2d 593, 597-98 (Tex. Civ. App. 1941) (testicular cancer)

Virginia

Ellis v. Commonwealth Dep’t of Highways, 28 S.E.2d 730, 731-32, 735 (Va. 1944) (accepting post-hoc reasoning “[f]acts prevail over possibilities or probabilities”)

Winchester Milling Corp. v. Sencindiver, 138 S.E. 479, 480-81 (Va. 1927)


[1] See, e.g., Melvin A. Shiffman, Can Trauma Cause or Accelerate the Growth of Cancer? Forensic Examiner 6 (Fall 2004).

[2] See Manasco v. Insurance Co. of State of Pennsylvania, 89 S.W.3d 239 (Tex. App. Texarkana 2002) (affirming denial of benefits to worker who claimed head injury caused brain tumor; citing to epidemiological studies that failed to show an association between trauma and brain tumors).

[3] See, e.g., George R. Parsons, “Sufficiency of Proof in Traumatic Cancer Cases,” 2 Tort & Med. Year Book 335 (1962); Stoll & Crissey, “Epithelioma from Single Trauma,” 62 N.Y. St. J. Med. 496 (Feb. 15, 1962); Wilhelm C. Hueper, Trauma and Cancer (1959); Arden R. Hedge, “Can a Single Injury Cause Cancer?” 90 Calif. Med. 55 (1959); R. Crane, “The Relationship of a Single Act of Trauma to Subsequent Malignancy,” in Alan R. Moritz & David S. Helberg, eds., Trauma and Disease 147 (1959); Shields Warren, M.D., “Minimal criteria required to prove causation of traumatic or occupational neoplasms,” Ann. Surgery 585 (1943); Bishop, “Cancer, Trauma, and Compensation,” 32 So. Med. J. 302 (1939); Knox, “Trauma and Malignant Tumors, 26 Am. J. Surg. 66, 69-70 (1934); William B. Coley & Norman L. Higinbotham, “Injury as a causative factor in the development of malignant tumors,” 98 Ann. Surg. 991 (1933); Wainwright, “Single Trauma, Carcinoma and Workman’s Compensation,” 5 Am. J. Surg. 433 (1928); Alson R. Kilgore & Curtis E. Smith, “Industrial liability for cancer,” 25 Calif. & Western Med. 70 (1926); Charles Phelps, “The relation of trauma to cancer formation,” 51 Ann. Surgery 609 (1910).

[4] James Ewing, “Modern Attitudes Toward Traumatic Cancer,” 19 Arch. Path. 690, 692 (1935); James Ewing, “The Relation of Trauma to Malignant Tumors,” Am. J. Surg. 30, 31-34 (Feb. 1926).

[5] See, e.g., James A. Tobey, Public Health Law 321 (3ed 1947) (“Although there is little, if any, scientific evidence to prove conclusively that malignant growths such as carcinoma, sarcoma, and other forms of cancer are ever caused by single blows, wounds, injuries, or other forms of trauma, the courts have awarded damages in a number of instances to persons who have developed cancers following single injuries.”) (internal citations omitted).

[6] George R. Monkman, Gregg Orwoll & John C. Ivins, “Trauma and Oncogenesis,” 49 Mayo Clinic Proc. 157 (1974).

[7] The trauma theory of carcinogenesis was discussed and questioned in several law review articles.  See, e.g., Orrin E. Tilevitz, “Judicial Attitudes Towards Legal and Scientific Proof of Cancer Causation,” 3 Colum. J. Envt’l L. 344 (1977); Donald J. Ladanyi, “Impact Trauma As ‘Legal Cause’ of Cancer,” 20 Cleveland State L. Rev. 409 (1971); Theodore Dyke, “Traumatic Cancer?” 15 Clev.-Marshall L. Rev. 472 (1966); Jerry G. Elliott, “Traumatic cancer and ‘an old misunderstanding between doctors and lawyers’,” 13 U. Kan. L. Rev. 79 (1964); Comment, Sufficiency of Proof in Traumatic Cancer: A Medico-Legal Quandary, 16 Ark. L. Rev. 243 (1962); Comment, “Sufficiency of Proof in Traumatic Cancer Cases,” 46 Cornell L.Q. 581 (1961); Adelson, Injury and Cancer, 5 Western Res. L. Rev. 150 (1954).

[8] State Compensation Ins. Fund v. Kindig, 445 P.2d 72 (Colo. 1968) (head injury held not to have caused leukemia 68 days later); Slack v. C.L. Percival Co., 198 Iowa 54, 199 N.W. 323, 326 (1924) (anticipating Daubert by rejecting expert witness opinion that was “wholly in the realm of conjecture, speculation, and surmise”); Ortner v. Zenith Carburetor Co., 207 Mich. 610, 175 N .W. 122 (1919) (holding that 30 months was too long for a claim that accident that crushed worker’s fingers caused blood poisoning and penile cancer); Stordahl v. Rush Implement Co., 417 P.2d 95 (Mont. 1966) (rejecting traumatic causation of malignant tumor); Tonkovich v. Dep’t of Lab. & Indus., 31 Wash. 2d 220, 195 P.2d 638 (1948) (injury to foot held not to have caused abdominal cancer)

[9] See Dennison v. Wing, 279 App. Div. 494, 110 N.Y.S.2d 811, 813 (1952) (rejecting cancer claim when latency was two months on grounds that cancer took longer to develop); Sikora v. Apex Beverage Corp., 282 App. Div. 193, 196-97 (1953) (reversing judgment for plaintiff based upon jury’s finding that slip and fall accelerated breast cancer based upon lack of evidentiary support), aff’d, 306 N.Y. 917, 119 N.E.2d 601 (1954); Frankenheim v. B. Altman & Co., 13 Misc. 2d 1079, 1080-81, 177 N.Y.S.2d 2 (Bronx Cty. S.Ct. 1958) (granting motion to set aside verdict for plaintiff based upon traumatic cancer claim on grounds of insufficient evidence), app. dism’d, 8 App. Div. 2d 809 (First Dep’t 1959). But see McGrath v. Irving, 24 App. Div. 2d 236, 265 N.Y.S.2d 376 (1965) (affirming jury verdict based upon claim that plaintiff’s swallowing glass in car accident caused or accelerated development of laryngeal cancer); Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 221 N.Y.S.2d 224, 224 (1st Dep’t 1961) (affirming award for traumatic cancer based upon the “usual” conflicting expert witness testimony) Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 942 (1961) (affirming workman’s compensation award for “aggravation” of cancer, which resulted after “the usual conflict of medical opinion”); Pezzolanti v. Green Bus Lines, 114 App. Div. 2d 553, 553-54, 494 N.Y.S.2d 168, 169 (1985) (affirming workman’s compensation award for disability to wrist, which resulted from “trauma” of hitting pothole, which in turn injured asymptomatic wrist destabilized by pre-existing cancer).

Intellectual Due Process in West Virginia and Beyond

June 1st, 2014

Harris v. CSX Transportation

I have borrowed and modified the phrase “Intellectual Due Process” from earlier writers because of its obvious implications for the presentation, interpretation, synthesis, and evaluation of scientific evidence in court. See Scott Brewer, “Scientific Expert Testimony and Intellectual Due Process,” 107 Yale L. J. 1535 (1998). The major reason courts write opinions is to explain and justify their decisions to litigants, present and future, and to a wider audience of lawyers, scholars, and the general public. Judicial opinions involving scientific evidence, whether in legislation, regulation, or litigation must satisfy the societal need to explain and justify the acceptance and rejection of scientific claims. Despite a great deal of hand waving that law and science are somehow different, in the end, when courts describe their acceptance or rejection of scientific claims, they are addressing the same epistemic warrant that scientists themselves employ. Even a cursory review of the judicial output reveals an unsatisfactory state of affairs in which many courts mangle scientific and statistical evidence and inference.  There is much that is needed to correct the problem.

One proposal would be to require that the parties file proposed findings of facts in connection with Rule 702 gatekeeping challenges.  Courts should file detailed findings of facts that underlie their decisions to admit or to exclude expert witness opinion testimony.  Another proposal would require courts to cite properly the scientific studies that they discuss in reaching a legal conclusion about sufficiency or admissibility.  These are small steps, but ones that would help reduce the gross inaccuracies and the glib generalizations, while increasing the opportunity for public scrutiny and criticism.

We do not think anything is amiss with special courts for tax, patent, family law, national security, equity, or commercial matters.  There is an even greater need for scientific skill, knowledge, and aptitude in a specialized science court.  The time has come for special courts to hear cases involving scientific claims in health effects and other litigation.

*   *   *   *   *   *   *

A decision of the West Virginia Supreme Court, late last year, illustrates the need for substantial reform of how claiming based upon “scientific evidence” is permitted and evaluated in court.  Mrs. Harris sued the railroad for the wrongful death of her husband, who died of multiple myeloma. Mr. Harris had been exposed, in his railroad workplace, to diesel exhaust, which Mrs. Harris claimed caused his cancer. See Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013). The trial court excluded Mrs. Harris’s expert witnesses. Harris v. CSX Transportation, Inc., No. 12-1135, 2012 WL 8899119 (Cir. Ct. Marshall Cty., W.Va. Aug. 21, 2012).

1. The West Virginia Supreme Court reversed the trial court’s exclusion of witnesses on the basis of an asymmetrical standard of review, which would allow de novo review of trial court decisions to exclude expert witness opinions, but which would privilege trial court decisions to admit opinions by limiting appellate review to abuse of discretion. This asymmetry was, of course, the same dodge that the Third and Eleventh Circuits had used to keep the “gates open,” regardless of validity or reliability concerns, and the same dodge that the Supreme Court shut down in General Electric v. Joiner. A single judge dissented in Harris, Justice Loughry, who took the majority to task for twisting facts and law to get to a desired result.

2. The Harris Court cited a federal court case for dicta that “Rule 702 reflects an attempt to liberalize the rules governing the admissibility of expert testimony.” See Harris, 753 S.E.2d at 279 (citing and quoting from Weisgram v. Marley Co., 169 F.3d 514, 523 (8th Cir.1999). Remarkably, the Harris Court omitted reference to the United States Supreme Court’s unanimous affirmance of Weisgram, which saw Justice Ginsburg write that “[s]ince Daubert, moreover, parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.” Weisgram v. Marley Co., 528 U.S. 440, 442 (2000).  The Harris Court’s lack of scholarship is telling.

3. Meta-analysis appeared to play a role in the case, but the judicial decisions in Harris fail to describe the proffered evidence. The majority in Harris noted that one of plaintiff’s expert witnesses, Dr. Infante, relied upon a meta-analysis referred to as “Sonoda 2001.” Harris, 753 S.E.2d at 309. Neither the Court nor the dissent cited the published meta-analysis in a way that would help an interested reader in finding the paper.  One could imagine the hue and cry if courts cited judicial cases or statutes by short-hand names without providing enough information to access the relied upon source.  In this case, a PubMed search reveals the source so perhaps the error is harmless. Tomoko Sonoda, Yoshie Nagata, Mitsuru Mori, Tadao Ishida & Kohzoh Imai, “Meta-analysis of multiple myeloma and benzene exposure,” 11. J. Epidemiol. 249 (2001).  Still, the time has come for courts to describe and report the scientific evidence with the same care and detail that they would use in a car collision case.

4. A quick read shows that the Sonoda meta-analysis supports the dissent’s assessment:

“‘Dr. Infante testified on direct examination that Sonoda 2001 considered 8 case-control studies specific to engine exhaust and stated it concluded that diesel and non-diesel engine exhaust causes multiple myeloma.’ Yet, as the trial court found, ‘[o]n cross examination Dr. Infante acknowledged that none of the 8 papers included in the Sonoda meta-analysis mention diesel exhaust’.”

Harris, 753 S.E.2d at 309.  The dissent would have been considerably more powerful had it actually adverted to the language of Sonoda 2001:

“These results suggested that benzene exposure itself was not likely to be a risk factor of MM [multiple myeloma]. It is thought that several harmful chemical agents in engine exhaust, other than benzene, could be etiologically related to the risk of MM. Further case-control studies on MM are needed to obtain more information about detailed occupational exposure to toxic substances.”

Sonoda at 249 (2001) (emphasis added).  Contrary to Infante’s asseveration, Sonoda and colleagues never concluded that diesel exhaust causes multiple myeloma.  The state of scholarship and “intellectual due process” makes it impossible to tell whether or not Dr. Infante was telling the truth or the Harris Court badly misunderstood the record. Either way, something must give.

The dissent went on to note that Dr. Infante conducted his own meta-analysis, which included studies that did not mention diesel exhaust. Harris, 753 S.E.2d at 309.  The railroad complained that some of the studies were small and had limited power, but that is exactly why a meta-analysis would be appropriate.  The more disturbing complaints were that the meta-analysis left out important studies, and that it included irrelevant studies of benzene exposure and myeloma, which raised insuperable problems of external validity.

5. A half empty glass that is always full.  According to the Harris Court, the West Virginia shadow of Rule 702 is a rule of “admissibility rather than exclusion.” Harris, 753 S.E.2d at 279 (citing and quoting from In re Flood Litig. Coal River Watershed, 222 W.Va. 574, 581, 668 S.E.2d 203, 210 (2008), which in turn quoted a federal case, Arcoren v. United States, 929 F.2d 1235, 1239 (8th Cir. 1991), decided before the Supreme Court decided Daubert.)  This is just silly hand waving and blatant partisanship.  A rule that sets out criteria or bases for admissibility also demarcates the inadmissible.

6. Cherry Picking. Dr. Infante was permitted by the Harris Court to aggregate data from studies that did not observe diesel exposure, while he failed to include, or he deliberately excluded data from, a large, powerful, exonerative study conducted by scientists from the National Cancer Institute, the International Agency for Research on Cancer (IARC), and the Karolinska Institute. See Paolo Boffetta, Mustafa Dosemeci, Gloria Gridley, Heather Bath, Tahere Moradi and Debra Silverman, “Occupational exposure to diesel engine emissions and risk of cancer in Swedish men and women,” 12 Cancer Causes Control 365 (2001). Dr. Infante inexplicably excluded this study, which found a risk ratio for men exposed to diesel exhaust that was below one, 0.98, with a very narrow 95% confidence interval, 0.92-1.05. Boffetta at 368, Table 2.

7. The West Virginia articulated an incohorent definition of “reliable,” designed to give itself the ability to reject gatekeeping completely. Citing its earlier decision in Flood, the Court offered its own ipse dixit:

“The assessment of whether scientifically-based expert testimony is “reliable,” as that term is used in [Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and Wilt v. Buracker, 191 W.Va. 39, 443 S.E.2d 196 (1993)], does not mean an assessment of whether the testimony is persuasive, convincing, or well-founded. Rather, assessing ‘reliability’ is a shorthand term of art for assessing whether the testimony is to a reasonable degree based on the use of knowledge and procedures that have been arrived at using the methods of science — rather than being based on irrational and intuitive feelings, guesses, or speculation. If the former is the case, then the jury may (or may not, in its sole discretion) ‘rely upon’ the testimony. In re Flood Litig., 222 W.Va. at 582 n. 5, 668 S.E.2d at 211 n. 5.”

Harris, 753 S.E.2d at 279-80. Surely, this is circular or vacuous or both. Opinions not “well-founded” will be ones that are based upon guesses or speculation.  Opinions arrived at by the “methods of science” will be ones that have an epistemic warrant that will survive a claim that they are not “well-founded.”

8. The Harris Court evidenced its hostility to scientific evidence by dredging up one of its own decisions involving a multiple myeloma causation claim, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  Wiseman involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation:

“Some research has suggested that people in some jobs may have an increased risk of developing multiple myeloma because they are exposed to certain chemicals. But the International Agency for Research on Cancer (IARC) states that the evidence is limited overall. It has been suggested that people may have an increased risk if they work in the petrol or oil industry, farming, wood working, the leather industry, painting and decorating, hairdressing, rubber manufacturing or fire fighting. But there is no evidence to prove that any of these occupations carry an increased risk of myeloma.”

Cancer Research UK, “Myeloma risks and causes” (last visited May 28, 2014). Even the most non-progressive jurisdictions have generally eradicated specious claiming for trauma-induced cancers, but West Virginia has carved out a place second to none in its race to the bottom.

9. WOE.  Not surprisingly, the Harris Court relied heavily on the First Circuit’s “weight of the evidence” end-run around the notion of epistemic warrant for scientific claims, citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir.2011), cert. denied sub nom., U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). The Harris Court went on to conflate and confuse WOE with Bradford Hill, and cited a recent New York case that confidently saw through WOE hand waving, while ignoring its devasting critique of expert witnesses’ attempts to pass off WOE for scientific, epistemic warrant.  Reeps ex rel. Reeps v. BMW of N. Am., LLC, No. 100725/08,

2013 WL 2362566, at *3, 2012 N.Y. Misc. LEXIS 5788; 2012 NY Slip Op 33030U  (N.Y. Sup. Ct. May 10, 2013).

10.  Link.  Dr. Infante links a lot, even when his sources do not:

“Dr. Infante testified that the International Agency for Research on Cancer issued Technical Publication Number 42 in 2009, and that the publication stated that diesel exhaust exposures have been linked to multiple myeloma and leukemia.”

Harris, 753 S.E.2d at 294. The Harris Court neglected to give the title of the publication, which tells a different story.  Identification of research needs to resolve the carcinogenicity of high-priority IARC carcinogens. The dissent was willing to go behind the conclusory and false characterization that Dr. Infante and plaintiff gave to this publication.  Harris, 753 S.E.2d at 309. The trial court’s finding (and the dissent’s assertion) that the IARC Technical Publication 42 intended to express a research agenda, not to make a causation statement, seems unassailable.  Furthermore, it appears to be precisely the sort of specious claim that a court should keep from a jury.  The cited IARC source actually notes that the then current IARC classification of diesel exhaust was of inadequate evidence for human carcinogenicity, with a focus on lung cancer, and barely a mention of multiple myeloma.

11.  The Benzene Connection. Plaintiffs’ expert witnesses, including Dr. Infante, argued that benzene was a component of diesel exhaust, and benzene caused multiple myeloma.  This move ignored not only the lack of evidence to implicate benzene in the causation of multiple myeloma, but it also ignored the large quantitative differences between the benzene occupational exposure studies and the very small amounts of benzene in diesel exhaust.  The Harris Court held that the trial court acted improperly by inquiring into and finding the following facts, which were “exclusively” for the jury:

  • “There is substantially more benzene in cigarette smoke than diesel exhaust.
  • Benzene is present only in trivial doses in diesel exhaust.
  • The hypothesis that diesel exhaust causes multiple myeloma is confounded by the fact that cigarette smoking does not.”

The Harris majority further chastised the trial court for adverting to the ten or so studies that failed to find a statistically significant association between benzene exposure and multiple myeloma.  Harris, 753 S.E.2d at 305-06.  This inquiry directly calls into question, however, Dr. Infante’s methodology.

If these facts, found by the trial court, were reasonably established, then Dr. Infante’s argument was less than bogus, and a major underpinning for inclusion of benzene studies in his meta-analysis was refuted.  These are precisely the sort of foundational facts that must be part of an inquiry into the methodological grounds of an expert witness’s opinion.

12.  The Harris Court confused “proving causation” with “showing a methodology that provides an epistemic warrant for concluding.” Harris, 753 S.E.2d at 300. The Harris Court asserted that the trial court exceeded its gatekeeping function by inquiring into whether Mrs. Harris’s expert witnesses “proved” causation. Harris, 753 S.E.2d at 300. Speaking of “proof of” or “proving” causation is an affectation of lawyers, who refer to their evidence as their “proofs.”  Epidemiologic articles and meta-analyses do not end with quod erat demonstrandum. Beyond the curious diction, there is a further issue in the majority’s suggestion that the trial court set the bar too high in declaring that the plaintiff failed to “prove” causation.  Even if we were to accept the continuous nature of strength of evidence for a causal conclusion, Dr. Infante and the other plaintiff’s witnesses, would be fairly low on the curve, and their lowly position must of necessity speak to the merits of the defense motion to exclude under Rule 702.

13. Purely Matters for Jury. The Harris Court criticized the trial court for conducting a “mini-trial,” which set out to “resolve issues that were purely matters for jury consideration.” Harris, 753 S.E.2d at 305. In holding that the matters addressed in the pre-trial hearing were “exclusively grist for the jury and which had no relevancy to the limited role the trial court had under the facts of this case,” the Harris Court displayed a profound disregard for what facts would be relevant for a challenge to the plaintiff’s expert witnesses’ methodology. Many of the facts found by the trial court were directly relevant to “general acceptance,” validity (internal and external) of studies relied upon, and reliability of reasoning and inferences drawn. Aside from the lack of general acceptance and peer review of the plaintiff’s claimed causal relationship, the proffered testimony was filled with gaps and lacunae, which are very much at issue in methodological challenges to an opinion of causality.

*   *   *   *   *   *   *

The Harris case has taken its place next to Milward in the litigation industry’s arsenal of arguments for abandoning meaningful judicial supervision and gatekeeping of expert witness opinion testimony.  See Andrew S. Lipton, “Proving Toxic Harm: Getting Past Slice and Dice Tactics,” 45 McGeorge L. Rev. 707, 731 (2014) (plaintiffs’ bar cheerleading for the Harris decision as “a lengthy and thoughtful analysis”, and for the Milward case as roadmap to evade meaningful judicial oversight).  Not all was perfect with the trial court’s opinion.  The defense seemed to have misled the court by asserting that “a difference between a case group and control group is not statistically significant then there is no difference at all.”  See Respondent’s Brief at 5, Harris v. CSX Transportation, Inc., 2013 WL 4747999 (filed (Feb. 4, 2013) (citing  App. 169, 228-230 (Shields) as having explained that the p-values greater than 0.05 do not support a causal association).

This is hardly true, and indeed, the lack of statistical significance does not lead to a claim that the null hypothesis of no association between exposure and outcome is correct.  The defense, however, did not have a burden of showing the null to be correct; only that there was no reliable method deployed to reject the null in favor an alternative that the risk ratio for myeloma was raised among workers exposed to diesel exhaust.

Still, the trial court did seem to understand the importance of replication, in studies free of bias and confounding. Courts generally will have to do better at delineating what are “positive” and “negative” studies, with citations to the data and the papers, so that judicial opinions provide a satisfactory statement of reasons for judicial decisions.

A Black Swan Case – Bayesian Analysis on Medical Causation

March 15th, 2014

Last month, I posted about an article that Professor Greenland wrote several years ago about his experience as a plaintiffs’ expert witness in a fenfluramine case. “The Infrequency of Bayesian Analyses in Non-Forensic Court Decisions (Feb. 16, 2014).” Greenland chided a defense expert for having declared that Bayesian analyses are rarely or never used in analyzing clinical trials or in assessments of pharmaco-epidemiologic data.  Greenland’s accusation of ludicrousness appeared mostly to blow back on him, but his stridency for Bayesian analyses did raise the question, whether such analyses have ever moved beyond random-match probability analyses in forensic evidence (DNA, fingerprint, paternity, etc.) or in screening and profiling cases.  I searched Google Scholar and Westlaw for counter-examples and found none, but I did solicit references to “Black Swan” cases. Shortly after I posted about the infrequency of Bayesian analyses, I came across a website that was dedicated to collecting legal citations of cases in which Bayesian analyses were important, but this website appeared to confirm my initial research.

Some months ago, Professor Brian Baigrie, of the Jackman Humanities Institute, at the University of Toronto, invited me to attend a meeting of an Institute working group on The Reliability of Evidence in Science and the Law.  The Institute fosters interdisciplinary scholarship, and this particular working group has a mission statement close to my interests:

The object of this series of workshops is to formulate a clear set of markers governing the reliability of evidence in the life sciences. The notion of evidence is a staple in epistemology and the philosophy of science; the notion of this group will be the way the notion of ‘evidence’ is understood in scientific contexts, especially in the life sciences, and in judicial form as something that ensures the objectivity of scientific results and the institutions that produce these results.

The Reliability of Evidence in Science and the Law. The faculty on the working group represent disciplines of medicine (Andrew Baines), philosophy (James R. Brown, Brian Baigrie), and law (Helena Likwornik, Hamish Stewart), with graduate students in the environmental science (Amy Lemay), history & philosophy of science and technology (Karolyn Koestler, Gwyndaf Garbutt ), and computer science (Maya Kovats).

Coincidentally, in preparation for the meeting, Professor Baigrie sent me links to a Canadian case, Goodman v. Viljoen, which turned out to be a black swan case! The trial court’s decision, in this medical malpractice case focused mostly on a disputed claim of medical causation, in which the plaintiffs’ expert witnesses sponsored a Bayesian analysis of the available epidemiologic evidence; the defense experts maintained that causation was not shown, and they countered with the unreliability of the proffered Bayesian analysis. The trial court resolved the causation dispute in favor of the plaintiffs, and their witnesses’ Bayesian approach. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII).  The Court of Appeals’ affirmance was issued over a lengthy, thoughtful dissent. The Canadian Supreme Court denied leave to appeal.

Goodman was a medical practice case. Mrs. Goodman alleged that her obstetrician deviated from the standard of care by failing to prescribe corticosteroids sufficiently early in advance of delivery to avoid or diminish the risk of cerebral palsy in her twins.  Damages were stipulated, and the breach of duty turned on a claim that Mrs. Goodman, in distress, called her obstetrician.  Given the decade that passed between the event and the lawsuit, the obstetrician was unable to document a response.  Duty and breach were disputed, but were not the focus of the trial.

The medical causation claim, in Goodman, turned upon a claim that the phone call to the obstetrician should have led to an earlier admission to the hospital, and the administration of antenatal corticosteroids.  According to the plaintiffs, the corticosteroids would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition.  The plaintiffs’ expert witnesses relied upon studies that suggested a 40% reduction and risk, and a probabilistic argument that they could infer from this risk ratio that the plaintiffs’ condition would have been avoided.  The case thus raises the issue whether evidence of risk can substitute for evidence of causation.  The Canadian court held that risk sufficed, and it went further, contrary to the majority of courts in the United States, to hold that a 40% reduction in risk sufficed to satisfy the more-likely-than-not standard.  See, e.g., Samaan v. St. Joseph Hosp., 670 F.3d 21 (1st Cir. 2012) (excluding expert witness testimony based upon risk ratios too small to support opinion that failure to administer intravenous tissue plasminogen activator (t-PA) to a patient caused serious stroke sequelae); see also “Federal Rule of Evidence 702 Requires Perscrutations — Samaan v. St. Joseph Hospital (2012)” (Feb. 4, 2012).

The Goodman courts, including the dissenting justice on the Ontario Court of Appeals, wrestled with a range of issues that warrant further consideration.  Here are some that come to mind from my preliminary read of the opinions:

1. Does evidence of risk suffice to show causation in a particular case?

2. If evidence of risk can show causation in a particular case, are there requirements that the magnitude of risk be quantified and of a sufficient magnitude to support the inference of causation in a particular case?

3. The judges and lawyers spoke of scientific “proof.”  When, if ever, is it appropriate to speak of scientific proof of a medical causal association?

4. Did the judges incorrectly dichotomize legal and scientific standards of causation?

5. Did the judges, by rejecting the need for “conclusive proof,” fail to articulate a meaningful standard for scientific evidence in any context, including judicial contexts?

6. What exactly does the “the balance of probabilities” mean, especially in the face of non-quantitative evidence?

7. What is the relationship between “but for” and “substantial factor” standards of causation?

8. Can judges ever manage to define “statistical significance” correctly?

9. What is the role of “common sense” in drawing inferences by judges and expert witnesses in biological causal reasoning?  Is it really a matter of common sense that if a drug did not fully avert the onset of a disease, it would surely have led to a less severe case of the disease?

10. What is the difference between “effect size” and the measure of random or sampling error?

11. Is scientific certainty really a matter of being 95% certain, or is this just another manifestation of the transposition fallacy?

12. Are Bayesian analyses acceptable in judicial settings, and if so, what information about prior probabilities must be documented before posterior probabilities can be given by expert witnesses and accepted by courts?

13. Are secular or ecological trends sufficiently reliable data for expert witnesses to rely upon in court proceedings?

14. Is the ability to identify biological plausibility sufficient to excuse the lack of statistical significance and other factors that are typically needed to support the causality of a putative association?

15. What are the indicia of reliability of meta-analyses used in judicial proceedings?

16. Should courts give full citations to scientific articles that are heavily relied upon as part of the requirement that they publicly explain and justify their decisions?

These are some of the questions that come to mind from my first read of the Goodman case.  The trial judge attempted to explain her decision in a fairly lengthy opinion. Unfortunately, the two judges, of the Ontario Court of Appeals, who voted to affirm, did not write at length. Justice Doherty wrote a thoughtful dissent, but the Supreme Court denied leave to appeal.  Many of the issues are not fully understandable from the opinions, but I hope to be able to read the underlying testimony before commenting.

Thanks to Professor Baigrie for the reference to this case.

The Rise and Rise of Junk Science

March 8th, 2014

Many authors attribute the term “junk science” to Peter Huber and his use of it in the term in his book, Galileo’s Revenge: Junk Science In The Courtroom (1991). As important as Huber’s book was to raising judicial consciousness to what was going on in courtrooms around the United States, the phrase “junk science” clearly predates Huber’s book.

Lawrence Hubert and Howard Wainer note that the phrase appears to have been in use by the early 1980’s, and sugggest that the first use of the pejorative phrase occurred in a Reagan administration white paper.  Lawrence Hubert and Howard Wainer, A Statistical Guide for the Ethically Perplexed 460 (Boca Raton 2013). The document cited by Hubert and Wainer notes:

“Another way in which causation often is undermined – also an increasingly serious problem in toxic tort cases – is the reliance by  judges and juries on non-credible scientific or medical testimony, studies or opinions. It has become all too common for “experts” or “studies” on the fringes of, or even well beyond the outer parameters of mainstream scientific or medical views, to be presented to juries as valid evidence from which conclusions may be drawn. The use of such invalid scientific evidence (commonly referred to as “junk science”) has resulted in findings of causation which simply cannot be justified or understood from the standpoint of the current state of credible scientific and medical knowledge. Most importantly, this development has led to a deep and growing cynicism about the ability of tort law to deal with difficult scientific and medical concepts in a principled and rational way.”

United States Dep’t of Justice, Tort Policy Working Group, Report of the Tort Policy Working Group on the causes, extent and policy implications of the current crisis in insurance availability and affordability at 35 (Report No. 027-000-01251-5) (Wash.DC 1986).  So according to the Justice Department authors, “junk science” was already in common use by 1986.  We really would not expect linguistic creativity in such a document.

Whence comes the phrase “junk science”?  Clearly, the phrase is an analogue of “junk food,” food that fills but fails to nourish.  Here is the Google ngram of the emergence of the phrase “junk food,” which shows the phrase took off in common use shortly before 1970: junk food Google NgramWhat then about junk science?

The Rise of Junk Science - Google Ngram Viewer

With a little tweaking of Google’s smoothing function, this search can be run to reveal more about the low end of the curve.

Junk Science without smoothing

This chart suggests that there was very small flurry of usage in the first half of the 1970s, with a re-emergence around 1982 or so, and then a re-introduction in 1985, with a steady increase every since.

Here is how “junk science” compares to “junk food” (with more smoothing to the curve added):

junk science vs junk food

Junk science seems to have overtaken junk food in books, at any rate.

Of course, “junk science” is an epithet to be hurled at science that the speaker dislikes.  It has an emotive content, but its persistence reflects that it has an epistemic content as well.  “Junk science” is science that lacks an epistemic warrant, and pretends to be something that it is not.  Honest scientists, engaged in hypothesis-generating work, should not be defamed as junk scientists, but the boundary between hypothesis generation and conclusion mongering is often blurred by advocate scientists of all political persuasions.

There are many synonyms for junk science, which has been with us ever science gained prestige and persuasiveness over religious pronouncements about the real world.  To avoid the politicization of the term “junk science,” here are some alternatives:

advertising

alternative medicine

alchemy

anti-vaccination movements

aroma therapy

astrology

baloney

blackguardism

blood letting

bogus science

bullshit

bunk

bunkum

cargo cult science

clinical ecology

cold fusion

creationism

cult

denialism

dodgy data

error

faith-based science

flim-flam

flotsam and jetsam

fraud

free-energy devices

fuzzy thinking

homeopathy

ignorance

intelligent design

junk science

Lysenkoism

magical thinking

magnetic therapy

miracles

misrepresentation

New Age science

nonsense on stilts

N rays

nostrums

not even wrong

paranormal phenomena

pathological science

phrenology

political science

propaganda

pseudoscience

pseudosymmetry

quackademic

parapsychology

quackery

rubbish

shamanism

spiritualism

voodoo science

Judicial Notice of Untruths

March 3rd, 2014

Judicial notice is a procedure for admitting facts the truth of which are beyond dispute. A special kind of magically thinking occurs when judges take judicial notice of falsehoods, myths, or lies.

In the federal judicial system, Federal Rule of Evidence 201 addresses judicial notice of adjudicative facts, and provides:

(b) Kinds of Facts That May Be Judicially Noticed. The court may judicially notice a fact that is not subject to reasonable dispute because it:

(1) is generally known within the trial court’s territorial jurisdiction; or

(2) can be accurately and readily determined from sources whose accuracy cannot reasonably be questioned.

Procedurally, Rule 201 provides that a court must take judicial upon the request of a party who has supplied any needed basis for the fact to be noticed.  A court may take notice sua sponte.  Rule 201(c)(1), (2).

In the Chantix litigation, counsel for Pfizer challenged plaintiffs’ expert witness, Curt Furberg, on Rule 702 grounds.  According the MDL judge, the Hon. Inge Prytz Johnson, Pfizer asserted that Furberg’s proferred testimony because the FDA approved Chantix as safe and effective. In re Chantix (Varenicline) Prods. Liab. Litig., 889 F. Supp. 2d 1272, 1285 n.8 (N.D. Ala. 2012).  Citing no authority or text, Judge Johnson announced that “[a]pproval by the FDA is not evidence of the safety of a medication.” Id.

To be sure, safety issue can sometimes arise after initial approval, but before the FDA or the manufacturer and sponsor of the medication can react to the new safety data.  The sweeping statement, however, that the FDA’s approval is not any evidence of safety seems bereft of factual support and common sense.

Judge Johnson went on, however, to invent supporting evidence out of thin air:

“The court takes judicial notice of such things as that at one time, thalidomide was used for morning sickness in pregnant women. Unfortunately, 10,000 children were born with birth defects from it before it was banned. And 50  years elapsed before doctors understood why thalidomide caused limbs to disappear. See e.g. http://www.nytimes.com/2010/03/16/science/16limb.html?pagewanted=all. Similarly, the fact that the FDA at one time approved Vioxx did not prevent the same being removed from the market due to growing concerns that it increased the risk of heart attacks and strokes. http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/ucm103420.htm. Hence, initial approval by the FDA is not proof of the safety of a medication.”

The point about the FDA’s approval not constituting evidence of safety may simply be sloppy writing and reasoning.  In the quote above, perhaps Her Honor merely meant to say that initial approval is not evidence that a medication is safe in view of later obtained data that were not available to the FDA on its review of the new drug application.  If so, fair enough, but the sweeping statement that the initial approval is no evidence of safety ignores the considerable time, cost, and energy that goes into the FDA’s review of safety before agency approves marketing.

More egregious, however, is Judge Johnson’s taking judicial notice of the marketing of thalidomide as though it had some relevancy and probative value for her claim about the inefficacy of the FDA’s safety reviews.[1]  Consider the recent review of the FDA’s handling of thalidomide by Margaret Hamburg, M.D., Commissioner of the U. S. Food and Drug Administration:

“Fifty years ago, the vigilance of FDA medical officer Dr. Frances Kelsey prevented a public health tragedy of enormous proportion by ensuring that the sedative thalidomide was never approved in the United States.  As many remember, in the early 1960’s, reports were coming in from around the world of countless women who were giving birth to children with extremely deformed limbs and other severe birth defects.  They had taken thalidomide. Although it was being used in many countries, Dr. Kelsey discovered that it hadn’t even been tested on pregnant animals.”

Margaret Hamburg, “50 Years after Thalidomide: Why Regulation Matters” (Feb. 7, 2012).

Judge Johnson took judicial notice of a non-fact. The FDA never approved thalidomide for use in the United States, back in the 1950s or 1960s.[2]



[1] Judge Johnson’s fantastical history of the FDA was recently cited by plaintiffs’ counsel in the Zoloft birth defects litigation.  See Plaintiffs’ Opposition to Defendants’ Motion to Exclude the Testimony of Anick Berard, Ph.D., at 13 (Filed Feb. 24, 2014), in In re Zoloft (sertraline hydrochloride) Prods. Liab. Litig., Case 2:12-md-02342-CMR Document 713.

[2] Judge Johnson’s errant history may have resulted from her European perspective of the thalidomide tragedy.  Judge Inge Prytz Johnson immigrated from Denmark, where she was born and educated. She became a U.S. citizen in 1978, and a state court judge one year later.  In 1998, she was nominated by President Clinton to the Northern District of Alabama.  In October 2012, Judge Johnson assumed senior status. See Kent Faulk, “U.S. District Judge Inge Johnson goes into semi-retirement” (Oct. 19, 2012) (quoting Judge Johnson as saying that “One thing I like about my job is I don’t have to take sides.”)

The Historical Intersection of Law and Epidemiology: Miller v National Cabinet (NY Court of Appeals 1960)

January 3rd, 2014

The history of statistics, epidemiology, and products liability are intertwined in ways that call for greater attention.  The 1950s and 1960s witnessed increasingly sophisticated statistical approaches to epidemiologic evidence. Starting in 1950, and continuing throughout the 1950s, Sir Richard Doll and Sir Austin Bradford Hill began their epidemiologic exploration of lung cancer among smokers. See, e.g., Richard Doll & A. Bradford Hill, “Smoking and Carcinoma of the Lung,” 2 Br. Med. J. 739 (1950); Richard Doll & A. Bradford Hill, “The mortality of doctors in relation to their smoking habits; a preliminary report,” 1 Br. Med. J. 1451 (1954).  In 1955, Sir Richard Doll published his important paper that suggested an association between asbestosis and lung cancer.  Richard Doll, “Mortality from Lung Cancer in Asbestos Workers,”  12 Br. J. Indus. Med. 81 (1955).  No disparity between observed and expected rates of lung cancer was observed among workers without asbestosis. Measures of p-values were used to assess the strength of the evidence against a null hypothesis of no association. As important an advance as was Doll’s paper, and as careful an investigator as he was, it is remarkable that Doll neglected to consider the potential role of smoking in producing the excess lung cancer rates among the factory workers with asbestosis. 

Starting in the 1960s, Dr. Irving Selikoff began publishing his epidemiologic studies of American asbestos insulators. See, e.g., Irving J. Selikoff , Jacob Churg,  and E. Cuyler Hammond, “Asbestos exposure and neoplasia,” 188 J. Am. Med. Ass’n 22 (1964). Selikoff neglected to stratify his observational data by the presence or absence of clinical asbestosis (although his later studies suggested that there was a very high prevalence of asbestosis after 20 years from first employment).  In addition, these insulator studies used crude measures of smoking, which lumped the very rare non-smoking insulators in with those who “never smoked regularly.”

In 1965, Sir Austin Bradford Hill published his lecture to the Royal Society of Medicine, in which he gave a spirited defense of inferring causality from observational epidemiologic studies. Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295 (1965). Hill was justly proud of the success of observational epidemiology, at least for very large effect sizes that made residual confounding and bias unlikely.

The same years as Hill’s lecture, the American Law Institute published the Restatement (Second) of Torts, with its controversial Section 402A. Before 1965, employee-plaintiff lawsuits against remote suppliers of raw materials and products to employers were a rarity in American law.  Bradford Hill’s lecture on causal assessments of “clear-cut” statistical associations came just as epidemiologic, statistical evidence was working its way into tort cases involving smoking and lung cancer.  Not surprisingly, some of the first uses of epidemiologic evidence occurred in cases involving claims that tobacco caused lung cancer. See, e.g., Lartigue v. R.J. Reynolds Tobacco Co., 317 F.2d 19 (1963) (affirming defense verdict in case noted for plaintiffs’ use of epidemiologic evidence) (“The plaintiff contends that the jury’s verdict was contrary to the manifest weight of the evidence. … The jury had the benefit of chemical studies, epidemiological studies, reports of animal experiments, pathological evidence, reports of clinical observations, and the testimony of renowned doctors. The plaintiff made a convincing case, in general, for the causal connection between tobacco and cancer and, in particular, for the causal connection between Lartigue’s smoking and his cancer. The defendants made a convincing case for the lack of any causal connection.”), cert. denied, 375 U.S. 865 (1963), and cert. denied, 379 U.S. 869 (1964).

Epidemiologic and statistical evidence in tort cases has become a commonplace, even when it is distorted and abused by litigants and judges. Recent decisions involving claims that benzene caused various cancers are illustrative.  See, e.g., Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009) (granting motion to exclude opinions that substantially distorted epidemiologic evidence under the vague rubric of “weight of the evidence”), rev’d, 639 F.3d 11 (1st Cir. 2011) (closing off scrutiny of expert witness’s abuse of epidemiologic evidence in one of the most controversial, reactionary decisions involving federal gatekeeping decisions of recent years), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). See also David E. Bernstein, “The Misbegotten Judicial Resistance to the Daubert Revolution,” 89 Notre Dame L. Rev. 27 (2013); “WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name” (Sept. 2, 2011); “Milward — Unhinging the Courthouse Door to Dubious Scientific Evidence” (Sept. 2, 2011)

Given the problematic Milward decision, we might wonder what challenges to benzene-leukemia cases looked like just before Hill’s defense of inferring causality from observational studies began to infuse the witches’ brew of Rule 402A. What did statistical evidence look like before Hill’s paper?  In court cases, typically, statistical evidence was presented crudely or not at all.

In 1960, there was little opportunity to challenge causation opinions on admissibility grounds; rather sufficiency of the evidence to support a verdict or judgment was the primary means to gain review of an adverse decision.  Case reports of leukemia in workers very heavily exposed to benzene appeared in the 1920s, but it was not until the 1960s that analytical epidemiologic evidence (case-control and cohort studies) of association between leukemia and benzene were published. See generally Deborah Glass, Christopher Gray, Damien Jolley, Carl Gibbons, and Malcolm Sim, “The health watch case – control study of leukemia and benzene: the story so far,” 1076 Ann. N.Y. Acad. Sci. 80-89 (2006).  Thus, when the New York Court of Appeals decided a case involving a claim of benzene-induced leukemia, in 1960, the judicial decision was driven largely by the absence of specific quantification of risk of leukemia among workers occupationally exposed to benzene.[1]  Miller v. National Cabinet Co., 8 N.Y.2d 277, 281, 168 N.E.2d 811, 813, 204 N.Y.S.2d 129, 132, modified on other grounds, 8 N.Y.2d 1025, 70 N.E.2d 214, 206 N.Y.S.2d 795 (1960). The New York high court wrestled with the formalistic aspects of the expert witnesses’ testimony, including whether they expressed themselves in terms of “possibilities” or “probabilities.” Miller, 8 N.Y.2d at 284, 168 N.E.2d  at 814-15, 204 N.Y.S.2d at 134.

Focusing in one of the more knowledgeable of plaintiffs’ expert witnesses, Dr. Reznikoff, the Miller court was impressed that this witness disclaimed any intent to support an inference, from statistical analyses, to the plaintiff’s decedent.  Id. at 283. Furthermore, the court suggested that Reznikoff’s evidence might have been sufficient were it not for concession:

“I am sorry I can’t give you any statistics, but we don’t have them.”

Id. at 283.  The court appeared also to be concerned that Reznikoff’s approach provided no mechanistic insight or understanding into why many cases of  leukemia followed benzene exposure. Id.  

Reznikoff’s qualifications to speak to the subject were not dispositive of the question; the court was looking for data that were not available in the 1950s, when the case was tried:

“Not every supposition of a witness concerning what might be has the force of evidence, even though he has been licensed to practice medicine. If the witness is unfamiliar with any statistical data in the medical literature or in his own practice to give an inkling either to himself or to the court or board of how high the incidence of these cases is in situations of this kind, then the doctor’s assumption that it is ‘quite high’ is without significance. The lack of any kind of statistical data, which in the absence of scientific understanding is all that there would be to go on, is the more inexplicable if the claim is well founded in view of the large number of persons who die of leukemia and of workers in industry who are exposed to benzol. If there were any observed correlation between the two, it is certain that a physician of Dr. Reznikoff’s standing would be in possession of the information.”

Id. at 283-84. The court did not excuse the claimant’s evidentiary display with the indulgent, “this was the best evidence available,” when the evidence was inadequate.  Nor did the court engage in soothsaying that causality would someday be demonstrated.  We might feel some frustration today, looking back, that the court missed this opportunity, but case reports, and even case reports and epidemiologic studies, have generated many false-positive associations.  Clearly, more is required, and the New York Court of Appeals recognized the necessity for more.

Traumatic Cancers Distinguished

In 1960, the courts still indulged the proto-scientific opinion that traumatic injury caused cancer.[2]  Some medical writers supported this opinion, but by 1960, the opinion was already falling out of favor due to an improved understanding of carcinogenesis.

There is much irony, therefore, in the Miller court drawing’s an invidious distinction between Reznikoff’s proto-epidemiologic evidence and traumatic cancer cases that were still prevalent in the 1960s.  Id. at 285-86. In the traumatic cancer cases, in the 1960s, and even in the 1970s, courts sustained verdicts for cancer claimants who had shown that their cancers were diagnosed very shortly after a traumatic blow to the precise portion of the body where cancer manifested. The Miller court referred to these traumatic cancer cases as presenting the kind of causal inferences that could be understood and made by judges and juries.  Today, 40 years later, we see those causal inferences as mostly rubbish, based upon incorrect, inadequate, and discarded theories of carcinogenesis.

The prospect of cancer cases sustained by epidemiologic (statistical) evidence clearly troubled the New York court:

“The courts have been confronted before with cancer cases, and this is not likely to be the last. This is not an isolated situation. Questions of causation are common to actions based on warranty, tort or workmen’s compensation proceedings. Would, for example, evidence that there are 4 to 11 times as many cases of lung cancer among cigarette smokers as among nonsmokers be sufficient to establish a cause of action for breach of warranty in the sale of cigarettes? … There appear to be no decisions upholding causation in so complex a variety of the disease as leukemia. The cancer decisions in the courts where recovery has been allowed have dealt almost entirely with trauma, and there only in instances where the trauma occurred in the spot in the body where the pre-existing cancer was and the symptoms of its aggravation were immediately apparent … . In all of those cases the immediacy of the symptoms of aggravation of the cancer by a traumatic injury suffered in the area where the cancer was located was accepted as a substitute for scientific evidence or understanding of cause and effect. Absent that, damage claims of this nature have been dismissed on the law for lack of evidence of causation.”

Id. at 285-86 (internal citations omitted). The Miller court went on to note that New York law required that the cancer must develop at the exact location of the injury.  Furthermore, latency between the traumatic blow and the clinical recognition of the cancer was fatal to the claim, even in the face of opinion testimony that a plaintiff’s cancer was a “very slow growing” tumor.  Today, we understand that latency between first-exposure and clinical manifestation is necessitated by the length of induction periods and the doubling time of solid tumor cancers.  As a result of the Miller court’s reliance upon some dodgy notions of cancer causation, it held that Mr. Miller’s latency period disqualified the case from the immediate impact rule of traumatic cancer cases. Id. at 287-88 (distinguishing Hagy v. Allied Chemical & Dye Corp., 122 Cal. App. 2d 361, 265 P.2d 86 (1953), which involved a diagnosis of laryngeal cancer following immediately upon exposure to sulfuric acid mists).

The majority in Miller further expressed its concern that the understanding of cancer causation was marked by such uncertainty that the mere possibilities of chemical carcinogenesis should not be tolerated in this and similar cases:

“… [F]or so long as the causes of a disease — like cancer — are unknown to science, everyone contracting the disease could secure medical testimony that it is ‘possible’ that the disease is contracted from a wide variety of causes, choosing in each instance the particular possibility having the greatest promise of holding liable some responsible defendant. Any cancer expert could readily state that cancer could be caused by virus infection or by exposure to automobile exhaust fumes, sunlight, radiation, smog, smoking, hormone imbalance or according to any other theory which has been entertained by researchers or specialists as a possibility. Is a malpractice suit pending against some doctor who has given cortisone or ACTH as medicine? Then appears a medical witness who testifies that possibly cancer is caused by hormone imbalance induced thereby. Is it an action for breach of an implied warranty in the sale of cigarettes? Then the medical witness will testify that cigarettes could be a cause of lung cancer. Is it X ray or working in a garage where there have been exhaust fumes? Then the ‘possibility’ doctrine is adapted to creating questions of fact in those fields — and the same with benzol exposure and leukemia. Such a doctrine would overturn the rule that the burden is on the party asserting that a disease is based on actionable facts to prove causation. It would mean that, wherever such a cause is possible, the burden rests on the opposite party to prove that the disease resulted from something else. Consequently, for so long as the causes of the disease are unknown to medical science, the claimant or plaintiff can always recover — if the trier of the fact is favorably disposed — since no one can prove that the disease had other causes. This is a perversion of the normal rule that the disease must have resulted from the occupation and that the burden of proving causation is upon the party asserting it. The law does not intend that the less that is known about a disease the greater shall be the opportunity of recovery in court.”

Id. at 289.

The Miller decision provoked a dissent, mostly on formalistic grounds.  Id. at 290. The dissenting judges asserted, without much analysis, that there was substantial evidence to support causality. Given that qualified expert witnesses showed up for the claimant seemed sufficient on this score for the dissenters.  To the extent that the claimant’s expert witnesses expressed themselves in terms of possibilities, the dissenters opined that possibilities are sufficient, especially in the context of workman’s compensation cases, in which the burden of proof standards are lower than in common law civil liability cases.

The majority opinion stands as an eloquent expression of concern about the need for quantitative evidence of statistical risk in chemical exposure cancer cases. The court also presciently saw what would become a plague of litigation involving claims of cancer causation.  In 1960, for benzene and leukemia, the evidence was clearly, even by the standards of the day, inadequate, and the claimant’s expert witnesses were appropriately modest about what inferences could be drawn both with respect to general and specific causation.  The 1970s would witness a growing immodesty among available expert witnesses, as well as an explosive growth in the techniques and applications of analytical epidemiology to many problems, including the relationship between benzene and leukemia.



[1] A decade or two later, the scientific community recognized high levels of exposure to benzene as a cause of certain kinds of leukemia, by virtue of epidemiologic studies. See, e.g., Fusun Yaris, Mustafa Dikici, Turhan Akbulut, Ersin Yaris, Hilmi Sabuncu, “Story of benzene and leukemia: epidemiologic approach of Muzaffer Aksoy,” 46 J. Occup. Health 244 (2004); Abdul Khalade, Maritta S Jaakkola, Eero Pukkala and Jouni JK Jaakkola, “Exposure to benzene at work and the risk of leukemia: a systematic review and meta-analysis,” 9 Envt’l Health 31 (2010).  See also Michael D. Green, The Paradox of Statutes of Limitations in Toxic Substances Litigation, 76 Cal. L. Rev. 965, 974 (1988).

[2] William B. Coley & Norman L. Higinbotham, “Injury as a Causative Factor in the Development of Malignant Tumors,” 98 Annals of Surgery 991 (1933); Shields Warren, “Minimal Criteria Required to Prove Causation of Traumatic or Occupational Neoplasms,” 117 Annals of Surgery 585 (April 1943); Shields Warren, “Criteria Required to Prove Causation of Occupational or Traumatic Tumors,” 10 U. Chi. L. Rev. 313, 318-20 (1943); Russell & Clark, “Medico-Legal Considerations of Trauma and Other External Influences in Relationship to Cancer,” 6 Vand. L. Rev. 868, 875 (1953); Arden R. Hedge, “Can a Single Injury Cause Cancer?” 90 California Medicine 55 (1959); Auster, “The Role of Trauma in Oncogenesis: A Juridical Consideration,” 175 J. Am. Med. Ass’n 940, 949 (1961); Comment, “Sufficiency of Proof in Traumatic Cancer Cases,” 46 Cornell L.Q. 581, 581-82 (1961); Comment, “Sufficiency of Proof in Traumatic Cancer: A Medico-Legal Quandary,” 16 Arkansas L. Rev. 243, 256 67 (1962); Dyke, “Traumatic Cancer,” 15 Clev.Mar. L. Rev 472, 484-94 (1977).  See also Comment, “Judicial Attitudes Towards Legal and Scientific Proof of Cancer Causation,” 3 Columbia J. Envt’l L. 344, 354-68 (1977).

 

Gastwirth on the Wells Case

December 27th, 2013

One of the high points of 2013 for me was the Practicing Law Institute seminar on expert witness gatekeeping.  I am indebted to my co-chair David Cohen, and the talented participants, Dr. David Garabrant, Joe Cecil, Mary Wells, and Mike Williams.  I learned a great deal from all of them, even when we disagreed.

At the seminar, which addressed the slippage in the federal judiciary’s fidelity to Rule 702 text and principles, Joe Cecil kindly reminded me of Professor Gastwirth’s contribution on the Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262, 298 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).  Joseph L. Gastwirth, “The need for careful evaluation of epidemiological evidence in product liablility cases: a reexamination of Wells v. Ortho and Key Pharmaceuticals,” 2 Law, Probability and Risk 151 (2003).

The Supreme Court recently reinvigorated the Wells case, and like Frankenstein’s monster, it walks again.  Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011).  See alsoMatrixx Unloaded.”  As a result of the Matrixx decision, I revisited the untoward Wells decision several times this year. SeeWells v. Ortho Pharmaceutical Corp. Reconsidered – Part 1”; and here, here, here, here, and here.  Joe’s reminder, however, provoked me to look one more time at Wells, through the lens of Gastwirth’s review.

Gastwirth is a well-known statistician, not a lawyer.  His article undertakes some interesting sensitivity analyses of the potential role of one known confounder in the epidemiologic studies of spermicides and birth defects, cited by witnesses in Wells. Not surprisingly, Gastwirth confuses important legal concepts, when he contends that science and law have different goals because science has the luxury of waiting for additional studies to resolve indeterminate datasets.  Gastwirth at 154. Gastwirth suggests that in law, causality determinations turn on fairness, but in science, they turn on the truth.  Gastwirth offers no support for his assertion, and there is much law to the contrary.  Federal Rule of Evidence 102, for instance, explains the “Purpose” of the Federal Rules and how they should be construed:

“These rules should be construed so as to administer every proceeding fairly, eliminate unjustifiable expense and delay, and promote the development of evidence law, to the end of ascertaining the truth and securing a just determination.”

Truth does count, at least in some courts, and it is hard to imagine a case in which the goals of truth and justice are bifurcated. Gastwirth sees law courts as unable to avoid judgment when the evidentiary display is weakly developed or not replicated, but he is wrong.  Courts can, do, and should avoid permitting judgments based upon indeterminate datasets.

Gastwirth also contends that the duty to warn in tort is based upon a possibility of causation, and he opines that this is perfectly satisfactory.  Gastwirth at 155.  Gastwirth cites the Restatement (Third) a/Torts for his view, but the Restatement does not couch the duty to warn as predicated upon merely possible risk:

“A defendant will not be liable under an implied warranty of merchantability for failure to warn about risks that were not reasonably foreseeable at the time of sale or could not have been discovered by way of reasonable testing prior to marketing the product. A manufacturer will be held to the standard of knowledge of an expert in the appropriate field, and will remain subject to a continuing duty to warn of risks discovered following the sale of the product at issue.”

Id. (emphasis added).  Gastwirth fails to explain why the epistemic standard of the Restatement should be diluted to include merely possible risks; nor does he explain why the law should impose a duty because a minority of so-called experts claim knowledge of the existence of a risk, whether potential or known.

Risks are based upon known causal relationships, viewed ex ante rather than post hoc.  Gastwirth attempts to defend the Wells decision on the basis that the trial judge, Judge Shoob, could have correctly found a possible risk, but this was not then or now the standard for imposing a duty to warn.  Even if possibilities were the basis for a duty to warn, the courts in Wells had to find sufficient evidence of cause in fact between the spermicidal jelly and the birth defect present in the infant plaintiff.  No merely possible risk could have satisfied this burden, and Gastwirth avoids this important issue in his review.  Ultimately, Gastwirth fails to come to grips with how the Wells courts artificially distinguished two causation standards – warning-harm and product-harm.  Gastwirth at 184.

When courts are required to work with an epistemic standard for evaluating opinions about causal claims, they will often confront sincere beliefs supported by “putative” justifications, but which turn out to be untrue.  The obvious implication is that the causal opinions were not “true” when offered, and the asserted, putative justification was no justification at all.  Twenty years post-Daubert should have made our courts more sensitive to the pathology of claiming, or pathoepistemology, as I call it.

Although Gastwirth is intent to show that the district and circuit court decisions in Wells were reasonable given the record evidence at the time of their decisions, Gastwirth’s defense is tepid at best. Gastwirth concedes that the plaintiffs’ evidence in the Wells case was not very convincing, and he is quick to point out that he does not conclude that spermicides cause limb reduction defects (although he does believe that the data were sufficient for his idiosyncratic diluted duty-to-warn legal standard:

 “The epidemiological evidence in the Wells case surely was not very convincing and it is understandable that scientists and legal scholars might desire stronger scientific evidence before holding a defendant liable. What seems unfortunate is the lack of discussion of the part of the record most supportive of the legal decisions from the published criticisms.201 While it is reasonable to ask how much evidence should be required before a warning is required, is it fair to rely on studies that were published after the time of exposure to assess the producer’s duty at the time?”

Gastwirth at 184-85.  The equivocation in this passage between assessing causation for a duty to warn and for product harm is typical of Gastwirth’s confusion throughout this article.  As noted, Gastwirth appears to believe it is sufficient to show the plausibility of the causal relationship such that the trial court’s findings of a duty to warn, and a breach of that duty, were reasonably supported:

“It should be stressed that our examination of the studies does not conclude that exposure to spermicides causes limb reduction defects. 202 Indeed, the studies published in the 1980s are sensitive to the potential effect of unmeasured known potential risk factors that were not controlled for in the analysis. Many studies observed that it is difficult to rule out small but meaningful risks given their sample size or that more studies including potential confounding variables be carried out before a scientific conclusion can be reached.203

Gastwirth at 185 & n.202.  So small increased risks were difficult to rule in or rule out, but then how can a judgment for plaintiffs be sustained unless, as I have shown elsewhere, Judge Shoob improperly placed the burden of proof upon the defendant?  The studies published in the 1980s may have been sensitive to the potential effect of unmeasured and uncontrolled known potential risks, but Judge Shoob clearly was not sensitive to the problem at all.

Other problems abound. The studies involved in the Wells case not only failed to account for the few known confounders at the time, but they could not and did not sort out the residual confounding.  Gastwirth concedes the point in a footnote, in which he cites a paper by Professor Gary Shaw on the role of multivitamin use in preventing birth defects. See Gastwirth at 185 & n. 202 (citing Shaw et al., “Maternal Periconceptional Use of Multivitamins and Reduced Risk for Conotruncal Heart Defects and Limb Deficiencies Among Offspring,” 59 Am. J. Med. Genetics 536 (1995) (showing that children of mothers who had taken vitamins and folic acid had a 30-35% lower risk of limb defects).  Gastwirth sees the discovery of causes not previously controlled for in the spermicide/birth defect studies of the 1980s as example of the differences between scientific and legal judgments.  Another way to look at such discoveries is to impose some standard of intellectual modesty upon courts in basing their findings upon speculative causal claims.

Gastwirth’s casual dismissiveness of the need for replication in a legal standard of causation is difficult to square with his acknowledgment that early studies often prove misleading about the existence of a true risk.  Gastwirth, for instance, acknowledges that later publications quite commonly do not support initial hypotheses.  Gastwirth at 179, 184 (citing Michael B. Bracken, “Spermicidal Contraceptives and Poor Reproductive Outcomes: The Epidemiologic Evidence Against an Association,” 151 Am. J. Obstet. & Gyncecol. 552, 555 (1985)).  Gastwirth channels the anti-manufacturer Zeitgeist for his claim that “firms” may not report studies that show risk, but he ignores the pervasive, prevalent practice of academic and advocacy scientists’ not publishing negative studies in the first place, and even later in the natural history of scientific controversies.

Ultimately, however, Gastwirth’s own judgment is a self-fulfilling prophecy of how the law and science should part company over false-positive assertions of causality.  Fortunately, the law has evolved in a different direction, towards convergence of scientific and legal modes of assessing causal claims.  Now judges must evolve as well.

Pennsylvania Superior Court Takes The Bite Out of Fixodent Claims

December 12th, 2013

In the spring of 2012, Judge Sandra Mazer Moss granted summary judgment to Proctor & Gamble, after excluding, on Frye grounds, plaintiff’s expert witnesses who opined that plaintiff suffered zinc neurotoxicity from his use of FixodentJacoby v. Rite Aid Corp., 2012 Phila. Ct. Com. Pl. LEXIS 208 (2012).  SeePhiladelphia Plaintiff’s Claims Against Fixodent Prove Toothless” (May 2, 2012).  Judge Moss’s exclusion of plaintiff’s expert witnesses involved a careful analysis of the evasive, hand-waving tactics of the witnesses.  Among the plaintiff’s team of expert witnesses was Dr. Martyn Smith, the chief hand waver and obscurantist in Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009), rev’d, 639 F.3d 11 (1st Cir. 2011), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012).

On Monday, December 9, 2013, after a careful review, the Pennsylvania Superior Court affirmed summary judgment for Proctor & Gamble in the Jacoby case. Jacoby v. Rite Aid Corp., Pa. Super. Ct. No. 1508 EDA 2012 (Dec. 9, 2013) [Slip op.]  The Superior Court panel, consisting of Judges Stevens, Lazarus, and Colville, largely adopted Judge Moss’s analysis and affirmed in a signed, but unpublished, opinion by Judge Lazarus. 

Like Judge Moss before them, the Panel saw through the attempt to pass off “Weight of the Evidence” (WOE) and “Totality of the Evidence” (TOE) as scientific methodologies.  The witnesses, Martyn Smith and others, failed to specify what evidence they weighed, how they weighed the evidence, and what the weights supposedly were.  Another expert witness vaguely pointed to the “Naranjo scale” in support of interpreting case reports to show causal association, but this scale was similarly incompetent other than as a crude “plausibility” scale for assessing case reports.

The Superior Court’s decision in Jacoby is noteworthy on several important issues. There was no material issue as to whether zinc at some dose and duration of exposure can cause neuropathies.  The Court saw, however, that the important issue was whether zinc in the form, dose, and duration ingested by plaintiff can cause the outcome he experienced, and whether his exposure to zinc in Fix-o-dent actually caused his alleged injury.  The Superior Court re-affirmed Pennsylvania’s case law that makes extrapolation from different doses, different durations, and different biological circumstances, a “novel” claim that is subject to the gatekeeping by the so-called Frye standard. Slip op. at 11.

The Superior Court’s opinion astutely observed that the issue was not whether WOE and TOE are accepted scientific methodologies, but whether expert witness Martyn Smith can “evade a reasoned Frye inquiry merely by making reference to accepted methods in the abstract.”  Id. at 12 (citing Betz, at 58).  When pressed, Martyn Smith’s invocation of WOE amounted to little more than a distortion and abridgment of the Bradford Hill factors.  The Superior Court recognized, however, that the Bradford Hill guidelines provide an evaluative process to consider whether an association, after first being shown to be clear cut and not attributable to chance, is causal or spurious.  Id. at 13.  As Bradford Hill postulated the question that arises before his famous nine factors come into the analysis:

“Disregarding then any such problem in semantics we have this situation. Our observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance. What aspects of that association should we especially consider before deciding that the most likely interpretation of it is causation?”

Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965) (emphasis added).

In this case, Smith never got off the dime with his evasive tactics.  He did not identify a quantified association that could be assessed for the play of random variation, or bias and confounding.  Smith tried to suggest that a mere possibility of an association was sufficient to invoke the Bradford Hill guidelines, but the Superior Court rejected this attempt to invent a new-age scientific method. Slip op. at 13.

The Superior Court acknowledged that the scientific literature describes WOE as varying from nothing more than “seat-of-the-pants qualitative assessment” to “aggregating diverse modalities.”  Id. at 14.  In Smith’s hands, WOE was little more than a personal, subjective opinion, an ipse dixit dressed as a scientific opinion.  Smith never defined his WOE approach, and the other witnesses never defined their TOE approach.  The plaintiffs’ witnesses in Jacoby failed to offer an accepted methodology when they failed to identify the forms of evidence they considered, and how they went about weighing the evidence upon which they had relied.  Id. at 15.

In a brief discussion, the Panel also embraced another basic evidentiary principle to dismiss a common tactic in specious claiming. The plaintiffs’ challenged defendants’ pharmokinetic study and tried to suggest that their deconstruction counted as affirmative evidence to support their own theory of biological fate and distribution. Id. at 16.  The Superior Court saw through the ruse; the plaintiffs had not created affirmative evidence for their theory by arguing that the defendants’ study was flawed. 

The Superior Court squarely confronted the limitations and inadequacy of relying upon descriptive, anecdotal case reports. Case reports provide a narrative and temporal history of events with respect to exposure and outcome, but they cannot fully account for confounding by known and unknown factors.  Case reports represent post hoc assessments that were not planned, and therefore lack data that would permit distinguishing coincidence from causality.  Id. at 17 (citing Dr. Lorene Nelson’s report).  See In re Denture Cream Prods. Liab. Litig., 795 F. Supp. 2d 1345 (S.D. Fla.2011).

Proctor & Gamble had the good fortunate to have obtained a good ruling in the MDL litigation in the Southern District of Florida, which no doubt helped focus the gatekeeping process in Pennsylvania state court. Unfortunately, the Superior Court Panel chose not to publish its decision.  This decision is regrettable for its inconsistency with the transparency and due process expected of all courts.  See Erica Weisgerber, “Unpublished Opinions: A Convenient Means to an Unconstitutional End,” 97 Georgetown L.J. 621 (2009).