Back in March, I wrote about a “Black Swan” case, in which litigants advanced a Bayesian analysis to support their claims. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).
Goodman was a complex medical practice case in which Mrs. Goodman alleged that her obstetrician, Dr. Johan Viljoen, deviated from the standard of care by failing to prescribe antenatal corticosteroids (ACS) sufficiently in advance of delivery to reduce the risks attendant early delivery for her twin boys, of early delivery. Both boys developed cerebral palsy (CP). The parties and their experts agreed that the administration of ACS reduced the risks of respiratory distress and other complications of pre-term birth, but they disputed the efficacy of ACS to avoid or diminish the risk of CP.
According to the plaintiffs, ACS would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition. Dr. Viljoen disputed both general and specific causation. Evidence of general causation came from both randomized clinical trials (RCTs) and observational studies.
Limitations Issue
There were many peculiar aspects to the Goodman case, not the least of which was that the twins sued Dr. Viljoen over a decade after they were born. Dr. Viljoen had moved his practice in the passage of time, and he was unable to produce crucial records that supported his account of how his staff responded to Mrs. Goodman’s telephone call about signs and symptoms of labor. The prejudice to Dr. Viljoen illustrates the harshness of broad tolling statutes, the unfairness of which could be reduced by requiring infant plaintiffs to give notice of their intent to sue, even if they wait until the age of majority before filing their complaints.
State of the Art Issue
Dr. Viljoen suffered perhaps a more serious prejudice in the form of hindsight bias that resulted from the evaluation of his professional conduct by evidence that was unavailable when the twins were born in 1995. The following roughly contemporaneous statement from the New England Journal of Medicine is typical of serious thinking at the time of the alleged malpractice:
“Antenatal glucocorticoid therapy decreases the incidence of several complications among very premature infants. However, its effect on the occurrence of cystic periventricular leukomalacia, a major cause of cerebral palsy, remains unknown.”
Olivier Baud, Laurence Laurence Foix l’Hélias, et al., “Antenatal Glucocorticoid- Treatment and Cystic Periventricular Leukomalacia in Very Premature Infants,” 341 New Engl. J. Med. 1190, 1190 (1999) (emphasis added). The findings of this observational study illustrate some of the difficulties with the claim that Dr. Viljoen failed to prevent an avoidable consequence of pre-term delivery:
“Our results suggest that exposure to betamethasone but not dexamethasone is associated with a decreased risk of cystic periventricular leukomalacia.”
Id. at 1194. Results varied among various corticosteroids, among doses, among timing regimens. There hardly seemed enough data in 1995 to dictate a standard of care.
Meta-Analysis Issues
Over ten years after the Goodman twins were born, the Cochrane collaboration published a meta-analysis that was primarily concerned with the efficacy of ACS for lung maturation. Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews Issue 3. Art. No. CD004454 (2006). The trials included mostly post-dated the birth of the twins, and the alleged malpractice. The relevance of the trials to address the causation of CP in infants who experienced periventricular leukomalacia (PVL) was hotly disputed, but for now, I will gloss over the external validity problem of the Cochrane meta-analysis.
The Cochrane Collaboration usually limits its meta-analyses to the highest quality evidence, or RCTs, but in this instance, the RCTs did not include CP in its primary pre-specified outcomes. Furthermore, the trials were generally designed to ascertain short-term benefits from ACS, and the data in the trials were uncertain with respect to longer-term outcomes, which may have been ascertained differentially. Furthermore, the trials were generally small and were plagued by sparse data. None of the individual trials was itself statistically significant at the 5 percent level. The meta-analysis did not show a statistically significant decrease in CP from ACS treatment. The authors reported:
“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”
Id. at 8 (emphasis added).
The Cochrane authors were appropriately cautious in interpreting the sparse data:
“Results suggest that antenatal corticosteroids result in less neurodevelopmental delay and possibly less cerebral palsy in childhood.”
Id. at 13-14 (emphasis added).
The quality of the trials included in the Cochrane meta-analysis varied, as did the trial methodologies. Despite the strong clinical heterogeneity, the Cochrane authors performed their meta-analysis with a fixed-effect model. The confidence interval, which included 1.0, reflected a p-value of 0.065, but that p-value would have certainly increased if a more appropriate random-effects model had been used.
Furthermore, the RCTs were often no better than observational studies on the CP outcome. The RCTs here perhaps should not have been relied upon to the apparent exclusion of observational epidemiology.
Relative Risk Less Than Two
There is much to be said about the handling of statistical significance, the Bayesian analysis, the arguments about causal inference, but for now, let us look at one of the clearest errors in the case: the inference of specific causation from a relative risk less than two. To be sure, the Cochrane meta-analysis reported a non-statistically significant 40% decrease, but if we were to look at this outcome in terms of the increase in risk of CP from the physician’s failure to administer ACS timely, then the risk ratio would be 1.67, or a 67% increase. On either interpretation, fewer than half the cases of CP can be attributed to the failure to administer ACS fully and timely in the case.
The parties tried their case before Justice Walters, in St. Catherines, Ontario. Goodman v. Viljoen, 2011 ONSC 821 (CanLII). Justice Walters recognized that specific causation was essential and at the heart of the parties’ disagreement:
“[47] In order to succeed, the plaintiffs must establish that the failure to receive a full course of ACS materially affected the twins’ outcome. That is, they must establish that “but for” the failureto receive a full course of ACS, the twins would not have suffered from the conditions they now do, or that the severity of these afflictions would have been materially reduced.
[48] Not surprisingly, this was the most contentious issue at trial and the court heard a good deal of evidence with respect to the issue of causation.”
One of the defendant’s expert witnesses, Robert Platt, a professor of statistics at McGill University School of Medicine, testified, according to Justice Walters:
“[144] Dr. Platt also stated that the absolute risk in and of itself does not tell us anything about what might have happened in a specific case absent clinical and mechanistic explanations for that specific case.”
The plaintiffs’ expert witnesses apparently conceded the point. Professor Andrew Willan, a statistician, testifying for the plaintiffs, attempted to brush Platt’s point aside by suggesting it would render clinical research useless, but that was hardly the point. Platt embraced clinical research for what it could show about the “averages” in a sample of the population, even if we cannot discern causal efficacy retrospectively in a specific patient:
“[133] Dr. Willan also responded to Dr. Platt’s criticism that it was impossible to determine the distribution of the effect across the population. Professor Willan felt this issue was a red herring, and if it were valid, it would render most clinical research useless. There is really no way of knowing who will benefit from a treatment and who will not. Unless there are reasons to believe otherwise, it is best to apply the population average effect to each person.”
Although Willan labeled Platt’s point as cold-blooded and fishy, he ultimately concurred that the population average effect should be applied to each person in the absence of evidence of risk being sequestered in a subgroup.
A closer look at Willan’s testimony at trial is instructive. Willan acknowledged, on direct examination, that the plaintiffs were at increased risk, even if their mother had received a full course of ACS. All he would commit to, on behalf of the plaintiffs, was that their risk would have been less had the ACS been given earlier:
“All we can say is that there’s a high probability that that risk would be reduced and that this is probably the best estimate of the excess risk for not being treated and I would say that puts that in the 70 percent range of excess risk and I would say the probability that the risk would have been reduced is into the 90 percentage points.”
Notes of Testimony of Andrew Willan at 62 (April 6, 2010). The 90 percentage points reference here was Willan’s posterior probability that the claimed effect was real.
On cross-examination, the defense pressed the point:
Q. What you did not do in this, in this report, is provide any quantification for the reduction in the risk, true?
A. That’s correct.
Notes of Testimony of Andrew Willan at 35 (April 9, 2010)
Q. And you stated that there is no evidence that the benefits of steroids is restricted to any particular subgroup of patients?
A. I wasn’t given any. I haven’t seen any evidence of that.
Id. at 43.
Q. And what you’re suggesting with that statement, is that the statistics should be generally, should be considered by the court to be generally applicable, true?
A. That’s correct.
Id. at 44.
Q. But given your report, you can’t offer assistance on the clinical application to the statistics, true?
A. That’s true.
Id. at 46.
With these concessions in hand, defense counsel elicited the ultimate concession relevant to the “but for” standard of causation:
Q. And to do that by looking at an increase in risk, the risk ratio from the data must achieve 2 in order for there to be a 50 percent change in the underlying data, true?
A. Yeah, to double the risk, the risk ratio would have to be 2, to double the risk.
Id. at 63.
* * *
Q. So, none of this data achieves the threshold of a 50 percent change in the underlying data, whether you look at it as an increase in risk or …
A. Sure.
Q …. a decrease in risk …
A. Yeah.
Id. at 66.
Leaping Inferences
The legal standard for causation in Canada is the same counterfactual requirement that applies in most jurisdictions in the United States. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), at ¶14, 47. The trial court well understood that the plaintiffs’ evidence left them short of showing that their CP would not have occurred but for the delay in administering ACS. Remarkably, the court permitted the plaintiffs to use non-existing evidence to bridge the gap.
According to Dr. Max Perlman, plaintiffs’ expert witness on neonatology and pediatrics, CP is not a dichotomous condition, but a spectrum that is manifested on a continuum of signs and symptoms. The RCTs relied upon had criteria for ascertaining CP and including it as an outcome. The result of these criteria was that CP was analyzed as a binary outcome. Dr. Perlman, however, held forth that “common sense and clinical experience” told him that CP is not a condition that is either present or not, but rather presented on a continuum. Id. at [74].
Without any evidence, Perlman testified that when CP is not avoided by ACS, “it is likely that it is less severe for those who do go on to develop it.” Id. [75]. Indeed, Perlman made the absence of evidence a claimed virtue; with all his experience and common sense, he “could not think of a single treatment which affects a basic biological process that has a yes or no effect; they are all on a continuum.” Id. From here, Perlman soared to his pre-specified conclusion that “that it is more likely than not that the twins would have seen a material advantage had they received the optimal course of steroids.” Id. at [76].
Perlman’s testimony is remarkable for inventing a non-existing feature of biological evidence: everything is a continuum. Justice Walters could not resist this seductive testimony:
“[195] The statistical information is but one piece of the puzzle; one way of assessing the impact of ACS on CP. Notably, the 40% reduction in CP attributable to ACS represents an all or nothing proposal. In other words, 93.5% of the time, CP is reduced in its entirety by 40%. It was the evidence of Dr. Perlman, which I accept, that CP is not a black and white condition, and, like all biological processes, it can be scaled on a continuum of severity. It therefore follows that in those cases where CP is not reduced in its entirety, it is likely to be less severe for those who go on to develop it. Such cases are not reflected in the Cochrane figure.
[196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.
[197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP. Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.”
Doubly unfortunate, because Perlman’s argument was premised on a counterfactual assumption. Many biological phenomena are dichotomous. Pregnancy, for instance, does not admit of degrees. Disease states are frequently dichotomous, and no evidence was presented that CP was not dichotomous. Threshold effects abound in living organisms. Perlman’s argument further falls apart when we consider that the non-experimental arm of the RCTs would also have had additional “less-severe” CP cases, with no evidence that they occurred disproportionately in the control arms of these RCTs. Furthermore, high-quality observational studies might have greater validity than post-hoc RCTs in this area, and there have been, and likely will continue to be, such studies to attempt better understanding of the efficacy of ACS, as well as differing effects among the various corticosteroids, doses, and patterns of administration.
On appeal, the Justice Walters’ verdict for plaintiffs was affirmed, but over a careful, thoughtful dissent. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) (Doherty, J., dissenting). Justice Doherty caught the ultimate futility of Dr. Perlman’s opinion based upon non-existent evidence: even if there were additional sub-CP cases in the treatment arms of the RCTs, and if they occurred disporportionately more often in the treatment than in the placebo arms, we are still left guessing about the quantitative adjustment to make to the 40% decrease, doubtful as it was, which came from the Cochrane review.
