So Ortho lost the liability issue in the Wells case, which turned on historical knowledge in 1980. How did Ortho lose medical causation?
The Wells case, as reported, is a fantastic if not terrifying insight into the mind of a judge, sitting as the trier of fact. It is not a legal determination of the reliability or validity of the expert witness opinions under Federal Rule of Evidence 702; nor is it an assessment of the reasonableness of any expert’s reliance upon any study or data. No Frye motions were made. (If such motions were made, the trial court did not reference them in its opinion.) Ortho defended with its expert witnesses, and argued its case as a battle of experts.
Many of the litigation and judicial decision-making themes on state of the art carried over to medical causation. Once Ortho lost the liability issue on such weak, controverted, and irrelevant evidence, its fate was likely sealed on causation, before Judge Shoob. The trial court, in resolving liability, had already determined that it would allow hints, hunches, and hypotheses to substitute for knowledge. Obscure and unpublished papers were accorded great weight, to the exclusion of peer-reviewed, published, data-driven, and carefully analyzed studies. Confounding in older studies with a mercury chemical not involved in Ortho’s product was ignored. Any evidence of fetal harm, even in non-validated animal models with extraordinary doses, was permitted to substitute for relevant congenital malformations. Exculpatory opinions, including those of the FDA’s Advisory Committee, were ignored.
In the litigation over medical causation, the parties agreed upon some facts and opinions. Timing of exposure within the embryological development is crucial to an assessment of causality. Some time windows are important, others irrelevant, to the development of specific malformations. Therefore, not only is the time window of exposure important, but so is the type of malformation at issue. Notwithstanding this agreement, the trial court was consistently vague in discussing the specific malformations in various studies. Not a single “effect size” is reported; and not a single p-value or confidence interval is indicated. There is no discussion or recognition of the role multiple testing and comparison may play in diluting the meaning of a particular p-value.
Physical Examinations
The court mentioned no less than four times that some of plaintiffs’ expert witnesses had Katie Wells, and that none of the defendant’s witnesses had. The court failed, however, to articulate what was learned in such examinations that could not be appreciated from the medical records themselves. This waving of hands about the “laying on of hands” reflects the trial court’s superficial approach to decision making.
Case Reports
The court mentioned that the plaintiffs’ expert witnesses relied upon the existence of case reports, but did not discuss what if any significance such case reports had for evaluating plaintiffs’ causal claims. 615 F. Supp. at 274.
Association Cannot Be Ruled Out — Further Research Is Needed
The trial court imposed a highly skewed approach in its evaluation of expert witnesses on either side of the case. One of the plaintiffs’ expert witnesses, Dr. John Holbrook had testified that “a definite relationship” existed:
“In Dr. Holbrook’s opinion, a ‘definite relationship’ exists between the use of spermicides and fetal malformations.”
615 F. Supp. at 275.
The court found Dr. Holbrook to be credible. Id. at 276. There are several interesting aspects of how the trial court reacted to such statements from the plaintiffs’ expert witnesses. First, the court tolerated an infuriating imprecision and inaccuracy in how the plaintiffs’ witnesses expressed their opinions. In the case of Dr. Holbrook, he was permitted to talk generally of “spermicides,” when it was clear that not all spermicides could be evaluated for causality together. Second, the court permitted Dr. Holbrook to speak of ill-defined “definite” relationship, without declaring whether that relationship was “causal,” or something else. When Holbrook, and the other plaintiffs’ witnesses, disagreed with the conclusions of articles upon which they relied, the trial court generally did not find that the discrepancies discredited the witnesses or their testimony.
The court’s approach to the plaintiffs’ witnesses should be compared with how it evaluated the defense expert witnesses’ testimony. Dr. Stolley was called by the defense, and he was the only witness who had any real training and expertise in epidemiology. Plaintiffs’ counsel questioned Dr. Stolley about statements made by Dr. Shapiro, an author of one of the studies relied upon by Dr. Stolley in his testimony. (Remarkably, and importantly, the trial court does not report what if anything the plaintiffs’ witnesses had to say about Dr. Shapiro’s study.)
The Shapiro study concluded as follows:
“We conclude that there is no satisfactory evidence to indicate that spermicides increase the overall risk of major birth defects. It is possible, however, that spermicides increase the risk of certain specific malformations. Our own study is not large enough to exclude such a possibility. To evaluate it, further studies are needed.”
Id. at 284. This sort of statement, an acknowledgement that further studies may be needed, is hardly an indication that the study had found an increased risk of a particular outcome. And yet, the trial court somehow thought that it was Dr. Stolley who equivocated by having relied upon the Shapiro study.
At the FDA hearing, Dr. Shapiro described his published study:
“There is one question that is unanswered and that we fully acknowledge, and that is that all of the cohort studies so far done, or the case control studies, for that matter, lack the power to evaluate in statistical terms that there is not an appreciable increase in the risk of specific birth defects.
….
I’m not for one moment claiming that this study rules out an increase in the risk of limb reduction deformities.”
Id. at 285 (emphasis added).
Dr. Stolley disagreed with Shapiro about a relatively unimportant linguistic issue of what the magnitude is of an “appreciable” increased risk; Dr. Stolley opined that the available studies could rule out an “appreciable increase” in risk. Note that Dr. Shapiro was addressing a power issue in his study, whereas Dr. Stolley was addressing the issue with respect to all the available studies. With respect to the supposed disagreement over “appreciable,” here is how Dr. Stolley expressed his view:
“I think that what [Shapiro] is saying is that, if there were a tiny increase in limb reduction defects, which is rare, you would need an extremely large study to absolutely rule that out and that can’t be done.
* * *
What he was trying to do is address the question of ruling out very, very tiny risks, and I think that I would agree that it’s very difficult to do that.
* * *
I think to me the data is quite conclusive in that it does not demonstrate — does not demonstrate an association of spermicides with limb reduction defects or with any other congenital anomaly.
* * *
I think that Doctor Shapiro would agree with my statement, and that is that no association between spermicides and birth defects has been shown, and I agree with him when he says a small increase cannot be ruled out. I think that’s true about a small increase in anything, but why should one even bother to do that given data that doesn’t show any substantial association as it now stands.”
Id. at 285-86.
The court found the above passages to be the basis for assigning “little weight” to Dr. Stolley’s opinion despite his “impressive credentials.” Here is how the court described what it perceived to be an “equivocation”:
“Several times during direct examination, Dr. Stolley testified emphatically that a specific study or studies showed that spermicides are not related to birth defects. ***. In the portions of his testimony on cross-examination quoted above, however, Dr. Stolley equivocated. His interpretation of the studies discussed was that ‘no association between spermicides and birth defects has been shown’, but on cross he conceded that ‘a small increase [in birth defects] cannot be ruled out’. Finally, by disagreeing with Shapiro’s warning that an ‘appreciable increase’ cannot be ruled out, Dr. Stolley in effect was expressing greater confidence in Shapiro’s findings that Shapiro himself thought was justified. *** In short, because of his apparent bias and his overstatements, the Court discounted Dr. Stolley’s conclusions.”
Id. at 286 (internal citations omitted). Putting aside the issue whether Dr. Stolley was assessing the Shapiro study alone or all the studies together, we can say that the trial court was clearly erroneous in finding an equivocation or an inconsistency between saying a study both failed to show an association and acknowledging that it also failed to rule out a small (or even an appreciable) increased risk. The trial court had shifted the burden of proof to the defense to show that there was no association.
The trial court, confused by its own conflation of failing to show and failing to rule out, proceeded to dismiss Dr. Stolley’s testimony as having little or no probative value:
“Although these scientific studies are invaluable aids, Dr. Stolley’s testimony demonstrates that the studies alone do not show conclusively whether or not Katie Wells’ birth defects were caused by Ortho-Gynol Contraceptive Jelly. Further, the Court reiterates that plaintiffs’ ultimate burden was not to produce a flawless epidemiological study, but rather to show from all the evidence presented, to a reasonable degree of medical certainty, that the Product caused some or all of Katie Wells’ birth defects in 1980.”
Id. at 286. Lovely rhetoric, but yet the trial court was prepared to hold the defense to a non-legal, insuperable burden to produce an infinitely powerful study that could rule out any increased risk. Dr. Robert Brent, the defense expert witness on teratology, similarly expressed the limitation of hypothesis testing to “prove” the hull hypothesis of no association:
“[T]here’s no way to prove that a substance is not teratogenic. It’s proving a negative, and, in fact, it’s true of water and vitamins and everything else that we are exposed to. All we can say is that after extensive evaluation, the risk appears no greater than if you didn’t expose yourself to that. That’s about as far as you can go.”
Id. at 289-90 (internal citations omitted). If the defense expert witnesses committed any mistake it was in not explaining hypothesis testing better in their direct examinations so that statements on cross-examination were not seen as “equivocations.” This tactical mistake, however, cannot excuse or erase the profoundly and clearly erroneous findings of the trial court.
Statistical Significance
It was, after all, Justice Sotomayor’s opinion in Matrixx Initiatives, with Her Honor’s broad dicta about statistical significance, which motivated my re-reading of the Wells case. The Supreme Court’s opinion cited the appellate court’s decision to affirm the judgment entered upon Judge Shoob’s verdict, but the essence of the Wells case was established in the proceedings in district court. Here is what Judge Shoob wrote about statistical significance:
“Plaintiffs’ burden of proving that Katie Wells’ defects were caused by the Product did not necessarily require them to produce scientific studies showing a statistically significant association between spermicides and congenital malformations in a large population.”
615 F. Supp. at 292. And later, in connection with one of the defense expert witnesses, the court noted:
“In Dr. Brent’s opinion, none of these three indicators suggests that non-ionic surfactants are teratogenic. Other than the Jick study, he was aware of no studies showing a statistically significant association between spermicides and birth defects.”
Id. at 289. Because there was at least one statistically significant outcome reported in one of the studies at issues, the Wells case cannot represent a legal precedent that established that statistically significant evidence is not necessary to a judgment of causation. Justice Sotomayor’s opinion in dictum to the contrary is not only wrong as a scientific generalization, it is wrong as a matter of basic legal process.
(to be continued)
