TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

November 21st, 2012

In 1984, before Judge Shoob gave his verdict in the Wells case, another firm filed a birth defects case against Ortho for failure to warn in connection with its non-ionic surfactant spermicides, in the same federal district court, the Northern District of Georgia. The mother in Smith used Ortho’s product about the same time as the mother in Wells (in 1980).  The case was assigned to Judge Shoob, who recused himself.  Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1562 n.1 (N.D. Ga. 1991) (no reasons for the recusal provided).  The Smith case was reassigned to Judge Horace Ward, who entertained Ortho’s motion for summary judgment in July 1988.  Two and one-half years later, Judge Ward granted summary judgment to Ortho on grounds that the plaintiffs’ expert witnesses’ testimony was not based upon the type of data reasonably relied upon by experts in the field, and was thus inadmissible under Federal Rule of Evidence 703. 770 F. Supp. at 1681.

A prevalent interpretation of the split between Wells and Smith is that the scientific evidence developed with new studies, and that the scientific community’s views matured in the five years between the two district court opinions. The discussion in Modern Scientific Evidence is typical:

“As epidemiological evidence develops over time, courts may change their view as to whether testimony based on other evidence is admissible. In this regard it is worth comparing Wells v. Ortho Pharmaceutical Corp., 788 F.2d 741 (11th Cir. 1986), with Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561 (N.D. Ga. 1991). Both involve allegations that the use of spermicide caused a birth defect. At the time of the Wells case there was limited epidemiological evidence and this type of claim was relatively novel.  In a bench trial the court found for the plaintiff.  *** The Smith court, writing five years later, noted that, ‘The issue of causation with respect to spermicide and birth defects has been extensively researched since the Wells decision.’ Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1563 (N.D. Ga. 1991).”

1 David L. Faigman, Michael J. Saks, Joseph Sanders, and Edward K. Cheng, Modern Scientific Evidence:  The Law and Science of Expert Testimony, “Chapter 23 – Epidemiology,” § 23:4, at 213 n.12 (West 2011) (internal citations omitted).

Although Judge Ward was being charitable to his judicial colleague, this attempt to reconcile Wells and Smith does a disservice to Judge Ward’s hard work in Smith, and Judge Shoob’s errors in Wells.

Even a casual reading of Smith and Wells reveals that the injuries were completely differently.  Plaintiff Crystal Smith was born with a chromosomal defect known as Trisomy-18; Plaintiff Katie Wells was born with limb reduction deficits.   Some studies relevant to one injury had no information about the other.  Other studies, which addressed both injuries, yielded different results for the different injuries.  Although some additional studies were available to Judge Ward in 1988, this difference is hardly the compelling difference between the two cases.

Perhaps the most important difference between the cases is that in Smith, the biologically plausibility that spermicides could cause a Trisomy-18 was completely absent.  The chromosomal defect arises from a meiotic disjunction, an error in meiosis that is part of the process in which germ cells are formed.  Simply put, spermicides arrive on the scene too late to cause a Trisomy-18.  Notwithstanding the profound differences between the injuries involved in Wells and Smith, the Smith plaintiffs sought the application of collateral estoppel.  Judge Ward refused this motion, on the basis of the factual differences in the cases, as well as the availability of new evidence.  770 F.Supp. at 1562.

The difference in injuries, however, was not the only important difference between these two cases.  Wells was actually tried, apparently without any challenge under Frye, or Rules 702 or 703, to the admissibility of expert witness testimony.  There is little to no discussion of scientific validity of studies, or analysis of the requisites for evaluating associations for causality.  It is difficult to escape the conclusion that Judge Shoob decided the Wells case on the basis of superficial appearances, and that he frequently ignored validity concerns in drawing invidious distinctions between plaintiffs’ and defendant’s expert witnesses and their “credibility.”  Smith, on the other hand, was never tried.  Judge Ward entertained and granted dispositive motions for summary judgment, on grounds that the plaintiffs’ expert witnesses’ testimony was inadmissible. Legally, the cases are light years apart.

In Smith, Judge Ward evaluated the same FDA reports and decisions seen by Judge Shoob.  Judge Ward did not, however, dismiss these agency materials simply because one or two of dozens of independent scientists involved had some fleeting connection with industry. 770 F.Supp. at 1563-64.

Judge Ward engaged with the structure and bases of the expert witnesses’ opinions, under Rules 702 and 703.  The Smith case thus turned on whether expert witness opinions were admissible, an issue not considered or discussed in Wells.  As was often the case before the Supreme Court decided Daubert in 1993, Judge Ward paid little attention to Rule 702’s requirement of helpfulness or knowledge.  The court’s 702 analysis was limited to qualifications.  Id. at 1566-67.  The qualifications of the plaintiffs’ witnesses were rather marginal.  They relied upon genetic and epidemiologic studies, but they had little training or experience in these disciplines. Finding the plaintiffs’ expert witnesses to meet the low threshold for qualification to offer an opinion in court, Judge Ward focused on Rule 703’s requirement that expert witnesses reasonably rely upon facts and data that are not otherwise admissible.

The trial court in Smith struggled with how it should analyze the underpinnings of plaintiffs’ witnesses’ proffered testimony.  The court acknowledged that conflicts between expert witnesses typically raise questions of weight, not admissibility.  Id. at 1569.  Ortho had, however, challenged plaintiffs’ witnesses for having given opinions that lacked a “sound underlying methodology.” Id.  The trial court found at least one Fifth Circuit case that suggested that Rule 703 requires trial courts to evaluate the reliability of expert witnesses’ sources.  Id. (citing Soden v. Freightliner Corp., 714 F.2d 498, 505 (5th Cir. 1983). Elsewhere, the trial court also found precedent from Judge Weinstein’s opinion in Agent Orange, as well as Court of Appeals decisions involving Bendectin, all of which turned to Rule 703 as the legal basis for reviewing, and in some cases limiting or excluding expert witness opinion testimony.  Id.

The defendant’s argument under Rule 703 was strained; Ortho argued that the plaintiffs’

“experts’ selection and use of the epidemiological data is faulty and thus provides an insufficient basis upon which experts in the field of diagnosing the source of birth defects normally form their opinions. The defendant also contends that the plaintiffs’ experts’ data on genetics is not of the kind reasonably relied upon by experts in field of determining causation of birth defects.”

Id. at 1572.  Nothing in Rule 703 addresses the completeness or thoroughness of expert witnesses in their consideration of facts and data; nor does Rule 703 address the sufficiency of data or the validity vel non of inferences drawn from facts and data considered.  Nonetheless, the trial court in Smith took Rule 703 as its legal basis for exploring the epistemic warrant for plaintiffs’ witnesses’ causation opinions.

Although plaintiffs’ expert witnesses stated that they had relied upon epidemiologic studies and method, the trial court in Smith went beyond their asseverations.  The Smith trial court explored the credibility of these witnesses at a whole other level.  The court reviewed and discussed the basic structure of epidemiologic studies, and noted that the objective of such studies is to provide a statistical analysis:

“The objective of both case-control and cohort studies is to determine whether the difference observed in the two groups, if any, is ‘statistically significant’, (that is whether the difference found in the particular study did not occur by chance alone).40 However, statistical methods alone, or the finding of a statistically significant association in one study, do not establish a causal relationship.41 As one authority states:

‘Statistical methods alone cannot establish proof of a causal relationship in an association’.42

As a result, once a statistical association is found in an epidemiological study, that data must then be evaluated in a systematic manner to determine causation. If such an association is present, then the researcher looks for ‘bias’ in the study.  Bias refers to the existence of factors in the design of a study or in the manner in which the study was carried out which might distort the result.43

If a statistically significant association is found and there is no apparent ‘bias’, an inference is created that there may be a cause-and-effect relationship between the agent and the medical effect. To confirm or rebut that inference, an epidemiologist must apply five criteria in making judgments as to whether the associations found reflect a cause-and-effect relationship.44 The five criteria are:

1. The consistency of the association;

2. The strength of the association;

3. The specificity of the association;

4. The temporal relationship of the association; and,

5. The coherence of the association.

Assuming there is some statistical association, it is these five criteria that provide the generally accepted method of establishing causation between drugs or chemicals and birth defects.45 ”

The Smith court acknowledged that there were differences of opinion in weighting these five factors, but that some of them were very important to drawing a reliable inference of causality.  Id. at 1775.

A major paradigm shift thus separates Wells and Smith.  The trial court in Wells contented itself with superficial and subjective indicia of witnesses’ personal credibility; the trial in Smith delved into the methodology of drawing an appropriate scientific conclusion about causation.  Telling was the Smith court’s citation to Moultrie v. Martin, 690 F.2d 1078, 1082 (4th Cir. 1982) (“In borrowing from another discipline. a litigant cannot be selective in which principles are applied.”).  770 F.Supp. at 1575 & n.45.  Gone is the Wells retreat from engagement with science, and the dodge that the court must make a legal, not a scientific decision.

Applying the relevant principles, the Smith court found that the plaintiffs’ expert witnesses had deviated from the scientific standards of reasoning and analysis:

“It is apparent to the court that the testimony of Doctors Bussey and Holbrook is insufficiently grounded in any reliable evidence. * * * The conclusions Doctors Bussey and Holbrook reach are also insufficient as a basis for a finding of causality because they fail to consider critical information, such as the most relevant epidemiologic studies and the other possible causes of disease.81

The court finds that the opinions of plaintiffs’ experts are not based upon the type of data reasonably relied upon by experts in determining the cause of birth defects. Experts in determining birth defects rely upon a consensus in genetic or epidemiological investigations or specific generally accepted studies in these fields. While a consensus in genetics or epidemiology is not a prerequisite to a finding of causation in any and all birth defect cases, Rule 703 requires some reliable evidence for the basis of an expert’s opinion.

Experts in determining birth defects also utilize methodologies and protocols not followed by plaintiffs’ experts. Without a well-founded methodology, opinions which run contrary to the consensus of the scientific community and are not supported by any reliable data are necessarily speculative and lacking in the type of foundation necessary to be admissible.

For the foregoing reasons, the court finds that plaintiffs have failed to produce admissible evidence sufficient to show that defendant’s product caused Crystal’s birth defects.”

Id. at 1581.  Rule 703 was forced into a service to filter out methodologically specious opinions.

Not all was smooth sailing for Judge Ward.  Like Judge Shoob, Judge Ward seemed to think that a physical examination of the plaintiff provided helpful, relevant evidence, but he never articulated what the basis for this opinion was. (His Honor did note that the parties agreed that the physical examination offered no probative evidence about causation.  Id. at 1572 n.32.) No harm came of this opinion.  Judge Ward wrestled with the lack of peer review in some unpublished studies, and the existence of a study only in abstract form.  See, e.g., id. at 1579 (“a scientific study not subject to peer review has little probative value”); id. at 1578 (insightfully noting that an abstract had insufficient data to permit a reader to evaluate its conclusions).  The Smith court recognized the importance of statistical analysis, but it confused Bayesian posterior probabilities with significance probabilities:

“Because epidemiology involves evidence on causation derived from group based information, rather than specific conclusions regarding causation in an individual case, epidemiology will not conclusively prove or disprove that an agent or chemical causes a particular birth defect. Instead, its probative value lies in the statistical likelihood of a specific agent causing a specific defect. If the statistical likelihood is negligible, it establishes a reasonable degree of medical certainty that there is no cause-and-effect relationship absent some other evidence.”

The confusion here is hardly unique, but ultimately it did not prevent Judge Ward from reaching a sound result in Smith.

What intervened between Wells and Smith was not any major change in the scientific evidence on spermicides and birth defects; the sea change came in the form of judicial attitudes toward the judge’s role in evaluating expert witness opinion testimony.  In 1986, for instance, after the Court of Appeals affirmed the judgment in Wells, Judge Higginbotham, speaking for a panel of the Fifth Circuit, declared:

“Our message to our able trial colleagues: it is time to take hold of expert testimony in federal trials.”

 In re Air Crash Disaster at New Orleans, 795 F.2d 1230, 1234 (5th Cir. 1986).  By the time the motion for summary judgment in Smith was decided, that time had come.

Posted in Causation, Data Sharing, Frye, Rule 702, Rule 703, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

November 21st, 2012

While the trial court was preparing its findings of fact and conclusions of law, Ortho moved to reopen to evidence to permit additional testimony based upon three new articles.  Ortho’s motion came three months after the close of evidence, and Judge Shoob’s announcement of his verdict. The court denied this motion without mentioning what the new articles purported to show.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262, 298 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).

What is remarkable in Wells, from the vantage point of current practice, is the absence of motions directed at the proffered expert witness opinion testimony.  On the basis of Judge Shoob’s opinion, there appears to have been no Frye motion, no motions to exclude expert witnesses based upon the Federal Rules of Evidence, and no motions to strike testimony after the fact for lack of a proper basis.

Having lost the verdict in a bench trial, Ortho had little chance for success in the Court of Appeals on a claim that the evidence supporting the plaintiffs’ verdict was legally insufficient.  The traditional standard, applied by the Court of Appeals, was to sustain the trier of fact’s decision as not “clearly erroneous” when there were two “permissible” views of the evidence. 788 F.2d 741, 743 (11th Cir. 1986).  Without some legal doctrine to filter out flawed, invalid, and inadequate expert witness opinion from permissible views of an evidentiary display, the Court of Appeals was left with only a rubber stamp, which it proceeded to use with alacrity.

Ortho attempted to turn its appellate argument about the sufficiency of the evidence into a legal principle about rejecting factual findings not based upon “scientifically reliable foundations.”  Id. at 744.  The appellate court framed the issue on appeal simply as a “battle of the experts,” which Ortho had lost.  Both sides had qualified expert witnesses, and thus, according to the appellate court, “the district court was forced to make credibility determinations to ‘decide the victor’.” Id. (citing Ferebee v. Chevron Chemical Co., 736 F.2d 1529, 1535 (D.C. Cir.), cert. denied, 469 U.S. 1062 (1984)).  The Court of Appeals thus acquiesced in Judge Shoob’s superficial analysis, which attempted to resolve a scientific issue by trial atmospherics, demeanor, and subjective impressions of witness confidence rather than the validity of the studies relied upon and inferences drawn therefrom.  The possibility that Judge Shoob might have evaluated the evidentiary basis underlying the expert witnesses’ opinions was not even acknowledged.

The Court of Appeals invoked the language from Ferebee on statistical significance, despite its irrelevance to the case before it:

“We recognize, as did the Ferebee court, that ‘a cause-effect relationship need not be clearly established by animal or epidemiological studies before a doctor can testify that, in his opinion, such a relationship exists. As long as the basic methodology employed to reach such a conclusion is sound, such as use of tissue samples, standard tests, and patient examination, products liability law does not preclude recovery until a “statistically significant” number of people have been injured or until science has had the time and resources to complete sophisticated laboratory studies of the chemical. Id. at 1535-36.”

Wells, 788 F.2d at 745 (quoting Ferebee). Ferebee involved an injury that all parties agreed could be attributed to paraquat exposure without the need for epidemiologic studies; statistical analysis was not particularly germane.  In Wells, on the other hand, both sides relied upon studies that required statistical analyses for any sensible interpretation, and some of the studies actually reported statistically significant results.  The appellate court’s rhetoric was empty and irrelevant.

(to be continued)

Posted in Causation, Expert Witnesses, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 5

Broadbent on the Relative Risk > 2 Argument

October 31st, 2012

Alex Broadbent, of the University of Johannesburg, Department of Philosophy, has published a paper that contributes to the debate over whether a relative risk (RR) greater than (>) two is irrelevant, helpful, necessary, or sufficient in inferring that an exposure more likely than not caused an individual claimant’s disease. Alex Broadbent, “Epidemiological Evidence in Proof of Specific Causation,” 17 Legal Theory 237 (2011) [cited as Broadbent].  I am indebted to his having called his paper to my attention. Professor Broadbent’s essay is clearly written, which is helpful in assessing the current use of the RR > 2 argument in judicial decisions.

General vs. Specific Causation

Broadbent carefully distinguishes between general and specific causation.  By focusing exclusively upon specific causation (and assuming that general causation is accepted), he avoids the frequent confusion over when RR > 2 might play a role in legal decisions. Broadbent also “sanitizes” his portrayal of RR by asking us to assume that “the RR is not due to anything other than the exposure.” Id. at 241. This is a BIG assumption and a tall order for observational epidemiologic evidence.  The study or studies that establishes the RR we are reasoning from must be free of bias and confounding. Id.  Broadbent does not mention, however, the statistical stability of the RR, which virtually always will be based upon a sample, and thus subject to the play of random error.  He sidesteps the need for statistical significance in comparing two proportions, but the most charitable interpretation of his paper requires us to assume further that the hypothetical RR from which we are reasoning is sufficiently statistically stable that random error, along with bias and confounding, can be also ruled out as likely explanations for the RR > 1.

Broadbent sets out to show that RR > 2 may, in certain circumstances, suffices to show specific causation, but he argues that RR > 2 is never logically necessary, and must never be required to support a claim of specific causation.  Broadbent at 237.  On the same page in which he states that epidemiologic evidence of increased risk is a “last resort,” Broadbent contradicts himself by stating RR > 2 evidence “must never be required,” and then, in an apparent about face, he argues:

“that far from being epistemically irrelevant, to achieve correct and just outcomes it is in fact mandatory to take (high-quality) epidemiological evidence into account in deciding specific causation. Failing to consider such evidence when it is available leads to error and injustice. The conclusion is that in certain circumstances epidemiological evidence of RR > 2 is not necessary to prove specific causation but that it is sufficient.”

Id. at 237 (emphasis added). I am not sure how epidemiologic evidence can be mandatory but never logically necessary, and something that we should never require.

Presumably, Broadbent is using “to prove” in its legal and colloquial sense, and not as a mathematician.  Let us also give Broadbent his assumptions of “high quality” epidemiologic studies, with established general causation, and ask why, and explore when and whether, RR > 2 is not necessary to show specific causation.

The Probability of Causation vs. The Fact of Causation

Broadbent notes that he is arguing against what he perceives to be Professor Haack’s rejection of probabilistic inference, which would suggest that epidemiologic evidence is “never sufficient to establish specific causation.” Id. at 239 & n.3 (citing Susan Haack, “Risky Business: Statistical Proof of Individual Causation,” in Causación y Atribucion de Responsabilidad (J. Beltran ed., forthcoming)). He correctly points out that sometimes the probabilistic inference is the only probative inference available to support specific causation.  His point, however, does not resolve the dispute; it suffices only to show that whether we allow the probabilistic inference may be outcome determinative in many lawsuits.  Broadbent characterizes Haack’s position as one of two “serious mistakes in judicial and academic literature on this topic.”  Broadbent at 239.  The other alleged mistake is the claim that RR > 2 is needed to show specific causation:

“What follows, I conclude, is that epidemiological evidence is relevant to the proof of specific causation. Epidemiological evidence says that a particular exposure causes a particular harm within a certain population. Importantly, it quantifies: it says how often the exposure causes the harm. However, its methods are limited: they measure only the net effect of the exposure, leaving open the possibility that the exposure is causing more harm than the epidemiological evidence suggests—but ruling out the possibility that it causes less. Accordingly I suggest that epidemiological evidence can be used to estimate a lower bound on the probability of causation but that no epidemiological measure can be required. Thus a relative risk (RR, defined in Section II) of greater than 2 can be used to prove causation when there is no other evidence; but RR < 2 does not disprove causation. Given high-quality epidemiological evidence, RR > 2 is sufficient for proof of specific causation when no other evidence is available but not necessary when other evidence is available.”

Some of this seems reasonable enough.  Contrary to the claims of authors such as Haack and Wright, Broadbent maintains that some RR evidence is relevant and indeed probative of specific causation.  In a tobacco lung cancer, with a plaintiff who has smoked three packs a day, for 50 years (and RR > 50), we can confidently attribute the lung cancer to smoking, and rest assured that background cosmic radiation did not likely play a substantial role. The RR quantifies the strength of the association, and it does lead us to a measure of “attributable risk” (AR), also known as the attributable fraction (AF):

AR = 1 – 1/RR.

So far, so good.

Among the perplexing statements above, however, Broadbent suggests that:

1. The methods of epidemiologic evidence measure only the net effect of the exposure.  Epidemiologic evidence (presumably the RR or other risk ratio) provides a lower bound on the probability of causation.  I take up this suggestion in discussing Broadbent’s distinction between the “excess fraction,” and the “etiologic fraction,” below.

2. A RR > 2 “can be used to prove causation when there is no other evidence; but RR < 2 does not disprove causation.” (My emphasis.) When an author is usually clear about his qualifications, and his language generally, it is distressing for him to start comparing apples to oranges.  Note that RR > 2 suffices “when there is no other evidence,” but the parallel statement about RR < 2 is not similarly qualified, and the statement about RR < 2 is framed in terms of disproof of causation. Even if the RR < 2 did not “disprove” specific causation, when there was no other evidence, it would not prove causation.  And if there is no other evidence, judgment for the defense must result. Broadbent fails to provide us a persuasive scenario in which a RR ≤ 2, with no other evidence, would support an inference of specific causation.

Etiological Fraction vs. Excess Fraction — Occam’s Disposable Razor

Broadbent warns that the expression “attributable risk” (AR or “attributable fraction,” AF) is potentially misleading.  The numerical calculation identifies the excess number of cases, above “expected” per base rate, and proceeds from there.  The AR thus identifies the “excess fraction,” and not the “etiological fraction,” which is the fraction of all cases in which exposure makes a contribution. Broadbent tells us that:

“Granted a sound causal inference, we can infer that all the excess cases are caused by the exposure. But we cannot infer that the remaining cases are not caused by the exposure. The etiologic fraction—the cases in which the exposure makes a causal contribution—could be larger. Roughly speaking, this is because, in the absence of substantive biological assumptions, it is possible that the exposure could contribute to cases that would have occurred12 even without the exposure.13 For example, it might be that smoking is a cause of lung cancer even among some of those who would have developed it anyway. The fact that a person would have developed lung cancer anyway does not offer automatic protection against the carcinogenic effects of cigarette smoke (a point we return to in Section IV).”

Id. at 241. In large measure here, Broadbent has adopted (and acknowledged) his borrowings from Professor Sander Greenland.  Id. at 242 n.11. The argument  still fails.  What Broadbent has interposed is a “theoretical possibility” that the exposure in question may contribute to those cases that would have occurred anyway.  Note that raising theoretical possibilities here now alters the hypothetical; Broadbent is no longer working from a hypothetical that we have a RR and no other evidence.  Even more important, we are left guessing what it means to say that an exposure causes some cases that would have occurred anyway.  If we accept the postulated new evidence at face value, we can say confidently that the exposure is not the “but for” cause of the case at issue.  Without sufficient evidence of “but for” causation, plaintiff will lose. Furthermore, we are being told to add a new fact to the hypothetical, namely that the non-excess cases are causally over-determined.  If this is the only additional new fact being added, a court might invoke the rule in Summers v. Tice, but even so, the defense will be entitled to a directed verdict if the RR < 2. (If the RR = 2, I suppose, the new fact, and the change in the controlling rule, might alter the result.)

Exposures that Cause Some and Prevent Some Cases of Disease

Broadbent raises yet another hypothetical possibility, which adds to, and materially alters,  his original hypothetical.  If the exposure in question, causes some cases, and prevents others, then the RR ≤ 2 will not permit us to infer that a given case is less likely than not the result of the exposure.  (Broadbent might have given an example of what he had in mind, from well-established biological causal relationships; I am skeptical that he would have found one that would have satisfactorily made his argument.) The bimodal distribution of causal effects is certainly not typical of biological processes, but even if we indulge the “possibility,” we are now firmly in the realm of speculation.  This is a perfectly acceptable realm for philosophers, but in court, we want evidence.  Assuming that the claimant could present such evidence, finders of fact would still founder because the new evidence would leave them guessing whether the claimant was a person who would have gotten the disease anyway, or got it because of the exposure, or even got it in spite of the exposure.

Many commentators who urge a “probability of [specific] causation” approach equate the probability of causation (PC) with the AR.  Broadbent argues that because of the possibility that some biological model results in the etiologic fraction exceeded the excess fraction, the usual equation of PC = AR, must be represented as an equality:

PC ≥ AR

While the point is logically unexceptional, Broadbent must concede that some other evidence, which supports and justifies the postulated biological model, is required to change the equality to an inequality.  If no other evidence besides the RR is available, we are left with the equality.  Broadbent tells us that the biological model “often” requires that the etiological fraction exceeds the excess fraction, but he never tells us how often, or how we would ascertain the margin of error.  Id. at 256.

Broadbent does not review any of the decided judicial cases to point out which ones involved biological models that invalidated the equality.  Doing so would be an important exercise because it might well show that even where PC ≥ AR, with a non-quantified upper bound, the plaintiff might still fail in presenting a prima facie case of specific causation.  Suppose the population RR for the exposure in question were 1.1, and we “know” (and are not merely speculating) that the etiological fraction > excess fraction.   Unless we know how much greater is the etiological fraction, such that we can recalculate the PC, then we are left agnostic about specific causation.

Broadbent treats us to several biological scenarios in which PC possibly is greater than AR.  All of these scenarios violate his starting premiss that we have a RR with no other evidence. For instance, Broadbent hypothesizes that exposure might accelerate onset of a disease.  Id. at 256. This biological model of acceleration can be established with the same epidemiologic evidence that established the RR for the population.  Epidemiologists will frequently look at time windows from onset of exposure to explore whether there is an acceleration of onset of cases in a younger age range that offsets a deficit later in the lives of the exposed population.  If there were firm evidence of such a phenomenon, then we would look to the RR within the relevant time window.  If the relevant RR ≤ 2, the biological model will have added nothing to the plaintiff’s case.

Broadbent cites Greenland for the proposition that PC > AR:

“We know of no cancer or other important chronic disease for which current biomedical knowledge allows one to exclude mechanisms that violate the assumptions needed to claim that PC = [AF].”

Id. at 259, quoting form Sander Greenland & James Robins, “Epidemiology, Justice, and the Probability of Causation,” 40 Jurimetrics J. 321, 325 (2000).  Here, not only has Broadbent postulated a mechanism that makes PC > AR, but he has shifted the burden of proof to the defense to exclude it!

The notion that the etiological fraction may exceed the excess fraction is an important caveat.  Courts and lawyers should take note.  It will not do, however, wave hands and exclaim that the RR > 2 is not a “litmus test,” and proceed to let any RR > 1, or even RR ≤ 1 support a verdict.  The biological models that may push the etiological fraction higher than the excess fraction can be tested, and quantified, with the same epidemiologic approaches that provided a risk ratio, in the first place.  Broadbent gives us an example of this sort of hand waving:

“Thus, for example, evidence that an exposure would be likely to aggravate an existing predisposition to the disease in question might suffice, along with RR between 1 and 2, to make it more likely than not that the claimant’s disease was caused by the exposure.”

Id. at 275. This is a remarkable, and unsupported claim.  The magnitude of the aggravation might still leave the RR ≤ 2.  What is needed is evidence that would allow quantification of the risk ratio in the scenario presented. Speculation will not do the trick; nor will speculation get the case to a jury, or support a verdict.

 

Posted in Causation, Rule 702, statistical evidence | Comments Off on Broadbent on the Relative Risk > 2 Argument

Old-Fashioned Probablism – Origins of Legal Probabilism

October 26th, 2012

In several posts, I have addressed Professor Haack’s attack on legal probabilism.  See

Haack Attack on Legal Probabilism (May 6, 2012).  The probabilistic mode of reasoning is not a modern innovation; nor is the notion that the universe is entirely determined, although revealed to humans as a stochastic phenomenon:

“I returned, and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.”

Ecclesiastes 9:11 King James Bible (Cambridge ed.)

The Old Testament describes the “casting of lots,” some sort of dice rolling or coin flipping, in a wide variety of human decision making.  The practice is described repeatedly in the Old Testament, and half a dozen times in the New Testament.

Casting of lots figures more prominently in the Old Testament, in the making of important decisions, and in attempting to ascertain “God’s will.”  The Bible describes matters of inheritance, Numbers 34:13; Joshua 14:2, and division of property, Joshua 14-21, Numbers 26:55, as decided by lots.  Elections to important office, including offices and functions in the Temple, were determined by lot. 1 Chronicles 24:5, 31; 25:8-9; 26:13-14; Luke 1:9.

Casting lots was an early form of alternative dispute resolution – alternative to slaying and smiting.  Proverbs describes the lot as used as a method to resolve quarrels.  Proverbs 18:18.  Lot casting determined fault in a variety of situations.  Lots were cast to identify the culprit who had brought God’s wrath upon Jonah’s ship. Jonah 1:7 (“Come, let us cast lots, that we may know on whose account this evil has come upon us.”).

What we might take as a form of gambling appeared to have been understood by the Israelites as a method for receiving instruction from God. Proverbs 16:33 (“The lot is cast into the lap, but its every decision is from the Lord.”).  This Old Testament fortune cookie suggests that the Lord knows the outcome of the lot casting, but mere mortals must wager.  I like to think the passage means that events that appear to be stochastic to humans may have a divinely determined mechanism.  In any event, the Bible describes various occasions on which lots were cast to access the inscrutable intentions and desires of the Lord.  Numbers 26:55; 33:54; 34:13; 36:2; Joshua 18:6-10; 1 Chronicles 24:5,31; 1 Samuel 14:42; Leviticus 16:8-10 (distinguishing between sacrificial and scape goat).

In the New Testament, the Apostles cast lots to decide upon a replacement for Judas (Acts 1:26). Matthias was the winner.  Matthew, Mark, and John describe Roman soldiers casting lots for Jesus’ garments (Matthew 27:35; Mark 15:24; John 19:24.  See also Psalm 22:18.  This use of lots by the Roman soldiers seems to have taken some of the magic out of lot casting, which fell into disrepute and gave way to consultations with the Holy Spirit for guidance on important decisions.

The Talmud deals with probabilistic inference in more mundane settings.  The famous “Nine Shops” hypothetical poses 10 butcher shops in a town, nine of which sell kosher meat.  The hypothetical addresses whether the dietary laws permit eating a piece of meat found in town, when its butchering cannot be attributed to either the nine kosher shops or the one non-kosher shop:

“A typical question involves objects whose identity is not known and reference is made to the likelihood that they derive from a specific type of source in order to determine their legal status, i.e. whether they be permitted or forbidden, ritually clean or unclean, etc. Thus, only meat which has been slaughtered in the prescribed manner is kasher, permitted for food. If it is known that most of the meat available in a town is kasher, there being, say, nine shops selling kasher meat and only one that sells non-kasher meat, then it can be assumed when an unidentified piece of meat is found in the street that it came from the majority and is therefore permitted.”

Nachum L. Rabinovitch, “Studies in the History of Probability and Statistics.  XXII Probability in the Talmud,” 56 Biometrika 437, 437 (1969).  Rabinovitch goes on to describe the Talmud’s resolution of this earthly dilemma:  “follow the majority” or the most likely inference.

A small digression on this Talmudic hypothetical.  First, why not try to find out whether someone has lost this package of meat? Or turn the package in to the local “lost and found.” Second, how can it be kosher to eat a piece of meat found lying around in the town?  This is really not very appetizing, and it cannot be good hygiene.  Third, why not open the package and determine whether it’s a nice pork tenderloin or a piece of cow?  This alone could resolve the issue. Fourth, the hypothetical posed asks us to assume a 9:1 ratio of kosher to non-kosher shops, but what if the one non-kosher shop had a market share equal to the other nine? The majority rule could lead to an untoward result for those who wish to keep kosher.

The Talmud’s proposed resolution is, nevertheless, interesting in anticipating the controversy over the use of “naked statistical inferences” in deciding specific causation or discrimination cases.  Of course, the 9:1 ratio is sufficiently high that it might allow an inference about the “likely” source of the meat.  The more interesting case would have been a town with 11 butcher shops, six of which were kosher.  Would the rabbis of old have had the intestinal fortitude to eat lost & found meat, on the basis of a ratio of 6:5?

In the 12th century, Maimonides rejected probabilistic conclusions for assigning criminal liability, at least where the death penalty was at issue:

“The 290th Commandment is a prohibition to carry out punishment on a high probability, even close to certainty . . .No punishment [should] be carried out except where . . . the matter is established in certainty beyond any doubt, and , moreover, it cannot be explained otherwise in any manner.  If we do not punish on very strong probabilities, nothing can happen other than a sinner be freed; but if punishment be done on probability and opinion it is possible that one day we might kill an innocent man — and it is better and more desirable to free a thousand sinners, than ever kill one innocent.”

Stephen E. Fienberg, ed., The Evolving Role of Statistical Assessments as Evidence in the Courts 213 (N.Y. 1989), quoting from Nachum Rabinovitch, Probability and Statistical Inference in Ancient and Medieval Jewish Literature 111 (Toronto 1973).

Indiana Senate candidate and theocrat, Republican Richard E. Mourdock, recently opined that conception that results from rape was God’s will:

“I’ve struggled with it myself for a long time, but I came to realize that life is that gift from God.  And even when life begins in that horrible situation of rape, that it is something that God intended to happen.”

Jonathan Weisman, “Rape Remark Jolts a Senate Race, and the Presidential One, Too,” N.Y. Times (Oct. 25, 2012 ).

Mourdock’s comments about pregnancies resulting from rape representing God’s will show that stochastic events continue to be interpreted as determined mechanistic events at some “higher plane.” Magical thinking is still with us.

Posted in statistical evidence | Comments Off on Old-Fashioned Probablism – Origins of Legal Probabilism

Siracusano Dicta Infects Daubert Decisions

September 22nd, 2012

Gatekeeping is sometimes  intellectually challenging, but the challenge does not excuse sloppy thinking.  Understandably, judges will sometimes misunderstand the relevant science.  The process, however, allows the public and the scientific community to see what is happening in court cases, rather than allowing the critical scientific reasoning to be hidden in the black box of jury determinations.  This transparency can and should invite criticism, commentary, corrections, and consensus, when possible.

Bad legal reasoning is much harder to excuse.  The Supreme Court, in Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011), unanimously affirmed the reversal of a trial court’s Rule 12(b)(6) dismissal of a securities fraud class action.  The corporate defendant objected that the plaintiffs failed to plead statistical significance in alleging causation between Zicam and the loss of the sense of smell.  The Supreme Court, however, made clear that causation was not required to make out a claim of securities fraud.  It was, and would be, sufficient for the company’s product to have raised sufficient regulatory concerns, which in turn would bring regulatory scrutiny and action that would affect the product’s marketability.

The Supreme Court could have disposed of the essential issue in a two page per curiam opinion.  Instead the Court issued an opinion signed by Justice Sotomayor, who waxed carelessly about causation and statistical significance, which discussion was not necessary to the holding.  Not only was Justice Sotomayor’s discussion obiter dicta, but the dicta were demonstrably incorrect. Matrixx Unloaded (Mar. 29, 2011).

The errant dicta in Siracusano has already led one MDL court astray:

“While the defendant repeatedly harps on the importance of statistically significant data, the United States Supreme Court recently stated that ‘[a] lack of statistically significant data does not mean that medical experts have no reliable basis for inferring a causal link between a drug and adverse events …. medical experts rely on other evidence to establish an inference of causation.’ Matrixx Initiatives, Inc. v. Siracsano, 131 S.Ct. 1309, 1319 (2011).”

Memorandum Opinion and Order at 22, In re Chantix (Varenicline) Products Liability Litigation, MDL No. 2092, Case 2:09-cv-02039-IPJ Document 642 (N.D. Ala. Aug. 21, 2012)[hereafter cited as Chantix].  See Open Admissions for Expert Witnesses in Chantix Litigation.

It was only a matter of time before the Supreme Court’s dictum would be put to this predictably erroneous interpretation.  See “The Matrixx Oversold” (April 4, 2011).  Within two weeks, the error in Chantix propagated itself in another MDL case, with another trial court succumbing to the misleading dicta in Justice Sotomayor’s opinion.  See Memorandum in Support of Separate Pretrial Order No. 8933, Cheek v. Wyeth Pharm. Inc. (E.D.Pa. Aug. 30, 2012)(Bartle, J.).

In Cheek, Judge Harvey Bartle rejected a Rule 702 challenge to plaintiffs’ expert witness’s opinion.  I confess that I do not know enough about the expert witness’s opinion or the challenge to assess Judge Bartle’s conclusion.  Judge Bartle, however, invoked the Matrixx decision for the dubious proposition that:

“Daubert does not require that an expert opinion regarding causation be based on statistical evidence in order to be reliable. Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, 1319 (2011). In fact, many courts have recognized that medical professionals often base their opinions on data other than statistical evidence from controlled clinical trials or epidemiological studies. Id. at 1320.”

Cheek at 16.  The Cheek decision is a welter of non-sequiturs.  The fact that in some instances statistical evidence is not necessary is hardly a warrant to excuse the lack of statistical evidence in every case. The truly disturbing gaps in reasoning, however, are not scientific, but legal. Siracusano was not a “Daubert” opinion; and Siracusano does not, and cannot, support the refusal to inquire whether statistical evidence was necessary in a causation opinion, in main part because causation was not at issue in Siracusano.

 

 

 

 

 

 

 

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on Siracusano Dicta Infects Daubert Decisions

Open Admissions for Expert Witnesses in Chantix Litigation

September 1st, 2012

Chantix is medication that helps people stop smoking.  Smoking kills people, but make a licensed drug and the lawsuits will come.

Earlier this month, Judge Inge Prytz Johnson, the MDL trial judge in the Chantix litigation, filed an opinion that rejected Pfizer’s challenges to plaintiffs’ general causation expert witnesses.  Memorandum Opinion and Order, In re Chantix (Varenicline) Products Liability Litigation, MDL No. 2092, Case 2:09-cv-02039-IPJ Document 642 (N.D. Ala. Aug. 21, 2012)[hereafter cited as Chantix].

Plaintiffs claimed that Chantix causes depression and suicidality, sometimes severe enough to result in suicide, attempted or completed.  Chantix at 3-4.  Others have written about Judge Johnson’s decision.  See Lacayo, “Win Some, Lose Some: Recent Federal Court Rulings on Daubert Challenges to Plaintiffs’ Experts,” (Aug. 30, 2012).

The breadth and depth of error of the trial court’s analysis, or lack thereof, remains, however, to be explored.

 

STATISTICAL SIGNIFICANCE

The Chantix MDL court notes several times that the defendant “harped” on this or that issue; the reader might think the defendant was a music label rather than a pharmaceutical manufacturer.  One of the defendant’s chords that failed to resonate with the trial judge was the point that the plaintiffs’ expert witnesses relied upon statistically non-significant results.  Here is how the trial court reported the issue:

“While the defendant repeatedly harps on the importance of statistically significant data, the United States Supreme Court recently stated that ‘[a] lack of statistically significant data does not mean that medical experts have no reliable basis for inferring a causal link between a drug and adverse events …. medical experts rely on other evidence to establish an inference of causation.’ Matrixx Initiatives, Inc. v. Siracsano, 131 S.Ct. 1309, 1319 (2011).”

Chantix at 22.

Well, it was only a matter of time before the Supreme Court’s dictum would be put to this predictably erroneous interpretation.  See “The Matrixx Oversold” (April 4, 2011).

Matrixx involved a motion to dismiss the complaint, which the trial court granted, but the Ninth Circuit reversed.  No evidence was offered; nor was any ruling that evidence was unreliable or insufficient at issue. The Supreme Court affirmed the Circuit on the issue whether pleading statistical significance was necessary.  Matrixx Initiatives took this position in the hopes of avoiding the merits, and so the issue of causation was never before the Supreme Court.  A unanimous Supreme Court held that because FDA regulatory action does not require reliable evidence to support a causal conclusion, pleading materiality for a securities fraud suit does not require an allegation of causation, and thus does not require an allegation of statistically significant evidence. Everything that the Court said about statistical significance and causation was obiter dictum, and rather ill-considered dictum at that.

The Supreme Court thus wandered far beyond its holding to suggest that courts “frequently permit expert testimony on causation based on evidence other than statistical significance.” Matrixx Initiatives, Inc. v. Siracsano, 131 S.Ct. 1309, 1319 (2011) (citing Wells v. Ortho Pharm. Corp., 788 F.2d 741, 744-745 (11th Cir.1986)).  But the Supreme Court’s citation to Wells, in Justice Sotomayor’s opinion, failed to support the point she was trying to make, or the decision that the trial court announced in Chantix.

Wells involved a claim of birth defects caused by the use of spermicidal jelly contraceptive.  At least one study reported a statistically significant increase in detected birth defects over the expected rate.  Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D.Ga. 1985), aff’d, and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).  Wells is not an example of a case in which an expert witness opined about causation in the absence of a scientific study with statistical significance. Of course, finding statistical significance is just the beginning of assessing the causality of an association; the Wells case was and remains notorious for the expert witness’s poor assessment of all the determinants of scientific causation, including the validity of the studies relied upon.

The Wells decision was met with severe criticism in the 1980s.  The decision was widely criticized for its failure to evaluate the entire evidentiary display, as well as for its failure to rule out bias and confounding in the studies relied upon by the plaintiff.  See, e.g., James L. Mills and Duane Alexander, “Teratogens and ‘Litogens’,” 15 New Engl. J. Med. 1234 (1986); Samuel R. Gross, “Expert Evidence,” 1991 Wis. L. Rev. 1113, 1121-24 (1991) (“Unfortunately, Judge Shoob’s decision is absolutely wrong. There is no scientifically credible evidence that Ortho-Gynol Contraceptive Jelly ever causes birth defects.”). See also Editorial, “Federal Judges v. Science,” N.Y. Times, December 27, 1986, at A22 (unsigned editorial);  David E. Bernstein, “Junk Science in the Courtroom,” Wall St. J. at A 15 (Mar. 24,1993) (pointing to Wells as a prominent example of how the federal judiciary had embarrassed the American judicial system with its careless, non-evidence based approach to scientific evidence). A few years later, another case in the same judicial district, against the same defendant, for the same product, resulted in the grant of summary judgment.  Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561 (N.D. Ga. 1991) (supposedly distinguishing Wells on the basis of more recent studies).

Neither the Justices in Matrixx Initiatives nor the trial court in Chantix can be excused for their poor scholarship, or their failure to note that Wells was overruled sub silentio by the Supreme Court’s own subsequent decisions in Daubert, Joiner, Kumho Tire, and Weisgram.  And if the weight of precedent did not kill the concept, then there is the simple matter of a supervening statute:  the 2000 amendment of Rule 702, of Federal Rules of Evidence.

 

CONFUSING REGULATORY ACTION WITH CAUSAL ASSESSMENTS

The Supreme Court in Matrixx Initiatives was careful to distinguish causal judgments from regulatory action, but then went on in dictum to conflate the two.  The trial judge in Chantix showed no similar analytical care.  Judge Johnson held that the asserted absence of statistical significance was not a basis for excluding plaintiffs’ expert witnesses’ opinions on general causation.  Her Honor adverted to the Matrixx Initiatives dictum that the FDA “does not apply any single metric for determining when additional inquiry or action is necessary.” Matrixx, 131 S.Ct. at 1320.  Chantix at 22.  Judge Johnson noted

“that ‘[n]ot only does the FDA rely on a wide range of evidence of causation, it sometimes acts on the basis of evidence that suggests, but does not prove, causation…. the FDA may make regulatory decisions against drugs based on postmarketing evidence that gives rise to only a suspicion of causation’.  Matrixx, id. The court declines to hold the plaintiffs’ experts to a more exacting standard as the defendant requests.”

Chantix at 23.

In the trial court’s analysis, the difference between regulatory action and civil litigation fact adjudication is obliterated.  This, however, is not the law of the United States, which has consistently acknowledged the difference. See, e.g., IUD v. API, 448 U.S. 607, 656 (1980)(“agency is free to use conservative assumptions in interpreting the data on the side of overprotection rather than underprotection.”)

As the Second Edition of the Reference Manual on Scientific Evidence (which was the out-dated edition cited by the court in Chantix) explains:

“[p]roof of risk and proof of causation entail somewhat different questions because risk assessment frequently calls for a cost-benefit analysis. The agency assessing risk may decide to bar a substance or product if the potential benefits are outweighed by the possibility of risks that are largely unquantifiable because of presently unknown contingencies. Consequently, risk assessors may pay heed to any evidence that points to a need for caution, rather than assess the likelihood that a causal relationship in a specific case is more likely than not.”

Margaret A. Berger, “The Supreme Court’s Trilogy on the Admissibility of Expert Testimony,” in Reference Manual On Scientific Evidence at 33 (Fed. Jud. Ctr. 2d. ed. 2000).

 

CONCLUSIONS VS. METHODOLOGY

Judge Johnson insisted that the “court’s focus was solely on the principles and methodology, not on the conclusions they generate.” Chantix at 9.  This insistence, however, is contrary to the established law of Rule 702.

Although the United States Supreme Court attempted, in Daubert, to draw a distinction between the reliability of an expert witness’s methodology and conclusion, that Court soon realized that the distinction was flawed. If an expert witness’s proffered testimony is discordant from regulatory and scientific conclusions, a reasonable, disinterested scientists would be led to question the reliability of the testimony’s methodology and its inferences from facts and data, to its conclusion.  The Supreme Court recognized this connection in General Electric v. Joiner, and the connection between methodology and conclusions was ultimately incorporated into a statute, the revised Federal Rule of Evidence 702:

“[I]f scientific, technical or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training or education, may testify thereto in the form of an opinion or otherwise, if

  1. the testimony is based upon sufficient fact or data,
  2. the testimony is the product of reliable principles and methods; and
  3. the witness has applied the principles and methods reliably to the facts.”

When the testimony is a conclusion about causation, the Rule 702 directs an inquiry into whether that conclusion is based upon sufficient fact or data, and whether that conclusion is the product of reliable principles and methods.  The court’s focus should indeed be on the conclusion as well the methodology claimed to generate the conclusion.  The Chantix MDL court thus ignored the clear mandate of a statute, Rule 702(1), and applied dictum from Daubert, superseded by Joiner, and an Act of Congress.  The ruling is thus legally invalid to the extent it departs from the statute.

 

EPIDEMIOLOGY

For obscure reasons, Judge Johnson sought to deprecate the need to rely upon epidemiologic studies, whether placebo-controlled clinical trials or observational studies.  See Chantix at 25 (citing Rider v. Sandoz Pharm. Corp., 295 F.3d 1194, 1198-99 (11 Cir.2002)). Of course, the language cited in Rider came from a pre-Daubert, pre-Joiner, case, Wells v. Ortho Pharm. Corp., 788 F.2d 741, 745 (11th Cir.1986) (holding that “a cause-effect relationship need not be clearly established by animal or epidemiological studies”).  This dubious legal lineage cannot support the glib dismissal of the need for epidemiologic evidence.

 

WEIGHT OF THE EVIDENCE (WOE)

According to Judge Johnson, plaintiffs’ expert witness Shira Kramer considered all the evidence relevant to Chantix and neuropsychiatric side effects, in what Kramer described as a “weight of the evidence” analysis.  Chantix at 26.  In her report, Kramer had written that determinations about the weight of evidence are “subjective interpretations” based upon “various lines of scientific evidence. Id. (citing and quoting Kramer’s report). Kramer also claimed that every scientist “brings a unique set of experiences, training and expertise …. Philosophical differences exist between experts…. Therefore, it is not surprising that differences of opinion exist among scientists. Such differences of opinion are not necessarily evidence of flawed scientific reasoning or methodology, but rather differences in judgment between scientists.” Id.

Without any support from scientific literature, or the Reference Manual on Scientific Evidence, Judge Johnson accepted Kramer’s explanation of a totally subjective, unprincipled approach as a scientific methodology.  Not surprisingly, Judge Johnson cited the First Circuit’s embrace of a similar vacuous embrace of a WOE analysis in Milward v. Acuity Specialty Products Group, Inc. 639 F.3d 11, 22 (1st Cir. 2011).  Chantix at 51.

 

CHERRY PICKING

Judge Johnson noted, contrary to her earlier suggestion that Shira Kramer had considered all the studies, that Kramer had excluded data from her analysis.  Kramer’s basis for excluding data may have been based upon pre-specified exclusionary principles, or they may have been completely ad hoc, as were the lack of weighting principles in her WOE analysis.  In its gatekeeping role, however, the trial court expressed complete indifference to Kramer’s selectivity in excluding data.  “Why Dr. Kramer chose to include or exclude data from specific clinical trials is a matter for cross-examination.”  Chantix at 27.  This indifference is an abdication of the court’s gatekeeping responsibility.

 

POWER

The trial court attempted to justify its willingness to mute defendant’s harping on statistical significance by adverting to the concept of statistical power:

“Oftentimes, epidemiological studies lack the statistical power needed for definitive conclusions, either because they are small or the suspected adverse effect is particularly rare. Id. [Michael D. Green et al., “Reference Guide on Epidemiology,” in Reference Manual on Scientific Evidence 333, 335 (Fed. Judicial Ctr. 2d ed. 2000)… .

Chantix at 29 n.16.

To be fair to the trial court, the Reference Manual invited this illegitimate use of statistical power because it, at times, omits the specification that statistical power requires not only a level of statistical significance to be attained, but also a specified alternative hypothesis to assess power.  See Power in the Courts — Part One; Power in the Courts — Part Two.  The trial court offered no alternative hypothesis against which any measure of power was to be assessed.

Judge Johnson did not report any power analyses, and she certainly did not report any quantification of power or lack thereof against some specific alternative hypothesis.  Judge Johnson’s invocation of power was just that – power used arbitrarily, without data, evidence, or reason.

 

CONFIDENCE INTERVALS

As with the invocation of statistical power, the trial also invoked the concept of confidence intervals to suggest that such intervals provide a more refined approach to assessing statistical significance:

“A study found to have ‘results that are unlikely to be the result of random error’ is ‘statistically significant’. Reference Guide on Epidemiology, supra, at 354. Statistical significance, however, does not indicate the strength of an association found in a study. Id. at 359. ‘A study may be statistically significant but may find only a very weak association; conversely, a study with small sample sizes may find a high relative risk but still not be statistically significant.’ Id. To reach a ‘more refined assessment of appropriate inferences about the association found in an epidemiologic study’, researchers rely on another statistical technique known as a confidence interval’. Id. at 360.”

Chantix at 30 n.17.  True, true, but immaterial.  The trial court, again, never carries through with the direction given by the Reference Manual.  Not a single confidence interval is presented.  No confidence intervals are subjected to this more refined assessment.  Why have more refined assessments when even the cruder assessments are not done?

 

OPEN ADMISSIONS IN SCHOOL OF EXPERT WITNESSING

The trial court somehow had the notion that all it had to do was state that every disputed fact and opinion went to the weight not the admissibility, and then pass to a presumably more scientifically literate jury.  To be sure, the court engaged in a good deal of hand waving, going through the motions of deciding a contested issues.  Not only did the Judge Johnson smash poor Pfizer’s harp, Her Honor unhinged the gate that federal judges are supposed to keep.  Chantix declares that it is now open admissions for expert witnesses testifying to causation in federal cases.  This is a judgment in search of an appeal.

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on Open Admissions for Expert Witnesses in Chantix Litigation

The Dow-Bears Debate the Decline of Daubert

August 10th, 2012

Last month, I posted a short screenplay about how judicial gatekeeping of expert witnesses has slackened recently.  See “Daubert Approaching the Age of Majority” (July 17, 2012).

Dr. David Schwartz, of Innovative Science Solutions, has adapted the screenplay to the cinematic screen, and directed a full-length feature movie, The Daubert Will Set Your Client Free, using text-to-talk technology. Dr. Schwartz is not only a first-rate scientist, but he is also an aspiring film maker and artist.

OK; full-length is only a little more than 90 seconds, but you may still enjoy our movie-making debut.  And it is coming to a YouTube screen near you, now.

Posted in Expert Witnesses, Rule 702, Scientific Evidence, statistical evidence | Comments Off on The Dow-Bears Debate the Decline of Daubert

Eighth Circuit Holds That Increased Risk Is Not Cause

August 4th, 2012

The South Dakota legislature took it upon itself to specify the “risks” to be included in the informed consent required by state law for an abortion procedure:

(1) A statement in writing providing the following information:
* * *
(e) A description of all known medical risks of the procedure and statistically significant risk factors to which the pregnant woman would be subjected, including:
(i) Depression and related psychological distress;
(ii) Increased risk of suicide ideation and suicide;
* * *

S.D.C.L. § 34-23A-10.1(1)(e)(i)(ii).  Planned Parenthood challenged the law on constitutional grounds, and the district court granted a preliminary injunction against the South Dakota statute, which a panel of the Eight Circuit affirmed, only to have that Circuit en banc reverse and remand the case for further proceedings.  Planned Parenthood Minn. v. Rounds, 530 F.3d 724 (8th Cir. 2008) (en banc).

On remand, the parties filed cross-motions for summary judgment.  The district court held that the so-called suicide advisory was unconstitutional.  On the second appeal to the Eight Circuit, a divided panel affirmed the trial court’s holding on the suicide advisory. 653 F.3d 662 (8th Cir. 2011).  The Circuit, however, again granted rehearing en banc, and reversed the summary judgment for Planned Parenthood on the advisory.  Planned Parenthood Minnesota v. Rounds, Slip op. July 24, 2012 (en banc)[Slip op.].

In support of the injunction, Planned Parenthood argued that the state’s mandatory suicide advisory violated women’s abortion rights and physicians’ free speech rights. The en banc court rejected this argument, holding that the required advisory was “truthful, non-misleading information,” which did not unduly burden abortion rights, even if it might cause women to forgo abortion.  See Planned Parenthood of Southeastern Pennsylvania v. Casey, 505 U.S. 833, 882-83 (1992).

Risk  ≠ Cause

Planned Parenthood’s success in the trial court turned on its identification of risk (or increased risk) with cause, and its expert witness evidence that causation had not been accepted in the medical literature. In other words, Planned Parenthood argued that the advisory required disclosure of a conclusive causal “link” between abortion and suicide or suicidal ideation.  See 650 F. Supp. 2d 972, 982 (D.S.D. 2009).  The en banc court, on the second appeal, sought to save the statute by rejecting Planned Parenthood’s reading.  The court parsed the statute to suggest that the term “increased risk” is more precise and limited than the umbrella term of “risk,” standing alone.  Slip op. at 6.  The statute does not define “increased risk,” which the en banc court noted had various meanings in medicine.  Id. at 7.

Reviewing the medical literature, the en banc court held that the term “increased risk” does not refer to causation but to a much more modest finding of “a relatively higher probability of an adverse outcome in one group compared to other groups—that is, to ‘relative risk’.”  Id.  The en banc majority seemed to embroil itself in some considerable semantic confusion.  One the hand, the majority, in a rhetorical rift proclaimed that:

“It would be nonsensical for those in the field to distinguish a relationship of ‘increased risk’ from one of causation if the term ‘risk’ itself was equivalent to causation.”

Id. at 9.  The majority’s nonsensical labeling is, well, … nonsensical.  There is a compelling difference in assessment of risk and causation.  Risk is an ex ante concept, applied before the effect has occurred. Assessment or attribution of causation takes place after the effect. Of course, there is a sense of risk or “increased risk,” which is epistemologically more modest, but that hardly makes the more rigorous use of risk as an ex ante cause, nonsensical.

The majority, however, is not content to leave the matter alone.  Elsewhere, the en banc court contradicts itself, and endorses a view that risk = causation.  For instance, in citing to a civil action involving a claimed causal relationship between Bendectin and a birth defect, the Eighth Circuit reduces risk to cause.  See Slip op. at 26 n. 9 (citing Brock v. Merrell Dow Pharms., Inc., 874 F.2d 307, 312 , modified on reh’g, 884 F.2d 166 (5th Cir. 1989)).  The en banc court’s “explanatory” parenthetical explains the depths of its confusion:

“explaining that if studies establish, within an acceptable confidence interval, that those who use a pharmaceutical have a relative risk of greater than 1.0—that is, an increased risk—of an adverse outcome, those studies might be considered sufficient to support a jury verdict of liability on a failure-to-warn claim.”

This reading of Brock is wrong on two counts.  First, the Fifth Circuit, in Brock, and consistently since, has required the relative risk greater than 1.0 to be statistically significant at the conventional significance probability, as well as other indicia of causality, such as the Bradford Hill factors.  So Brock and its progeny did not confuse or conflate risk with cause, or dilute the meaning of cause such that it could be satisfied by a mere showing of an increased relative risk.

Second, Brock itself made a serious error in interpreting statistical significance and confidence intervals. The Bendectin studies at issue in Brock were not statistically significant, and the confidence intervals did not include a measure of no association (relative risk = one). Brock, however, in notoriously incorrect dicta claimed that the computation of confidence intervals took into account bias and confounding as well as sampling variability.  Brock v. Merrill Dow Pharmaceuticals, Inc., 874 F.2d 307, 311-12 (5th Cir. 1989)(“Fortunately, we do not have to resolve any of the above questions [as to bias and confounding], since the studies presented to us incorporate the possibility of these factors by the use of a confidence interval.”)(emphasis in original).  See, e.g., David H. Kaye, David E. Bernstein, and Jennifer L. Mnookin, The New Wigmore – A Treatise on Evidence:  Expert Evidence § 12.6.4, at 546 (2d ed. 2011); Michael O. Finkelstein, Basic Concepts of Probability and Statistics in the Law 86-87 (2009)(criticizing the over-interpretation of confidence intervals by the Brock court); Schachtman, “Confidence in Intervals and Diffidence in the Courts” (Mar. 4, 2012).

The en banc majority’s discussion of the studies of abortion and suicidality make clear that the presence of bias and confounding in a study may prevent inference of causation, but they do not undermine the conclusion that the studies show an increased risk.  A conclusion that the body of epidemiologic studies was inconclusive, and that it failed to “to disentangle confounding factors and establish relative risks of abortion compared to its alternatives,” did not, therefore, render the suicide advisory about risk or increased risk unsupported, untruthful, or misleading.  Slip op. at 20.  Indeed, the en banc court provided an example, outside the context of abortion, to illustrate its meaning.  The en banc court’s use of the example of prolonged television viewing and “increased risk” of mortality suggests that the court took risk to mean any association, no matter how likely it was the result of bias or confounding.  See id. at 10 n. 3 (citing Anders Grøntved, et al., “Television Viewing and Risk of Type 2 Diabetes, Cardiovascular Disease, and All-Cause Mortality, 305 J. Am. Med. Ass’n 2448 (2011). The en banc majority held that the advisory would be misleading only if Planned Parenthood could show that the available epidemiologic studies conclusively ruled out causation.  Slip op. at 24-25.

The Suicide Advisory Has Little Content Because Risk Is Not Cause

The majority decision clarified that the mandatory disclosure does not require a physician to inform a patient that abortion causes suicide or suicidal thoughts.  Slip op. at 25.  The en banc court took solace in its realization that physicians’ reviewing the available studies could provide a disclosure that captures the difference between risk, relative risk, and causation.  In other words, physicians are free to tell patients that this thing called increased risk is not concerning because the studies are highly confounded, and they do not show causation.  Id. at 25-26.  Indeed, it would be hard to imagine an ethical physician telling patients anything else.

Dissent

Four of the Eight Circuit judges dissented, pointing to evidence that the South Dakota legislators intended to mandate a disclosure about causality.  Slip op. at 29.  Putting aside whether the truthfulness of the suicide advisory can be saved by reverting to a more modest interpretation of risk or of increased risk, the dissenters appear to have the better argument that the advisory is misleading.  The majority, however, by driving its wedge between causation and increased risk have allowed physicians to explain that the advisory has little or no meaning.

NOCEBO

The nocebo effect is the dark side of the placebo effect.  As pointed out recently in the Journal of the American Medical Association, nocebos can induce harmful outcomes because of the expectation of injury from the “psychosocial context or therapeutic environment” affecting patients’ perception of their health.  Luana Colloca & Damien Finniss, “Nocebo Effects, Patient-Clinician Communication, and Therapeutic Outcomes,” 307 J. Am. Med. Ass’n 567, 567 (2012).  It is fairly well accepted that clinicians can inadvertently prejudice health outcomes by how they frame outcome information to patients.  Colloca and Finniss note that the negative expectations created by nocebo communication can take place in the process of obtaining informed consent.

Unfortunately, there is no discussion of nocebo effects in the Eight Circuit’s decision. Planned Parenthood might well consider the role the nocebo effect has on the risk-benefit of an informed consent disclosure about a risk that really is not a risk, or is not a risk in the sense that it is a factor that will result in the putative cause, but rather only something that is under study and which cannot be separated from many confounding factors.  Surely, physicians in South Dakota will figure out how to give truthful, non-misleading disclosures that incorporate the mandatory suicide advisory, as well as the scientific evidence.

Posted in Causation, Risk and Risk Factor, Scientific Evidence, statistical evidence | Comments Off on Eighth Circuit Holds That Increased Risk Is Not Cause

Statistical Significance – Will Judicial Notice Substitute for An Expert Witness?

July 23rd, 2012

Do litigants in civil and criminal proceedings need statistical expert witnesses to present statistical analyses? Or, can lawyers take the data that are in evidence, and present their own statistical analyses?

Surely, lawyers could add figures to arrive at a sum, which is relevant to the issues in dispute.  Some lawyers and judges might be able to take model assumptions, and compare two means or two proportions, to show that they the statistics did not likely come from the same populations. Indeed, some lawyers may be able to do such analyses better than some expert witnesses, but this begs the question:  is it legally permissible?

In In re Pfizer Inc. Securities Litig., 584 F.Supp. 2d 621 (S.D.N.Y. 2008), defendant Pfizer filed a motion to dismiss a securities class action complaint.  The court found that Pfizer’s motion would require it to interpret statistical significance, and that it could not accept the parties’ non-expert assertions of the meaning of the concept; nor could the court take judicial notice of the meaning:

“The Court declines to take judicial notice of the meaning of statistical significance or of the data interpretations proffered by Defendants in the context of this motion practice. Rule 201 of the Federal Rules of Evidence provides that courts may only take notice of facts ‘either (1) generally known . . . or (2) capable of accurate and ready determination by resort to sources whose accuracy cannot reasonably be questioned’. Fed. R. Evid. 201(b). While statistical significance may have certain characteristics capable of general abstraction, it is far beyond the scope of Rule 201 to accept as fact the particular definitions of statistical significance proffered by Defendants as either facts generally known or as drawn from sources whose accuracy cannot reasonably be questioned. It is one thing to take notice of the fact that an author has written that 5% is the threshold for statistical significance. It is quite another thing entirely to use that 5% figure as a basis for rejecting the significance of complicated medical studies.”

Id. at 634. Similarly, the court refused to look at specific studies and conclude that they failed to find a statistically significant association between Celebrex and cardiovascular adverse events:

“A motion to dismiss a complaint is not an appropriate vehicle for determination as to the weight of the evidence, expert or otherwise. Clearly, the Court cannot take judicial notice that the three studies show a lack of any statistically significant link between Celebrex/Bextra and adverse cardiovascular events because that supposed fact is neither generally known nor capable of accurate and ready determination by reference to unquestionably accurate sources. Moreover, the Court cannot determine as a matter of law whether such links were statistically insignificant because statistical significance is a question of fact.”

Id. at 635.

In Bristol-Myers Squibb v. AIU Insurance Co., et al.,  Cause No. A-145,672, Jefferson County, 58th Judicial District, Texas, plaintiff’s counsel made a Batson challenge to the defendants’ exercise of peremptory challenges.  See Daily Transcript in (May 13, 1997).  Not having expected the defense counsel to exercise their peremptory challenges in an apparently discriminatory fashion, the plaintiff’s counsel did not have a statistician ready to analyze the pattern of challenges.  One of the plaintiff’s counsel presented the analysis in his oral argument to the court.  The venire panel was made up of 49 persons, 18 black, and 31 white.  The defense exercised 6 of their 7 peremptory challenges to black veniremen.  Based upon these numbers, plaintiff’s counsel presented a calculation of the probability that defense counsel would have exercised their challenges in such an extreme fashion if they made their choices independent of race. The defense objected to plaintiff’s counsel’s calculations, but the trial court overruled the objection and noted that the laws of probability were subject to judicial notice.  Id. at 828-30.

The Texas trial court found a prima facie case of discrimination, and permitted plaintiff’s counsel to cross-examine defense counsel about their peremptory and selection decisions.  Id. at 858.  The case settled shortly afterwards.  See also Andrew T. Berry, “Selecting Jurors,” 24 Litigation 8, 9 (Fall 1997)(“For example, in a recent (unreported) large civil case in the Southwest, the defendants successfully defeated a Batson challenge to their use of 85 percent of their peremptory challenges against protected class members. The successful defense? That the two dozen eminent counsel …, given less than a quarter-hour to exercise their peremptories were simply too disorganized to have struck jurors in violation of Batson.”)

In the welding fume MDL 1535, the plaintiffs persisted in challenges to a particular industry-funded, published epidemiologic study, which reported findings of no increased risks for Parkinson’s disease and parkinsonism among non-shipyard Danish welders.  Jon Fryzek, J. Hansen, S. Cohen, J. Bonde, et al., “A cohort study of Parkinson’s disease and other neurodegenerative disorders in Danish welders,” 47 J. Occup. & Envt’l Med. 466 (2005).  Plaintiffs’ counsel went to the extreme of traveling to Denmark, with one of their expert witnesses in tow, to analyze the underlying data for this study. Upon returning to the United States, the plaintiffs moved to bar reliance upon the Fryzek study, on the theory that the statistical analysis concerning the article’s finding of no statistically significant difference in the age of onset was incorrect.  In support of their argument, one of the plaintiff’s counsel, a law professor who was assisting plaintiffs in the welding fume litigation, submitted an affidavit in support of the motion in limine to bar defense witness’s testimony.  See Affidavit of Theodore Eisenberg, in In re Welding Fume Products Liability Litigation, Case No.: 1:03-cv-17000, MDL No. 1535, Document 1862 Filed 08/07/2006.

Eisenberg’s affidavit reported analyses of the Danish data, apparently based upon work done by an unnamed “programmer” at the Danish Cancer Society.  The affidavit included truncated computer program output, without identification of the statistical tests, or of the statistical software, used. Eisenberg interpreted the p-value result of the attached statistical analysis to show that there was a statistically significant difference in the age of onset of Parkinson’s disease between welders and non-welders.

The defense opposed the motion on grounds that Eisenberg’s affidavit was an ethically impermissible attempt by a lawyer in the case to present an expert witness opinion.  The defense also countered substantively with an affidavit from one of its expert witnesses, who analyzed the affidavit and realized that Eisenberg and the anonymous programmer had not presented the complete software output from their analyses, and that they had used a different test from that used in the published paper.  Eisenberg’s affidavit therefore had not identified an error in the published paper.  Declaration of Timothy L. Lash (Sept. 11, 2006), filed in In re Welding Fume Products Liability Litigation, Case No.: 1:03-cv-17000, MDL No. 1535.  The trial court denied the plaintiff’s motion to bar reliance upon the Fryzek study, without comment on the propriety of Eisenberg’s affidavit.

The MTBE mass tort litigation gave rise a peculiar instance in which a trial court held that a real estate value appraiser had departed from the level of intellectual rigor used in assessing property value changes, claimed to have resulted from a gas station’s pollution of the ground water in a small town in Orange County, New York.  The witness opined that the plaintiffs’ property suffered a 15% decline in market value, but he failed to identify the methods he used to arrive at his opinion. In re Methyl Tertiary Butyl Ether (“MTBE”) Prods. Liab. Litig., 2008 U.S. Dist. LEXIS 44216 (S.D.N.Y. June 4, 2008)(Scheindlin, J.).  The expert witness did explain that there were so few sales in the affected town that he could not use regression analysis, and that it was thus necessary to look at “trend data on sales by sub-markets, sales/list price analysis and days on the market comparisons.” Id. at *5.  Even so, the trial court could not otherwise discern what method the witness did use:

“In this case, I am unable to discern any method — much less a reliable method — that Langer used to reach his conclusion that the value of plaintiffs’ property decreased by fifteen percent because of MTBE contamination. Rather, Langer has merely compiled market data and then offered his conclusions, yet he has failed to explain the relationship between the two.”

Id. at *11.

Although the expert witness’s departure from the professional standard of care rendered his opinion inadmissible, the trial court decided that the would-be expert witness could still testify as a fact witness to the facts that he had collected about sales trends in the affected community and elsewhere.  According to the court, the statistics gathered by this witness were relevant, and the plaintiffs’ counsel could argue plausible inferences to the jury, from the sales figures.  Id. at *16-17.  The court thus remarkably permitted the plaintiffs’ counsel to provide the statistical analysis that his designated expert witness had failed to give in a legally reliably form.

Posted in Expert Witnesses, statistical evidence | Comments Off on Statistical Significance – Will Judicial Notice Substitute for An Expert Witness?

Discovery of Statistician Expert Witnesses

July 19th, 2012

This post has been updated and superseded by ““

Posted in Expert Witnesses, Rule 702, statistical evidence, Underlying Data | Comments Off on Discovery of Statistician Expert Witnesses

« Older Entries
Newer Entries »