The new, third edition of the Reference Manual on Scientific Evidence (RMSE) appears to get off to a good start in the Preface by Judge Kessler and Dr. Kassirer, when they note that the Supreme Court mandated federal courts to
“examine the scientific basis of expert testimony to ensure that it meets the same rigorous standard employed by scientific researchers and practitioners outside the courtroom.”
RMSE at xiii. The preface falters, however, on two key issues, causation and conflicts of interest, which are taken up as an introduction to the new volume.
1. CAUSATION
The authors tell us in squishy terms that causal assessments are judgments:
“Fundamentally, the task is an inferential process of weighing evidence and using judgment to conclude whether or not an effect is the result of some stimulus. Judgment is required even when using sophisticated statistical methods. Such methods can provide powerful evidence of associations between variables, but they cannot prove that a causal relationship exists. Theories of causation (evolution, for example) lose their designation as theories only if the scientific community has rejected alternative theories and accepted the causal relationship as fact. Elements that are often considered in helping to establish a causal relationship include predisposing factors, proximity of a stimulus to its putative outcome, the strength of the stimulus, and the strength of the events in a causal chain.”
RMSE at xiv.
The authors leave the inferential process as a matter of “weighing evidence,” but without saying anything about how the scientific community does its “weighing.” Language about “proving” causation is also unclear because “proof” in scientific parlance connotes a demonstration, which we typically find in logic or in mathematics. Proving empirical propositions suggests a bar set too high such that the courts must inevitable lower the bar considerably. The question is, of course, how low will judges go to admit evidence.
The authors thus introduce hand waving and excuses for why evidence can be weighed differently in court proceedings from the world of science:
“Unfortunately, judges may be in a less favorable position than scientists to make causal assessments. Scientists may delay their decision while they or others gather more data. Judges, on the other hand, must rule on causation based on existing information. Concepts of causation familiar to scientists (no matter what stripe) may not resonate with judges who are asked to rule on general causation (i.e., is a particular stimulus known to produce a particular reaction) or specific causation (i.e., did a particular stimulus cause a particular consequence in a specific instance). In the final analysis, a judge does not have the option of suspending judgment until more information is available, but must decide after considering the best available science.”
RMSE at xiv. But the “best available science” may be pretty crummy, and the temptation to turn desperation into evidence (“well, it’s the best we have now”) is often severe. The authors of the Preface signal that “inconclusive” is not a judgment open to judges charged with expert witness gatekeeping. If the authors truly mean to suggest that judges should go with whatever is dished out as “the best available science,” then they have overlooked the obvious: Rule 702 opens the door to “scientific, technical, or other specialized knowledge,” not to hunches, suggestive but inconclusive evidence, and wishful thinking about how the science may turn out when further along. Courts have a choice to exclude expert witness opinion testimony that is based upon incomplete or inconclusive evidence.
2. CONFLICTS OF INTEREST
Surprisingly, given the scope of the scientific areas covered in the RMSE, the authors discuss conflicts of interest (COI) at some length. Conflicts of interest are a fact of life in all endeavors, and it is understandable counsel judges and juries to try to identify, assess, and control them. COIs, however, are weak proxies for unreliability. The emphasis given here is undue because federal judges are misled into thinking that they can discern unreliability from COI, when they should be focused on the data and the analysis.
The authors of the Preface set about to use COI as a basis for giving litigation plaintiffs a pass, and for holding back studies sponsored by corporate defendants.
“Conflict of interest manifests as bias, and given the high stakes and adversarial nature of many courtroom proceedings, bias can have a major influence on evidence, testimony, and decisionmaking. Conflicts of interest take many forms and can be based on religious, social, political, or other personal convictions. The biases that these convictions can induce may range from serious to extreme, but these intrinsic influences and the biases they can induce are difficult to identify. Even individuals with such prejudices may not appreciate that they have them, nor may they realize that their interpretations of scientific issues may be biased by them. Because of these limitations, we consider here only financial conflicts of interest; such conflicts are discoverable. Nonetheless, even though financial conflicts can be identified, having such a conflict, even one involving huge sums of money, does not necessarily mean that a given individual will be biased. Having a financial relationship with a commercial entity produces a conflict of interest, but it does not inevitably evoke bias. In science, financial conflict of interest is often accompanied by disclosure of the relationship, leaving to the public the decision whether the interpretation might be tainted. Needless to say, such an assessment may be difficult. The problem is compounded in scientific publications by obscure ways in which the conflicts are reported and by a lack of disclosure of dollar amounts.
Judges and juries, however, must consider financial conflicts of interest when assessing scientific testimony. The threshold for pursuing the possibility of bias must be low. In some instances, judges have been frustrated in identifying expert witnesses who are free of conflict of interest because entire fields of science seem to be co-opted by payments from industry. Judges must also be aware that the research methods of studies funded specifically for purposes of litigation could favor one of the parties. Though awareness of such financial conflicts in itself is not necessarily predictive of bias, such information should be sought and evaluated as part of the deliberations.”
RMSE at xiv-xv. All in all, rather misleading advice. Financial conflicts are not the only conflicts that can be “discovered.” Often expert witnesses will have political and organizational alignments, which will show deep-seated ideological alignments with the party for which they are testifying. For instance, in one silicosis case, an expert witness in the field of history of medicine testified, at an examination before trial, that his father suffered from a silica-related disease. This witness’s alignment with Marxist historians and his identification with radical labor movements made his non-financial conflicts obvious, although these COI would not necessarily have been apparent from his scholarly publications alone.
How low will the bar be set for discovering COI? If testifying expert witnesses are relying upon textbooks, articles, essays, will federal courts open the authors/hearsay declarants up to searching discovery of their finances?
Also misleading is the suggestion that “entire fields of science seem to be co-opted by payments from industry.” Do the authors mean to exclude the plaintiffs’ lawyer litigation industry, which has grown so large and politically powerful in this country? In litigations in which I have been involved, I have certainly seen plaintiffs’ counsel, or their proxies – labor unions or “victim support groups” provide substantial funding for studies. The Preface authors themselves show an untoward bias by their pointing out industry payments without giving balanced attention to other interested parties’ funding of scientific studies.
The attention to COI is also surprising given that one of the key chapters, for toxic tort practitioners, was written by Dr. Bernard D. Goldstein, who has testified in toxic tort cases, mostly (but not exclusively) for plaintiffs. See, e.g., Parker v. Mobil Oil Corp., 7 N.Y.3d 434, 857 N.E.2d 1114, 824 N.Y.S.2d 584 (2006); Exxon Corp. v. Makofski, 116 SW 3d 176 (Tex. Ct. App. 2003). The Makofsky case is particularly interesting because Dr. Goldstein was forced to explain why he was willing to opine that benzene caused acute lymphocytic leukemia, despite the plethora of published studies finding no statistically significant relationship. Dr. Goldstein resorted to the inaccurate notion that scientific “proof” of causation requires 95 percent certainty, whereas he imposed only a 51 percent certainty for his medico-legal testimonial adventures. Dr. Goldstein also attempted to justify the discrepancy from the published literature by adverting to the lower standards used by federal regulatory agencies and treating physicians. Id.
These explanations are particularly concerning because they reflect basic errors in statistics and in causal reasoning. The 95 percent derives from the use of the same percentage in confidence intervals, but the probability involved there is not the probability of the association’s being correct, and it has nothing to do with the probability in the belief that an association is real or is causal. (Thankfully the RMSE chapter on statistics gets this right, but my fear is that judges will skip over the more demanding chapter on statistics and place undue weight on the toxicology chapter, written by Dr. Goldstein.) The reference to federal agencies (OSHA, EPA, etc.) and to treating physicians was meant, no doubt, to invoke precautionary principle concepts as a justification for some vague, ill-defined, lower standard of causal assessment.
The Preface authors might well have taken their own counsel and conducted a more searching assessment of COI among authors of Reference Manual. Better yet, the authors might have focused the judiciary on the data and the analysis.
