In the law of expert witness gatekeeping, the distinction between scientific claims made in support of litigation positions and claims made in support of regulations is fundamental. In re Agent Orange Product Liab. Litig., 597 F. Supp. 740, 781 (E.D.N.Y. 1984) (“The distinction between avoidance of risk through regulation and compensation for injuries after the fact is a fundamental one”), aff’d 818 F.2d 145 (2d Cir. 1987), cert. denied sub nom. Pinkney v. Dow Chemical Co., 487 U.S. 1234 (1988). Although scientists proffer opinions in both litigation and regulatory proceedings, their opinions are usually evaluated by substantially different standards. In federal litigation, civil and criminal, expert witnesses must be qualified and have an epistemic basis for their opinions, to satisfy the statutory requirements of Federal Rule of Evidence 702, and they must have reasonably relied upon otherwise inadmissible evidence (such as the multiple layers of hearsay involved in an epidemiologic study) under Rule 703. In regulatory proceedings, scientists are not subject to admissibility requirements and the sufficiency requirements set by the Administrative Procedures Act are extremely low[1].
Some industry stakeholders are aggrieved by the low standards for scientific decision making in certain federal agencies, and they have urged that the more stringent litigation evidentiary rules be imported into regulatory proceedings. There are several potential problems with such reform proposals. First, the epistemic requirements of science generally, or of Rules 702 and 703 in particular, are not particularly stringent. Scientific method leads to plenty of false positive and false negative conclusions, which are subject to daily challenge and revision. Scientific inference is not necessarily so strict, as much as ordinary reasoning is so flawed, inexact, and careless. Second, the call for “regulatory Daubert” ignores mandates of some federal agency enabling statutes and guiding regulations, which call for precautionary judgments, and which allow agencies to decide issues on evidentiary display that fall short of epistemic warrants for claims of knowledge.
Many lawyers who represent industry stakeholders have pressed for extension of Daubert-type gatekeeping to federal agency decision making. The arguments for constraining agency action find support in the over-extended claims that agencies and so-called public interest science advocates make in support of agency measures. Advocates and agency personnel seem to believe that worst-case scenarios and overstated safety claims are required as “bargaining” positions to achieve the most restrictive and possibly the most protective regulation that can be gotten from the administrative procedure, while trumping industry’s concerns about costs and feasibility. Still, extending Daubert to regulatory proceedings could have the untoward result of lowering the epistemic bar for both regulators and litigation fact finders.
In a recent article, Professor David Bernstein questions the expansion of Daubert into some regulatory realms. David E. Bernstein, “What to Do About Federal Agency Science: Some Doubts About Regulatory Daubert,” 22 Geo. Mason L. Rev. 549 (2015)[cited as Bernstein]. His arguments are an important counterweight to those who insist on changing agency rulemaking and actions at every turn. As an acolyte and a defender of scientific scruples and reasoning in the courts, Bernstein’s arguments are worth taking seriously.
Bernstein reminds us that bad policy, as seen in regulatory agency rulemaking or decisions, is not always a scientific issue. In any event, regulatory actions, unlike jury decisions, are not, or at least should not be, “black boxes.” The agency’s rationale and reasoning are publicly stated, subject to criticism, and open to revision. Jury decisions are opaque, non-transparent, potentially unreasoned, not carefully articulated, and not subject to revision absent remarkable failures of proof.
One line of argument[2] pursued by Professor Bernstein follows from his observation that Daubert procedures are required to curtail litigation expert witness “adversarial bias.” Id. at 555. Bernstein traces adversarial bias to three sources:
(1) conscious bias;
(2) unconscious bias; and
(3) selection bias.
Id. Conscious bias stems from deliberate attempts by “hired guns” to deliver opinions that satisfy the lawyers who retained them. The problem of conscious bias is presented by “hired guns” who will adapt their opinions to the needs of the attorney who hires them. Unconscious biases are the more subtle, but no less potent determinants of expert witness behavior, which are created by financial dependence upon, and allegiance to, the witness’s paymaster. Selection bias results from lawyers’ ability to choose expert witnesses to support their claims, regardless whether those witnesses’ opinions are representative of the scientific community. Id.
Professor Bernstein’s taxonomy of bias is important, but incomplete. First, the biases he identifies operate fulsomely in regulatory settings. Although direct financial remuneration is usually not a significant motivation for a scientist to testify before an agency, or to submit a whitepaper, professional advancement and cause advocacy are often powerful incentives at work. These incentives for self-styled public interest zealots may well create more powerful distortions of scientific judgment than any monetary factors in private litigation settings. As for selection bias, lawyers are ethically responsible for screening their expert witnesses, and there can be little doubt that once expert witnesses are disclosed, their opinions will align with their sponsoring parties’ interests. This systematic bias, however, does not necessarily mean that both side’s expert witnesses will necessarily be unrepresentative or unscientific. In the silicone gel breast implant litigation (MDL 926), Judge Pointer, the presiding judge, insisted that both sides’ witnesses were “too extreme,” and he was stunned when his court-appointed expert witnesses filed reports that vindicated the defendants’ expert witnesses’ positions[3]. The defendants had selected expert witnesses who analyzed the data on sound scientific principles; the plaintiffs had selected expert witnesses who overreached in their interpretation of the evidence. Furthermore, many scientific disputes, which find their way into the courtroom, will not have the public profile of silicone gel breast implants, and for which there may be no body of scientific community opinion from which lawyers could select “outliers,” even if they wished to do so.
Professor Bernstein’s offered taxonomy of bias is incomplete because it does not include the most important biases that jurors (and many judges) struggle to evaluate:
random errors;
systematic biases;
confounding; and
cognitive biases.
These errors and biases, along with their consequential fallacies of reasoning, apply with equal force to agency and litigation science. Bernstein does point out, however, an important institutional difference between jury or judge trials and agency review and decisions based upon scientific evidence: agencies often have extensive in-house expertise. Although agency expertise may sometimes be blinded by its policy agenda, agency procedures usually afford the public and the scientific community to understand what the agency decided, and why, and to respond critically when necessary. In the case of the Food and Drug Administration, agency decisions, whether pro- or contra-industry positions are dissected and critiqued by the scientific and statistical community with great care and relish. Nothing of the same sort is possible in response to a jury verdict.
Professor Bernstein is not a science nihilist, and he would not have reviewing courts give a pass to whatever nonsense federal agencies espouse. He calls for enforcement of available statutory requirements that agency action be based upon the “best available science,” and for requiring agencies to explicitly separate and state their policy and scientific judgments. Bernstein also urges greater use of agency peer review, such as occasionally seen from the Institute of Medicine (soon to be the National Academy of Medicine), and the use of Daubert-like criteria for testimony at agency hearings. Bernstein at 554.
Proponents of regulatory Daubert should take Professor Bernstein’s essay to heart, with a daily dose of atorvastatin. Importing Rule 702 into agency proceedings may well undermine the rule’s import in litigation, civil and criminal, while achieving little in the regulatory arena. Consider the pending OSHA rulemaking for lowering the permissible exposure limit (PEL) of crystalline silica in the workplace. OSHA, and along with some public health organizations, has tried to justify this rulemaking on the basis of many overwrought claims of the hazards of crystalline silica exposure at current levels. Clearly, there are some workers who continue to work in unacceptably hazardous conditions, but the harms sustained by these workers can be tied to violations of the current PEL; they are hardly an argument for lowering that current PEL. Contrary to the OSHA’s parade of horribles, silicosis mortality in the United States has steadily declined over the last several decades. The following chart draws upon NIOSH and other federal governmental data:
Silicosis deaths, crude and age-adjusted death rates, for U.S. residents age 15 and over, 1968–2007
from Susan E. Dudley & Andrew P. Morriss, “Will the Occupational Safety and Health Administration’s Proposed Standards for Occupational Exposure to Respirable Crystalline Silica Reduce Workplace Risk?” 35 Risk Analysis (2015), in press, doi: 10.1111/risa.12341 (NIOSH reference number: 2012F03–01, based upon multiple cause-of-death data from National Center for Health Statistics, National Vital Statistics System, with population estimates from U.S. Census Bureau).
The decline in silicosis mortality is all the more remarkable because it occurred in the presence of stimulated reporting from silicosis litigation, and misclassification of coal workers’ pneumoconiosis in coal-mining states.
The decline in silicosis mortality may be helpfully compared with the steady rise in mortality from accidental falls among men and women 65 years old, or older:
Yahtyng Sheu, Li-Hui Chen, and Holly Hedegaard, “QuickStats: Death Rates* from Unintentional Falls† Among Adults Aged ≥ 65 Years, by Sex — United States, 2000–2013,” 64 CDC MMWR 450 (May 1, 2015). Over the observation period, these death rates roughly doubled in both men and women.
Is there a problem with OSHA rulemaking? Of course. The agency has gone off on a regulatory frolic and detour trying to justify an onerous new PEL, without any commitment to enforcing its current silica PEL. OSHA has invoked the prospect of medical risks, many of which are unproven, speculative, and remote, such as lung cancer, autoimmune disease, and kidney disease. The agency, however, is awash with PhDs, and I fear that Professor Bernstein is correct that the distortions of the science are not likely to be corrected by applying Rule 702 to agency factfinding. Courts, faced with the complex prediction models, with disputed medical claims made by agency and industry scientists, will do what they usually do, shrug and defer. And the blow back of the “judicially approved” agency science in litigation contexts will be a cure worse than the disease. At bottom, the agency twisting of science is driven by policy goals and considerations, which require public debate and scrutiny, sound executive judgment, with careful legislative oversight and guidance.
[1] Even under the very low evidentiary and procedural hurdles, federal agencies still manage to outrun their headlights on occasion. See, e.g., Industrial Union Department v. American Petroleum Institute, 448 U.S. 607 (1980) (The Benzene Case); Gulf South Insulation v. U.S. Consumer Product Safety Comm’n, 701 F.2d 1137 (5th Cir. 1983); Corrosion Proof Fittings v. EPA, 947 F2d 1201 (5th Cir 1991).
[2] See also David E. Bernstein, “The Misbegotten Judicial Resistance to the Daubert Revolution,” 89 Notre Dame L. Rev. 27, 31 (2013); David E. Bernstein, “Expert Witnesses, Adversarial Bias, and the (Partial) Failure of the Daubert Revolution,” 93 Iowa L. Rev. 451, 456–57 (2008).
[3] Judge Pointer was less than enthusiastic about performing any gatekeeping role. Unlike most of today’s MDL judges, he was content to allow trial judges in the transferor districts to decide Rule 702 and other pre-trial issues. See Note, “District Judge Takes Issue With Circuit Courts’ Application of Gatekeeping Role” 3 Federal Discovery News (Aug. 1997) (noting that Chief Judge Pointer had criticized appellate courts for requiring district judges to serve as gatekeepers of expert witness testimony).
