TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Earthquake-Induced Data Loss – We’re All Shook Up

June 26th, 2015

Adam Marcus and Ivan Oransky are medical journalists who publish the Retraction Watch blog. Their blog’s coverage of error, fraud, plagiarism, and other publishing disasters is often first-rate, and a valuable curative for the belief that peer review publication, as it is now practiced, ensures trustworthiness.

Yesterday, Retraction Watch posted an article on earthquake-induced data loss. Shannon Palus, “Lost your data? Blame an earthquake” (June 25, 2015). A commenter on PubPeer raised concerns about a key figure in a paper[1]. The authors acknowledged a problem, which they traced to their loss of data in an earthquake. The journal retracted the paper.

This is not the first instance of earthquake-induced loss of data.

When John O’Quinn and his colleagues in the litigation industry created the pseudo-science of silicone-induced autoimmunity, they recruited Nir Kossovsky, a pathologist at UCLA Medical Center. Although Kossovsky looked a bit like Pee-Wee Herman, he was a graduate of the University of Chicago Pritzker School of Medicine, and the U.S. Naval War College, and a consultant to the FDA. In his dress whites, Kossovsky helped O’Quinn sell his silicone immunogenicity theories to juries and judges around the country. For a while, the theories sold well.

In testifying and dodging discovery for the underlying data in his silicone studies, Kossovsky was as slick as silicone itself. Ultimately, when defense counsel subpoenaed the underlying data from Kossovsky’s silicone study, Kossovsky shrugged and replied that the Northridge Earthquake destroyed his data. Apparently coffee cups and other containers of questionable fluids spilled on his silicone data in the quake, and Kossovsky’s emergency response was to obtain garbage cans and throw out the data. For the gory details, see Gary Taubes, “Silicone in the System: Has Nir Kossovsky really shown anything about the dangers of breast implants?” Discover Magazine (Dec. 1995).

As Mr. Taubes points out, Kossovsky’s paper was rejected by several journals before being published in the Journal of Applied Biomaterials, of which Kossovsky was a member of the editorial board. The lack of data did not, however, keep Kossovsky from continuing to testify, and from trying to commercialize, along with his wife, Beth Brandegee, and his father, Ram Kossowsky[2], an ELISA-based silicone “antibody” biomarker diagnostic test, Detecsil. Although Rule 702 had been energized by the Daubert decision in 1993, many judges were still not willing to take a hard look at Kossovsky’s study, his test, or to demand the supposedly supporting data. The Food and Drug Administration, however, eventually caught up with Kossovsky, and the Detecsil marketing ceased. Lillian J. Gill, FDA Acting Director, Office of Compliance, Letter to Beth S. Brandegee, President, Structured Biologicals (SBI) Laboratories: Detecsil Silicone Sensitivity Test (July 15, 1994); see Taubes, Discover Magazine.

After defense counsel learned of the FDA’s enforcement action against Kossovsky and his company, the litigation industry lost interest in Kossovsky, and his name dropped off trial witness lists. His name also dropped off the rolls of tenured UCLA faculty, and he apparently left medicine altogether to become a business consultant. Dr. Kossovsky became “an authority on business process risk and reputational value.” Kossovsky is now the CEO and Director of Steel City Re, which specializes in strategies for maintaining and enhancing reputational value. Ironic; eh?

A review of PubMed’s entries for Nir Kossovsky shows that his run in silicone started in 1983, and ended in 1996. He testified for plaintiffs in Hopkins v. Dow Corning Corp., 33 F.3d 1116 (9th Cir.1994) (tried in 1991), and in the infamous case of Johnson v. Bristol-Myers Squibb, CN 91-21770, Tx Dist. Ct., 125th Jud. Dist., Harris Cty., 1992.

A bibliography of Kossovsky silicone oeuvre is listed, below.

[1] Federico S. Rodríguez, Katterine A. Salazar, Nery A. Jara, María A García-Robles, Fernando Pérez, Luciano E. Ferrada, Fernando Martínez, and Francisco J. Nualart, “Superoxide-dependent uptake of vitamin C in human glioma cells,” 127 J. Neurochemistry 793 (2013).

[2] Father and son apparently did not agree on how to spell their last name.

Nir Kossovsky, D. Conway, Ram Kossowsky & D. Petrovich, “Novel anti-silicone surface-associated antigen antibodies (anti-SSAA(x)) may help differentiate symptomatic patients with silicone breast implants from patients with classical rheumatological disease,” 210 Curr. Topics Microbiol. Immunol. 327 (1996)

Nir Kossovsky, et al., “Preservation of surface-dependent properties of viral antigens following immobilization on particulate ceramic delivery vehicles,” 29 J. Biomed. Mater. Res. 561 (1995)

E.A. Mena, Nir Kossovsky, C. Chu, and C. Hu, “Inflammatory intermediates produced by tissues encasing silicone breast prostheses,” 8 J. Invest. Surg. 31 (1995)

Nir Kossovsky, “Can the silicone controversy be resolved with rational certainty?” 7 J. Biomater. Sci. Polymer Ed. 97 (1995)

Nir Kossovsky & C.J. Freiman, “Physicochemical and immunological basis of silicone pathophysiology,” 7 J. Biomater. Sci. Polym. Ed. 101 (1995)

Nir Kossovsky, et al., “Self-reported signs and symptoms in breast implant patients with novel antibodies to silicone surface associated antigens [anti-SSAA(x)],” 6 J. Appl. Biomater. 153 (1995), and “Erratum,” 6 J. Appl. Biomater. 305 (1995)

Nir Kossovsky & J. Stassi, “A pathophysiological examination of the biophysics and bioreactivity of silicone breast implants,” 24s1 Seminars Arthritis & Rheum. 18 (1994)

Nir Kossovsky & C.J. Freiman, “Silicone breast implant pathology. Clinical data and immunologic consequences,” 118 Arch. Pathol. Lab. Med. 686 (1994)

Nir Kossovsky & C.J. Freiman, “Immunology of silicone breast implants,” 8 J. Biomaterials Appl. 237 (1994)

Nir Kossovsky & N. Papasian, “Mammary implants,” 3 J. Appl. Biomater. 239 (1992)

Nir Kossovsky, P. Cole, D.A. Zackson, “Giant cell myocarditis associated with silicone: An unusual case of biomaterials pathology discovered at autopsy using X-ray energy spectroscopic techniques,” 93 Am. J. Clin. Pathol. 148 (1990)

Nir Kossovsky & R.B. Snow RB, “Clinical-pathological analysis of failed central nervous system fluid shunts,” 23 J. Biomed. Mater. Res. 73 (1989)

R.B. Snow & Nir Kossovsky, “Hypersensitivity reaction associated with sterile ventriculoperitoneal shunt malfunction,” 31 Surg. Neurol. 209 (1989)

Nir Kossovsky & Ram Kossowsky, “Medical devices and biomaterials pathology: Primary data for health care technology assessment,” 4 Internat’l J. Technol. Assess. Health Care 319 (1988)

Nir Kossovsky, John P. Heggers, and M.C. Robson, “Experimental demonstration of the immunogenicity of silicone-protein complexes,” 21 J. Biomed. Mater. Res. 1125 (1987)

Nir Kossovsky, John P. Heggers, R.W. Parsons, and M.C. Robson, “Acceleration of capsule formation around silicone implants by infection in a guinea pig model,” 73 Plastic & Reconstr. Surg. 91 (1984)

John Heggers, Nir Kossovsky, et al., “Biocompatibility of silicone implants,” 11 Ann. Plastic Surg. 38 (1983)

Nir Kossovsky, John P. Heggers, et al., “Analysis of the surface morphology of recovered silicone mammary prostheses,” 71 Plast. Reconstr. Surg. 795 (1983)

Posted in Expert Witness Hall of Shame, Expert Witnesses, Rule 702, Scientific Evidence, Scientific Misconduct | Comments Off on Earthquake-Induced Data Loss – We’re All Shook Up

The One Percent Non-solution – Infante Fuels His Own Exclusion in Gasoline Leukemia Case

June 25th, 2015

Most epidemiologic studies are not admissible. Such studies involve many layers of hearsay evidence, measurements of exposures, diagnoses, records, and the like, which cannot be “cross-examined.” Our legal system allows expert witnesses to rely upon such studies, although clearly inadmissible, when “experts in the particular field would reasonably rely on those kinds of facts or data in forming an opinion on the subject.” Federal Rule of Evidence 703. One of the problems that judges face in carrying out their gatekeeping duties is to evaluate whether challenged expert witnesses have reasonably relied upon particular studies and data. Judges, unlike juries, have an obligation to explain their decisions, and many expert witness gatekeeping decisions by judges fall short by failing to provide citations to the contested studies at issue in the challenge. Sometimes the parties may be able to discern what is being referenced, but the judicial decision has a public function that goes beyond speaking to the litigants before the court. Without full citations to the studies that underlie an expert witness’s opinion, the communities of judges, lawyers, scientists, and others cannot evaluate the judge’s gatekeeping. Imagine a judicial opinion that vaguely referred to a decision by another judge, but failed to provide a citation? We would think such an opinion to be a miserable failure of the judge’s obligation to explain and justify the resolution of the matter, as well as a case of poor legal scholarship. The same considerations should apply to the scientific studies relied upon by an expert witness, whose opinion is being discussed in a judicial opinion.

Judge Sarah Vance’s opinion in Burst v. Shell Oil Co., C. A. No. 14–109, 2015 WL 3755953 (E.D. La. June 16, 2015) [cited as Burst], is a good example of judicial opinion writing, in the context of deciding an evidentiary challenge to an expert witness’s opinion, which satisfies the requirements of judicial opinion writing, as well as basic scholarship. The key studies relied upon by the challenged expert witness are identified, and cited, in a way that permits both litigants and non-litigants to review Her Honor’s opinion, and evaluate both the challenged expert witness’s opinion, and the trial judge’s gatekeeping performance. Citations to the underlying studies creates the delicious possibility that the trial judge might actually have read the papers to decide the admissibility question. On the merits, Judge Vance’s opinion in Burst also serves as a good example of judicial scrutiny that cuts through an expert witness’s hand waving and misdirection in the face of inadequate, inconsistent, and insufficient evidence for a causal conclusion.

Burst is yet another case in which plaintiff claimed that exposure to gasoline caused acute myeloid leukemia (AML), one of several different types of leukemia[1]. The claim is fraught with uncertainty and speculation in the form of extrapolations between substances, from high to low exposures, and between diseases.

Everyone has a background exposure to benzene from both natural and anthropogenic sources. Smoking results in approximately a ten-fold elevation of benzene exposure. Agency for Toxic Substances and Disease Registry (ATSDR) Public Health Statement – Benzene CAS#: 71-43-2 (August 2007). Gasoline contains small amounts of benzene, on the order of 1 percent or less. U.S. Environmental Protection Agency (EPA), Summary and Analysis of the 2011 Gasoline Benzene Pre-Compliance Report (2012).

Although gasoline has always contained benzene, the quantitative difference in levels of benzene exposure involved in working with concentrated benzene and with gasoline has led virtually all scientists and regulatory agencies to treat the two exposures differently. Benzene exposure is a known cause of AML; gasoline exposure, even in occupational contexts, is not taken to be a known cause of AML. Dose matters.

Although the reviews of the International Agency for Research on Cancer (IARC) are sometimes partisan, incomplete, and biased towards finding carcinogenicity, the IARC categorizes benzene as a known human carcinogen, in large part because of its known ability to cause AML, but regards the evidence for gasoline as inadequate for making causal conclusions. IARC, Monographs on the Evaluation of Carcinogenic Risks to Humans, Vol. 45, Occupational Exposures in Petroleum Refining; Crude Oil and Major Petroleum Fuels (1989) (“There is inadequate evidence for the carcinogenicity in humans of gasoline.”) (emphasis in original)[2].

To transmogrify a gasoline case into a benzene case, plaintiff called upon Peter F. Infante, a fellow of the white-hat conspiracy, Collegium Ramazzini, and an adjunct professor at George Washington University School of Public Health and Health Services. Previously, Dr. Infante was Director of OHSA’s Office of Standards Review (OSHA). More recently, Infante is known as the president and registered agent of Peter F. Infante Consulting, LLC, in Falls Church, Virginia, and a go-to expert witness for plaintiffs in toxic tort litigation[3].

In the Burst case, Infante started out in trouble, by claiming that he had he “followed the methodology of the International Agency for Research on Cancer (IARC) and of the Occupational Safety and Health Administration (OSHA) in evaluating epidemiological studies, case reports and toxicological studies of benzene exposure and its effect on the hematopoietic system.” Burst at *4. Relying upon the IARC’s methodology might satisfy some uncritical courts, but here the IARC itself sharply distinguished its characterizations of benzene and gasoline in separate reviews. Infante’s opinion ignored this divide, although it ultimately had to connect gasoline exposure to the claimed injury[4].

Judge Vance found that Infante’s proffered opinions ransacked the catalogue of expert witness errors. Infante:

  • relied upon studies of benzene exposure and diseases other than the outcome of interest, AML. Burst at *4, *10, *13.
  • relied upon studies of benzene exposure rather than gasoline exposure. Burst at *9.
  • relied upon studies that assessed outcomes in groups with multiple exposures, which studies were hopelessly confounded. Burst at *7.
  • failed to acknowledge the inconsistency of outcomes in the studies of the relevant exposure, gasoline. Burst at *9.
  • relied upon studies that lacked adequate exposure measurements and characterizations, which lack was among the reasons that the ATSDR declined to label gasoline a carcinogen. Burst at *12.
  • relied upon studies that did not report statistically significant associations between gasoline exposure and AML. Burst at *10, *12
  • cherry picked studies and failed to explain contrary results. Burst at *10.
  • cherry picked data from within studies that did not otherwise support his conclusion. Burst at *10.
  • interpreted studies at odds with how the authors of published papers interpreted their own studies. Burst at *10.
  • failed to reconcile conflicting studies. Burst at *10.
  • manipulated data without sufficient explanation or justification. Burst at *14.
  • failed to conduct an appropriate analysis of the entire dataset, along the lines of Sir Austin Bradford Hill’s nine factors. Burst at *10.

The manipulation charge is worth further discussion because it reflects upon the trial court’s acumen and the challenged witness’s deviousness. Infante combined the data from two exposure subgroups from one study[5] to claim that the study actually had a statistically significant association. The trial court found that Dr. Infante failed to explain or justify the recalculation. Burst at *14. At the pre-trial hearing, Dr. Infante offered that he performed the re-calculation on a “sticky note,” but failed to provide his calculations. The court might also have been concerned about the misuse of claiming statistical significance in a post-hoc, non-prespecified analysis that would have clearly raised a multiple comparisons issue. Infante also combined two separate datasets from an unpublished study (the Spivey study for Union Oil), which the court found problematic for his failure to explain and justify the aggregation of data. Id. This recalculation raises the issue whether the two separate datasets could be appropriately combined.

For another study[6], Infante adjusted the results based upon his assessment that the study was biased by a “healthy worker effect[7].” Burst at *15. Infante failed to provide any explanation of how he adjusted for the healthy worker effect, thus giving the court no basis for evaluating the reliability of his methodology. Perhaps more telling, the authors of this study acknowledged the hypothetical potential for healthy worker bias, but chose not to adjust for it because their primary analyses were conducted internally within the working study population, which fully accounted for the potential bias[8].

The court emphasized that it did not question whether combining datasets or adjusting for bias was accepted or proper methodology; rather it focused its critical scrutiny on Infante’s refusal or failure to explain and justify his post-hoc “manipulations of published data.” Burst at *15. Without a showing that AML is more common among non-working, disabled men, the health worker adjustment could well be questioned.

In the final analysis, Infante’s sloppy narrative review could not stand in the face of obviously inconsistent epidemiologic data. Burst at *16. The trial court found that Dr. Infante’s methodology of claiming reliance upon multiple studies, which did not reliably (validly) support his claims or “fit” his conclusions, failed to satisfy the requirements of Federal Rule of Evidence 702. The analytical gap between the data and the opinion were too great. Id. at *8. Infante’s opinion fell into the abyss[9].

[1] See, e.g., Castellow v. Chevron USA, 97 F. Supp. 2d 780, 796 (S.D.Tex.2000) (“Plaintiffs here have not shown that the relevant scientific or medical literature supports the conclusion that workers exposed to benzene, as a component of gasoline, face a statistically significant risk of an increase in the rate of AML.”); Henricksen v. Conoco Phillips Co., 605 F.Supp.2d 1142, 1175 (E.D.Wa. 2009) (“None of the studies relied upon have concluded that gasoline has the same toxic effect as benzene, and none have concluded that the benzene component of gasoline is capable of causing AML.”); Parker v. Mobil Oil Corp., 7 N.Y.3d 434, 450 (N.Y.2006) (“[N]o significant association has been found between gasoline exposure and AML. Plaintiff’s experts were unable to identify a single epidemiologic study finding an increased risk of AML as a result of exposure to gasoline.”).

[2] See also ATSDR Toxicological Profile for Gasoline (1995) (concluding “there is no conclusive evidence to support or refute the carcinogenic potential of gasoline in humans or animals based on the carcinogenicity of one of its components, benzene”); ATSDR, Public Health Statement for Automotive Gasoline (June 1995) (“[However, there is no evidence that exposure to gasoline causes cancer in humans. There is not enough information available to determine if gasoline causes birth defects or affects reproduction.”).

[3] See, e.g., Harris v. CSX Transp., Inc., 753 SE 2d 275, 232 W. Va. 617 (2013); Henricksen v. ConocoPhillips Co., 605 F. Supp. 2d 1142 (E.D. Wash. 2009); Roney v. GENCORP, Civil Action No. 3: 05-0788 (S.D.W. Va. Sept. 18, 2009); Chambers v. Exxon Corp., 81 F. Supp. 2d 661 (M.D. La. 2000).

[4] Judge Vance did acknowledge that benzene studies were relevant to Infante’s causation opinion, but emphasized that such studies could not suffice to show that all gasoline exposures could cause AML. Burst at *10 (citing Dickson v. Nat’l Maint. & Repair of Ky., Inc., No. 5:08–CV–00008, 2011 WL 12538613, at *6 (W.D. Ky. April 28, 2011) (“Benzene may be considered a causative agent despite only being a component of the alleged harm.”).

[5] L. Rushton & H. Romaniuk, “A Case-Control Study to Investigate the

Risk of Leukaemia Associated with Exposure to Benzene in Petroleum Marketing and Distribution Workers in the United Kingdom,” 54 Occup. & Envt’l Med. 152 (1997).

[6] Otto Wong, et al., “Health Effects of Gasoline Exposure. II. Mortality Patterns of Distribution Workers in the United States,” 101 Envt’l Health Persp. 6 (1993).

[7] Burst at *15, citing and quoting from John Last, A Dictionary of Epidemiology (3d ed.1995) (“Workers usually exhibit lower overall death rates than the general population because the severely ill and chronically disabled are ordinarily excluded from employment.”).

[8] Wong, supra.

[9] In a separate opinion, Judge Vance excluded a physician, Dr. Robert Harrison, who similarly opined that gasoline causes AML, and Mr. Burst’s AML, without the benefit of sound science to support his opinion. Burst v. Shell Oil Co., C. A. No. 14–109, 2015 WL 2015 WL 3620111 (E.D. La. June 9, 2015).

Posted in Causation, Expert Witnesses, Rule 702 | Comments Off on The One Percent Non-solution – Infante Fuels His Own Exclusion in Gasoline Leukemia Case

Diclegis and Vacuous Philosophy of Science

June 24th, 2015

Just when you thought that nothing more could be written intelligently about the Bendectin litigation, you find out you are right. Years ago, Michael Green and Joseph Sanders each wrote thoughtful, full-length books[1] about the litigation assault on the morning-sickness (nausea and vomiting of pregnancy) medication, which was voluntarily withdrawn by its manufacturer from the United States market. Dozens, if not hundreds, of law review articles discuss the scientific issues, the legal tactics, and the judicial decisions in the U.S. Bendectin litigation, including the Daubert case in Supreme Court and in the Ninth Circuit. But perhaps fresh eyes might find something new and fresh to say.

Boaz Miller teaches social epistemology and philosophy of science, and he recently weighed in on the role that scientific consensus played in resolving the Bendectin litigation. Miller, “Scientific Consensus and Expert Testimony in Courts: Lessons from the Bendectin Litigation,” Foundations of Science (2014) (Oct. 17, 2014) (in press) [cited as Miller]. Miller astutely points out that scientific consensus may or may not be epistemic, that is, based upon robust, valid, sufficient scientific evidence of causality. Scientists are people, and sometimes they come to conclusions based upon invalid evidence, or because of cognitive biases, or social pressures, and the like. Sometimes scientists get the right result for the wrong reasons. From this position he argues that adverting to scientific consensus is fraught with danger of being misled, and that the Bendectin ligitation specifically is an example of courts led astray by a “non-epistemic” scientific consensus. Miller at 1.

Miller is correct that the scientific consensus on Bendectin’s safety, which emerged after the initiation of litigation, played a role in resolving the litigation, id. at 8, but he badly misunderstands how the consensus actually operated to bring closure to the birth defects litigation. Remarkably, he pays no attention to the consolidated trial of over 800 cases before the Hon. Carl B. Rubin, in the Southern District of Ohio. This trial resulted in a defense verdict in March 1985, and judgment that withstood appellate review. Assoc. Press, “U.S. Jury Clears a Nausea Drug in Birth Defects,” N.Y. Times (Mar. 13, 1985). The subsequent litigation turned into guerilla warfare based upon relatively few remaining individual cases in state and federal courts. In one of the state court cases, the trial court appointed neutral expert witnesses, who opined that plaintiffs had failed to make out their causal claims of teratogenicity in human beings. DePyper v. Navarro, No. 83–303467-NM, 1995 WL 788828 (Mich. Cir. Ct. Nov. 27, 1995).

To be sure, plaintiffs’ expert witnesses and plaintiffs’ counsel continued in their campaign to manufacture “reasonable medical certainty” of Bendectin’s teratogenicity, well after a scientific consensus emerged. Boaz Miller makes the stunning claim that this consensus was not a “knowledge-based” consensus because:

(1) the research was controlled by parties to the dispute (Miller at 10);

(2) the consensus ignored or diminished the “value” of in vitro toxicology (Miller at 13);

(3) the consensus relied upon most heavily upon the epidemiologic evidence (Miller at 14);

(4) the animal toxicology research was “prematurely” abandoned when the U.S. withdrew its product from the market (Miller at 15); and

(5) the withdrawal ended the “threat” to public health, and the concerns about teratogenicity (Miller at 15).

Miller’s asserted reasons are demonstrably incorrect. Although Merrell Richardson funded some studies early on, by the time the scientific consensus emerged, many studies funded by neutral sources, and conducted by researchers of respected integrity, were widely available. The consensus did not diminish the value of in vivo toxicology; rather scientists evaluated the available evidence through their understanding of epidemiology’s superiority in assessing actual risks in human populations. Animal studies were not prematurely abandoned; more accurately, the animal studies gave way to more revealing, valid studies in humans about human outcomes. The sponsor’s withdrawal of Bendectin in the United States was not the cause of any abandonment of research. The drug remained available outside the United States, in countries with less rapacious tort systems, and researchers would have, in any event, continued animal studies if there were something left open by previous research. A casual browse through PubMed’s collection of articles on thalidomide shows that animal research continued well after that medication had been universally withdrawn for use in pregnant women. Given that thalidomide was unquestionably a human teratogen, there was a continued interest in understanding its teratogenicity. No such continued interest existed for Bendectin after the avalanche of exculpatory human data.

What sort of inquiry permitted Miller to reach his conclusions? His article cites no studies, no whole-animal toxicology, no in vitro research, no epidemiologic studies, no systematic reviews, no regulatory agency reviews, and no meta-analysis. All exist in abundance. The full extent of his engagement with the actual scientific data and issues is a reference to an editorial and two letters to the editor[2]! From the exchange of views in one set of letters in 1985, Miller infers that there was “clear dissent within the community of toxicologists.” Miller at 13. The letters in question, however, were written in a journal of teratology, which was not limited to toxicology, and the interlocutors were well aware of the hierarchy of evidence that placed human observational studies at the top of the evidential pyramid.

Miller argues that it was possible that the consensus was not knowledge-based because it might have reflected the dominance of epidemiology over the discipline of toxicology. Again, he ignores the known dubious validity of inferring human teratogenicity from high dose whole animal or in vitro toxicology. By the time the scientific consensus emerged with respect to Bendectin’s safety, this validity point was widely appreciated by all but the most hardened rat killers, and plaintiffs’ counsel. In less litigious countries, the drug never left the market. No regulatory agency ever called for its withdrawal.

Miller might have tested whether the scientific community’s consensus on Bendectin, circa 1992 (when Daubert was being briefed in the Supreme Court) was robust by looking to how well it stood up to further testing. He did not, but he could easily have found the following. The U.S. sponsor of Diclegis, Duchesnay USA, sought and obtained the indication for its medication in pregnancy. Under U.S. law, Duchesnay’s new drug application had to establish safety and efficacy for this indication. In 2013, the U.S. FDA approved Bendectin, under the tradename, Diclegis[3], as a combination of doxylamine succinate and pyridoxine hydrochloride for sale in the United States. Under the FDA’s pregnancy labeling system, Diclegis is a category A, with a specific indication for use in pregnancy. The FDA’s review of the actual data is largely available for all to see. See, e.g., Center for Drug Evaluation and Research, Other Reviews (Aug. 2012); Summary Review (April 2013); Pharmacology Review (March 2013); Medical Review (March 2013); Statistical Review (March 2013); Cross Discipline Team Leader Review (April 2013). Given the current scientific record, the consensus that emerged in the early 1990s looks strong. Indeed, the consensus was epistemically strong when reached two decades ago.

Miller is certainly correct that reliance upon consensus entails epistemic risks. Sometimes the consensus has not looked very hard or critically at all the evidence. Political, financial, and cognitive biases can be prevalent. Miller fails to show that any such biases were prevalent in the early 1990s, or that they infected judicial assessments of the plaintiffs’ causal claims in Bendectin litigation. Miller is also wrong to suggest that courts did not look beyond the consensus to the actual evidential base for plaintiffs’ claims. Through the lens of expert witness testimony, both party and court-appointed expert witnesses, courts and juries had a better view of the causation issues than Miller appreciates. Miller’s philosophy of science might be improved by his rolling up his sleeves and actually looking at the data[4].

[1] See Joseph Sanders, Bendectin on Trial: A Study of Mass Tort Litigation (1998); Michael D. Green, Bendectin and Birth Defects: The Challenges of Mass Toxic Substances Litigation (1996).

[2] Robert Brent, “Editorial comment on comments on ‘Teratogen Update: Bendectin’,” 31 Teratology 429 (1985); Kenneth S. Brown, John M. Desesso, John Hassell, Norman W. Klein, Jon M. Rowland, A. J. Steffek, Betsy D. Carlton, Cas. Grabowski, William Slikker Jr. and David Walsh, “Comments on ‘Teratogen Update: Bendectin’,” 31Teratology 431 (1985); Lewis B. Holmes, “Response to comments on ‘Teratogen Update: Bendectin’,” 31 Teratology 432 (1985).

[3] See FDA News Release, “FDA approves Diclegis for pregnant women experiencing nausea and vomiting,” (April 8, 2013). The return of this drug to the United States market was held up as a triumph of science over the will of the industry litigation. See Gideon Koren, “The Return to the USA of the Doxylamine-Pyridoxine Delayed Release Combination (Diclegis®) for Morning Sickness — A New Morning for American Women,” 20 J. Popul. Ther. Clin. Pharmacol. e161 (2013).

[4] See “Bendectin, Diclegis & The Philosophy of Science” (Oct 26, 2013).

Posted in Rule 702, Scientific Evidence | Comments Off on Diclegis and Vacuous Philosophy of Science

Government Secrecy That Denies Defendant A Fair Trial – Because of Reasons

June 20th, 2015

In Davis v. Ayala, defendant Hector Ayala challenged the prosecutor’s use of preemptory challenges in an apparently racially motivated fashion. The trial judge allowed the prosecutor to disclose his reasons in an ex parte session, without the defense present. Under the Supreme Court’s decision in Batson, the defendant should have had the opportunity to inquiry into the bona fides of the prosecutor’s claimed motivations. Based upon the prosecutor’s one-sided presentation, the trial judge ruled that the prosecutor had valid, race-neutral grounds for the contested strikes. After a trial, the empanelled jury convicted Ayala of murder, and sentenced him to death. In a 5-4 decision, the Supreme Court held that the trial court’s error was harmless. Davis v. Ayala, Supreme Court, No. 13–1428 (June 18, 2015). Justice Kennedy issued a concurrence. His conscience was curiously not troubled by the Star Chamber proceedings, but the facts of Ayala’s post-conviction incarceration, which has taken place largely in solitary confinement.

Remarkably, the New York Times weighed in on the Ayala case, but not to castigate the Court for rubber-stamping Kafkaesque Rules of Procedure that permits the defense to be excluded and prevented from exercising its Constitutionally protected role. The Times chose to spill ink instead on Justice Kennedy’s concurrence on the length of solitary confinement. Editorial, “Justice Kennedy on Solitary Confinement,” N.Y. Times (June 19, 2015).

What is curious about Justice Kennedy’s focus, and the Times’ cheerleading, is that they run roughshod over a procedural error that excused prosecutorial secrecy and that affected the adjudication of guilt or innocence, only to obsess about whether a man, taken to be guilty, has been treated inhumanely by the California prison system. Even more curious is the willingness to the Times to castigate, on bogus legal grounds, Justice Thomas for responding to Justice Kennedy:

“In a brief, sour retort that read more like a comment to a blog post, Justice Clarence Thomas quipped that however small Mr. Ayala’s current accommodations may be, they are ‘a far sight more spacious than those in which his victims, Ernesto Dominguez Mendez, Marcos Antonio Zamora, and Jose Luis Rositas, now rest’. It was a bizarre and unseemly objection. The Eighth Amendment does not operate on a sliding scale depending on the gravity of a prisoner’s crime.”

Id. (emphasis added). Except, of course, the Eight Amendment’s requirement of proportionality does operate on a sliding scale[1]. In Kennedy v. Louisiana, 554 U.S. 407 (2008), for instance, the Court held that the Eighth Amendment’s Cruel and Unusual Punishments Clause prohibited a state from imposing the death penalty to punish a child rapist because of the sanction’s disproportionality[2].

Perhaps the New York Times could hire a struggling young lawyer to fact check its legal pronouncements? Both Justice Kennedy and Justice Thomas were in the same majority that would tolerate denying the defendant of his constitutional right to examine prosecutor’s motivation for striking black and Hispanic jurors. What a “sour note” for the Times to sound over Justice Thomas’s empathy for the victims of the defendant’s crimes.

[1] William W. Berry III, “Eighth Amendment Differentness,” 78 Missouri L. Rev. 1053 (2013); Charles Walter Schwartz, “Eighth Amendment Proportionality Analysis and the Compelling Case of William Rummel,” 71 J. Crim. L. & Criminology 378 (1980); John F. Stinneford, “Rethinking Proportionality Under the Cruel and Unusual Punishments Clause,” 97 Va. L. Rev. 899 (2011).

[2] Also curious was that then Senator Barack Obama criticized the Supreme Court for its decision in the Kennedy case. See Sara Kugler “Obama Disagrees With High Court on Child Rape Case,” ABC News (June 25, 2008) (archived from the original).

Posted in Uncategorized | Comments Off on Government Secrecy That Denies Defendant A Fair Trial – Because of Reasons

Daubert’s Error Rate

June 16th, 2015

In Daubert, the Supreme Court came to the realization that expert witness opinion testimony was allowed under the governing statute, Federal Rule of Evidence 702, only when that witness’s “scientific, technical, or other specialized knowledge” would help the fact finder. Knowledge clearly connotes epistemic warrant, and some of the Court’s “factors” speak directly to this warrant, such as whether the claim has been tested, and whether the opinion has an acceptable rate of error. The Court, however, continued to allow some proxies for that warrant, in the form of “general acceptance,” or “peer review.”

The “rate of error” factor has befuddled some courts in their attempt to apply the statutory requirements of Rule 702, especially when statistical evidence is involved. Some litigants have tried to suggest that a statistically significant result suffices alone to meet the demands of Rule 702, but this argument is clearly wrong. See, e.g., United States v. Vitek Supply Corp., 144 F.3d 476, 480, 485–86 (7th Cir. 1998) (stating that the purpose of the inquiry into rate of error is to determine whether tests are “accurate and reliable”) (emphasis added). See also “Judicial Control of the Rate of Error in Expert Witness Testimony” (May 28, 2015). The magnitude of tolerable actual or potential error rate remains, however, a judicial mystery[1].

Sir Austin Bradford Hill described ruling out bias, confounding, and chance (or random error) as essential prerequisites to considering his nine factors used to assess whether an association is causal:

“Disregarding then any such problem in semantics we have this situation. Our observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance. What aspects of that association should we especially consider before deciding that the most likely interpretation of it is causation.”

Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965). The better reasoned cases agree. See, e.g., Frischhertz v. SmithKline Beecham Corp., 2012 U.S. Dist. LEXIS 181507, *6 (E.D.La. 2012) (“The Bradford-Hill criteria can only be applied after a statistically significant association has been identified.”) (citing and quoting among other sources, Federal Judicial Center, Reference Manual on Scientific Evidence, 599 & n.141 (3d. ed. 2011)).

Citing the dictum in Matrixx Initiatives[2] as though it were a holding is not only ethically dubious, but also ignores the legal and judicial context of the Court’s statements[3]. There are, after all, some circumstances such as cases of death by blunt-force trauma, or bullet wounds, when epidemiological and statistical evidence is not needed. The Court did not purport to speak to all causation assessments; nor did it claim that it was addressing only instances in which there were “expected cases,” and “base-line risks,” in diseases that have an accepted occurrence and incidence among unexposed persons. It is, of course, in exactly those cases that statistical consideration of bias, confounding, and chance are essential before Bradford Hill’s factors can be parsed.

Lord Rutherford[4] is often quoted as having said that “[i]f your experiment needs statistics, you ought to have done a better experiment.” Today, physics and chemistry have dropped their haughty disdain for statistics in the face of their recognition that some processes can be understood only as stochastic and rate driven. In biology, we are a long way from being able to describe the most common disease outcomes as mechanistic genetic or epigenetic events. Statistical analyses, with considerations of random and systematic error, will be with us for a long time, whether the federal judiciary acknowledges this fact or not.

*        *        *        *        *        *        *        *        *        *        *         *        *       

Cases Discussing Error Rates in Rule 702 Decisions

SCOTUS

Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593 (1993) (specifying the “the known or potential rate of error” as one of several factors in assessing the scientific reliability or validity of proffered expert witness’s opinion)

Kumho Tire Co. v. Carmichael, 526 U.S. 137, 151 (1999) (suggesting that reliability in the form of a known and an acceptable error rate is an important consideration for admissibility)

US Court of Appeals

FIRST CIRCUIT

United States v. Shea, 957 F. Supp. 331, 334–45 (D.N.H. 1997) (rejecting criminal defendant’s objection to government witness’s providing separate match and error probability rates)

SECOND CIRCUIT

Rabozzi v. Bombardier, Inc., No. 5:03-CV-1397 (NAM/DEP), 2007 U.S. Dist. LEXIS 21724, at *7, *8, *20 (N.D.N.Y. Mar. 27, 2007) (excluding testimony from civil engineer about boat design, in part because witness failed to provide rate of error)

Sorto-Romero v. Delta Int’l Mach. Corp., No. 05-CV-5172 (SJF) (AKT), 2007 U.S. Dist. LEXIS 71588, at *22–23 (E.D.N.Y. Sept. 24, 2007) (excluding engineering opinion that defective wood-carving tool caused injury because of lack of error rate)

In re Ephedra Products Liability Litigation, 393 F. Supp. 2d 181, 184 (S.D.N.Y. 2005) (confusing assessment of random error with probability that statistical estimate of true risk ratio was correct)

Roane v. Greenwich Swim Comm., 330 F. Supp. 2d 306, 309, 319 (S.D.N.Y. 2004) (excluding mechanical engineer, in part because witness failed to provide rate of error)

Nook v. Long Island R.R., 190 F. Supp. 2d 639, 641–42 (S.D.N.Y. 2002) (excluding industrial hygienist’s opinion in part because witness was unable to provide a known rate of error).

United States v. Towns, 19 F. Supp. 2d 67, 70–72 (W.D.N.Y. 1998) (permitting clinical psychologist to opine about defendant’s mens rea and claimed mental illness causing his attempted bank robbery, in part because the proffer of opinion maintained that the psychologist would provide an error rate)  

Meyers v. Arcudi, 947 F. Supp. 581 (D. Conn. 1996) (excluding polygraph in civil action in part because of error rate)

THIRD CIRCUIT

United States v. Ewell, 252 F. Supp. 2d 104, 113–14 (D.N.J. 2003) (rejecting criminal defendant’s objection to government’s failure to quantify laboratory error rate)

Soldo v. Sandoz Pharmaceuticals Corp., 244 F. Supp. 2d 434, 568 (W.D. Pa. 2003) (excluding plaintiffs’ expert witnesses in part because court, and court-appointed expert witnesses, were unable to determine error rate).

Pharmacia Corp. v. Alcon Labs., Inc., 201 F. Supp. 2d 335, 360 (D.N.J. 2002) (excluding ; error too high).

FOURTH CIRCUIT

United States v. Moreland, 437 F.3d 424, 427–28, 430–31 (4th Cir. 2006) (affirming district court’s allowance of forensic chemist’s testimony that could not provide error rate because reviews of witness’s work found it to be free of error)

Buckman v. Bombardier Corp., 893 F. Supp. 547, 556–57 (E.D.N.C. 1995) (ruling that an expert witness may opine about comparisons between boat engines in rough water but only as a lay witness, because the comparison tests were unreliable, with a high estimated rate of error)

FIFTH CIRCUIT

Albert v. Jordan, Nos. 05CV516, 05CV517, 05CV518, 05CV519, 2007 U.S. Dist. LEXIS 92025, at *2–3 (W.D. La. Dec. 14, 2007) (allowing testimony of vocational rehabilitation expert witness, over objection, because witness provided “reliable” information, with known rate of error)

SIXTH CIRCUIT

United States v. Leblanc, 45 F. App’x 393, 398, 400 (6th Cir. 2002) (affirming exclusion of child psychologist, whose testimony about children’s susceptibility to coercive interrogation was based upon “‘soft science’ . . . in which ‘error is . . . rampant’.” (quoting the district court))

United States v. Sullivan, 246 F. Supp. 2d 696, 698–99 (E.D. Ky. 2003) (admitting expert witness’s opinion on the unreliability of eyewitness identification; confusing error rate of witness’s opinion with accuracy of observations made based upon order of presentation of photographs of suspect)

SEVENTH CIRCUIT

United States v. Vitek Supply Corp., 144 F.3d 476, 480, 485–86 (7th Cir. 1998) (affirming denial of defendant’s Rule 702 challenge based in part upon error rates; the purpose of the inquiry into rate of error is to determine whether tests are “accurate and reliable”; here the government’s expert witnesses used adequate controls and replication to ensure an acceptably low rate of error)

Phillips v. Raymond Corp., 364 F. Supp. 2d 730, 732–33, 740-41 (N.D. Ill. 2005) (excluding biomechanics expert witness who had not reliably tested his claims in a way to produce an accurate rate of error)

EIGHTH CIRCUIT

Bone Shirt v. Hazeltine, 461 F.3d 1011, 1020 (8th Cir. 2006) (affirming district court’s ruling to admit testimony of expert witness’s regression analysis in vote redistricting case); see id. at 1026 (Gruender, J., concurring) (expressing concern with the questioned testimony’s potential rate of error because it is “difficult to weigh this factor in Daubert’s analysis if ‘the effect of that error is unknown’.” (quoting court below, Bone Shirt v. Hazeltine, 336 F. Supp. 2d 976, 1002 (D.S.D. 2004))

United States v. Beasley, 102 F.3d 1440, 1444, 1446–48 (8th Cir. 1996) (confusing random error with general error rate) (affirming admissibility of expert witness testimony based upon DNA testing, because such testing followed acceptable standards in testing for contamination and “double reading”)

NINTH CIRCUIT

United States v. Chischilly, 30 F.3d 1144, 1148, 1152, 1154–55 (9th Cir. 1994) (affirming admissibility of testimony based upon DNA match in sex crime, noting that although error rate of error was unquantified, the government had made a sufficient showing of rarity of false positives to support an inference of low error rate)

Cascade Yarns, Inc. v. Knitting Fever, Inc., No. C10–861RSM, 2012 WL 5194085, at *7 (W.D. Wash. Oct. 18. 2012) (excluding expert witness opinion because error rate was too high)

United States v. Microtek Int’l Dev. Sys. Div., Inc., No. 99-298-KI, 2000 U.S. Dist. LEXIS 2771, at *2, *10–13, *15 (D. Or. Mar. 10, 2000) (excluding polygraph data based upon showing that claimed error rate came from highly controlled situations, and that “real world” situations led to much higher error (10%) false positive error rates)

TENTH CIRCUIT

Miller v. Pfizer, Inc., 356 F.3d 1326, 1330, 1334 (10th Cir. 2004) (affirming exclusion of plaintiffs’ expert witness, Dr. David Healy, based upon district court’s findings, made with the assistance of court-appointed expert witnesses, that Healy’s opinion was based upon studies that lacked sufficient sample size, adequate controls, and freedom from study bias, and thus prone to unacceptable error rate)

ELEVENTH CIRCUIT

Quiet Tech. DC-8, Inc. v. Hurel-Duboi U.K., Ltd., 326 F.3d 1333, 1343–45 (11th Cir. 2003) (affirming trial court’s admission of defendant’s aerospace engineer’s testimony, when the lower court had found that the error rate involved was “relatively low”; rejecting plaintiff’s argument that the witness had entered data incorrectly on ground that the asserted error would not affect the validity of the witness’s opinions)

Wright v. Case Corp., No. 1:03-CV-1618-JEC, 2006 U.S. Dist. LEXIS 7683, at *14 (N.D. Ga. Feb. 1, 2006) (granting defendant’s motion to exclude plaintiff’s mechanical engineering expert, because the expert’s alternative designs for the seat safety bar were not reliable due to potential feasibility issues, and because the associated error rate was therefore unquantifiable but potentially very high)

Benkwith v. Matrixx Initiatives, Inc., 467 F. Supp. 2d 1316, 1326, 1330, 1332 (M.D. Ala. 2006) (granting defendant’s motion to exclude testimony of an expert in the field of epidemiology regarding Zicam nasal spray’s causing plaintiff’s anosmia, because the opinions had not been tested and a rate of error could not be provided).

D.C. CIRCUIT

Ambrosini v. Upjohn Co., No. 84-3483 (NHJ), 1995 U.S. Dist. LEXIS 21318, at *16, *22–24 (D.D.C. Oct. 18, 1995) (finding that plaintiff’s teratology expert was not permitted to testify, because the methodology used was found to be unreliable and could not yield an accurate error rate)

[1] Jed S. Rakoff, “Science and the Law: Uncomfortable Bedfellows,” 38 Seton Hall L. Rev. 1379, 1382–83 (2008) (observing that an error rate of 13 percent in polygraph interpretation would likely be insufficiently reliable to support admissibility of testimony based upon polygraph results).

[2] Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, 1319 (2011) (suggesting that courts “frequently permit expert testimony on causation based on evidence other than statistical significance”).

[3] See, e.g., WLF Legal Backgrounder on Matrixx Initiatives (June 20, 2011); “The Matrixx – A Comedy of Errors”; Matrixx Unloaded (Mar. 29, 2011)”; “The Matrixx Oversold” (April 4, 2011); “De-Zincing the Matrixx.”

[4] Ernest Rutherford, a British chemist, investigated radioactivity. He won the Nobel Prize in chemistry, in 1908.

Posted in Expert Witnesses, Rule 702 | Comments Off on Daubert’s Error Rate

How to Fake a Moon Landing

June 8th, 2015

“The meaning of the world is the separation of wish and fact.”
Kurt Gödel

Everyone loves science except when science defeats wishes for a world not known. Coming to accept the world based upon evidence requires separating wish from fact. And when the evidence is lacking in quality or quantity, then science requires us to have the discipline to live with uncertainty rather than wallow in potentially incorrect beliefs.

Darryl Cunningham has written an engaging comic graphics book about science and the scientific worldview. Darryl Cunningham, How to Fake a Moon Landing: Exposing the Myths of Science Denial (2013). Through pictorial vignettes, taken from current controversies, Cunningham has created a delightful introduction to scientific methodology and thinking. Cunningham has provided chapters on several modern scandalous deviations from the evidence-based understanding of the world, including:

  • The Moon Hoax
  • Homeopathy
  • Chiropractic
  • The MMR Vaccination Scandal
  • Evolution
  • Fracking
  • Climate Change, and
  • Science Denial

Most people will love this book. Lawyers will love the easy-to-understand captions. Physicians will love the debunking of chiropractic. Republicans will love the book’s poking fun at (former Dr.) Andrew Wakefield and his contrived MMR vaccination-autism scare, and the liberal media’s unthinking complicity in his fraud. Democrats will love the unraveling of the glib, evasive assertions of the fracking industry. New Agers will love the book because of its neat pictures, and they probably won’t read the words anyway, and so they will likely miss the wonderful deconstruction of homeopathy and other fashionable hokum. Religious people, however, will probably hate the fun poked at all attempts to replace evidence with superstitions.Roblox HackBigo Live Beans HackYUGIOH DUEL LINKS HACKPokemon Duel HackRoblox HackPixel Gun 3d HackGrowtopia HackClash Royale Hackmy cafe recipes stories hackMobile Legends HackMobile Strike Hack

Without rancor, Cunningham pillories all true believers who think that they can wish the facts of the world. At $16.95, the book is therapeutic and a bargain.

Posted in Scientific Evidence | Comments Off on How to Fake a Moon Landing

The Eleventh Circuit Confuses Adversarial and Methodological Bias, Manifestly Erroneously

June 6th, 2015

The Eleventh Circuit’s decision in Adams v. Laboratory Corporation of America, is disturbing on many levels. Adams v. Lab. Corp. of Am., 760 F.3d 1322 (11th Cir. 2014). Professor David Bernstein has already taken the Circuit judges to task for their failure to heed the statutory requirements of Federal Rule of Evidence 702. See David Bernstein, “A regrettable Eleventh Circuit expert testimony ruling, Adams v. Lab. Corp. of America,” Wash. Post (May 26, 2015). Sadly, the courts’ strident refusal to acknowledge the statutory nature of Rule 702, and the Congressional ratification of the 2000 amendments to Rule 702, have become commonplace in the federal courts. Ironically, the holding of the Supreme Court’s decision in Daubert itself was that the lower courts were not free to follow common law that had not been incorporated into the first version of the Rule 702.

There is much more wrong with the Adams case than just a recalcitrant disregard for the law: the Circuit displayed an equally distressing disregard for science. The case started as a negligent failure to diagnose cervical cancer claim against defendant Laboratory Corporation of America. Plaintiffs claimed that the failure to diagnose cancer led to delays in treatment, which eroded Mrs. Adam’s chance for a cure.

Before the Adams case arose, two professional organizations, the College of American Pathologists (CAP) and the American Society of Cytopathology (ASC) issued guidelines about how an appropriate retrospective review should be conducted. Both organizations were motivated by two concerns: protecting their members from exaggerated, over-extended, and bogus litigation claims, as well as by a scientific understanding that a false-negative finding by a cytopathologist does not necessarily reflect a negligent interpretation of a Pap smear[1]. Both organizations called for a standard of blinded review in litigation to protect against hind-sight bias. The Adams retained a highly qualified pathologist, Dr. Dorothy Rosenthal, who with full knowledge of the later diagnosis and the professional guidelines, reviewed the earlier Pap smears that were allegedly misdiagnosed as non-malignant. 760 F.3d at 1326. Rosenthal’s approach violated the CAP and ASC guidelines, as well as common sense.

The district judge ruled that Rosenthal’s approach was little more than an ipse dixit, and a subjective method that could not be reviewed objectively. Adams v. Lab. Corp. of Am., No. 1:10-CV-3309-WSD, 2012 WL 370262, at *15 (N.D. Ga. Feb. 3, 2012). In a published per curiam opinion, the Eleventh Circuit reversed, holding that the district judge’s analysis of Rosenthal’s opinion was “manifestly erroneous.” 760 F.3d at 1328. Judge Garza, of the Fifth Circuit, sitting by designation, concurred to emphasize his opinion that Rosenthal did not need a methodology, as long as she showed up with her qualifications and experience to review the contested Pap smears.

The Circuit opinion is a model of conceptual confusion. The judges refer to the professional society guidelines, but never provide citations. (See note 1, infra.). The Circuit judges are obviously concerned that the professional societies are promulgating standards to be used in judging claims against their own members for negligent false-negative interpretations of cytology or pathology. What the appellate judges failed to recognize, however, is that the professional societies had a strong methodological basis for insisting upon “blinded” review of the slides in controversy. Knowledge of the outcome must of necessity bias any subsequent review, such as plaintiffs’ expert witness, Rosenthal. Even a cursory reading of the two guidelines would have made clear that they had been based on more than simply a desire to protect members; they were designed to protect members against bogus claims, and cited data in support of their position[2]. Subsequent to the guidelines, several publications have corroborated the evidence-based need for blinded review[3].

The concepts of sensitivity, specificity, and positive predictive value are inherent in any screening procedure; they are very much part of the methodology of screening. These measures, along with statistical analyses of concordance and discordance among experienced cytopathologists, can be measured and assessed for accuracy and reliability. The Circuit judges in Adams, however, were blinded (in a bad way) to the scientific scruples that govern screenings. The per curiam opinion suggests that:

“[t]he only arguably appreciable differences between Dr. Rosenthal’s method and the review method for LabCorp’s cytotechnologists is that Dr. Rosenthal (1) already knew that the patient whose slides she was reviewing had developed cancer and (2) reviewed slides from just one patient. Those differences relate to the lack of blinded review, which we address later.”

760 F.3d at 1329 n. 10. And when the judges addressed the lack of blinded review, they treated hindsight bias, a cognitive bias and methodological flaw in the same way as they would have trial courts and litigants treat Dr. Rosenthal’s “philosophical bent” in favor of cancer patients — as “a credibility issue for the jury.” Id. at 1326-27, 1332. This conflation of methodological bias with adversarial bias, however, is a prescription for eviscerating judicial gatekeeping of flawed opinion testimony. Judge Garza, in a concurring opinion, would have gone further and declared that plaintiffs’ expert witness Rosenthal had no methodology and thus she was free to opine ad libitum.

Although Rosenthal’s “philosophical bent” might perhaps be left to the crucible of cross-examination, hindsight review bias could and should have been eliminated by insisting that Rosenthal wear the same “veil of ignorance” of Mrs. Adam’s future clinical course, which the defendant wore when historically evaluating the plaintiff’s Pap smears. Here Rosenthal’s adversarial bias was very probably exacerbated by her hindsight bias, and the Circuit missed a valuable opportunity to rein in both kinds of bias.

Certainly in other areas of medicine, such as radiology, physicians are blinded to the correct interpretation and evaluated on their ability to line up with a gold standard. The NIOSH B-reader examination, for all its problems, at least tries to qualify physicians in the use of the International Labor Organization’s pneumoconiosis scales for interpreting plain-film radiographs for pulmonary dust diseases, by having them read and interpret films blinded to the NIOSH/ILO consensus interpretation.

[1] See Patrick L. Fitzgibbons & R. Marshall Austin, “Expert review of histologic slides and Papanicolaou tests in the context of litigation or potential litigation — Surgical Pathology Committee and Cytopathology Committee of the College of American Pathologists,” 124 Arch. Pathol. Lab. Med. 1717 (2000); American Society of Cytopathology, “Guidelines for Review of Gyn Cytology Samples in the Context of Litigation or Potential Litigation” (2000).

[2] The CAP guideline, for instance, cited R. Marshall Austin, “Results of blinded rescreening of Papanicolaou smears versus biased retrospective review,” 121 Arch. Pathol. Lab. Med. 311 (1997).

[3] Andrew A. Renshaw, K.M Lezon, and D.C. Wilbur, “The human false-negative rate of rescreening Pap tests: Measured in a two-arm prospective clinical trial,” 93 Cancer (Cancer Cytopathol.) 106 (2001); Andrew A. Renshaw, Mary L. Young, and E. Blair Holladay, “Blinded review of Papanicolaou smears in the context of litigation: Using statistical analysis to define appropriate thresholds,” 102 Cancer Cytopathology 136 (2004) (showing that data from blinded reviews can be interpreted in a statistically appropriate way, and defining standards to improve the accuracy and utility of blinded reviews); D. V. Coleman & J. J. R. Poznansky, “Review of cervical smears from 76 women with invasive cervical cancer: cytological findings and medicolegal implications,” 17 Cytopathology 127 (2006); Andrew A. Renshaw, “Comparing Methods to Measure Error in Gynecologic Cytology and Surgical Pathology,” 130 Arch. Path. & Lab. Med. 626 (2009).

Posted in Cognitive Biases, Rule 702 | Comments Off on The Eleventh Circuit Confuses Adversarial and Methodological Bias, Manifestly Erroneously

Judicial Control of the Rate of Error in Expert Witness Testimony

May 28th, 2015

In Daubert, the Supreme Court set out several criteria or factors for evaluating the “reliability” of expert witness opinion testimony. The third factor in the Court’s enumeration was whether the trial court had considered “the known or potential rate of error” in assessing the scientific reliability of the proffered expert witness’s opinion. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 593 (1993). The Court, speaking through Justice Blackmun, failed to provide much guidance on the nature of the errors subject to gatekeeping, on how to quantify the errors, and on to know how much error was too much. Rather than provide a taxonomy of error, the Court lumped “accuracy, validity, and reliability” together with a grand pronouncement that these measures were distinguished by no more than a “hen’s kick.” Id. at 590 n.9 (1993) (citing and quoting James E. Starrs, “Frye v. United States Restructured and Revitalized: A Proposal to Amend Federal Evidence Rule 702,” 26 Jurimetrics J. 249, 256 (1986)).

The Supreme Court’s failure to elucidate its “rate of error” factor has caused a great deal of mischief in the lower courts. In practice, trial courts have rejected engineering opinions on stated grounds of their lacking an error rate as a way of noting that the opinions were bereft of experimental and empirical evidential support[1]. For polygraph evidence, courts have used the error rate factor to obscure their policy prejudices against polygraphs, and to exclude test data even when the error rate is known, and rather low compared to what passes for expert witness opinion testimony in many other fields[2]. In the context of forensic evidence, the courts have rebuffed objections to random-match probabilities that would require that such probabilities be modified by the probability of laboratory or other error[3].

When it comes to epidemiologic and other studies that require statistical analyses, lawyers on both sides of the “v” frequently misunderstand p-values or confidence intervals to provide complete measures of error, and ignore the larger errors that result from bias, confounding, study validity (internal and external), inappropriate data synthesis, and the like[4]. Not surprisingly, parties fallaciously argue that the Daubert criterion of “rate of error” is satisfied by expert witness’s reliance upon studies that in turn use conventional 95% confidence intervals and measures of statistical significance in p-values below 0.05[5].

The lawyers who embrace confidence intervals and p-values as their sole measure of error rate fail to recognize that confidence intervals and p-values are means of assessing only one kind of error: random sampling error. Given the carelessness of the Supreme Court’s use of technical terms in Daubert, and its failure to engage in the actual evidence at issue in the case, it is difficult to know whether the Court intended to suggest that random error was the error rate it had in mind[6]. The statistics chapter in the Reference Manual on Scientific Evidence helpfully points out that the inferences that can be drawn from data turn on p-values and confidence intervals, as well as on study design, data quality, and the presence or absence of systematic errors, such as bias or confounding.  Reference Manual on Scientific Evidence at 240 (3d 2011) [Manual]. Random errors are reflected in the size of p-values or the width of confidence intervals, but these measures of random sampling error ignore systematic errors such as confounding and study biases. Id. at 249 & n.96.

The Manual’s chapter on epidemiology takes an even stronger stance: the p-value for a given study does not provide a rate of error or even a probability of error for an epidemiologic study:

“Epidemiology, however, unlike some other methodologies—fingerprint identification, for example—does not permit an assessment of its accuracy by testing with a known reference standard. A p-value provides information only about the plausibility of random error given the study result, but the true relationship between agent and outcome remains unknown. Moreover, a p-value provides no information about whether other sources of error – bias and confounding – exist and, if so, their magnitude. In short, for epidemiology, there is no way to determine a rate of error.”

Manual at 575. This stance seems not entirely justified given that there are Bayesian approaches that would produce credibility intervals accounting for sampling and systematic biases. To be sure, such approaches have their own problems and they have received little to no attention in courtroom proceedings to date.

The authors of the Manual’s epidemiology chapter, who are usually forgiving of judicial error in interpreting epidemiologic studies, point to one United States Court of Appeals case that fallaciously interpreted confidence intervals magically to quantify bias and confounding in a Bendectin birth defects case. Id. at 575 n. 96[7]. The Manual could have gone further to point out that, in the context of multiple studies, of different designs and analyses, cognitive biases involved in evaluating, assessing, and synthesizing the studies are also ignored by statistical measures such as p-values and confidence intervals. Although the Manual notes that assessing the role of chance in producing a particular set of sample data is “often viewed as essential when making inferences from data,” the Manual never suggests that random sampling error is the only kind of error that must be assessed when interpreting data. The Daubert criterion would appear to encompass all varieties or error, not just random error.

The Manual’s suggestion that epidemiology does not permit an assessment of the accuracy of epidemiologic findings misrepresents the capabilities of modern epidemiologic methods. Courts can, and do, invoke gatekeeping approaches to weed out confounded study findings. See “Sorting Out Confounded Research – Required by Rule 702” (June 10, 2012). The “reverse Cornfield inequality” was an important analysis that helped establish the causal connection between tobacco smoke and lung cancer[8]. Olav Axelson studied and quantified the role of smoking as a confounder in epidemiologic analyses of other putative lung carcinogens.[9] Quantitative methods for identifying confounders have been widely deployed[10].

A recent study in birth defects epidemiology demonstrates the power of sibling cohorts in addressing the problem of residual confounding from observational population studies with limited information about confounding variables. Researchers looking at various birth defect outcomes among offspring of women who used certain antidepressants in early pregnancy generally found no associations in pooled data from Iceland, Norway, Sweden, Finland, and Denmark. A putative association between maternal antidepressant use and a specific kind of cardiac defect (right ventricular outflow tract obstruction or RVOTO) did appear in the overall analysis, but was reversed when the analysis was limited to the sibling subcohort. The study found an apparent association between RVOTO defects and first trimester maternal exposure to selective serotonin reuptake inhibitors, with an adjusted odds ratio of 1.48 (95% C.I., 1.15, 1.89). In the adjusted analysis for siblings, the study found an OR of 0.56 (95% C.I., 0.21, 1.49) in an adjusted sibling analysis[11]. This study and many others show how creative analyses can elucidate and quantify the direction and magnitude of confounding effects in observational epidemiology.

Systematic bias has also begun to succumb to more quantitative approaches. A recent guidance paper by well-known authors encourages the use of quantitative bias analysis to provide estimates of uncertainty due to systematic errors[12].

Although the courts have failed to articulate the nature and consequences of erroneous inference, some authors would reduce all of Rule 702 (and perhaps 704, 403 as well) to a requirement that proffered expert witnesses “account” for the known and potential errors in their opinions:

“If an expert can account for the measurement error, the random error, and the systematic error in his evidence, then he ought to be permitted to testify. On the other hand, if he should fail to account for any one or more of these three types of error, then his testimony ought not be admitted.”

Mark Haug & Emily Baird, “Finding the Error in Daubert,” 62 Hastings L.J. 737, 739 (2011).

Like most antic proposals to revise Rule 702, this reform vision shuts out the full range of Rule 702’s remedial scope. Scientists certainly try to identify potential sources of error, but they are not necessarily very good at it. See Richard Horton, “Offline: What is medicine’s 5 sigma?” 385 Lancet 1380 (2015) (“much of the scientific literature, perhaps half, may simply be untrue”). And as Holmes pointed out[13], certitude is not certainty, and expert witnesses are not likely to be good judges of their own inferential errors[14]. Courts continue to say and do wildly inconsistent things in the course of gatekeeping. Compare In re Zoloft (Setraline Hydrochloride) Products, 26 F. Supp. 3d 449, 452 (E.D. Pa. 2014) (excluding expert witness) (“The experts must use good grounds to reach their conclusions, but not necessarily the best grounds or unflawed methods.”), with Gutierrez v. Johnson & Johnson, 2006 WL 3246605, at *2 (D.N.J. November 6, 2006) (denying motions to exclude expert witnesses) (“The Daubert inquiry was designed to shield the fact finder from flawed evidence.”).

[1] See, e.g., Rabozzi v. Bombardier, Inc., No. 5:03-CV-1397 (NAM/DEP), 2007 U.S. Dist. LEXIS 21724, at *7, *8, *20 (N.D.N.Y. Mar. 27, 2007) (excluding testimony from civil engineer about boat design, in part because witness failed to provide rate of error); Sorto-Romero v. Delta Int’l Mach. Corp., No. 05-CV-5172 (SJF) (AKT), 2007 U.S. Dist. LEXIS 71588, at *22–23 (E.D.N.Y. Sept. 24, 2007) (excluding engineering opinion that defective wood-carving tool caused injury because of lack of error rate); Phillips v. Raymond Corp., 364 F. Supp. 2d 730, 732–33 (N.D. Ill. 2005) (excluding biomechanics expert witness who had not reliably tested his claims in a way to produce an accurate rate of error); Roane v. Greenwich Swim Comm., 330 F. Supp. 2d 306, 309, 319 (S.D.N.Y. 2004) (excluding mechanical engineer, in part because witness failed to provide rate of error); Nook v. Long Island R.R., 190 F. Supp. 2d 639, 641–42 (S.D.N.Y. 2002) (excluding industrial hygienist’s opinion in part because witness was unable to provide a known rate of error).

[2] See, e.g., United States v. Microtek Int’l Dev. Sys. Div., Inc., No. 99-298-KI, 2000 U.S. Dist. LEXIS 2771, at *2, *10–13, *15 (D. Or. Mar. 10, 2000) (excluding polygraph data based upon showing that claimed error rate came from highly controlled situations, and that “real world” situations led to much higher error (10%) false positive error rates); Meyers v. Arcudi, 947 F. Supp. 581 (D. Conn. 1996) (excluding polygraph in civil action).

[3] See, e.g., United States v. Ewell, 252 F. Supp. 2d 104, 113–14 (D.N.J. 2003) (rejecting defendant’s objection to government’s failure to quantify laboratory error rate); United States v. Shea, 957 F. Supp. 331, 334–45 (D.N.H. 1997) (rejecting objection to government witness’s providing separate match and error probability rates).

[4] For a typical judicial misstatement, see In re Zoloft Products, 26 F. Supp.3d 449, 454 (E.D. Pa. 2014) (“A 95% confidence interval means that there is a 95% chance that the ‘‘true’’ ratio value falls within the confidence interval range.”).

[5] From my experience, this fallacious argument is advanced by both plaintiffs’ and defendants’ counsel and expert witnesses. See also Mark Haug & Emily Baird, “Finding the Error in Daubert,” 62 Hastings L.J. 737, 751 & n.72 (2011).

[6] See David L. Faigman, et al. eds., Modern Scientific Evidence: The Law and Science of Expert Testimony § 6:36, at 359 (2007–08) (“it is easy to mistake the p-value for the probability that there is no difference”)

[7] Brock v. Merrell Dow Pharmaceuticals, Inc., 874 F.2d 307, 311-12 (5th Cir. 1989), modified, 884 F.2d 166 (5th Cir. 1989), cert. denied, 494 U.S. 1046 (1990). As with any error of this sort, there is always the question whether the judges were entrapped by the parties or their expert witnesses, or whether the judges came up with the fallacy on their own.

[8] See Joel B Greenhouse, “Commentary: Cornfield, Epidemiology and Causality,” 38 Internat’l J. Epidem. 1199 (2009).

[9] Olav Axelson & Kyle Steenland, “Indirect methods of assessing the effects of tobacco use in occupational studies,” 13 Am. J. Indus. Med. 105 (1988); Olav Axelson, “Confounding from smoking in occupational epidemiology,” 46 Brit. J. Indus. Med. 505 (1989); Olav Axelson, “Aspects on confounding in occupational health epidemiology,” 4 Scand. J. Work Envt’l Health 85 (1978).

[10] See, e.g., David Kriebel, Ariana Zeka1, Ellen A Eisen, and David H. Wegman, “Quantitative evaluation of the effects of uncontrolled confounding by alcohol and tobacco in occupational cancer studies,” 33 Internat’l J. Epidem. 1040 (2004).

[11] Kari Furu, Helle Kieler, Bengt Haglund, Anders Engeland, Randi Selmer, Olof Stephansson, Unnur Anna Valdimarsdottir, Helga Zoega, Miia Artama, Mika Gissler, Heli Malm, and Mette Nørgaard, “Selective serotonin reuptake inhibitors and ventafaxine in early pregnancy and risk of birth defects: population based cohort study and sibling design,” 350 Brit. Med. J. 1798 (2015).

[12] Timothy L.. Lash, Matthew P. Fox, Richard F. MacLehose, George Maldonado, Lawrence C. McCandless, and Sander Greenland, “Good practices for quantitative bias analysis,” 43 Internat’l J. Epidem. 1969 (2014).

[13] Oliver Wendell Holmes, Jr., Collected Legal Papers at 311 (1920) (“Certitude is not the test of certainty. We have been cock-sure of many things that were not so.”).

[14] See, e.g., Daniel Kahneman & Amos Tversky, “Judgment under Uncertainty:  Heuristics and Biases,” 185 Science 1124 (1974).

Posted in Cognitive Biases, Expert Witnesses, Reference Manual on Scientific Evidence, Rule 702, Scientific Evidence | Comments Off on Judicial Control of the Rate of Error in Expert Witness Testimony

Can an Expert Witness Be Too Biased to Be Allowed to Testify

May 20th, 2015

The Case of Barry Castleman

Barry Castleman has been a fixture in asbestos litigation for over three decades. By all appearances, he was the creation of the litigation industry. Castleman received a bachelor of science degree in chemical engineering in 1968, and a master’s degree in environmental engineering, in 1972. In 1975, he started as a research assistant to plaintiffs’ counsel in asbestos litigation, and in 1979, he commenced his testimonial adventures as a putative expert witness for plaintiffs’ counsel. Enrolled in a doctoral program, Castleman sent chapters of his thesis to litigation industry mentors for review and edits. In 1985, Castleman received a doctorate degree, with the assistance of a Ron Motley fellowship. See John M. Fitzpatrick, “Digging Deep to Attack Bias of Plaintiff Experts,” DRI Products Liability Seminar (2013).

Castleman candidly testified, on many occasions, that he was not an epidemiologist, a biostatistician, a toxicologist, a physician, a pathologist, or any other kind of healthcare professional. He is not a trained historian. Understandably, courts puzzled over exactly what someone like Castleman should be allowed to testify about. Many courts limited or excluded Castleman from remunerative testimonial roles[1]. Still, in the face of his remarkably inadequate training, education, and experience, Castleman persisted, and often prevailed, in making a living at testifying about the historical “state of the art” of medical knowledge about asbestos over time.

The result was often not pretty. Castleman worked not just as an expert witness, but also as an agent of plaintiffs’ counsel to suppress evidence. “The Selikoff – Castleman Conspiracy” (May 13, 2011). As a would-be historian, Castleman was controlled and directed by the litigation industry to avoid inconvenient evidence. “Discovery into the Origin of Historian Expert Witnesses’ Opinions” (Jan. 30, 2012). Despite his covert operations, and his exploitation of defendants’ internal documents, Castleman complained more than anyone about the scrutiny created by his self-chosen litigation roles. In 1985, pressed for materials he had considered in formulating his “opinions,” Castleman wrote a personal letter to the judge, the Hon. Hugh Gibson of Galveston, Texas, to object to lawful discovery into his activities:

“1. It threatens me ethically through demands that I    divulge material submitted in confidence, endangering my good name and reputation.
2. It exposes me to potential liability arising from the release of correspondence and other materials provided to me by others who assumed I would honor their confidence.
3. It jeopardizes my livelihood in that material requested reveals strategies of parties with whom I consult, as well as other materials of a confidential nature.
4. It is far beyond the scope of relevant material to my qualifications and the area of expert testimony offered.
5. It is unprecedented in 49 prior trials and depositions where I have testified, in federal and state courts all over the United States, including many cases in Texas. Never before have I had to produce such voluminous and sensitive material in order to be permitted to testify.
6. It is excessively and unjustifiably intrusive into my personal and business life.
7. I have referenced most of the information I have in my 593-page book, “Asbestos: Medical and Legal Aspects.” The great majority of the information I have on actual knowledge of specific defendants has come from the defendants themselves.
8. All information that I have which is relevant to my testimony and qualifications has been the subject of numerous trials and depositions since 1979.”

Castleman Letter to Hon. Hugh Gibson (Nov. 5, 1985).

Forty years later, Castleman is still working for the litigation industry, and courts are still struggling to figure out what role he should be allowed as a testifying expert witness.

Last year, the Delaware Supreme Court had to order a new trial for R. T. Vanderbilt, in part because Castleman had blurted out non-responsive, scurrilous hearsay statements that:

(1) employees of Johns-Manville (a competitor of R.T. Vanderbilt) had called employees of Vanderbilt “liars;”

(2) R.T. Vanderbilt spent a great amount of money on studies and activities to undermine federal regulatory action on talc; and

(3) R.T. Vanderbilt was “buying senators and lobbying the government.”

The Delaware court held that Castleman’s gratuitous, unsolicited testimony on cross-examination was inadmissible, and that his conduct required a new trial.  R.T. Vanderbilt Co. v. Galliher, No. 510, 2013, 2014 WL 3674180 (Del. July 24, 2014).

Late last year, a federal court ruled, pre-trial, that Castleman may testify over Rule 702 objections because he “possesses ‘specialized knowledge’ regarding the literature relating to asbestos available during the relevant time periods,” and that his testimony “could be useful to the jury as a ‘sort of anthology’ of the copious available literature.” Krik v. Crane Co., No. 10-cv-7435, – F. Supp. 2d -, 2014 WL 5350463, *3 (N.D. Ill. Oct. 21, 2014). Because Castleman was little more than a sounding board for citing and reading sections of the historical medical literature, the district court prohibited him from testifying as to the accuracy of any conclusions in the medical literature. Id.

Last week, another federal court took a different approach to keeping Castleman in business. In ruling on defendant’s Rule 702 objections to Castleman, the court held:

“I agree with defendant that plaintiffs have made no showing that Castleman is qualified to explain the meaning and significance of medical literature. Further, there is no suggestion in Krik that Castleman is qualified as an expert in that respect. To the extent that plaintiffs want Castleman simply to read excerpts from medical articles, they do not explain how doing so could be helpful to the jury. Accordingly, I am granting defendant’s motion as it relates to Castleman’s discussion of the medical literature.

***

However, Castleman’s report also includes discussions of articles in trade journals and government publications, which, presumably, would not require medical expertise to understand or summarize.”

Suoja v. Owens-Illinois, Inc., 2015 U.S. Dist. LEXIS 63170, at *3 (W.D.Wisc. May 14, 2015). Judge Barbara Crabb thus disallowed medical state of the art testimony from Castleman, but permitted him to resume his sounding board role for non-medical and other historical documents referenced in his Rule 26 report.

The strange persistence of Barry Castleman, and the inconsistent holdings of dozens of opinions strewn across the asbestos litigation landscape, raise the question whether someone so biased, so entrenched in a litigation role, so lacking in the requisite expertise, should simply be expunged from the judicial process. Rather than struggling to find some benign, acceptable role for Barry Castleman, perhaps courts should just say no. “How Testifying Historians Are Like Lawn-Mowing Dogs” (May 24, 2010).

[1] See, e.g., Van Harville v. Johns-Manville Sales Corp., CV-78-642-H (S.D. Ala 1979); In re Related Asbestos Cases, 543 F. Supp. 1142, 1149 (N.D. Cal. 1982) (rejecting Castleman’s bid to be called an “expert”) (holding that the court was “not persuaded that Mr. Castleman, as a layperson, possesses the expertise necessary to read complex, technical medical articles and discern which portions of the articles would best summarize the authors’ conclusions”); Kendrick v. Owens-Corning Fiberglas Corp., No. C-85-178-AAm (E.D. Wash. 1986); In re King County Asbestos Cases of Levinson, Friedman, Vhugen, Duggan, Bland and Horowitz, No. 81-2-08702-7, (Washington St. Super. Ct. for King Cty.1987); Franze v. Celotex Corp., C.A. No. 84-1316 (W.D. Pa.); Dunn v. Hess Oil Virgin Islands Corp., C.A. No. 1987-238 (D.V.I. May 16, 1989) (excluding testimony of Barry Castleman); Rutkowski v. Occidental Chem. Corp., No. 83 C 2339, 1989 WL 32030, at *1 (N.D. Ill. Feb. 16, 1989) (“Castleman lacks the medical background and experience to evaluate and analyze the articles in order to identify which parts of the articles best summarize the authors’ conclusions.”); In re Guam Asbestos Litigation, 61 F.3d 910, 1995 WL 411876 (9th Cir. 1995) (Kozinski, J., dissenting) (“I would also reverse because Barry Castleman was not qualified to testify as an expert witness on the subject of medical state of the art or anything else; he appears to have read a number of articles for the sole purpose of turning himself into an expert witness. Reductio ad absurdum.”); McClure v. Owens Corning Fiberglas Corp. 188 Ill. 2d 102, 720 N.E.2d 242 (1999) (rejecting probativeness of Castleman’s testimony about company conduct).

Posted in Expert Witnesses | Comments Off on Can an Expert Witness Be Too Biased to Be Allowed to Testify

Professor Bernstein’s Critique of Regulatory Daubert

May 15th, 2015

In the law of expert witness gatekeeping, the distinction between scientific claims made in support of litigation positions and claims made in support of regulations is fundamental. In re Agent Orange Product Liab. Litig., 597 F. Supp. 740, 781 (E.D.N.Y. 1984) (“The distinction between avoidance of risk through regulation and compensation for injuries after the fact is a fundamental one”), aff’d 818 F.2d 145 (2d Cir. 1987), cert. denied sub nom. Pinkney v. Dow Chemical Co., 487 U.S. 1234 (1988). Although scientists proffer opinions in both litigation and regulatory proceedings, their opinions are usually evaluated by substantially different standards. In federal litigation, civil and criminal, expert witnesses must be qualified and have an epistemic basis for their opinions, to satisfy the statutory requirements of Federal Rule of Evidence 702, and they must have reasonably relied upon otherwise inadmissible evidence (such as the multiple layers of hearsay involved in an epidemiologic study) under Rule 703. In regulatory proceedings, scientists are not subject to admissibility requirements and the sufficiency requirements set by the Administrative Procedures Act are extremely low[1].

Some industry stakeholders are aggrieved by the low standards for scientific decision making in certain federal agencies, and they have urged that the more stringent litigation evidentiary rules be imported into regulatory proceedings. There are several potential problems with such reform proposals. First, the epistemic requirements of science generally, or of Rules 702 and 703 in particular, are not particularly stringent. Scientific method leads to plenty of false positive and false negative conclusions, which are subject to daily challenge and revision. Scientific inference is not necessarily so strict, as much as ordinary reasoning is so flawed, inexact, and careless. Second, the call for “regulatory Daubert” ignores mandates of some federal agency enabling statutes and guiding regulations, which call for precautionary judgments, and which allow agencies to decide issues on evidentiary display that fall short of epistemic warrants for claims of knowledge.

Many lawyers who represent industry stakeholders have pressed for extension of Daubert-type gatekeeping to federal agency decision making. The arguments for constraining agency action find support in the over-extended claims that agencies and so-called public interest science advocates make in support of agency measures. Advocates and agency personnel seem to believe that worst-case scenarios and overstated safety claims are required as “bargaining” positions to achieve the most restrictive and possibly the most protective regulation that can be gotten from the administrative procedure, while trumping industry’s concerns about costs and feasibility. Still, extending Daubert to regulatory proceedings could have the untoward result of lowering the epistemic bar for both regulators and litigation fact finders.

In a recent article, Professor David Bernstein questions the expansion of Daubert into some regulatory realms. David E. Bernstein, “What to Do About Federal Agency Science: Some Doubts About Regulatory Daubert,” 22 Geo. Mason L. Rev. 549 (2015)[cited as Bernstein]. His arguments are an important counterweight to those who insist on changing agency rulemaking and actions at every turn. As an acolyte and a defender of scientific scruples and reasoning in the courts, Bernstein’s arguments are worth taking seriously.

Bernstein reminds us that bad policy, as seen in regulatory agency rulemaking or decisions, is not always a scientific issue. In any event, regulatory actions, unlike jury decisions, are not, or at least should not be, “black boxes.” The agency’s rationale and reasoning are publicly stated, subject to criticism, and open to revision. Jury decisions are opaque, non-transparent, potentially unreasoned, not carefully articulated, and not subject to revision absent remarkable failures of proof.

One line of argument[2] pursued by Professor Bernstein follows from his observation that Daubert procedures are required to curtail litigation expert witness “adversarial bias.” Id. at 555. Bernstein traces adversarial bias to three sources:

(1) conscious bias;

(2) unconscious bias; and

(3) selection bias.

Id. Conscious bias stems from deliberate attempts by “hired guns” to deliver opinions that satisfy the lawyers who retained them. The problem of conscious bias is presented by “hired guns” who will adapt their opinions to the needs of the attorney who hires them. Unconscious biases are the more subtle, but no less potent determinants of expert witness behavior, which are created by financial dependence upon, and allegiance to, the witness’s paymaster. Selection bias results from lawyers’ ability to choose expert witnesses to support their claims, regardless whether those witnesses’ opinions are representative of the scientific community. Id.

Professor Bernstein’s taxonomy of bias is important, but incomplete. First, the biases he identifies operate fulsomely in regulatory settings. Although direct financial remuneration is usually not a significant motivation for a scientist to testify before an agency, or to submit a whitepaper, professional advancement and cause advocacy are often powerful incentives at work. These incentives for self-styled public interest zealots may well create more powerful distortions of scientific judgment than any monetary factors in private litigation settings. As for selection bias, lawyers are ethically responsible for screening their expert witnesses, and there can be little doubt that once expert witnesses are disclosed, their opinions will align with their sponsoring parties’ interests. This systematic bias, however, does not necessarily mean that both side’s expert witnesses will necessarily be unrepresentative or unscientific. In the silicone gel breast implant litigation (MDL 926), Judge Pointer, the presiding judge, insisted that both sides’ witnesses were “too extreme,” and he was stunned when his court-appointed expert witnesses filed reports that vindicated the defendants’ expert witnesses’ positions[3]. The defendants had selected expert witnesses who analyzed the data on sound scientific principles; the plaintiffs had selected expert witnesses who overreached in their interpretation of the evidence. Furthermore, many scientific disputes, which find their way into the courtroom, will not have the public profile of silicone gel breast implants, and for which there may be no body of scientific community opinion from which lawyers could select “outliers,” even if they wished to do so.

Professor Bernstein’s offered taxonomy of bias is incomplete because it does not include the most important biases that jurors (and many judges) struggle to evaluate:

random errors;

systematic biases;

confounding; and

cognitive biases.

These errors and biases, along with their consequential fallacies of reasoning, apply with equal force to agency and litigation science. Bernstein does point out, however, an important institutional difference between jury or judge trials and agency review and decisions based upon scientific evidence: agencies often have extensive in-house expertise. Although agency expertise may sometimes be blinded by its policy agenda, agency procedures usually afford the public and the scientific community to understand what the agency decided, and why, and to respond critically when necessary. In the case of the Food and Drug Administration, agency decisions, whether pro- or contra-industry positions are dissected and critiqued by the scientific and statistical community with great care and relish. Nothing of the same sort is possible in response to a jury verdict.

Professor Bernstein is not a science nihilist, and he would not have reviewing courts give a pass to whatever nonsense federal agencies espouse. He calls for enforcement of available statutory requirements that agency action be based upon the “best available science,” and for requiring agencies to explicitly separate and state their policy and scientific judgments. Bernstein also urges greater use of agency peer review, such as occasionally seen from the Institute of Medicine (soon to be the National Academy of Medicine), and the use of Daubert-like criteria for testimony at agency hearings. Bernstein at 554.

Proponents of regulatory Daubert should take Professor Bernstein’s essay to heart, with a daily dose of atorvastatin. Importing Rule 702 into agency proceedings may well undermine the rule’s import in litigation, civil and criminal, while achieving little in the regulatory arena. Consider the pending OSHA rulemaking for lowering the permissible exposure limit (PEL) of crystalline silica in the workplace. OSHA, and along with some public health organizations, has tried to justify this rulemaking on the basis of many overwrought claims of the hazards of crystalline silica exposure at current levels. Clearly, there are some workers who continue to work in unacceptably hazardous conditions, but the harms sustained by these workers can be tied to violations of the current PEL; they are hardly an argument for lowering that current PEL. Contrary to the OSHA’s parade of horribles, silicosis mortality in the United States has steadily declined over the last several decades. The following chart draws upon NIOSH and other federal governmental data:

 

Silicosis Deaths by Year

 

Silicosis deaths, crude and age-adjusted death rates, for U.S. residents age 15 and over, 1968–2007

from Susan E. Dudley & Andrew P. Morriss, “Will the Occupational Safety and Health Administration’s Proposed Standards for Occupational Exposure to Respirable Crystalline Silica Reduce Workplace Risk?” 35 Risk Analysis (2015), in press, doi: 10.1111/risa.12341 (NIOSH reference number: 2012F03–01, based upon multiple cause-of-death data from National Center for Health Statistics, National Vital Statistics System, with population estimates from U.S. Census Bureau).

The decline in silicosis mortality is all the more remarkable because it occurred in the presence of stimulated reporting from silicosis litigation, and misclassification of coal workers’ pneumoconiosis in coal-mining states.

The decline in silicosis mortality may be helpfully compared with the steady rise in mortality from accidental falls among men and women 65 years old, or older:

CDC MMWR Death Rates from Unintentional Falls 2015

Yahtyng Sheu, Li-Hui Chen, and Holly Hedegaard, “QuickStats: Death Rates* from Unintentional Falls† Among Adults Aged ≥ 65 Years, by Sex — United States, 2000–2013,” 64 CDC MMWR 450 (May 1, 2015). Over the observation period, these death rates roughly doubled in both men and women.

Is there a problem with OSHA rulemaking? Of course. The agency has gone off on a regulatory frolic and detour trying to justify an onerous new PEL, without any commitment to enforcing its current silica PEL. OSHA has invoked the prospect of medical risks, many of which are unproven, speculative, and remote, such as lung cancer, autoimmune disease, and kidney disease. The agency, however, is awash with PhDs, and I fear that Professor Bernstein is correct that the distortions of the science are not likely to be corrected by applying Rule 702 to agency factfinding. Courts, faced with the complex prediction models, with disputed medical claims made by agency and industry scientists, will do what they usually do, shrug and defer. And the blow back of the “judicially approved” agency science in litigation contexts will be a cure worse than the disease. At bottom, the agency twisting of science is driven by policy goals and considerations, which require public debate and scrutiny, sound executive judgment, with careful legislative oversight and guidance.

[1] Even under the very low evidentiary and procedural hurdles, federal agencies still manage to outrun their headlights on occasion. See, e.g., Industrial Union Department v. American Petroleum Institute, 448 U.S. 607 (1980) (The Benzene Case); Gulf South Insulation v. U.S. Consumer Product Safety Comm’n, 701 F.2d 1137 (5th Cir. 1983); Corrosion Proof Fittings v. EPA, 947 F2d 1201 (5th Cir 1991).

[2] See also David E. Bernstein, “The Misbegotten Judicial Resistance to the Daubert Revolution,” 89 Notre Dame L. Rev. 27, 31 (2013); David E. Bernstein, “Expert Witnesses, Adversarial Bias, and the (Partial) Failure of the Daubert Revolution,” 93 Iowa L. Rev. 451, 456–57 (2008).

[3] Judge Pointer was less than enthusiastic about performing any gatekeeping role. Unlike most of today’s MDL judges, he was content to allow trial judges in the transferor districts to decide Rule 702 and other pre-trial issues. See Note, “District Judge Takes Issue With Circuit Courts’ Application of Gatekeeping Role” 3 Federal Discovery News (Aug. 1997) (noting that Chief Judge Pointer had criticized appellate courts for requiring district judges to serve as gatekeepers of expert witness testimony).

Posted in Rule 702, Scientific Evidence, Silica | Comments Off on Professor Bernstein’s Critique of Regulatory Daubert

« Older Entries
Newer Entries »

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.