TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Intellectual Due Process in West Virginia and Beyond

June 1st, 2014

Harris v. CSX Transportation

I have borrowed and modified the phrase “Intellectual Due Process” from earlier writers because of its obvious implications for the presentation, interpretation, synthesis, and evaluation of scientific evidence in court. See Scott Brewer, “Scientific Expert Testimony and Intellectual Due Process,” 107 Yale L. J. 1535 (1998). The major reason courts write opinions is to explain and justify their decisions to litigants, present and future, and to a wider audience of lawyers, scholars, and the general public. Judicial opinions involving scientific evidence, whether in legislation, regulation, or litigation must satisfy the societal need to explain and justify the acceptance and rejection of scientific claims. Despite a great deal of hand waving that law and science are somehow different, in the end, when courts describe their acceptance or rejection of scientific claims, they are addressing the same epistemic warrant that scientists themselves employ. Even a cursory review of the judicial output reveals an unsatisfactory state of affairs in which many courts mangle scientific and statistical evidence and inference. There is much that is needed to correct the problem.

One proposal would be to require that the parties file proposed findings of facts in connection with Rule 702 gatekeeping challenges. Courts should file detailed findings of facts that underlie their decisions to admit or to exclude expert witness opinion testimony. Another proposal would require courts to cite properly the scientific studies that they discuss in reaching a legal conclusion about sufficiency or admissibility. These are small steps, but ones that would help reduce the gross inaccuracies and the glib generalizations, while increasing the opportunity for public scrutiny and criticism.

We do not think anything is amiss with special courts for tax, patent, family law, national security, equity, or commercial matters. There is an even greater need for scientific skill, knowledge, and aptitude in a specialized science court. The time has come for special courts to hear cases involving scientific claims in health effects and other litigation.

* * * * * * *

A decision of the West Virginia Supreme Court, late last year, illustrates the need for substantial reform of how claiming based upon “scientific evidence” is permitted and evaluated in court. Mrs. Harris sued the railroad for the wrongful death of her husband, who died of multiple myeloma. Mr. Harris had been exposed, in his railroad workplace, to diesel exhaust, which Mrs. Harris claimed caused his cancer. See Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013). The trial court excluded Mrs. Harris’s expert witnesses. Harris v. CSX Transportation, Inc., No. 12-1135, 2012 WL 8899119 (Cir. Ct. Marshall Cty., W.Va. Aug. 21, 2012).

1. The West Virginia Supreme Court reversed the trial court’s exclusion of witnesses on the basis of an asymmetrical standard of review, which would allow de novo review of trial court decisions to exclude expert witness opinions, but which would privilege trial court decisions to admit opinions by limiting appellate review to abuse of discretion. This asymmetry was, of course, the same dodge that the Third and Eleventh Circuits had used to keep the “gates open,” regardless of validity or reliability concerns, and the same dodge that the Supreme Court shut down in General Electric v. Joiner. A single judge dissented in Harris, Justice Loughry, who took the majority to task for twisting facts and law to get to a desired result.

2. The Harris Court cited a federal court case for dicta that “Rule 702 reflects an attempt to liberalize the rules governing the admissibility of expert testimony.” See Harris, 753 S.E.2d at 279 (citing and quoting from Weisgram v. Marley Co., 169 F.3d 514, 523 (8th Cir.1999). Remarkably, the Harris Court omitted reference to the United States Supreme Court’s unanimous affirmance of Weisgram, which saw Justice Ginsburg write that “[s]ince Daubert, moreover, parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.” Weisgram v. Marley Co., 528 U.S. 440, 442 (2000). The Harris Court’s lack of scholarship is telling.

3. Meta-analysis appeared to play a role in the case, but the judicial decisions in Harris fail to describe the proffered evidence. The majority in Harris noted that one of plaintiff’s expert witnesses, Dr. Infante, relied upon a meta-analysis referred to as “Sonoda 2001.” Harris, 753 S.E.2d at 309. Neither the Court nor the dissent cited the published meta-analysis in a way that would help an interested reader in finding the paper. One could imagine the hue and cry if courts cited judicial cases or statutes by short-hand names without providing enough information to access the relied upon source. In this case, a PubMed search reveals the source so perhaps the error is harmless. Tomoko Sonoda, Yoshie Nagata, Mitsuru Mori, Tadao Ishida & Kohzoh Imai, “Meta-analysis of multiple myeloma and benzene exposure,” 11. J. Epidemiol. 249 (2001). Still, the time has come for courts to describe and report the scientific evidence with the same care and detail that they would use in a car collision case.

4. A quick read shows that the Sonoda meta-analysis supports the dissent’s assessment:

“‘Dr. Infante testified on direct examination that Sonoda 2001 considered 8 case-control studies specific to engine exhaust and stated it concluded that diesel and non-diesel engine exhaust causes multiple myeloma.’ Yet, as the trial court found, ‘[o]n cross examination Dr. Infante acknowledged that none of the 8 papers included in the Sonoda meta-analysis mention diesel exhaust’.”

Harris, 753 S.E.2d at 309. The dissent would have been considerably more powerful had it actually adverted to the language of Sonoda 2001:

“These results suggested that benzene exposure itself was not likely to be a risk factor of MM [multiple myeloma]. It is thought that several harmful chemical agents in engine exhaust, other than benzene, could be etiologically related to the risk of MM. Further case-control studies on MM are needed to obtain more information about detailed occupational exposure to toxic substances.”

Sonoda at 249 (2001) (emphasis added). Contrary to Infante’s asseveration, Sonoda and colleagues never concluded that diesel exhaust causes multiple myeloma. The state of scholarship and “intellectual due process” makes it impossible to tell whether or not Dr. Infante was telling the truth or the Harris Court badly misunderstood the record. Either way, something must give.

The dissent went on to note that Dr. Infante conducted his own meta-analysis, which included studies that did not mention diesel exhaust. Harris, 753 S.E.2d at 309. The railroad complained that some of the studies were small and had limited power, but that is exactly why a meta-analysis would be appropriate. The more disturbing complaints were that the meta-analysis left out important studies, and that it included irrelevant studies of benzene exposure and myeloma, which raised insuperable problems of external validity.

5. A half empty glass that is always full. According to the Harris Court, the West Virginia shadow of Rule 702 is a rule of “admissibility rather than exclusion.” Harris, 753 S.E.2d at 279 (citing and quoting from In re Flood Litig. Coal River Watershed, 222 W.Va. 574, 581, 668 S.E.2d 203, 210 (2008), which in turn quoted a federal case, Arcoren v. United States, 929 F.2d 1235, 1239 (8th Cir. 1991), decided before the Supreme Court decided Daubert.) This is just silly hand waving and blatant partisanship. A rule that sets out criteria or bases for admissibility also demarcates the inadmissible.

6. Cherry Picking. Dr. Infante was permitted by the Harris Court to aggregate data from studies that did not observe diesel exposure, while he failed to include, or he deliberately excluded data from, a large, powerful, exonerative study conducted by scientists from the National Cancer Institute, the International Agency for Research on Cancer (IARC), and the Karolinska Institute. See Paolo Boffetta, Mustafa Dosemeci, Gloria Gridley, Heather Bath, Tahere Moradi and Debra Silverman, “Occupational exposure to diesel engine emissions and risk of cancer in Swedish men and women,” 12 Cancer Causes Control 365 (2001). Dr. Infante inexplicably excluded this study, which found a risk ratio for men exposed to diesel exhaust that was below one, 0.98, with a very narrow 95% confidence interval, 0.92-1.05. Boffetta at 368, Table 2.

7. The West Virginia articulated an incohorent definition of “reliable,” designed to give itself the ability to reject gatekeeping completely. Citing its earlier decision in Flood, the Court offered its own ipse dixit:

“The assessment of whether scientifically-based expert testimony is “reliable,” as that term is used in [Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and Wilt v. Buracker, 191 W.Va. 39, 443 S.E.2d 196 (1993)], does not mean an assessment of whether the testimony is persuasive, convincing, or well-founded. Rather, assessing ‘reliability’ is a shorthand term of art for assessing whether the testimony is to a reasonable degree based on the use of knowledge and procedures that have been arrived at using the methods of science — rather than being based on irrational and intuitive feelings, guesses, or speculation. If the former is the case, then the jury may (or may not, in its sole discretion) ‘rely upon’ the testimony. In re Flood Litig., 222 W.Va. at 582 n. 5, 668 S.E.2d at 211 n. 5.”

Harris, 753 S.E.2d at 279-80. Surely, this is circular or vacuous or both. Opinions not “well-founded” will be ones that are based upon guesses or speculation. Opinions arrived at by the “methods of science” will be ones that have an epistemic warrant that will survive a claim that they are not “well-founded.”

8. The Harris Court evidenced its hostility to scientific evidence by dredging up one of its own decisions involving a multiple myeloma causation claim, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002). Wiseman involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation:

“Some research has suggested that people in some jobs may have an increased risk of developing multiple myeloma because they are exposed to certain chemicals. But the International Agency for Research on Cancer (IARC) states that the evidence is limited overall. It has been suggested that people may have an increased risk if they work in the petrol or oil industry, farming, wood working, the leather industry, painting and decorating, hairdressing, rubber manufacturing or fire fighting. But there is no evidence to prove that any of these occupations carry an increased risk of myeloma.”

Cancer Research UK, “Myeloma risks and causes” (last visited May 28, 2014). Even the most non-progressive jurisdictions have generally eradicated specious claiming for trauma-induced cancers, but West Virginia has carved out a place second to none in its race to the bottom.

9. WOE. Not surprisingly, the Harris Court relied heavily on the First Circuit’s “weight of the evidence” end-run around the notion of epistemic warrant for scientific claims, citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir.2011), cert. denied sub nom., U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). The Harris Court went on to conflate and confuse WOE with Bradford Hill, and cited a recent New York case that confidently saw through WOE hand waving, while ignoring its devasting critique of expert witnesses’ attempts to pass off WOE for scientific, epistemic warrant. Reeps ex rel. Reeps v. BMW of N. Am., LLC, No. 100725/08,

2013 WL 2362566, at *3, 2012 N.Y. Misc. LEXIS 5788; 2012 NY Slip Op 33030U (N.Y. Sup. Ct. May 10, 2013).

10. Link. Dr. Infante links a lot, even when his sources do not:

“Dr. Infante testified that the International Agency for Research on Cancer issued Technical Publication Number 42 in 2009, and that the publication stated that diesel exhaust exposures have been linked to multiple myeloma and leukemia.”

Harris, 753 S.E.2d at 294. The Harris Court neglected to give the title of the publication, which tells a different story. Identification of research needs to resolve the carcinogenicity of high-priority IARC carcinogens. The dissent was willing to go behind the conclusory and false characterization that Dr. Infante and plaintiff gave to this publication. Harris, 753 S.E.2d at 309. The trial court’s finding (and the dissent’s assertion) that the IARC Technical Publication 42 intended to express a research agenda, not to make a causation statement, seems unassailable. Furthermore, it appears to be precisely the sort of specious claim that a court should keep from a jury. The cited IARC source actually notes that the then current IARC classification of diesel exhaust was of inadequate evidence for human carcinogenicity, with a focus on lung cancer, and barely a mention of multiple myeloma.

11. The Benzene Connection. Plaintiffs’ expert witnesses, including Dr. Infante, argued that benzene was a component of diesel exhaust, and benzene caused multiple myeloma. This move ignored not only the lack of evidence to implicate benzene in the causation of multiple myeloma, but it also ignored the large quantitative differences between the benzene occupational exposure studies and the very small amounts of benzene in diesel exhaust. The Harris Court held that the trial court acted improperly by inquiring into and finding the following facts, which were “exclusively” for the jury:

“There is substantially more benzene in cigarette smoke than diesel exhaust.

Benzene is present only in trivial doses in diesel exhaust.

The hypothesis that diesel exhaust causes multiple myeloma is confounded by the fact that cigarette smoking does not.”

The Harris majority further chastised the trial court for adverting to the ten or so studies that failed to find a statistically significant association between benzene exposure and multiple myeloma. Harris, 753 S.E.2d at 305-06. This inquiry directly calls into question, however, Dr. Infante’s methodology.

If these facts, found by the trial court, were reasonably established, then Dr. Infante’s argument was less than bogus, and a major underpinning for inclusion of benzene studies in his meta-analysis was refuted. These are precisely the sort of foundational facts that must be part of an inquiry into the methodological grounds of an expert witness’s opinion.

12. The Harris Court confused “proving causation” with “showing a methodology that provides an epistemic warrant for concluding.” Harris, 753 S.E.2d at 300. The Harris Court asserted that the trial court exceeded its gatekeeping function by inquiring into whether Mrs. Harris’s expert witnesses “proved” causation. Harris, 753 S.E.2d at 300. Speaking of “proof of” or “proving” causation is an affectation of lawyers, who refer to their evidence as their “proofs.” Epidemiologic articles and meta-analyses do not end with quod erat demonstrandum. Beyond the curious diction, there is a further issue in the majority’s suggestion that the trial court set the bar too high in declaring that the plaintiff failed to “prove” causation. Even if we were to accept the continuous nature of strength of evidence for a causal conclusion, Dr. Infante and the other plaintiff’s witnesses, would be fairly low on the curve, and their lowly position must of necessity speak to the merits of the defense motion to exclude under Rule 702.

13. Purely Matters for Jury. The Harris Court criticized the trial court for conducting a “mini-trial,” which set out to “resolve issues that were purely matters for jury consideration.” Harris, 753 S.E.2d at 305. In holding that the matters addressed in the pre-trial hearing were “exclusively grist for the jury and which had no relevancy to the limited role the trial court had under the facts of this case,” the Harris Court displayed a profound disregard for what facts would be relevant for a challenge to the plaintiff’s expert witnesses’ methodology. Many of the facts found by the trial court were directly relevant to “general acceptance,” validity (internal and external) of studies relied upon, and reliability of reasoning and inferences drawn. Aside from the lack of general acceptance and peer review of the plaintiff’s claimed causal relationship, the proffered testimony was filled with gaps and lacunae, which are very much at issue in methodological challenges to an opinion of causality.

* * * * * * *

The Harris case has taken its place next to Milward in the litigation industry’s arsenal of arguments for abandoning meaningful judicial supervision and gatekeeping of expert witness opinion testimony. See Andrew S. Lipton, “Proving Toxic Harm: Getting Past Slice and Dice Tactics,” 45 McGeorge L. Rev. 707, 731 (2014) (plaintiffs’ bar cheerleading for the Harris decision as “a lengthy and thoughtful analysis”, and for the Milward case as roadmap to evade meaningful judicial oversight). Not all was perfect with the trial court’s opinion. The defense seemed to have misled the court by asserting that “a difference between a case group and control group is not statistically significant then there is no difference at all.” See Respondent’s Brief at 5, Harris v. CSX Transportation, Inc., 2013 WL 4747999 (filed (Feb. 4, 2013) (citing App. 169, 228-230 (Shields) as having explained that the p-values greater than 0.05 do not support a causal association).

This is hardly true, and indeed, the lack of statistical significance does not lead to a claim that the null hypothesis of no association between exposure and outcome is correct. The defense, however, did not have a burden of showing the null to be correct; only that there was no reliable method deployed to reject the null in favor an alternative that the risk ratio for myeloma was raised among workers exposed to diesel exhaust.

Still, the trial court did seem to understand the importance of replication, in studies free of bias and confounding. Courts generally will have to do better at delineating what are “positive” and “negative” studies, with citations to the data and the papers, so that judicial opinions provide a satisfactory statement of reasons for judicial decisions.

Posted in Causation, Expert Witnesses, Meta-analysis, Rule 702, Scientific Evidence | Comments Off on Intellectual Due Process in West Virginia and Beyond

The Fallacy of Cherry Picking As Seen in American Courtrooms

May 3rd, 2014

After a long winter, the cherry trees are finally managing to blossom. Before we know it, it will be cherry-picking time.

Cherry picking is a good thing; right? Cherry picking yields cherries, and cherries are good. Selective cherry picking yields the best, ripest, sweetest, tastiest cherries. Cherry picking data no doubt yields the best, unbiased, unconfounded, most probative data to be had. Well, maybe not.

What could be wrong with picking cherries? At the end of the process you have cherries, and if you do it right, you have all ripe, and no rotten, cherries. Your collection of ripe cherries, however, will be unrepresentative of the universe of cherries, but at least we understand how and why your cherries were selected.

Elite colleges cherry pick the best high school students; leading law schools cherry pick the top college students; and top law firms and federal judges cherry pick the best graduates from the best law schools. Lawyers are all-too-comfortable with “cherry picking.” Of course, the cherry-picking process here has at least some objective criteria, which can be stated in advance of the selection.

In litigation, each side is expected to “cherry pick” the favorable evidence, and ignore or flyblow the contrary evidence. Perhaps this aspect of the adversarial system induces complacency in judges about selectivity in the presentation of evidence by parties and their witnesses. In science, this kind of adversarial selectivity is a sure way to inject bias and subjectivity into claims of knowledge. And even in law, there are limits to this adversarial system. Undue selectivity in citing precedent can land a lawyer in a heap of trouble. See Thul v. OneWest Bank, FSB, No. 12 C 6380, 2013 WL 212926 (N.D. Ill. Jan. 18, 2013) (failure to cite relevant judicial precedent constitutes an ethical offense)

In science, the development of the systematic review, in large measure, has been supported by the widespread recognition that studies cannot be evaluated with post hoc, subjective evaluative criteria. See generally Matthias Egger, George Davey Smith, and Douglas Altman, Systematic Reviews in Health Care: Meta-Analysis in Context (2001).

Farmers pick the cherries they want to go to market, to make money and satisfy customers. The harvesters’ virtue lies in knowing what to pick to obtain the best crop. The scientist’s virtue lies in the disinterested acquisition of data pursuant to a plan, and the evaluation of the data pursuant to pre-specified criteria.

The scientist’s virtue is threatened by motivations that are all-too human, and all-too common. The vice in science is wanting data that yields marketable publications, grants, promotions, awards, prizes, and perhaps a touch of fame. Picking data based upon a desired outcome is at the very least scientific fallacy if not scientific fraud. Cherry picking does not necessarily imply scienter, but in science, it is a strict liability offense.

The metaphor of cherry picking, mixed as it may be, thus gives us a label for fallacy and error. Cherry picking incorporates sampling bias, selection bias, confirmation bias, hasty generalization, and perhaps others as well. As explained recently, in Nature:

“Data can be dredged or cherry picked. Evidence can be arranged to support one point of view. * * * The question to ask is: ‘What am I not being told?’”

William J. Sutherland, David Spiegelhalter & Mark Burgman, “Policy: Twenty tips for interpreting scientific claims,” 503 Nature 335, 337 (2013).

Cherry picking in the orchard may be a good thing, but in the scientific world, it refers to the selection of studies or data within studies to yield results desired results, however misleading or counterfactual. See Ben Goldacre, Bad Science 97-99 (2008). The selective use of evidence is not a fallacy unique to science. Cherry picking is widely acknowledged to seriously undermine the quality of public debate See Gary Klass, “Just Plain Data Analysis: Common Statistical Fallacies in Analyses of Social Indicator Data” (2008). See generally Bradley Dowden, “Fallacies,” in James Fieser & Bradley Dowden, eds., Internet Encyclopedia of Philosophy.

The International Encyclopedia of Philosophy describes “cherry picking” as a fallacy, “a kind of error in reasoning.” Cherry-picking the evidence, also known as “suppressed evidence,” is:

“[i]ntentionally failing to use information suspected of being relevant and significant is committing the fallacy of suppressed evidence. This fallacy usually occurs when the information counts against one’s own conclusion. * * * If the relevant information is not intentionally suppressed but rather inadvertently overlooked, the fallacy of suppressed evidence also is said to occur, although the fallacy’s name is misleading in this case.”

Bradley Dowden, “Suppressed Evidence,” International Encyclopedia of Philosophy (Last updated: December 31, 2010). See also “Cherry picking (fallacy),” Wikipedia (describing cherry picking as the pointing to data that appears to confirm one’s opinion, while ignoring contradictory data).

In 1965, in his landmark paper, Sir Austin Bradford Hill described some important factors to consider in determining whether a clear-cut association, beyond that which we would attribute to chance, was a causal association. Hill, Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965).

One of the key Hill factors is, of course, consistent, replicated results. Surely, an expert witness should not be permitted to manufacture a faux consistency by conducting a partial review. In birth defects litigation, the problem of “cherry picking” is so severe that one of the leading professional societies concerned with birth defects has issued a position paper to remind its members, other scientists, and the public that “[c]ausation determinations are made using all the scientific evidence”:

“Causation determinations are made using all the scientific evidence. This evidence is derived from correctly interpreted papers that have been published in the peer-reviewed literature. Unpublished data may be useful if available in sufficient detail for an evaluation and if derived from a source that is known to use reliable internal or external review standards. A National Toxicology program report would be an example of an unpublished source that is typically reliable. All available papers are considered in a scientific deliberation; selective consideration of the literature is not a scientific procedure.”

The Public Affairs Committee of the Teratology Society, “Teratology Society Public Affairs Committee Position Paper Causation in Teratology-Related Litigation,” 73 Birth Defects Research (Part A) 421, 422 (2005) (emphasis added).

* * * * * *

Cherry picking is a main rhetorical device for the litigator. Given the pejorative connotations of “cherry picking,” no one should be very surprised that lawyers and judges couch their Rule 702 arguments and opinions in terms of whether expert witnesses engaged in this fulsome fruitful behavior.

The judicial approach to cherry picking is a just a little schizophrenic. Generally, in the context of exercising its gatekeeping function for expert witnesses, the elimination of cherry picking is an important goal. Lust v. Merrell Dow Pharmaceuticals, Inc., 89 F.3d 594, 596-98 (9th Cir. 1996) (affirming exclusion of Dr. Done in a Chlomid birth defects case; district court found that “Dr. Done has seen fit to ‘pick and chose’ [sic] from the scientific landscape and present the Court with what he believes the final picture looks like. This is hardly scientific.”) (internal citation omitted); Barber v. United Airlines, Inc., 17 Fed. Appx. 433, 437 (7th Cir. 2001) (holding that a “selective use of facts fails to satisfy the scientific method and Daubert”). See also Crawford v. Indiana Harbor Belt Railroad Co., 461 F.3d 844 (7th Cir. 2006) (affirming summary judgment in disparate treatment discharge case, and noting judicial tendency to require “comparability” between plaintiffs and comparison group as a “natural response to cherry-picking by plaintiffs”); Miller v. Pfizer, Inc., 196 F. Supp. 2d 1062, (D. Kan. 2002) (excluding, with aid of independent, court-appointed expert witnesses, a party expert witness, David Healy, who failed to reconcile the fact that other research is contrary to his conclusion), aff’d, 356 F.3d 1326 (10th Cir.), cert denied, 125 S. Ct. 40 (2004).

In Ellis v. Barnhart, the Eighth Circuit affirmed a district court’s reversal of an Administrative Law Judge for “cherry picking” the record in a disability case. 392 F.3d 988 (8th Cir. 2005). Clearly cherry picking was a bad thing for a judicial officer to do when charged with the administration of justice. Several years later, however, the Eighth Circuit held that a trial court erred in excluding an expert witness for having offered an opinion that ignored the witness’s own prior, contrary opinions, a key National Institutes of Health clinical trial, and multiple other studies. The adversary’s charges of “cherry picking” were to no avail. Kuhn v. Wyeth, Inc., 686 F.3d 618, 633 (8th Cir. 2012) (“There may be several studies supporting Wyeth’s contrary position, but it is not the province of the court to choose between the competing theories when both are supported by reliable scientific evidence.”), rev’g Beylin v. Wyeth, 738 F.Supp. 2d 887, 892 (E.D.Ark. 2010) (MDL court) (Wilson, J. & Montgomery, J.) (excluding proffered testimony of Dr. Jasenka Demirovic who appeared to have “selected study data that best supported her opinion, while downplaying contrary findings or conclusions.”).

But wait, the court in Kuhn did not cite its own published opinion on cherry picking in Ellis. Some might say that the Circuit cherry picked its own precedents to get to a desired result. Anthony Niblett, “Do Judges Cherry Pick Precedents to Justify Extralegal Decisions?: A Statistical Examination,” 70 Maryland L. Rev. 234 (2010) (reviewing charges of cherry picking, and examining data [cherry picked?] from California).

The situation in the federal trial courts is chaotic. Most of the caselaw recognizes the fallacy of an expert witness’s engaging in ad hoc selection of studies upon which to rely. Federal courts, clear on their gatekeeping responsibilities and aware of the selection fallacy, have condemned cherry-picking expert witnesses. Judge Lewis Kaplan, in the Southern District of New York, expressed the proper judicial antipathy to cherry picking:

“[A]ny theory that fails to explain information that otherwise would tend to cast doubt on that theory is inherently suspect,” and “courts have excluded expert testimony ‘where the expert selectively chose his support from the scientific landscape.’”

In re Rezulin Prod. Liab. Litig., 369 F. Supp. 2d 398, 425 & n.164 (S.D.N.Y. 2005) (citation omitted).

Judge Breyer, of the Northern District of California, expressed similar sentiments in ruling on Rule 702 motions in the Celebrex personal injury litigation:

“these experts ignore the great weight of the observational studies that contradict their conclusion and rely on the handful that appear to support their litigation-created opinion.”

In re Bextra & Celebrex Mktg. Sales Pracs. & Prods. Liab. Litig., 524 F. Supp. 2d 1166, 1181 (N.D. Cal. 2007). The “cherry-picking” of favorable data “does not reflect scientific knowledge, is not derived by the scientific method, and is not ‘good science.’” Id. at 1176.

Other illustrative federal cases include:

In re Bausch & Lomb, Inc., 2009 WL 2750462 at *13-14 (D.S.C. 2009) (“Dr. Cohen did not address [four contradictory] studies in her expert reports or affidavit, and did not include them on her literature reviewed list [. . .] This failure to address this contrary data renders plaintiffs’ theory inherently unreliable.”)

Rimbert v. Eli Lilly & Co., No. 06-0874, 2009 WL 2208570, *19 (D.N.M. July 21, 2009) )(“Even more damaging . . . is her failure to grapple with any of the myriad epidemiological studies that refute her conclusion.”), aff’d, 647 F.3d 1247 (10th Cir. 2011) (affirming exclusion but remanding to permit plaintiff to find a new expert witness)

LeClercq v. The Lockformer Co., No. 00C7164, 2005 WL 1162979, at *4, 2005 U.S. Dist. LEXIS 7602, at *15 (N.D. Ill. Apr. 28, 2005) (“failure to discuss the import of, or even mention … material facts in [expert] reports amounts to ‘cherry-pick[ing]’ … and such selective use of facts fail[s] to satisfy the scientific method and Daubert.”) (internal citations and quotations omitted)

Contractors Ass’n of E. Pa. Inc. v. City of Philadelphia, 893 F. Supp. 419, 436 (E.D. Pa., 1995) (holding that expert witness opinion was unreliable when witness’s conclusions rested on incomplete factual data)

Galaxy Computer Servs. Inc. v. Baker, 325 B.R. 544 (E.D. Va. 2005) (excluding expert witness when witness relied upon incomplete data in reaching a valuation assessment).

Dwyer v. Sec’y of Health & Human Servs., No. 03-1202V, 2010 WL 892250, at *14 (Fed. Cl. Spec. Mstr. Mar. 12, 2010)(recommending rejection of thimerosal autism claim)(“In general, respondent’s experts provided more responsive answers to such questions. Respondent’s experts were generally more careful and nuanced in their expert reports and testimony. In contrast, petitioners’ experts were more likely to offer opinions that exceeded their areas of expertise, to “cherry-pick” data from articles that were otherwise unsupportive of their position, or to draw conclusions unsupported by the data cited… .”)

Holden Metal & Aluminum Works, Ltd. v. Wismarq Corp., No. 00C0191, 2003 WL 1797844, at *2 (N.D. Ill. Apr. 3, 2003) (“Essentially, the expert ‘cherrypicked’ the facts he considered to render his opinion, and such selective use of facts failed to satisfy the scientific method and Daubert.”) (internal citation omitted).

Flue-Cured Tobacco Cooperative Stabilization Corp. v. EPA, 4 F. Supp. 2d 435, 459 – 60 (M.D.N.C. 1998) (finding that EPA’s selection of studies for inclusion in a meta-analysis to be “disturbing,” and that agency’s selective, incomplete inclusion of studies violated its own guidelines for conducting risk assessments), rev’d on other grounds, 313 F.3d 852, 862 (4th Cir. 2002) (Widener, J.) (holding that the issuance of the report was not “final agency action”)

Fail-Safe, LLC v. AO Smith Corp., 744 F. Supp. 2d 870, 889 (E.D. Wis. 2010) (“the court also finds the witness’s methodology unreliable because of how Dr. Keegan uniformly treated all evidence that undermined his underlying conclusion: unwarranted dismissal of the evidence or outright blindness to contrary evidence. … In fact, it is readily apparent that Dr. Keegan all but ‘cherry picked’ the data he wanted to use, providing the court with another strong reason to conclude that the witness utilized an unreliable methodology. * * * Dr. Keegan’s two reports are rich with examples of his ‘cherry picking’ of the evidence.”)

As noted, however, there are federal trial courts that are all too willing to suspend judgment and kick the case to the jury. Here is a sampler of cases that found cherry picking to be an acceptable methodology, or at least a methodology sufficient to require that the case be submitted to the finder of fact.

In Berg v. Johnson & Johnson, the district court noted the defendants’ argument that proffered testimony is unreliable because witness “cherry-picked” data in order to form an opinion solely for purposes of litigation. 940 F.Supp. 2d 983, 991-92 (D.S.D. 2013). The trial judge, however, was not willing to look particularly closely at what was excluded or why:

“The only difference between his past and present research seems to exist in how he categorized his data. Defendants label this ‘cherry-picking’. The court views it as simply looking at the existing data from a different perspective.”

Id. Of course, expert witnesses on opposite sides look at the case from different perspectives, but the question begged was whether the challenged expert witness had categorized data in an unprincipled way. Other cases of this ilk include:

United States v. Paracha, 2006 WL 12768, at *20 (S.D. N.Y. Jan. 3, 2006) (rejecting challenge to terrorism expert witness on grounds that he cherry picked evidence in conspiracy prosecution involving al Queda)

In re Chantix (Varenicline) Products Liab. Litig., 889 F. Supp. 2d 1272, 1288 (N.D. Ala. 2012) (“Why Dr. Kramer chose to include or exclude data from specific clinical trials is a matter for cross-examination, not exclusion under Daubert.“)

Bouchard v. Am. Home Prods. Corp., 2002 WL 32597992 at *7 (N.D. Ohio May 24, 2002) (“If Bouchard believes that [the expert]… ignored evidence that would have required him to substantially change his opinion, that is a fit subject for cross-examination, not a grounds for wholesale rejection of an expert opinion.”)

In re Celexa & Lexapro Prods. Liab. Litig., 927 F. Supp. 2d 758, 2013 WL 791780, at *5, *7, *8 (E.D. Mo. 2013) (Sippel, J.) (rejecting challenge to David Healy in antidepressant suicide case)

Allen v. Takeda Pharms., MDL No. 6:11-md-2299, No. 12-cv-00064, 2013 WL 6825953, at *11 (W.D. La. Dec. 20, 2013) (challenged expert witness in Actos litigation sufficiently explained his choices to be exonerated from charges of cherry picking)

In re NuvaRing Prods. Liab. Litig., No. 4:08–MD–1964 RWS, 2013 WL 791787 (E.D. Mo. Mar. 4, 2013) (“As to cherry picking data, the Eighth Circuit has recently made clear that such allegations should be left for crossexamination.”)

McClellan v. I-Flow Corp., 710 F. Supp. 2d 1092, 1114 (D. Ore. 2010) (“Defendants are correct that plaintiffs’ experts must elucidate how the relevant evidence lends support to their opinions by explaining…..”) (rejecting cherry picking but denying Rule 702 challenge based in part upon alleged cherry picking)

Rich v. Taser Internat’l, Inc., No. 2:09–cv–02450–ECR–RJJ, 2012 WL 1080281, at *6 (D. Nev. March 30, 2012) (noting the objection to cherry picking but holding that it was an issue for cross-examination)

In re Urethane Antitrust Litig., No. 04-1313-JWL, MDL No. 1616, 2012 WL 6681783, at *3 (D. Kan. Dec. 21, 2012) (allowing expert testimony that “certain events are consistent with collusion”; “the extent to which [an expert] considered the entirety of the evidence in the case is a matter for cross-examination.”)

In re Titanium Dioxide Antitrust Litig., No. RDB-10-0318, 2013 WL 1855980, 2013 U.S. Dist. LEXIS 62394 (D. Md. May 1, 2013) (rejecting Rule 702 cherry-picking challenge to an expert who cherry picked; witness’s selection of documents upon which to rely from a record that exceeded 14 million pages was not unreliable. “ If important portions of the record were overlooked, then the Defendants may address that issue at trial.”)

STATE COURTS

The situation in state courts is similarly chaotic and fragmented.

In Lakey v. Puget Sound Energy, Inc., the Washington Supreme Court resoundingly rejected “cherry picking” by expert witnesses in a public and private nuisance case against a local utility for fear of future illnesses from exposure to electro-magnetic frequency radiation (EMF). Lakey v. Puget Sound Energy, Inc., 176 Wn.2d 909 (2013). The court held that the plaintiffs’ expert witnesses’ cherry-picking approach to data and studies was properly excluded under Rule 702. Their selective approach vitiated the reliability of his opinion with the consequence of :

“seriously tainting his conclusions because epidemiology is an iterative science relying on later studies to refine earlier studies in order to reach better and more accurate conclusions. Carpenter refused to account for the data from the toxicological studies, which epidemiological methodology requires unless the evidence for the link between exposure and disease is unequivocal and strong, which is not the case here. Carpenter also selectively sampled data within one of the studies he used, taking data indicating an EMF-illness link and ignoring the larger pool of data within the study that showed no such link, Carpenter’s treatment of this data created an improper false impression about what the study actually showed.”

Id.; see also “Washington Supreme Court Illustrates the Difference Between Frye and Rule 702” (April 15, 2013).

Other state Supreme Courts have recognized and rejected the gerrymandering of scientific evidence. Betz v. Pneumo Abex LLC, 2012 WL 1860853, *16 (May 23, 2012 Pa. S. Ct.)(“According to Appellants, moreover, the pathologist’s self-admitted selectivity in his approach to the literature is decidedly inconsistent with the scientific method. Accord Brief for Amici Scientists at 17 n.2 (“‘Cherry picking’ the literature is also a departure from ‘accepted procedure’.”)); George v. Vermont League of Cities and Towns, 2010 Vt. 1, 993 A.2d 367, 398 (Vt. 2010)(expressing concern about how and why plaintiff’s expert witnesses selected some studies to include in their “weight of evidence” methodology. Without an adequate explanation of selection and weighting criteria, the choices seemed “arbitrary” “cherry picking.”); Bowen v. E.I. DuPont de Nemours & Co., 906 A.2d 787, 797 (Del. 2006) (noting that expert witnesses cannot ignore studies contrary to their opinions).

Lower state courts have also quashed the cherry-picking harvest. Scaife v. AstraZeneca LP, 2009 WL 1610575, at *8 (Del. Super. June 9, 2009) (“Simply stated, the expert cannot accept some but reject other data from the medical literature without explaining the bases for her acceptance or rejection.”); see also In re Bextra & Celebrex Prod. Liab. Litig., No. 762000/2006, 2008 N.Y. Misc. LEXIS 720, at *47 (Sup. Ct. N.Y. Co. Jan 7, 2008) (stating that plaintiffs must show that their experts “do not ignore contrary data”).

The Nebraska Supreme Court appears to recognize the validity of considering the existence of cherry-picking in expert witness gatekeeping. In practice, however, that Court has shown an unwillingness to tolerate close scrutiny into what was included and excluded from the expert witness’s consideration. King v. Burlington No. Santa Fe Ry, ___N.W.2d___, 277 Neb. Reports 203, 234 (2009)(noting that the law does “not preclude a trial court from considering as part of its reliability inquiry whether an expert has cherry-picked a couple of supporting studies from an overwhelming contrary body of literature,” but ignoring the force of the fallacious expert witness testimony by noting that the questionable expert witness (Frank) had some studies that showed associations between exposure to diesel exhaust or benzene and multiple myeloma).

“Of all the offspring of time, Error is the most ancient, and is so old and familiar an acquaintance, that Truth, when discovered, comes upon most of us like an intruder, and meets the intruder’s welcome.”

Charles MacKay, Extraordinary Popular Delusions and the Madness of Crowds (1841)

Posted in Causation, Rule 702 | Comments Off on The Fallacy of Cherry Picking As Seen in American Courtrooms

Relative Risk of Two in Medical Malpractice Cases

April 14th, 2014

Counsel for plaintiffs and defendants in toxic tort cases are well aware of the need to show a sufficiently large relative risk, greater than two, to have sufficient evidence to satisfy the burden of proof on proximate causation between a known causal exposure and a specific plaintiff’s injury. As Judge Jack Weinstein wrote 30 years ago, “[a] government administrative agency may regulate or prohibit the use of toxic substances through rulemaking, despite a very low probability of any causal relationship. A court, in contrast, must observe the tort law requirement that a plaintiff establish a probability of more than 50% that the defendant’s action injured him. … This means that at least a two-fold increase in incidence of the disease attributable to Agent Orange exposure is required to permit recovery if epidemiological studies alone are relied upon.” In re Agent Orange Product Liab. Litig., 597 F. Supp. 740, 785, 836 (E.D.N.Y. 1984), aff’d 818 F.2d 145, 150-51 (2d Cir. 1987)(approving district court’s analysis), cert. denied sub nom. Pinkney v. Dow Chemical Co., 487 U.S. 1234 (1988).

In toxic tort cases, the risk ratio at issue allegedly results from a higher incidence of the disease in exposed persons compared to the incidence in unexposed persons. A similar risk ratio issue occurs in medical malpractice cases when a healthcare provider negligently fails to administer a therapy, or fails to administer a therapy in a timely fashion, to the detriment of the plaintiff. In instances in which the therapy is almost always efficacious, the risk ratio of a bad patient outcome will be very high, and the corresponding probability that the bad outcome would have been avoided by proper or timely therapy will be close to 100 percent. On the other hand, for some therapies, even timely administration is efficacious in a limited number of cases, less often than the 50-plus percent of cases that would support a proximate cause opinion between the allegedly negligent failure to administer therapy and the patient’s bad health outcome.

Unfortunately, the relative risk issue goes unlitigated in many cases, in New York and elsewhere. One recurring malpractice claim involves the alleged failure to administer clot-busting drugs to ischemic stroke patients. One such drug, tissue plasminogen activator (t-PA), which was approved by the Food and Drug Administration in 1996, can substantially reduce brain damage if administered within three hours of stroke onset. Even if administered within the crucial therapeutic time window, however, t-PA will benefit only about 30 percent of patients, and there is no medical “fingerprint”that identifies who has benefitted from the t-PA. In Samaan v. St. Joseph Hospital, 670 F.3d 21 (1st Cir. 2012), the First Circuit acted on its gatekeeping responsibilities to perscrutate the risk evidence and found that it fell short of what is required by Federal Rule of Evidence 702, and the “more likely than not” standard for civil cases. See also Smith v. Bubak, 643 F.3d 1137, 1141–42 (8th Cir.2011) (rejecting relative benefit testimony and suggesting in dictum that absolute benefit “is the measure of a drug’s overall effectiveness”); Young v. Mem’l Hermann Hosp. Sys., 573 F.3d 233, 236 (5th Cir.2009) (holding that Texas law requires a doubling of the relative risk of an adverse outcome to prove causation), cert. denied, ___ U.S. ___, 130 S.Ct. 1512 (2010).

Samaan and the cases cited by the First Circuit are hardly unique; the size of the relative risk issue has helped the defense prevail in other t-PA and similar malpractice cases around the country. Kava v. Peters, 450 Fed.Appx. 470, 478-79 (6th Cir. 2011) (affirming summary judgment for defendants when plaintiffs expert witnesses failed to provide clear testimony that plaintiff specific condition would have been improved by timely administration of therapy); Bonesmo v. The Nemours Foundation, 253 F.Supp. 2d 801, 809 (D.Del. 2003); Joshi v. Providence Health System of Oregon Corp., 342 Or. 152, 156, 149 P. 3d 1164, 1166 (2006) (affirming directed verdict for defendants when expert witness testified that he could not state, to a reasonable degree of medical probability, beyond 30%, that administering t-PA, or other anti-coagulant would have changed the outcome and prevented death); Ensink v. Mecosta County Gen. Hosp., 262 Mich.App. 518, 687 N.W.2d 143 (Mich.App. 2004) (affirming summary judgment for hospital and physicians when patient could not greater than 50% probability of obtaining a better result had emergency physician administered t-PA within three hours of stroke symptoms); Merriam v. Wanger, 757 A.2d 778, 2000 Me. 159 (2000) (reversing judgment on jury verdict for plaintiff on grounds that plaintiff failed to show that defendant failure to act were, more likely than not, a cause of harm). In Michigan, the holding of the t-PA and similar medical malpractice cases has been codified by statute:

“In an action alleging medical malpractice, the plaintiff has the burden of proving that he or she suffered an injury that more probably than not was proximately caused by the negligence of the defendant or defendants. In an action alleging medical malpractice, the plaintiff cannot recover for loss of an opportunity to survive or an opportunity to achieve a better result unless the opportunity was greater than 50%.”

Mich. Comp. Laws § 600.2912a(2) (2009). But see O’Neal v. St. John Hosp. & Med. Ctr., 487 Mich. 485, 791 N.W.2d 853 (Mich. 2010) (affirming denial of summary judgment when failure to administer therapy (not t-PA) in a timely fashion more than doubled the risk of stroke). In one unpublished Kentucky case, involving t-PA, the court seemed to acknowledge the general principle, but became confused as to whether 30 percent, was a reasonable probability. Lake Cumberland, LLC v. Dishman, 2007 WL 1229432, *5 (Ky. Ct. App. 2007) (unpublished) (citing without critical discussion an apparently innumerate opinion of expert witness Dr. Lawson Bernstein).

Despite the success of medical malpratice defense counsel in litigating dispositive motions in t-PA cases, the issue seems to go unnoticed in New York cases. For instance, in Gyani v. Great Neck Medical Group, a stroke victim sued on various allegations of medical malpractice, including failure to administer t-PA. N.Y. S.Ct. for Nassau Cty, 2011 WL 1430037 (April 4, 2011). The trial court denied summary judgment on proximate cause grounds, and noted that

“[t]he plaintiffs’ expert ultimately opines that the failure to administer t-PA allowed Gyani’s stroke to go untreated and progress to the point of her being locked-in permanently which would not have happened had t-PA been administered.”

From the court’s opinion, it would appear that defense counsel never pressed beyond this conclusory opinion, devoid of quantified relative risk. Behind the curtain of “ultimate” opinion is an expert without a meaningful basis for his opinion. It is time to pull the curtain.

Posted in Causation, Expert Witnesses, Risk and Risk Factor, Rule 702 | Comments Off on Relative Risk of Two in Medical Malpractice Cases

A Black Swan Case – Bayesian Analysis on Medical Causation

March 15th, 2014

Last month, I posted about an article that Professor Greenland wrote several years ago about his experience as a plaintiffs’ expert witness in a fenfluramine case. “The Infrequency of Bayesian Analyses in Non-Forensic Court Decisions (Feb. 16, 2014).” Greenland chided a defense expert for having declared that Bayesian analyses are rarely or never used in analyzing clinical trials or in assessments of pharmaco-epidemiologic data. Greenland’s accusation of ludicrousness appeared mostly to blow back on him, but his stridency for Bayesian analyses did raise the question, whether such analyses have ever moved beyond random-match probability analyses in forensic evidence (DNA, fingerprint, paternity, etc.) or in screening and profiling cases. I searched Google Scholar and Westlaw for counter-examples and found none, but I did solicit references to “Black Swan” cases. Shortly after I posted about the infrequency of Bayesian analyses, I came across a website that was dedicated to collecting legal citations of cases in which Bayesian analyses were important, but this website appeared to confirm my initial research.

Some months ago, Professor Brian Baigrie, of the Jackman Humanities Institute, at the University of Toronto, invited me to attend a meeting of an Institute working group on The Reliability of Evidence in Science and the Law. The Institute fosters interdisciplinary scholarship, and this particular working group has a mission statement close to my interests:

“The object of this series of workshops is to formulate a clear set of markers governing the reliability of evidence in the life sciences. The notion of evidence is a staple in epistemology and the philosophy of science; the notion of this group will be the way the notion of ‘evidence’ is understood in scientific contexts, especially in the life sciences, and in judicial form as something that ensures the objectivity of scientific results and the institutions that produce these results.”

The Reliability of Evidence in Science and the Law. The faculty on the working group represent disciplines of medicine (Andrew Baines), philosophy (James R. Brown, Brian Baigrie), and law (Helena Likwornik, Hamish Stewart), with graduate students in the environmental science (Amy Lemay), history & philosophy of science and technology (Karolyn Koestler, Gwyndaf Garbutt ), and computer science (Maya Kovats).

Coincidentally, in preparation for the meeting, Professor Baigrie sent me links to a Canadian case, Goodman v. Viljoen, which turned out to be a black swan case! The trial court’s decision, in this medical malpractice case focused mostly on a disputed claim of medical causation, in which the plaintiffs’ expert witnesses sponsored a Bayesian analysis of the available epidemiologic evidence; the defense experts maintained that causation was not shown, and they countered with the unreliability of the proffered Bayesian analysis. The trial court resolved the causation dispute in favor of the plaintiffs, and their witnesses’ Bayesian approach. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII). The Court of Appeals’ affirmance was issued over a lengthy, thoughtful dissent. The Canadian Supreme Court denied leave to appeal.

Goodman was a medical practice case. Mrs. Goodman alleged that her obstetrician deviated from the standard of care by failing to prescribe corticosteroids sufficiently early in advance of delivery to avoid or diminish the risk of cerebral palsy in her twins. Damages were stipulated, and the breach of duty turned on a claim that Mrs. Goodman, in distress, called her obstetrician. Given the decade that passed between the event and the lawsuit, the obstetrician was unable to document a response. Duty and breach were disputed, but were not the focus of the trial.

The medical causation claim, in Goodman, turned upon a claim that the phone call to the obstetrician should have led to an earlier admission to the hospital, and the administration of antenatal corticosteroids. According to the plaintiffs, the corticosteroids would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition. The plaintiffs’ expert witnesses relied upon studies that suggested a 40% reduction and risk, and a probabilistic argument that they could infer from this risk ratio that the plaintiffs’ condition would have been avoided. The case thus raises the issue whether evidence of risk can substitute for evidence of causation. The Canadian court held that risk sufficed, and it went further, contrary to the majority of courts in the United States, to hold that a 40% reduction in risk sufficed to satisfy the more-likely-than-not standard. See, e.g., Samaan v. St. Joseph Hosp., 670 F.3d 21 (1st Cir. 2012) (excluding expert witness testimony based upon risk ratios too small to support opinion that failure to administer intravenous tissue plasminogen activator (t-PA) to a patient caused serious stroke sequelae); see also “Federal Rule of Evidence 702 Requires Perscrutations — Samaan v. St. Joseph Hospital (2012)” (Feb. 4, 2012).

The Goodman courts, including the dissenting justice on the Ontario Court of Appeals, wrestled with a range of issues that warrant further consideration. Here are some that come to mind from my preliminary read of the opinions:

1. Does evidence of risk suffice to show causation in a particular case?

2. If evidence of risk can show causation in a particular case, are there requirements that the magnitude of risk be quantified and of a sufficient magnitude to support the inference of causation in a particular case?

3. The judges and lawyers spoke of scientific “proof.” When, if ever, is it appropriate to speak of scientific proof of a medical causal association?

4. Did the judges incorrectly dichotomize legal and scientific standards of causation?

5. Did the judges, by rejecting the need for “conclusive proof,” fail to articulate a meaningful standard for scientific evidence in any context, including judicial contexts?

6. What exactly does the “the balance of probabilities” mean, especially in the face of non-quantitative evidence?

7. What is the relationship between “but for” and “substantial factor” standards of causation?

8. Can judges ever manage to define “statistical significance” correctly?

9. What is the role of “common sense” in drawing inferences by judges and expert witnesses in biological causal reasoning? Is it really a matter of common sense that if a drug did not fully avert the onset of a disease, it would surely have led to a less severe case of the disease?

10. What is the difference between “effect size” and the measure of random or sampling error?

11. Is scientific certainty really a matter of being 95% certain, or is this just another manifestation of the transposition fallacy?

12. Are Bayesian analyses acceptable in judicial settings, and if so, what information about prior probabilities must be documented before posterior probabilities can be given by expert witnesses and accepted by courts?

13. Are secular or ecological trends sufficiently reliable data for expert witnesses to rely upon in court proceedings?

14. Is the ability to identify biological plausibility sufficient to excuse the lack of statistical significance and other factors that are typically needed to support the causality of a putative association?

15. What are the indicia of reliability of meta-analyses used in judicial proceedings?

16. Should courts give full citations to scientific articles that are heavily relied upon as part of the requirement that they publicly explain and justify their decisions?

These are some of the questions that come to mind from my first read of the Goodman case. The trial judge attempted to explain her decision in a fairly lengthy opinion. Unfortunately, the two judges, of the Ontario Court of Appeals, who voted to affirm, did not write at length. Justice Doherty wrote a thoughtful dissent, but the Supreme Court denied leave to appeal. Many of the issues are not fully understandable from the opinions, but I hope to be able to read the underlying testimony before commenting.

Thanks to Professor Baigrie for the reference to this case.

Posted in Causation, Scientific Evidence, statistical evidence | Comments Off on A Black Swan Case – Bayesian Analysis on Medical Causation

Judicial Notice of Untruths

March 3rd, 2014

Judicial notice is a procedure for admitting facts the truth of which are beyond dispute. A special kind of magically thinking occurs when judges take judicial notice of falsehoods, myths, or lies.

In the federal judicial system, Federal Rule of Evidence 201 addresses judicial notice of adjudicative facts, and provides:

(b) Kinds of Facts That May Be Judicially Noticed. The court may judicially notice a fact that is not subject to reasonable dispute because it:

(1) is generally known within the trial court’s territorial jurisdiction; or

(2) can be accurately and readily determined from sources whose accuracy cannot reasonably be questioned.

Procedurally, Rule 201 provides that a court must take judicial upon the request of a party who has supplied any needed basis for the fact to be noticed. A court may take notice sua sponte. Rule 201(c)(1), (2).

In the Chantix litigation, counsel for Pfizer challenged plaintiffs’ expert witness, Curt Furberg, on Rule 702 grounds. According the MDL judge, the Hon. Inge Prytz Johnson, Pfizer asserted that Furberg’s proferred testimony because the FDA approved Chantix as safe and effective. In re Chantix (Varenicline) Prods. Liab. Litig., 889 F. Supp. 2d 1272, 1285 n.8 (N.D. Ala. 2012). Citing no authority or text, Judge Johnson announced that “[a]pproval by the FDA is not evidence of the safety of a medication.” Id.

To be sure, safety issue can sometimes arise after initial approval, but before the FDA or the manufacturer and sponsor of the medication can react to the new safety data. The sweeping statement, however, that the FDA’s approval is not any evidence of safety seems bereft of factual support and common sense.

Judge Johnson went on, however, to invent supporting evidence out of thin air:

“The court takes judicial notice of such things as that at one time, thalidomide was used for morning sickness in pregnant women. Unfortunately, 10,000 children were born with birth defects from it before it was banned. And 50 years elapsed before doctors understood why thalidomide caused limbs to disappear. See e.g. http://www.nytimes.com/2010/03/16/science/16limb.html?pagewanted=all. Similarly, the fact that the FDA at one time approved Vioxx did not prevent the same being removed from the market due to growing concerns that it increased the risk of heart attacks and strokes. http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/ucm103420.htm. Hence, initial approval by the FDA is not proof of the safety of a medication.”

The point about the FDA’s approval not constituting evidence of safety may simply be sloppy writing and reasoning. In the quote above, perhaps Her Honor merely meant to say that initial approval is not evidence that a medication is safe in view of later obtained data that were not available to the FDA on its review of the new drug application. If so, fair enough, but the sweeping statement that the initial approval is no evidence of safety ignores the considerable time, cost, and energy that goes into the FDA’s review of safety before agency approves marketing.

More egregious, however, is Judge Johnson’s taking judicial notice of the marketing of thalidomide as though it had some relevancy and probative value for her claim about the inefficacy of the FDA’s safety reviews.[1] Consider the recent review of the FDA’s handling of thalidomide by Margaret Hamburg, M.D., Commissioner of the U. S. Food and Drug Administration:

“Fifty years ago, the vigilance of FDA medical officer Dr. Frances Kelsey prevented a public health tragedy of enormous proportion by ensuring that the sedative thalidomide was never approved in the United States. As many remember, in the early 1960’s, reports were coming in from around the world of countless women who were giving birth to children with extremely deformed limbs and other severe birth defects. They had taken thalidomide. Although it was being used in many countries, Dr. Kelsey discovered that it hadn’t even been tested on pregnant animals.”

Margaret Hamburg, “50 Years after Thalidomide: Why Regulation Matters” (Feb. 7, 2012).

Judge Johnson took judicial notice of a non-fact. The FDA never approved thalidomide for use in the United States, back in the 1950s or 1960s.[2]

[1] Judge Johnson’s fantastical history of the FDA was recently cited by plaintiffs’ counsel in the Zoloft birth defects litigation. See Plaintiffs’ Opposition to Defendants’ Motion to Exclude the Testimony of Anick Berard, Ph.D., at 13 (Filed Feb. 24, 2014), in In re Zoloft (sertraline hydrochloride) Prods. Liab. Litig., Case 2:12-md-02342-CMR Document 713.

[2] Judge Johnson’s errant history may have resulted from her European perspective of the thalidomide tragedy. Judge Inge Prytz Johnson immigrated from Denmark, where she was born and educated. She became a U.S. citizen in 1978, and a state court judge one year later. In 1998, she was nominated by President Clinton to the Northern District of Alabama. In October 2012, Judge Johnson assumed senior status. See Kent Faulk, “U.S. District Judge Inge Johnson goes into semi-retirement” (Oct. 19, 2012) (quoting Judge Johnson as saying that “One thing I like about my job is I don’t have to take sides.”)

Posted in Causation, Scientific Evidence, Trial Procedure | Comments Off on Judicial Notice of Untruths

“Judges and other lawyers must learn how to deal with scientific evidence and inference.”

March 1st, 2014

Late last year, a panel of 7th Circuit reversed an Administrative Law Judge (ALJ) who had upheld a citation and fine against Caterpillar Logistics, Inc. (Cat). The panel, in a wonderfully succinct, but meaty decision by Judge Easterbrook, wrote of the importance of judges’ and lawyers’ learning to deal with scientific and statistical evidence. Caterpillar Logistics, Inc. v. Perez, 737 F.3d 1117 (7th Cir. 2013)

Pseudonymous MK, a worker in Cat’s packing department, developed epidcondylitis (tennis elbow). Id. at 1118. OSHA regulations require employers to report injuries “the work environment either caused or contributed to the resulting condition”. 29 C.F.R. § 1904.5(a). MK’s work required her to remove items from containers and place items in shipping cartons. The work was repetitive, but MK acknowledged that the work involved little or no impact or force. Apparently, Cat gave some rather careful consideration to whether MK’s epidcondylitis was work related; it assembled a panel of three specialists in musculoskeletal disorders and two generalists to consider the matter. The panel, relying upon NIOSH and AMA guidelines, rejected MK’s claim of work relatedness. Both the NIOSH and the AMA guidelines conclude that repetitive motion in the absence of weight or impact does not cause epicondylitis. Id.

MK called an expert witness, Dr. Robert Harrison, a clinical professor of medicine, at the University of California, San Francisco. Id. at 1118-1119. Harrison unequivocally attributed MK’s condition to her work at Cat, but he failed to explain why no one else in Cat’s packing department ever developed the condition. Id. at 1119.

Harrison acknowledged that epidemiologic evidence could confirm his opinion, but he dismissed such evidence as being able to disconfirm his opinion. The ALJ echoed Dr. Harrison in holding epidemiologic evidence to be irrelevant:

“none of these [other] people are [sic] MK. Similar to the concept of the ‘eggshell skull’ plaintiff in civil litigation, you take your workers as they are.”

Id. at 1119-20, citing ALJ, at 2012 OSAHRC LEXIS 118 at *32.

Judge Easterbrook found this attempt to disqualify any opposing evidence to lie beyond the pale:

“Judges and other lawyers must learn how to deal with scientific evidence and inference.”

Id. (citing Jackson v. Pollion, 733 F.3d 786 (7th Cir. 2013).

Judge Easterbrook called out the ALJ for misunderstanding the nature of epidemiology and the role of statistics, in the examination of causation of health outcomes that have a baseline incidence or prevalence in the population:

“The way to test whether Harrison is correct is to look at data from thousands of workers in hundreds of workplaces—or at least to look at data about hundreds of worker-years in Caterpillar’s own workplace. Any given worker may have idiosyncratic susceptibility, though there’s no evidence that MK does. But the antecedent question is whether Harrison’s framework is sound, and short of new discoveries about human physiology only statistical analysis will reveal the answer. Any large sample of workers will contain people with idiosyncratic susceptibilities; the Law of Large Numbers ensures that their experience is accounted for. If studies of large numbers of workers show that the incidence of epicondylitis on jobs that entail repetitive motion but not force is no higher than for people who do not work in jobs requiring repetitive motion, then Harrison’s view has been refuted.”

Id. at 1120.

Judge Easterbrook acknowledged that Cat’s workplace evidence may have been a sample too small from which to draw a valid statistical inference, given the low base rate of epicondylitis in the general population. Dr. Harrison’s and the ALJ’s stubborn refusal, however, to consider any disconfirming evidence, obviating the need to consider sample size and statistical power issues.

Finally, Judge Easterbrook chastised the ALJ for dismissing Cat’s experience as irrelevant because many other employers will not have sufficient workforces or record keeping to offer similar evidence. In Judge Easterbrook’s words:

“This is irrational. If the camera in a police car captures the events of a highspeed chase, the judiciary would not ignore that video just because other police cars lack cameras; likewise, if the police record an interrogation, courts will consider that information rather than wait for the day when all interrogations are recorded.”

Id. This decision illustrates why some commentators at places such as the Center for Progressive Reform get their knickers in a knot over the prospect of applying the strictures of Rule 702 to agency fact finding; they know it will make a difference.

As for the “idiosyncratic gambit,” this argument is made all too frequently in tort cases, with similar lack of predicate. Plaintiffs claim that there may be a genetic or epigenetic susceptibility in a very small subset of the population, and that epidemiologic studies may miss this small, sequestered risk. Right, and the light in the refrigerator may stay on when you close the door. Prove it!

Posted in Causation, statistical evidence | Comments Off on “Judges and other lawyers must learn how to deal with scientific evidence and inference.”

Pennsylvania Superior Court Takes The Bite Out of Fixodent Claims

December 12th, 2013

In the spring of 2012, Judge Sandra Mazer Moss granted summary judgment to Proctor & Gamble, after excluding, on Frye grounds, plaintiff’s expert witnesses who opined that plaintiff suffered zinc neurotoxicity from his use of Fixodent. Jacoby v. Rite Aid Corp., 2012 Phila. Ct. Com. Pl. LEXIS 208 (2012). See “Philadelphia Plaintiff’s Claims Against Fixodent Prove Toothless” (May 2, 2012). Judge Moss’s exclusion of plaintiff’s expert witnesses involved a careful analysis of the evasive, hand-waving tactics of the witnesses. Among the plaintiff’s team of expert witnesses was Dr. Martyn Smith, the chief hand waver and obscurantist in Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009), rev’d, 639 F.3d 11 (1st Cir. 2011), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012).

On Monday, December 9, 2013, after a careful review, the Pennsylvania Superior Court affirmed summary judgment for Proctor & Gamble in the Jacoby case. Jacoby v. Rite Aid Corp., Pa. Super. Ct. No. 1508 EDA 2012 (Dec. 9, 2013) [Slip op.] The Superior Court panel, consisting of Judges Stevens, Lazarus, and Colville, largely adopted Judge Moss’s analysis and affirmed in a signed, but unpublished, opinion by Judge Lazarus.

Like Judge Moss before them, the Panel saw through the attempt to pass off “Weight of the Evidence” (WOE) and “Totality of the Evidence” (TOE) as scientific methodologies. The witnesses, Martyn Smith and others, failed to specify what evidence they weighed, how they weighed the evidence, and what the weights supposedly were. Another expert witness vaguely pointed to the “Naranjo scale” in support of interpreting case reports to show causal association, but this scale was similarly incompetent other than as a crude “plausibility” scale for assessing case reports.

The Superior Court’s decision in Jacoby is noteworthy on several important issues. There was no material issue as to whether zinc at some dose and duration of exposure can cause neuropathies. The Court saw, however, that the important issue was whether zinc in the form, dose, and duration ingested by plaintiff can cause the outcome he experienced, and whether his exposure to zinc in Fix-o-dent actually caused his alleged injury. The Superior Court re-affirmed Pennsylvania’s case law that makes extrapolation from different doses, different durations, and different biological circumstances, a “novel” claim that is subject to the gatekeeping by the so-called Frye standard. Slip op. at 11.

The Superior Court’s opinion astutely observed that the issue was not whether WOE and TOE are accepted scientific methodologies, but whether expert witness Martyn Smith can “evade a reasoned Frye inquiry merely by making reference to accepted methods in the abstract.” Id. at 12 (citing Betz, at 58). When pressed, Martyn Smith’s invocation of WOE amounted to little more than a distortion and abridgment of the Bradford Hill factors. The Superior Court recognized, however, that the Bradford Hill guidelines provide an evaluative process to consider whether an association, after first being shown to be clear cut and not attributable to chance, is causal or spurious. Id. at 13. As Bradford Hill postulated the question that arises before his famous nine factors come into the analysis:

“Disregarding then any such problem in semantics we have this situation. Our observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance. What aspects of that association should we especially consider before deciding that the most likely interpretation of it is causation?”

Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965) (emphasis added).

In this case, Smith never got off the dime with his evasive tactics. He did not identify a quantified association that could be assessed for the play of random variation, or bias and confounding. Smith tried to suggest that a mere possibility of an association was sufficient to invoke the Bradford Hill guidelines, but the Superior Court rejected this attempt to invent a new-age scientific method. Slip op. at 13.

The Superior Court acknowledged that the scientific literature describes WOE as varying from nothing more than “seat-of-the-pants qualitative assessment” to “aggregating diverse modalities.” Id. at 14. In Smith’s hands, WOE was little more than a personal, subjective opinion, an ipse dixit dressed as a scientific opinion. Smith never defined his WOE approach, and the other witnesses never defined their TOE approach. The plaintiffs’ witnesses in Jacoby failed to offer an accepted methodology when they failed to identify the forms of evidence they considered, and how they went about weighing the evidence upon which they had relied. Id. at 15.

In a brief discussion, the Panel also embraced another basic evidentiary principle to dismiss a common tactic in specious claiming. The plaintiffs’ challenged defendants’ pharmokinetic study and tried to suggest that their deconstruction counted as affirmative evidence to support their own theory of biological fate and distribution. Id. at 16. The Superior Court saw through the ruse; the plaintiffs had not created affirmative evidence for their theory by arguing that the defendants’ study was flawed.

The Superior Court squarely confronted the limitations and inadequacy of relying upon descriptive, anecdotal case reports. Case reports provide a narrative and temporal history of events with respect to exposure and outcome, but they cannot fully account for confounding by known and unknown factors. Case reports represent post hoc assessments that were not planned, and therefore lack data that would permit distinguishing coincidence from causality. Id. at 17 (citing Dr. Lorene Nelson’s report). See In re Denture Cream Prods. Liab. Litig., 795 F. Supp. 2d 1345 (S.D. Fla.2011).

Proctor & Gamble had the good fortunate to have obtained a good ruling in the MDL litigation in the Southern District of Florida, which no doubt helped focus the gatekeeping process in Pennsylvania state court. Unfortunately, the Superior Court Panel chose not to publish its decision. This decision is regrettable for its inconsistency with the transparency and due process expected of all courts. See Erica Weisgerber, “Unpublished Opinions: A Convenient Means to an Unconstitutional End,” 97 Georgetown L.J. 621 (2009).

Posted in Causation, Expert Witnesses, Frye, Scientific Evidence | Comments Off on Pennsylvania Superior Court Takes The Bite Out of Fixodent Claims

Pharmacovigilantism – Avandia Litigation

November 27th, 2013

Six and one-half years ago, I gave a presentation on the then newly emerging controversy over Avandia (rosiglitazone). Plaintiffs’ counsel Vance Andrus chaired the program, Mealey’s™ Avandia Litigation Conference, in Chicago on July 13, 2007. Vance was a gracious host despite my skepticism about the potential for plaintiffs to cash in on their use of Avandia.

Despite Vance’s best efforts, the program was one of those lopsided affairs, and I was the only presenter who came prepared to address the scientific evidence from a skeptical perspective. The remaining presenters were mostly cheerleaders for their declaration of war against GlaxoSmithKline over claims of heart attack and stroke from the use of Avandia.

This week, a Food and Drug Administration announcement sent me back to my presentation slides, which were provocatively titled “Pharmacovigilantism and Avandia.” Dr. Steven Nissen had published a meta-analysis in the New England Journal of Medicine in May 2007, and it had all the appearances of a contrived effort to embarrass GSK. See Steven E. Nissen, M.D., and Kathy Wolski, M.P.H., “Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes,” 356 New Engl. J. Med. 2457 (2007). A few weeks later, Dr. George Diamond published a thorough debunking of the Nissen meta-analysis, by showing that the statistically significant result in Nissen’s meta-analysis could be achieved only by choosing an inappropriate meta-analytic method. Any other choice resulted in a result that lacked statistical significance for the rate of heart attack among patients taking Avandia.

Litigation, of course, followed and the Rule 702 hearings and decision resulted in a serious abridgement of the scientific process. The federal MDL trial judge denied GSK’s motions to exclude plaintiffs’ causation witnesses in an opinion that has become a model for Rule 702 avoidance. In re Avandia Marketing, Sales Practices and Product Liability Litigation, 2011 WL 13576, *12 (E.D. Pa. 2011) (Rufe, J.). See “Learning to Embrace Flawed Evidence – The Avandia MDL’s Daubert Opinion” (Jan. 10, 2011)

This week, without much fanfare, the FDA announced that maybe the evidence supporting the claims that Avandia causes heart attacks is not so strong, after all. See “FDA Drug Safety Communication: FDA requires removal of some prescribing and dispensing restrictions for rosiglitazone-containing diabetes medicines” (Nov. 25, 2011). The Avandia MDL stands out as an expensive, negligent rush to judgment; a case more of phamacovigilantism than of pharmacovigilance.

Posted in Causation, Meta-analysis, Rule 702 | Comments Off on Pharmacovigilantism – Avandia Litigation

Bendectin, Diclegis & The Philosophy of Science

October 26th, 2013

In April of this year, the United States Food and Drug Administration (FDA) approved Diclegis, a combination of doxylamine succinate and pyridoxine hydrochloride for sale in the United States, for pregnant women experiencing nausea and vomiting. See FDA News Release, “FDA approves Diclegis for pregnant women experiencing nausea and vomiting,” (April 8, 2013). The return of this drug to the United States market was held up as a triumph of science over the will of the lawsuit industry. See Gideon Koren, “The Return to the USA of the Doxylamine-Pyridoxine Delayed Release Combination (Diclegis®) for Morning Sickness — A New Morning for American Women,” 20 J. Popul. Ther. Clin. Pharmacol. e161 (2013).

The sponsor of the drug, Duchesnay USA, wisely did not use the medication’s former name, Bendectin, which was the victim of a litigation industry jihad in the late 1970s through the mid-1990s. The plaintiffs’ lawyers’ war against Bendectin and its United States manufacturer is chronicled in two book-length accounts, and hundreds of articles. See Joseph Sanders, Bendectin on Trial: A Study of Mass Tort Litigation (Ann Arbor 1998); Michael D. Green, Bendectin and Birth Defects: The Challenges of Mass Toxic Substances Litigation (Philadelphia 1996).

As recently approved by the FDA, Declegis is categorized as “Pregnancy Category A,” which means that it is medication indicated for use in pregnant women. Most drugs are not tested in pregnant women in randomized clinical trials for obvious ethical and practical reasons. Perhaps one of the good things that came out of the Bendectin litigation wars was that Bendectin became one of the most intensely studied medications available for pregnant women. Another good thing was the achievement of evidence-based standards for expert witness opinion testimony in federal court. See David Bernstein, “Bendectin is Back” (April 9, 2013).

According to FDA regulations, Category A is defined:

“(1) Pregnancy category A. If adequate and well-controlled studies in pregnant women have failed to demonstrate a risk to the fetus in the first trimester of pregnancy (and there is no evidence of a risk in later trimesters), the labeling must state: ‘Pregnancy Category A. Studies in pregnant women have not shown that (name of drug) increases the risk of fetal abnormalities if administered during the first (second, third, or all) trimester(s) of pregnancy. If this drug is used during pregnancy, the possibility of fetal harm appears remote. Because studies cannot rule out the possibility of harm, however, (name of drug) should be used during pregnancy only if clearly needed.’ The labeling must also contain a description of the human studies. If animal reproduction studies are also available and they fail to demonstrate a risk to the fetus, the labeling must also state: ‘Reproduction studies have been performed in (kinds of animal(s)) at doses up to (x) times the human dose and have revealed no evidence of impaired fertility or harm to the fetus due to (name of drug).’ The labeling must also contain a description of available data on the effect of the drug on the later growth, development, and functional maturation of the child.

21 CFR § 201.57 (c)(9)(i)(A)(1) (April 2012).

A Litmus Test for Philosophy of Science?

The inability to discriminate between valid and invalid science should be a disqualifying characteristic in a putative philosopher of science, or a putative expert, for that matter. Professor Susan Haack, whose writings provide both insight and confusion on the role of science in the law, revealed her robust biases and prejudices in commenting upon the Bendectin litigation. These revelations should raise red flags about her objectivity in commenting on the legal process. See Susan Haack, “Irreconcilable Differences? The Troubled Marriage of Science and Law,” 72 Law & Contemporary Problems 1 (2009).

Haack’s paper on the marital discord was based upon her presentation at the Fourth Coronado Conference, organized by SKAPP (The Project on Scientific Knowledge and Public Policy), an ideological group dedicated to opening the courthouse doors to every quackacademic theory, and shadily funded by the litigation industry of plaintiffs’ lawyers from their left-over spoils from the silicone breast implant litigation. See SKAPP A LOT (April 30, 2010); “Haacking at the Truth – Part Two” (Oct. 31, 2010).

Haack provided examples of “marginal” science and witnesses who disturb her for biases and prejudices she perceives in these witnesses. Haack focuses upon Dr. Robert Brent, a toxicologist, who appears to her as Merrell Dow’s expert witness “always ready to testify that Bendectin does not cause birth defects.” Id. At 17. Haack presented no evidence or basis to suggest that Brent was wrong, and indeed, Brent published widely on his views of the subject. Multiple publications do not necessarily mean that Brent was right, but at least he was willing to subject himself to professional peer review, and post-publication, professional challenges. Still, Haack is distressed that Dr Robert Brent opines with “unwarranted certainty” that Bendectin does not cause birth defects, but she offers no suggestion or support that his certainty was or is misplaced.

In stark contrast, Haack expressed no discomfort with Bendectin plaintiffs’ expert witness, Dr Done, or with the facile ease with which he opined with scientific certainty that Bendectin causes birth defects. Here there really is a great deal of empirical evidence, and along with the FDA’s recent approval of Diclegis for use in pregnant women, the evidence has vindicated Dr. Brent’s views on the safety and efficacy of Bendectin/Diclegis. Dr. Done’s subjective appreciation of “flaws” in some clinical studies does not turn criticism into affirmative evidence in favor of the opinion that he so zealously, and overzealously, advocated in many Bendectin cases, for his own substantial pecuniary benefit. What is remarkable about Haack’s article is that she singles out Dr. Brent in the context of a discussion of “marginal” and “willing” testifying scientists, but she omits any mention of the plaintiffs’ cadre of ready, willing, and somewhat disreputable testifiers. Perhaps even more remarkable is that Haack overlooks that Dr. Done was essentially fired from his university for his entrepreneurial testimonial activities of dubious scientific worth, and that he probably lied about his credentials. See Michael Green, Bendectin and Birth Defects: The Challenges of Mass Toxic Substances Litigation 280 – 82 (Philadelphia 1996) (citing decisional law in which Done’s lack of veracity was judicially noted).

Of course, what is most remarkable about Haack’s infatuation with Dr. Alan Done and his mosaic theory is that the theory was the concoction of plaintiffs’ lawyer, Barry Nace, and that the theory leads to such a palpably incorrect result. Barry Nace was one of the lead plaintiffs’ counsel in the Bendectin litigation. Nace was also formerly President of the litigation industry’s principal lobbying organization, the American Trial Lawyers Association (now the AAJ). After the second Ninth Circuit Daubert decision, Nace declined to pursue one of his Bendectin cases, and his client sued him. Nace’s attorney moved for summary judgment in August 1998, on grounds that included the assertion that “courts soundly and uniformly reject the notion that Bendectin causes birth defects.” See David Bernstein, “A Day Late and a Dollar Short on Bendectin” (June 28, 2005).

This is the same Barry Nace lauded by Michael Green for having devised the notorious matrix theory of scientific evidence (a.k.a. the tsumish theory). Michael D. Green, “Pessimism About Milward,” 3 Wake Forest J. Law & Policy41, 62-63 (2013). Professor Haack sees Nace’s matix theory as the practical application of some of her theories. Susan Haack, “Irreconcilable Differences? The Troubled Marriage of Science and Law,” 72 Law & Contemporary Problems 1, 17 (2009); Susan Haack, “Proving Causation: The Holism of Warrant and the Atomism of Daubertm” 4 J. Health & Biomedical Law 273, 274-78 (2008). Haack’s embrace of the dubious Bendectin causal claims as supported by her matrix theory of causal inference raises the issue why we should credit a theory in the face of such a compelling counter example? As Professor Ronald Allen put the matter, before Bendectin was reintroduced into the United States market this year:

“Given the weight of evidence in favor of Bendectin’s safety, it seems peculiar to argue for mosaic evidence from a case in which it would have plainly been misleading.”

Ronald J. Allen and Esfand Nafisi, “Daubert and its Discontents,” 76 Brooklyn L. Rev. 132, 148 (2010). Peculiar indeed. Professor Allen’s point is important for its wide-ranging implications. Methodologies that yield false-positive results are unreliable. Perhaps a methodology can be saved if we could quantify that a given methodology rarely yields such false results, but the matrix theory of Barry Nace and his expert witnesses seems so vague and insubstantial that no one, in all likelihood, could frame a test for the generalized approach. Expert witnesses perhaps should be judged by their track record over time, as well. See David Kaye, “The Experts in Daubert.”[1]

[1] Dr. Alan Done (pediatrician, pharmacology, toxicology); Dr. Jay Glasser (biostatistician, epidemiologist); Dr. Adrian Gross (veterinarian); Dr. Stuart Newman (developmental biologist); Dr. Wayne Snodgrass (Assoc. Professor of Pediatrics, Pharmacology, and Toxicology); Dr. Shanna Swan (epidemiologist); Dr. Johannes Thiersch (pathologist and pharmacologist); Dr. John Palmer (Professor of pharmacology).

Posted in Causation, Expert Witnesses, Rule 702 | Comments Off on Bendectin, Diclegis & The Philosophy of Science

Christopher Bryson and the Problem of Political Science

October 12th, 2013

Fluoridation of water has long been a “political science” issue, with radical libertarians and anarchists viewing fluoridation as the high-water mark of state paternalism. The motive to misstate and misrepresent the science may at times be obvious, but individual statements, standing alone, may be difficult to judge.

Fluorine chemistry and toxicology are sufficiently advanced that misrepresentations should be easy to detect. Fluorine is a halogen; the lightest in the series. As a gas, fluorine is extremely reactive and toxic, as are other halogen gases, such as chlorine. Fluorine gas was used for uranium enrichment in the Manhattan project, and project scientists conducted research on fluorine toxicity to help them set exposure limits in a new manufacturing process.

As devotees of Breaking Bad no doubt have learned, hydrogen fluoride is extremely toxic and corrosive. Other halogen-based acids are, of course, toxic and corrosive, such as hydrogen chloride.

Compounds of fluorine are generically fluorides, and the properties of the salts and compounds varies considerably with the cation and the chemical structures involved. Many modern medications, such as atorvastatin and fluoxetine contain fluorine in their chemical structures. The toxicology of the fluorine compounds must, therefore, take into account the variability of structure and function of fluorine. Toxicity of fluorine gas or of hydrogen fluoride cannot be “extrapolated” to a simple sodium salt, and more than the toxicity of chlorine gas can be imply the toxicity or ordinary table salt, sodium chloride. The allergenicity of a compound such as potassium aluminumtetrafluoride cannot be the basis for asserting the immunogenicity of a simple alkali salt.

Recently, I came across a YouTube video of a journalist, Christopher Bryson, holding forth on his perception of a vast conspiracy to poison people by the fluoridation of drinking water. Bryson’s passion and selectivity in making his case resembles the deep flaws of our tort system, which allows lawyers and expert witnesses to overwhelm judges and juries with emotion, selectivity, and overstatement. Bryson refers to all fluorine chemicals, whether the elemental gas, the acid, or the many complex and variable salts as “fluoride.” Occupational and environmental exposures to hydrogen fluoride are equated with micromolar levels of sodium fluoride in drinking water. Never once does he actually quantitate the exposures he labels as “toxic.” Largely, Bryson proceeds by ad hominems, ad nauseam. If scientists have industry connections, they are bad, and their science is corrupt. If a scientist has ever done something productive (e.g., George L. Waldbott), and he opines that water fluoridation is bad, then that scientist must be correct. Apparently, Bryson has never heard of Linus Pauling and his Vitamin C fiasco. See K. Frank Austen, M. Dworetzky, Richard S. Farr, G.B. Logan, S. Malkiel, E. Middleton Jr., M.M. Miller, Roy Patterson, C.E. Reed, S.C. Siegel, and P.P. Van Arsdel Jr., “A statement on the question of allergy to fluoride as used in the fluoridation of community water supplies,” 47 J. Allergy & Clinical Immunology 347 (1971) (“no”).

Bryson makes for an interesting case study in hysteria. He is also very much a public example of the tone and substance of many of the plaintiffs’ theories that clog the civil dockets of our court system. Bryson’s passion and intensity — heat without illumination — are reminiscent of the courtroom antics in many a so-called “toxic tort” case. Bryson’s video is thus a good place to start to try to understand science in the courtroom, and the need for strong gatekeeping. The potential for inflammatory advocacy, distortion, and misrepresentation have always been part of legal proceedings, but when it comes to advocacy about claims that turn on “scientific” evidence, there is a difference. Juries in common law cases, in 1789, were not confronted with the abuses of the sort that Bryson so well exemplifies.

The Bryson video led me to look at Bryson’s book, The Fluoride Deception. The book starts with “Notes on Terminology,” which warns that

“THE TERMS fluorine and fluoride should not be confused in a book about chemical toxicity.”

* * *

“In these pages I’ve tried to be clear when I’m referring to the element fluorine or to a compound, a fluoride. And because different fluoride compounds often have unique toxicities, where relevant or possible, I have also given the compound’s specific name.”

So far so good, but then Bryson, having baited, switches:

“Mostly, however, for simplicity’s sake, I have followed convention and used the shorthand fluoride when referring to the element and its multiple manifestations, a procedure approved and used by the U.S. National Academy of Sciences.”

Christopher Bryson, The Fluoride Deception at xi (2004).

Fluoride deception indeed Mr. Bryson. The cited source for the indiscriminate use of fluoride makes clear that it uses “fluoride” as a general term when differentiation is not necessary for its discussion. It is not, however, a basis for conflating or confusing the toxicities of fluorine species or doses. National Research Council, Biological Effects of Atmospheric Pollutants: Fluorides 3 (1971). Bryson provides an apt example of how science communication works in politicized contexts, such as the courtroom or the legislature.

Posted in Causation, Rule 702, Scientific Evidence | Comments Off on Christopher Bryson and the Problem of Political Science