TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

Zoloft MDL Excludes Proffered Testimony of Anick Bérard, Ph.D.

June 27th, 2014

Anick Bérard is a Canadian perinatal epidemiologist in the Université de Montréal.  Bérard was named by plaintiffs’ counsel in the Zoloft MDL to offer an opinion that selective serotonin reuptake inhibitor (SSRI) antidepressants as a class, and Zoloft (sertraline) specifically, cause a wide range of birth defects. Bérard previously testified against GSK about her claim that paroxetine, another SSRI antidepressant is a teratogen.

Pfizer challenged Bérard’s proffered testimony under Federal Rules of Evidence 104(a), 702, 703, and 403.  Today, the Zoloft MDL transferee court handed down its decision to exclude Dr. Bérard’s testimony at the time of trial.  In re Zoloft (Sertraline Hydrochloride) Prods. Liab. Litig., MDL 2342, Document 979 (June 27, 2014).  The MDL court acknowledged the need to consider the selectivity (“cherry picking”) of studies upon which Dr. Bérard relied, as well as her failure to consider multiple comparisons, ascertainment bias, confounding by indication, and lack of replication of specific findings across the different SSRI medications, and across studies. Interestingly, the MDL court recognized that Dr. Bérard’s critique of studies as “underpowered” was undone by her failure to consider available meta-analyses or to conduct one of her own. The MDL court seemed especially impressed by Dr. Bérard’s having published several papers that rejected a class effect of teratogenicity for all SSRIs, as recently as 2012, while failing to identify anything that was published subsequently that could explain her dramatic change in opinion for litigation.

Posted in Causation, Expert Witnesses, Meta-analysis, Rule 702, Scientific Misconduct | Comments Off on Zoloft MDL Excludes Proffered Testimony of Anick Bérard, Ph.D.

Substituting Risk for Specific Causation

June 15th, 2014

Specious, Speculative, Spurious, and Sophistical

Some legal writers assert that all evidence is ultimately “probable,” but that assertion appears to be true only to the extent that the evidentiary support for any claim can be mapped on scale from 0 to 1, much as probability is.  Probability thus finds its way into discussions of burdens of persuasion as requiring the claim to be shown more probably than not, and expert witness certitude as requiring “reasonable degree of scientific probability.”

There is a contrary emphasis in the law on “actual truth,” which is different from “mere probability.”  The rejection of probabilism can be seen in some civil cases, in which courts have emphasized the need for individualistic data and conclusions, beyond generalizations that might be made about groups that clearly encompass the individual at issue. For example, the Supreme Court has held that charging more for funding a woman’s pension than a man’s is discriminatory because not all women will outlive all men, or the men’s average life expectancy. City of Los Angeles Dep’t of Water and Power v. Manhart, 435 U.S. 702, 708 (1978) (“Even a true generalization about a class is an  insufficient reason for disqualifying an individual to whom the generalization does not apply.”). See also El v. Southeastern Pennsylvania Transportation Authority, 479 F.3d 232, 237 n.6 (3d Cir. 2007) (“The burden of persuasion … is the obligation to convince the factfinder at trial that a litigant’s necessary propositions of fact are indeed true.”).

Specific causation is the soft underbelly of the toxic tort world, in large measure because courts know that risk is not specific causation. In the context of risk of disease, which is usually based upon a probabilistic group assessment, courts occasionally distinguish between risk and specific causation. See “Probabilism Case Law” (Jan. 28, 2013) (collecting cases for and against probabilism).

In In re Fibreboard Corp., 893 F. 2d 706, 711-12 (5th Cir. 1990), the court rejected a class action approach to litigating asbestos personal injury claims because risk could not substitute for findings of individual causation:

“That procedure cannot focus upon such issues as individual causation, but ultimately must accept general causation as sufficient, contrary to Texas law. It is evident that these statistical estimates deal only with general causation, for ‘population-based probability estimates do not speak to a probability of causation in any one case; the estimate of relative risk is a property of the studied population, not of an individual’s case.’ This type of procedure does not allow proof that a particular defendant’s asbestos ‘really’ caused a particular plaintiff’s disease; the only ‘fact’ that can be proved is that in most cases the defendant’s asbestos would have been the cause.”

Id. at 711-12 (citing Steven Gold, “Causation in Toxic Torts: Burdens of Proof, Standards of Persuasion, and Statistical Evidence,” 96 Yale L.J. 376, 384, 390 (1986). See also Guinn v. AstraZeneca Pharms., 602 F.3d 1245, 1255 (11th Cir. 2010) (“An expert, however, cannot merely conclude that all risk factors for a disease are substantial contributing factors in its development. ‘The fact that exposure to [a substance] may be a risk factor for [a disease] does not make it an actual cause simply because [the disease] developed.’”) (internal citation omitted).

Specific causation is the soft underbelly of the toxic tort world, in large measure because courts know that risk is not specific causation. The analytical care of the Guinn case and others is often abandoned when it will stand in the way of compensation. The conflation of risk and (specific) causation is prevalent precisely because in many cases there is no scientific or medical way to discern what antecedent risks actually played a role in causing an individual’s disease.  Opinions about specific causation are thus frequently devoid of factual or logical support, and are propped up solely by hand waving about differential etiology and inference to the best explanation.

In the scientific world, most authors recognize that risk, even if real and above baseline, regardless of magnitude, does not support causal attribution in a specific case.[1]  Sir Richard Doll, who did so much to advance the world’s understanding of asbestosis as a cause of lung cancer, issued a caveat about the limits of specific causation inference. Richard Doll, “Proof of Causality: Deduction from Epidemiological Observation,” 45 Perspectives in Biology & Medicine 499, 500 (2002) (“That asbestos is a cause of lung cancer in this practical sense is incontrovertible, but we can never say that asbestos was responsible for the production of the disease in a particular patient, as there are many other etiologically significant agents to which the individual may have been exposed, and we can speak only of the extent to which the risk of the disease was increased by the extent of his or her exposure.”)

Similarly, Kenneth Rothman, a leading voice among epidemiologists, cautioned against conflating epidemiologic inferences about groups with inferences about causes in individuals. Kenneth Rothman, Epidemiology: An Introduction 44 (Oxford 2002) (“An elementary but essential principal that epidemiologists must keep in mind is that a person may be exposed to an agent and then develop disease without there being any causal connection between exposure and disease.”  … “In a courtroom, experts are asked to opine whether the disease of a given patient has been caused by a specific exposure.  This approach of assigning causation in a single person is radically different from the epidemiologic approach, which does not attempt to attribute causation in any individual instance.  Rather, the epidemiologic approach is to evaluate the proposition that the exposure is a cause of the disease in a theoretical sense, rather than in a specific person.”) (emphasis added).

The late David Freedman, who was the co-author of the chapters on statistics in all three editions of the Reference Manual on Scientific Evidence, was also a naysayer when it came to transmuting risk into cause:

“The scientific connection between specific causation and a relative risk of two is doubtful. *** Epidemiologic data cannot determine the probability of causation in any meaningful way because of individual differences.”

David Freedman & Philip Stark, “The Swine Flu Vaccine and Guillaine-Barré Syndrome:  A Case Study in Relative Risk and Specific Causation,” 64 Law & Contemporary Problems 49, 61 (2001) (arguing that proof of causation in a specific case, even starting with a relative risk of four, was “unconvincing”; citing Manko v. United States, 636 F. Supp. 1419, 1437 (W.D. Mo. 1986) (noting relative risk of 3.89–3.92 for GBS from swine-flu vaccine), aff’d in part, 830 F.2d 831 (8th Cir. 1987)).

Graham Colditz, who testified for plaintiffs in the hormone therapy litigation, similarly has taught that an increased risk of disease cannot be translated into the “but-for” standard of causation.  Graham A. Colditz, “From epidemiology to cancer prevention: implications for the 21st Century,” 18 Cancer Causes Control 117, 118 (2007) (“Knowledge that a factor is associated with increased risk of disease does not translate into the premise that a case of disease will be prevented if a specific individual eliminates exposure to that risk factor. Disease pathogenesis at the individual level is extremely complex.”)

Another epidemiologist, who wrote the chapter in the Federal Judicial Center’s Reference Manual on Scientific Evidence, on epidemiology, put the matter thus:

“However, the use of data from epidemiologic studies is not without its problems. Epidemiology answers questions about groups, whereas the court often requires information about individuals.”

Leon Gordis, Epidemiology 362 (5th ed. 2014) (emphasis in original).

=========================================================

In New Jersey, an expert witness’s opinion that lacks a factual foundation is termed a “net opinion.” Polzo v. County of Essex, 196 N.J. 569, 583 (2008) (explaining New Jersey law’s prohibition against “net opinions” and “speculative testimony”). Under federal law, Rule 702, such an opinion is simply called inadmissible.

Here is an interesting example of a “net opinion” from an expert witness, in the field of epidemiology, who has testified in many judicial proceedings:

 

                                                                                          November 12, 2008

George T. Brugess, Esq.
Hoey & Farina, Attorneys at Law
542 South Dearborn Street, Suite 200
Chicago, IL 60605

Ref: Oscar Brooks v. Ingram Barge and Jantran Inc.

* * * *

Because [the claimant] was employed 28 years, he falls into the greater than 20 years railroad employment category (see Table 3 of Garshick’s 2004 paper) which shows a significant risk for lung cancer that ranges from 1.24 to 1.50. This means that his diesel exposure was a significant factor in his contracting lung cancer. His extensive smoking was also a factor in his lung cancer, and diesel exposure combined with smoking is an explanation for the relatively early age, 61 years old, of his diagnosis.

Now assuming that diesel exposure truly causes lung cancer, what was the basis for this witness (David F. Goldsmith, PhD) to opine that diesel exposure was a “significant factor” in the claimant’s developing lung cancer?  None really.  There was no basis in the report, or in the scientific data, to transmute an exposure that yielded a risk ratio of 1.24 to 1.50 for lung cancer, in a similarly exposed population to diesel emissions, into a “significant factor.” The claimant’s cancer may have arisen from background, baseline risk.  The cancer may have arisen from the risk due to smoking, which would have been on the order of a 2,000% increase, or so.  The cancer may have arisen from the claimed carcinogenicity of diesel emissions, on the order of 25 to 50%, which was rather insubstantial compared with his smoking risk.  Potentially, the cancer arose from a combination of the risk from both diesel emissions and tobacco smoking. In the population of men who looked like Mr. Oscar Brooks, by far, the biggest reduction in incidence would be achieved by removing tobacco smoking.

There were no biomarkers that identified the claimant’s lung cancer as having been caused by diesel emissions.  The expert witness’s opinion was nothing more than an ipse dixit that equated a risk, and a rather small risk, with specific causation.  Notice how a 24% increased risk from diesel emissions was a “significant factor,” but the claimant’s smoking history was merely “a factor.”

Goldsmith’s report on specific causation was a net opinion that exemplifies what is wrong with a legal system that encourages and condones baseless expert witness testimony. In Agent Orange, Judge Weinstein pointed out that the traditional judicial antipathy to probabilism would mean no recovery in many chemical and medicinal exposure cases.  If the courts lowered their scruples to permit recovery on a naked statistical inference of greater than 50%, from relative risks greater than two, some cases might remain viable (but alas not the Agent Orange case itself). Judge Weinstein was, no doubt, put off by the ability of defendants, such as tobacco companies, to avoid liability because plaintiffs would never have more than evidence of risk.  In the face of relative risks often in excess of 30, with attributable risks in excess of 95%, this outcome was disturbing.

Judge Weinstein’s compromise was a pragmatic solution to the problem of adjudicating specific causation on the basis of risk evidence. Although as noted above, many scientists rejected any use of risk to support specific causation inferences, some scientists agreed with this practical solution.  Ironically, David Goldsmith, the author of the report in the Oscar Brooks case, supra, was one such writer who had embraced the relative risk cut off:

“A relative risk greater than 2.0 produces an attributable risk (sometimes called attributable risk percent10) or an attributable fraction that exceeds 50%.  An attributable risk greater than 50% also means that ‘it is more likely than not’, or, in other words, there is a greater than 50% probability that the exposure to the risk factor is associated with disease.”

David F. Goldsmith & Susan G. Rose, “Establishing Causation with Epidemiology,” in Tee L. Guidotti & Susan G. Rose, eds., Science on the Witness Stand:  Evaluating Scientific Evidence in Law, Adjudication, and Policy 57, 60 (OEM Press 2001).

In the Brooks case, Goldsmith did not have an increased risk even close to 2.0. The litigation industry ultimately would not accept anything other than full compensation for attributable risks greater than 0%.

[1] See, e.g., Sander Greenland, “Relation of the Probability of Causation to Relative Risk and Doubling Dose:  A Methodologic Error that Has Become a Social Problem,” 89 Am. J. Pub. Health 1166, 1168 (1999)(“[a]ll epidemiologic measures (such as rate ratios and rate fractions) reflect only the net impact of exposure on a population”); Joseph V. Rodricks & Susan H. Rieth, “Toxicological Risk Assessment in the Courtroom:  Are Available Methodologies Suitable for Evaluating Toxic Tort and Product Liability Claims?” 27 Regulatory Toxicol. & Pharmacol. 21, 24-25 (1998)(noting that a population risk applies to individuals only if all persons within the population are the same with respect to the influence of the risk on outcome); G. Friedman, Primer of Epidemiology 2 (2d ed. 1980)(epidemiologic studies address causes of disease in populations, not causation in individuals)

 

Posted in Causation, Expert Witnesses, Risk and Risk Factor, Rule 702 | Comments Off on Substituting Risk for Specific Causation

Hysterical Histortions

June 12th, 2014

Ramses Delafontaine is a young, aspiring historian. In his graduate thesis, Historicizing the Forensification of History: A Study of Historians as Expert Witnesses in Tobacco Litigation in the United States of America (Univ. Ghent 2013), discusses my commentary on Marxist historians, David Rosner and Gerald Markowitz, and suggests that I claim that lawyers without historical training or experience can do the job of historians.  Id. at 98-100.

Given their training and skills in documenting and recounting narratives, lawyers do, indeed, often do the job of historians, and they often do it very well. Of course, lawyers are often guided, inspired, and assisted by professional historians. Sometimes that guidance is necessary. Lawyers’ narratives, unlike historians’, are also subject to judicial control in the form of evidentiary rules about speculation, relevance, reliability, authentication, and trustworthiness.

Regurgitating Historical Evidence

American courts have thus appropriately limited the use of expert witnesses to present historical narratives in judicial proceedings.  See In re Fosamax Prods. Liab. Litig., 645 F. Supp. 2d 164, 192 (S.D.N.Y. 2009) (‘‘[A]n expert cannot be presented to the jury solely for the purpose of constructing a factual narrative based upon record evidence.’’) (internal citations omitted).[1] In Fosamax, Judge Keenan excluded Dr. Susan Parisian’s proffered narrative account of the development and approval of a bisphosphonate medication for osteoporosis in a case involving a claim of osteonecrosis of the jaw (“phossy jaw”).  Judge Keenan detailed the problems that arise from using expert witnesses to present partisan historical narratives:

“In detailing the factual basis for her opinions, Dr. Parisian’s report presents a narrative of select regulatory events through the summary or selective quotation from internal Merck documents, regulatory filings, and the deposition testimony of Merck employees.

The Court agrees with Merck that, to the extent such evidence is admissible, it should be presented to the jury directly. Dr. Parisian’s commentary on any documents and exhibits in evidence will be limited to explaining the regulatory context in which they were  created, defining any complex or specialized terminology, or drawing inferences that would not be apparent without the benefit of experience or specialized knowledge. She will not be permitted to merely read, selectively quote from, or ‘regurgitate’ the evidence.”

Id.[2]

Ramses Delafontaine is wrong, however, to opine that my rants against Rosner and Markowitz suggest that I have ruled out any role for historians in litigation.   Lisa K. Walker is an historian, who trained at the University of California, Berkeley.  In the welding fume litigation, plaintiffs’ counsel weaved a complex narrative of conspiracy allegations, based in large measure upon the absence of evidence. At the request of Metropolitan Life Insurance Company, Professor Walker researched the dates of publication for various editions of a booklet, Health Protection in Welding, which formed the basis for the plaintiffs’ speculations. Walker found and analyzed eight separate editions, and dated each by internal and external references.  Based upon her research, Walker submitted a declaration, which ultimately was immensely helpful to the resolution of the issues. See In re Welding Rod Prods. Liab. Litig., Case No. 1:03-CV-17000 (MDL Docket No. 1535) (N.D.Ohio Nov. 24, 2004; In re Welding Fume Prods. Liab. Litig., 2007 WL 1087605 (N.D. Ohio April 9, 2007)  (O’Malley, J.) (granting summary judgment in favor of Metropolitan Life Insurance Company).

Although Metropolitan Life should not have had to disprove the allegations, Walker’s research showed the plaintiffs’ historical speculation to be clearly wrong. Sometimes historians can and do contribute valuably to the resolution of legal issues, but the issues are usually more modest than the ones that social and labor historians want to see resolved in favor of their pet theories.

Delafontaine also has a website on the role of historians as expert witnesses in United States tobacco cases.  One of his theses is that “historians who have been involved as expert witnesses for the tobacco industry have been in it for the money and have sold their professional integrity as a historian and an academic.” Delafontaine’s approach is a bit one-sided in that he sees only defendants’ expert witnesses as being “in it for the money,” despite the substantial billings of plaintiffs’ expert witnesses, and their ideological biases. Somehow Delafontaine has missed how even the feel-good advocacy of anti-tobacco activists occasionally outruns its evidentiary headlights.  See, e.g., Michael Siegel, “Is the tobacco control movement misrepresenting the acute cardiovascular health effects of secondhand smoke exposure? An analysis of the scientific evidence and commentary on the implications for tobacco control and public health practice,” 4 Epidem. Persp. & Innov. 12 (2007).

[1] See also In re Prempro Prods. Liab. Litig., 554 F.Supp. 2d 871, 880, 886 (E.D.Ark.2008) (overturning a punitive damages award based on Dr. Parisian’s testimony in part because she ‘‘did not explain the documents, provide summaries, or tie them in to her proposed regulatory testimony’’ and ‘‘did not provide analysis, opinion, or expertise’’); Highland Capital Management, L.P. v. Schneider, 379 F.Supp. 2d 461, 469 (S.D.N.Y.2005)(‘‘[A]n expert cannot be presented to the jury solely for the purpose of constructing a factual narrative based upon record evidence.’’); In re Rezulin Products Liab. Litig., 309 F.Supp. 2d 531, 546 (S.D.N.Y.2004) (rejecting portion of expert report presenting history of Rezulin for no purpose but to ‘‘provid[e] an historical commentary of what happened,’’ along with subjective assessments of intent, motives, and states of mind); In re Diet Drugs Prods. Liab. Litig., MDL No. 1203, 2000 WL 876900, at *9 (E.D.Pa. June 20, 2000) (same); Taylor v. Evans, 1997 WL 154010, at *2 (S.D.N.Y. Apr.1, 1997) (rejecting portions of expert report on the ground that the testimony consisted of ‘‘a narrative of the case which a lay juror is equally capable of constructing’’).

[2] Dr. Parisian appears to be a serial narrative abuser, who has been repeatedly but not consistently excluded. See Scheinberg v. Merck & Co. 924 F.Supp. 2d 477, 497 (S.D.N.Y. 2013); Pritchett v. I-Flow Corp., 2012 WL 1059948, at *7 (D. Colo. Mar. 28, 2012); Miller v. Stryker Instruments, 2012 WL 1718825, at *10-12 (D. Ariz. Mar. 29, 2012) (excluding narrative testimony); Kaufman v. Pfizer Pharms., Inc., 2011 WL 7659333, at *6-10 (S.D. Fla. Aug. 4, 2011), reh’g denied, 2011 WL 10501233 (S.D. Fla. Aug. 10, 2011)(narrative testimony); Hines v. Wyeth, 2011 WL 2680842, at *7 (S.D.W. Va. July 8, 2011), reh’g granted in part, 2011 WL 2730908, at *2 (S.D.W. Va. July 13, 2011); In re Heparin Prods. Liab. Litig., 2011 WL 1059660, at *8 (N.D. Ohio March 21, 2011); Lopez v. I-Flow Inc., 2011 WL 1897548, at *9-10 (D. Ariz. Jan. 26, 2011) (narrative testimony); In re Trasylol Prods. Liab. Litig., 709 F.Supp.2d 1323, 1351 (S.D. Fla. 2010)(tendentious narrative testimony) (“Plainly stated, Dr. Parisian is an advocate, presented with the trappings of an expert but with no expectation or intention of abiding by the opinion constraints of Rule 702.”), reh’g denied, 2010 WL 2541892 (S.D. Fla. June 22, 2010); In re Gadolinium-Based Contrast Agents Prods. Liab. Litig., 2010 WL 1796334, at *13 (N.D. Ohio May 4, 2010); Bessemer v. Novartis Pharms. Corp., 2010 WL 2300222 (N.J. Super. Law Div. April 30, 2010); In re Prempro Prods. Liab. Litig., 554 F. Supp. 2d 871, 879-87 (E.D. Ark. 2008)(reversing judgment on grounds of erroneous admission of narrative testimony), aff’d  in relevant part, 586 F.3d 547, 571 (8th Cir. 2009). Occasionally, Dr. Parisian slips through the gate.  See, e.g., Block v. Woo Young Medical Co., 937 F.Supp.2d 1028, 1044-47 (2013)

Posted in Historian Testimony, Rule 702 | Comments Off on Hysterical Histortions

Goodman v Viljoen – Statistical Fallacies from Both Sides

June 8th, 2014

There was a deep irony to the Goodman[1] case.  If a drug company, in 1995, marketed antenatal corticosteroid (ACS) for the prevention of cerebral palsy (CP) in the United States, the government might well have prosecuted the company for misbranding.  The company might also be subject to a False Claims Act case as well. No clinical trial had found ACS efficacious for the prevention of CP at the significance level typically required by the FDA; no meta-analysis had found ACS statistically significantly better than placebo for this purpose.  In the Goodman case, however, failure to order a full course of ACS was malpractice with respect to the claimed causation of CP in the Goodman twins.

The Goodman case also occasioned a well-worn debate over the difference between scientific and legal evidence, inference, and standards of “proof.” The plaintiffs’ case rested upon a Cochrane review of ACS with respect to various outcomes. For CP, the Cochrane meta-analyzed only clinical trial data, and reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”[2]

The defendant, Dr. Viljoen, appeared to argue that the Cochrane meta-analysis must be disregarded because it did not provide a showing of efficacy for ACS in preventing CP, at a significance probability less than 5 percent.  Here is the trial court’s characterization of Dr. Viljoen’s argument:

“[192] The argument that the Cochrane data concerning the effects of ACS on CP must be ignored because it fails to reach statistical significance rests on the flawed premise that legal causation requires the same standard of proof as medical/scientific causation. This is of course not the case; the two standards are in fact quite different. The law is clear that scientific certainty is not required to prove causation to the legal standard of proof on a balance of probabilities (See: Snell v. Farrell, [1990] 2 S.C.R. 311, at para. 34). Accordingly, the defendant’s argument in this regard must fail and for the purposes of this court, I accept the finding of the Cochrane analysis that ACS reduces the instance [sic] of CP by 40%.”

“Disregard” seems extreme for a meta-analysis that showed a 40% reduction in risk of a serious central nervous system disorder, with p = 0.065.  Perhaps Dr. Viljoen might have tempered his challenge some by arguing that the Cochrane analysis was insufficient.  One problem with Dr. Viljoen’s strident argument about statistical significance was that it overshadowed the more difficult, qualitative arguments about threats to validity in the Cochrane finding from loss to follow up in the aggregated trial data. These threats were probably stronger arguments against accepting the Cochrane “trend” as a causal conclusion. Indeed, the validity and the individual studies and the meta-analyses, along with questions about the accuracy of data, were not reflected in Bayesian analysis.

Another problem is that Dr. Viljoen’s strident assertion that p < 0.05 was absolutely necessary fed plaintiffs’ argument that the defendant was attempting to change the burden of proof for plaintiffs from greater than 50% to 95% or greater.  Given the defendant’s position, great care was required to prevent the trial court from committing the transposition fallacy.

Justice Walters rejected the suggestion that a meta-analysis with a p-value of 6.5% should be disregarded, but the court’s discussion skirts the question whether and how the Cochrane data can be sufficient to support a conclusion of ACS efficacy. Aside from citing a legal case, however, Justice Walters provided no basis for suggesting that the scientific standard of proof was different from the legal standard. From the trial court’s opinion, the parties or their expert witnesses appeared to conflate “confidence,” a technical term when used to describe intervals or random error around sample statistics, with “level of certainty” in the obtained result.

Justice Walters is certainly not the first judge to fall prey to the fallacious argument that the scientific burden of proof is 95%.[3]  The 95% is, of course, the coefficient of confidence for the confidence interval that is based upon a p-value of 5%. No other explanation for why 95% is a “scientific” standard of proof was offered in Goodman; nor is it likely that anyone could point to an authoritative source for the claim that scientists actually adjudge facts and theories by this 95 percent probability level.

Justice Walters’ confusion was led by the transposition fallacy, which confuses posterior and significance probabilities.  Here is a sampling from Her Honor’s opinion, first from Dr. Jon Barrett, one of the plaintiffs’ expert witnesses, an obstetrician and fetal maternal medicine specialist at Sunnybrook Hospital, in Toronto, Ontario:

“[85] Dr. Barrett’s opinion was not undermined during his lengthy cross-examination. He acknowledged that the scientific standard demands 95% certainty. He is, however, prepared to accept a lower degree of certainty. To him, 85 % is not merely a chance outcome.

                                                                                        * * *

[87] He acknowledged that scientific evidence in support of the use of corticosteroids has never shown statistical significance with respect to CP. However, he explained it is very close at 93.5%. He cautioned that if you use a black and white outlook and ignore the obvious trends, you will falsely come to the conclusion that there is no effect.”

Dr. Jon (Yoseph) Barrett is a well-respected physician, who specializes in high-risk pregnancies, but his characterization of a black-white outlook on significance testing as leading to a false conclusion of no effect was statistically doubtful.[4]  Dr. Barrett may have to make divinely inspired choices in surgery, but in a courtroom, expert witnesses are permitted to say that they just do not know. Failure to achieve statistical significance, with p < 0.05, does support a conclusion that there is no effect.

Professor Andrew Willan was plaintiffs’ testifying expert witness on statistics.  Here is how Justice Walters summarized Willan’s testimony:

“[125] Dr. Willan described different statistical approaches and in particular, the frequentist or classical approach and the Bayesian approach which differ in their respective definitions of probability. Simply, the classical approach allows you to test the hypothesis that there is no difference between the treatment and a placebo. Assuming that there is no difference, allows one to make statements about the probability that the results are not due to chance alone.

To reach statistical significance, a standard of 95% is required. A new treatment will not be adopted into practice unless there is less than a 5% chance that the results are due to chance alone (rather than due to true treatment effect).

[127] * * * The P value represents the frequentist term of probability. For the CP analysis [from the Cochrane meta-analysis], the P value is 0.065. From a statistical perspective, that means that there is a 6.5% chance that the differences that are being observed between the treatment arm versus the non-treatment arm are due to chance rather than the treatment, or conversely, a 93.5% chance that they are not.”

Justice Walters did not provide transcript references for these statements, but they are clear examples of the transposition fallacy. The court’s summary may have been unfair to Professor Willan, who seems to have taken care to avoid the transposition fallacy in his testimony:

“And I just want to draw your attention to the thing in parenthesis where it says, “P = 0.065.” So, basically that is the probability of observing data this extremely, this much in favor of ACS given, if, if in fact the no [sic, null] hypothesis was true. So, if, if the no hypothesis was true, that is there was no difference, then the probability of observing this data is only 6.5 percent.”

Notes of Testimony of Andrew Willan at 26 (April , 2010). In this quote, Professor Willan might have been more careful to point out that the significance probability of 6.5%  is a cumulative probability by describing the data observed “this extremely” and more. Nevertheless, Willan certainly made clear that the probability measure was based upon assuming the correctness of the null hypothesis. The trial court, alas, erred in stating the relevant statistical concepts.

And then there was the bizarre description by Justice Walters, of the Cochrane data, as embodying a near-uniform distribution represented by the Cochrane data:

“[190] * * * The Cochrane analysis found that ACS reduced the risk of CP (in its entirety) by 40%, 93.5% of the time.”

The trial court did not give the basis for this erroneous description of the Cochrane ACS/CP data.[5] To be sure, if the Cochrane result were true, then 40% reduction might be the expected value for all trials, but it would be a remarkable occurrence for 93.5% of the trials to obtain the same risk ratio as the one observed in the meta-analysis.

The defendant’s expert witness on statistical issues, Prof. Robert Platt, similarly testified that the significance probability reported by the Cochrane was dependent upon an assumption of the null hypothesis of no association:

“What statistical significance tells us, and I mentioned at the beginning that it refers to the probability of a chance finding could occur under the null-hypothesis of no effect. Essentially, it provides evidence in favour of there being an effect.  It doesn’t tell us anything about the magnitude of that effect.”

Notes of Testimony of Robert Platt at 11 (April 19, 2010)

Perhaps part of the confusion resulted from Prof. Willan’s sponsored Bayesian analysis, which led him to opine that the Cochrane data permitted him to state that there was a 91 to 97 percent probability of an effect, which might have appeared to the trial court to be saying the same thing as interpretation of the Cochrane’s p-value of 6.5%.  Indeed, Justice Walters may have had some assistance in this confusion from the defense statistical expert witness, Prof. Platt, who testified:

“From the inference perspective the p-value of 0.065 that we observe in the Cochrane review versus a 91 to 97 percent probability that there is an effect, those amount to the same thing.”

Notes of Testimony of Robert Platt at 50 (April 19, 2010).  Now the complement of the p-value, 93.5%, may have fallen within the range of posterior probabilities asserted by Professor Willan, but these probabilities are decidedly not the same thing.

Perhaps Prof. Platt was referring only to the numerical equivalence, but his language, “the same thing,” certainly could have bred misunderstanding.  The defense apparently attacked the reliability of the Bayesian analysis before trial, only to abandon the challenge by the time of trial.  At trial, defense expert witness Prof. Platt testified that he did not challenge Willan’s Bayesian analysis, or the computation of posterior probabilities.  Platt’s acquiescence in Willan’s Bayesian analysis is unfortunate because the parties never developed testimony exactly as to how Willan arrived at his posterior probabilities, and especially as to what prior probability he employed.

Professor Platt went on to qualify his understanding of Willan’s Bayesian analysis as providing a posterior probability that there is an effect, or in other words, that the “effect size” is greater than 1.0.  At trial, the parties spent a good deal of time showing that the Cochrane risk ratio of 0.6 represented the decreased risk for CP of administering a full course of ACS, and that this statistic could be presented as an increased CP risk ratio of 1.7, for not having administered a full course of ACS.  Platt and Willan appeared to agree that the posterior probability described the cumulative posterior probabilities for increased risks above 1.0.

“[T]he 91% is a probability that the effect is greater than 1.0, not that it is 1.7 relative risk.”

Notes of Testimony of Robert Platt at 51 (April 19, 2010); see also Notes of Testimony of Andrew Willan at 34 (April 9, 2010) (concluding that ACS reduces risk of CP, with a probability of 91 to 97 percent, depending upon whether random effects or fixed effect models are used).[6]

One point on which the parties’ expert witnesses did not agree was whether the failure of the Cochrane’s meta-analysis to achieve statistical significance was due solely to the sparse data aggregated from the randomized trials. Plaintiffs’ witnesses appeared to have testified that had the Cochrane been able to aggregate additional clinical trial data, the “effect size” would have remained constant, and the p-value would have shrunk, ultimately to below the level of 5 percent.  Prof. Platt, testifying for the defense, appropriately criticized this hand-waving excuse:

“Q. and the probability factor, the P value, was 0.065, which the previous witness had suggested is an increase in probability of our reliability on the underlying data.  Is it reasonable to assume that this data that a further increase in the sample size will achieve statistical significance?

A. No, that’s not a reasonable assumption….”

Notes of Testimony of Robert Platt at 29 (April 19, 2010).

Positions on Appeal

Dr. Viljoen continued to assert the need for significance on appeal. As appellant, he challenged the trial court’s finding that the Cochrane review concluded that there was a 40% risk reduction. See Goodman v. Viljoen, 2011 ONSC 821, at ¶192 (CanLII) (“I accept the finding of the Cochrane analysis that ACS reduces the instance of CP by 40%”). Dr. Viljoen correctly pointed out that the Cochrane review never reached such a conclusion. Appellant’s Factum, 2012 CCLTFactum 20936, ¶64.  It was the plaintiffs’ expert witnesses, not the Cochrane reviewers, who reached the conclusion of causality from the Cochrane data.

On appeal, Dr. Viljoen pressed the point that his expert witnesses described statistical significance in the Cochrane analysis would have been “a basic and universally accepted standard” for showing that ACS was efficacious in preventing CP or PVL. Id. at ¶40. The appellant’s brief then commits to the very error that Dr. Barrett complained would follow from a finding that did not have statistical significance; Dr. Viljoen maintained that the “trend” of reduced CP reduced CD rates from ACS administration “is the same as a chance occurrence.” Defendant (Appellant), 2012 CCLTFactum 20936, at ¶40; see also id. at ¶14(e) (arguing that the Cochrane result for ACS/CP “should be treated as pure chance given it was not a statistically significant difference”).

Relying upon the Daubert decision from the United States, as well as Canadian cases, Dr. Viljoen framed one of his appellate issues as whether the trial court had “erred in relying upon scientific evidence that had not satisfied the benchmark of statistical significance”:

“101. Where a scientific effect is not shown to a level of statistical significance, it is not proven. No study has demonstrated a reduction in cerebral palsy with antenatal corticosteroids at a level of statistical significance.

102. The Trial Judge erred in law in accepting that antenatal corticosteroids reduce the risk of cerebral palsy based on Dr. Willan’s unpublished Bayesian probability analysis of the 48 cases of cerebral palsy reviewed by Cochrane—an analysis prepared for the specific purpose of overcoming the statistical limitations faced by the Plaintiffs on causation.”

Defendant (Appellant), 2012 CCLTFactum 20936. The use of the verb “proven” is problematic because it suggests a mathematical demonstration, which is never available for empirical propositions about the world, and especially not for the biological world.  The use of a mathematical standard begs the question whether the Cochrane data were sufficient to establish a scientific conclusion of the efficacy of ACS in preventing CP.

In opposing Dr. Viljoen’s appeal, the plaintiffs capitalized upon his assertion that science requires a very high level of posterior probability for establishing a causal claim, by simply agreeing with it. See Plaintiffs’ (Respondents’) Factum,  2012 CCLTFactum 20937, at ¶31 (“The scientific method requires statistical significance at a 95% level.”).  By accepting the idealized notion that science somehow requires 95% certainty (as opposed to 95% confidence levels as a test for assessing random error), the plaintiffs made the defendant’s legal position untenable.

In order to keep the appellate court thinking that the defendant was imposing an extra-legal, higher burden of proof upon plaintiffs, the plaintiffs went so far as to misrepresent the testimony of their own expert witness, Professor Willan, as having committed the transposition fallacy:

“49. Dr. Willan provided the frequentist explanation of the Cochrane analysis on CP:

a. The risk ratio (RR) is .060 which means that there is a 40% risk reduction in cerebral palsy where there has been administration of antenatal corticosteroids;

b. The upper limit of the confidence interval (CI) barely crosses 1 so it just barely fails to meet the rigid test of statistical significance;

c. The p value represents the frequentist term of probability;

d. In this case the p value is .065;

e. From a statistical perspective that means that there is a 6.5% chance that the difference observed in CP rates is due to chance alone;

f. Conversely there is a 93.5% chance that the result (the 40% reduction in CP) is due to a true treatment effect of ACS.”

2012 CCLTFactum 20937, at ¶49 (citing Evidence of Dr. Willan, Respondents’ Compendium, Tab 4, pgs. 43-52).

Although Justice Doherty dissented from the affirmance of the trial court’s judgment, he succumbed to the parties’ misrepresentations about scientific certainty, and their prevalent commission of the transposition fallacy. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36 (“Scientists will draw a cause and effect relationship only when a result follows at least 95 per cent of the time. The results reported in the Cochrane analysis fell just below that standard.”), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

The statistical errors on both sides redounded to the benefit of the plaintiffs.

[1] Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[2] Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews, at 8, Issue 3. Art. No. CD004454 (2006).

[3] See, e.g., In re Ephedra Prods. Liab. Litig., 393 F.Supp. 2d 181, 191, 193 (S.D.N.Y. 2005) (fallaciously arguing that the use of a critical value of less than 5% of significance probability increased the “more likely than not” burden of proof upon a civil litigant.  Id. at 188, 193.  See also Michael O. Finkelstein, Basic Concepts of Probability and Statistics in the Law 65 (2009) (criticizing the Ephedra decision for confusing posterior probability with significance probability).

[4] I do not have the complete transcript of Dr. Barrett’s testimony, but the following excerpt from April 9, 2010, at page 100, suggests that he helped lead Justice Walters into error: “When you say statistical significance, if you say that something is statistically significance, it means you’re, for the scientific notation, 95 percent sure. That’s the standard we use, 95 percent sure that that result could not have happened by chance. There’s still a 5 percent chance it could. It doesn’t mean for sure, but 95 percent you’re sure that the result you’ve got didn’t happen by chance.”

[5] On appeal, the dissenting judge erroneously accepted Justice Walters’ description of the Cochrane review as having supposedly reported a 40% reduction in CP incidence, 93.5% of the time, from use of ACS. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36, leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[6] The Bayesian analysis did not cure the attributability problem with respect to specific causation.

 

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Goodman v Viljoen – Statistical Fallacies from Both Sides

Recrudescence of Traumatic Cancer Claims

June 4th, 2014

In 1991, Peter Huber, discussing traumatic cancer claims, wrote:

“After years of floundering in the junk science morass of traumatic cancer, judges slowly abandoned sequence-of-events logic, turned away from the sympathetic speculations of family doctors, and struggled on to the higher and firmer ground of epidemiology and medical science.  Eventually, the change of heart among appellate judges was communicated back down to trial judges and worker’s compensation boards, and traumatic cancer went into almost complete remission.”

Peter W. Huber, Galileo’s Revenge: Junk Science in the Courtroom 55-56 (1991).

With the advent of Daubert and meaningful gatekeeping of expert witness opinion testimony, the traumatic cancer claims did recede. For a while. Plaintiffs’ counsel, and stalwart opponent of epistemic standards for scientific claims in court, Kenneth Chesebro attacked Huber’s précis of the traumatic cancer law and science. Kenneth J. Chesebro, “Galileo’s Retort: Peter Huber’s Junk Scholarship,” 42 Am. Univ. L. Rev. 1637 (1993). Defenses of the dubious science continue to appear, although mostly in non-peer-reviewed publications.[1]

One of the more disturbing implications of the West Virginia Supreme Court’s decision in Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013), was the Court’s reliance upon its own, recent approval of traumatic cancer claims.  The Harris Court cited, with approval, a 2002 traumatic cancer case, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  The Wiseman case involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation.  The West Virginia Supreme Court blinked at the challenge to the physician expert witness who advanced the causal claim in Wiseman; and in Harris, the Court made clear that blinking is what trial courts should do when confronted with methodological challenges to far-fetched causal opinions.

A couple of years ago, the New York Times ran an article about traumatic cancer. C. Claiborne Ray, “Injury and Insult” (Nov. 5, 2012), responding to the question “Is it possible for cancer to develop as a result of an injury?” Here is how Times science reporter responded:

A. “It’s a common myth that injuries can cause cancer,” the American Cancer Society says on its Web site. Until the 1920s, some doctors believed trauma did cause cancer, “despite the failure of injury to cause cancer in experimental animals.” But most medical authorities, including the cancer society and the National Cancer Institute, see no such link. The more likely explanation, the society suggests, is that a visit to the doctor for an injury could lead to finding an existing cancer.

Other possibilities are that scar tissue from an old trauma could look like a cancerous lesion and that an injured breast or limb would be more closely watched for cancer to develop.

Ms. Ray went on to note a published study, in which would-be myth-busters presented observational data purportedly showing a relationship between physical injury and subsequent breast cancer.  The paper cited by Ms. Ray was a report on a small case-control study done by investigators at the Department of Geography, Lancaster University. See Jan Rigby, et al., “Can physical trauma cause breast cancer?” 11 Eur. J. Cancer. Prev. 307 (2002). The study consisted of 67 breast cancer cases and 134 controls, matched on age, family history, age of menarche, parity, age at first birth, and menopausal status.

Not surprisingly, considering its small size, the Rigby study reported no statistically significant differences for several factors known to be associated with breast cancer: social class, education, residence, smoking and alcohol consumption.  Although lacking power to detect differences of known risk factors, this study turned up a large, statistically significant association between physical trauma and breast cancer:

“Women with breast carcinoma were more likely to report physical trauma to the breast in the previous 5 years than were the controls (odds ratio (OR) 3.3, 95% confidence interval (CI) 1.3-10.8, P < 0.0001).”

* * * * *

“More likely to [self-]report” hardly implies causation, but the authors jumped not only to a causal explanation but to a causal conclusion:

* * * * *

“In conclusion, recall bias is an unlikely explanation for these results in view of the nature and severity of physical trauma. Models of epithelial cell generation indicate that a causal link between physical trauma and cancer is plausible. A latent interval between cancer onset and presentation of under 5 years is also plausible. The most likely explanation of the findings is that physical trauma can cause breast cancer.”

Rigby at 307.

The Rigby study is a valuable demonstration of how malleable researchers can be in discovering plausible explanations for their data.  The authors fail to discuss the natural history of breast carcinoma, such as tumor doubling time, which would make their five-year window decidedly implausible.  The Rigby paper also demonstrates how strident researchers can be in claiming that they have produced a study that has eliminated bias in observational research, when they have barely scratched the surface of bias or confounding. Magical thinking is not the exclusive domain of lawyers.

Until reading the Harris and Wiseman cases, I had thought that the legal system had graduated from the “mythology” of traumatic cancer cases.[2]  To be sure, in the past, any number of physicians have supported traumatic cancer claims, in print and in the courtroom.[3] Some authors attempted to put some rational limits on the extent of the traumatic cancer claims.[4] By 1947, at least, the trauma theory was criticized in leading texts.[5]  In 1974, the Mayo Clinic published a review that emphasized the lack of experimental evidence to support the claim that uncomplicated trauma causes cancer.[6] The law review literature attempted to make sense of the compensation-frenzied courts, without much success.[7]

Many cases from most jurisdictions have approved traumatic cancer claims.  Some are set out below. Some courts heroically resisted the pro-compensation Zeitgeist, usually on case-specific evidentiary issues.[8]

In New York, judges seem to be well aware that post hoc ergo propter hoc is a fallacy.  Cassano v. Hagstrom, 5 N.Y.2d 643, 159 N.E.2d 348, 187 N.Y.S.2d 1 (1959) (affirming dismissal of case based because of plaintiffs’ attempt to use fallacious reasoning in the form of  “post hoc ergo propter hoc”); Holzberg v. Flower & Fifth Ave. Hosps., 39 AD 2d 526 (N.Y. 1st Dep’t 1972). Still, the New York courts struggled with traumatic cancer claims, and appeared to oscillate wildly without clear guidance on whether or to what extent the courts could reject specious claiming supported by speculative or unreliable expert witness opinion testimony.[9] Given the current hostility to gatekeeping of expert witness opinion, a recrudescence of traumatic cancer claims is likely.

Opinions Approving Causation in Traumatic Cancer Cases

California

Santa Ana Sugar Co. v. Industrial Accid. Comm’n, 170 P. 630, 630 (Cal. Dist. Ct. App. 1917)

Colorado

Canon Reliance Coal Co. v. Indus. Comm’n, 72 Colo. 477, 211 P. 868, 869-70 (1922) (cancer caused by being hit on cheek with a lump of coal)

Georgia

National Dairy Prods. Corp. v. Durham, 154 S.E.2d 752, 753-54 (Ga. Ct. App. 1967)

Kentucky

Louisville Ry v. Steubing’s Adm’r, 136 S.W. 634, 634 (Ky. Ct. App. 1911)

Louisiana

Reed v. Mullin Wood Co., 274 So. 2d 845, 846-47 (La. Ct. App. 1972), cert. denied, 275 So. 2d 729, 791 (La. 1973);

Thompson v. New Orleans Ry. & Light Co., 83 So. 19, 20 (La. 1919)

Michigan

Wilson v. Doehler-Jarvis Div. of Nat’l Lead Co., 353 Mich. 363, 91 N.W.2d 538, 539-40 (1958) (blow to lip caused cancer)

Mooney v. Copper Range RR, 27 N.W.2d 603, 604 (Mich. 1947)

Minnesota

Daly v. Bergstedt, 267 Minn. 244, 126 N.W.2d 242, 247–48 (1964) (affirming jury finding of causation between traumatic leg fracture and breast cancer; six physicians testified against causation; one stated cancer “could” result from trauma; imagining that scientific and legal standards of causation differ)

Pittman v. Pillsbury Flour Mills, Inc., 48 N.W.2d 735, 736 (Minn. 1951)

Hertz v. Watab Pulp & Paper Co., 237 N.W. 610, 611 (Minn. 1931)

Austin v. Red Wing Sewer Pipe Co., 163 Minn. 397, 204 N.W. 323, 323-24 (Minn. 1925) (cancer developed one year after worker was hit in the face with coal)

Gaetz v. City of Melrose, 193 N.W. 691, 692 (Minn. 1923)

Missouri

Vitale v. Duerbeck, 338 Mo. 536, 92 S.W.2d 691, 695 (1936)

New Hampshire

Jewell v. Grand Trunk Ry, 55 N.H. 84 (1874) (reversing traumatic cancer verdict on other grounds)

New Mexico

White v. Valley Land Co., P.2d 707, 708-10 (N.M. 1957)

Ohio

Hanna v. Aetna Ins., 24 Ohio Misc. 27, 52 Ohio Op. 2d 316, 259 N.E.2d 177, 177-79 (Ohio Mun. Ct. Dayton 1970)(breast lump found three months after car accident)

Glenn v. National Supply, 129 N.E.2d 189, 190-91 (Ohio Ct. App. 1954)

Oregon

Devine v. Southern Pacific Co., 207 Or. 261, 295 P.2d 201 (1956) (holding that physician’s testimony as to “probable” causation between shoulder fracture and lung cancer was sufficient; jury verdict for plaintiff reversed on other grounds).

Pennsylvania

Baker v. DeRosa, 413 Pa. 164, 196 A.2d 387, 389–90 (Pa. 1964)

Menarde v. Philadelphia Transp. Co., 376 Pa. 497, 103 A.2d 681, 684(1954) (the fact that breast cancer was found in the same place as the injury-caused bruise helped establish causation);

Southern S.S. Co. v. Norton, 41 F. Supp. 103 (E.D. Pa. 1940) (trauma to skull and lower back held to have caused lung cancer)

Tennessee

Koehring-Southern & Am. Mut. Ins. Co. v. Burnette, 464 S.W.2d 820, 821 (Tenn. 1970)

Boyd v. Young, 193 Tenn. 272, 246 S.W.2d 10, 10 (Tenn. 1951)

Rhode Island

Valente v. Bourne Mills, 77 R.I. 274, 278-79, 75 A.2d 191, 193-94 (1950) (adopting house of cards position in which any rational inference suffices even if not supported by expert medical opinion)

Emma v. A.D. Julliard & Co., 75 R.I. 94, 63 A.2d 786, 787-89 (R.I. 1949)(plaintiff had malignant tumor removed from her breast seven weeks after being hit with a can of juice)

Texas

Traders & General Insur. Co. v. Turner, 149 S.W.2d 593, 597-98 (Tex. Civ. App. 1941) (testicular cancer)

Virginia

Ellis v. Commonwealth Dep’t of Highways, 28 S.E.2d 730, 731-32, 735 (Va. 1944) (accepting post-hoc reasoning “[f]acts prevail over possibilities or probabilities”)

Winchester Milling Corp. v. Sencindiver, 138 S.E. 479, 480-81 (Va. 1927)

[1] See, e.g., Melvin A. Shiffman, Can Trauma Cause or Accelerate the Growth of Cancer? Forensic Examiner 6 (Fall 2004).

[2] See Manasco v. Insurance Co. of State of Pennsylvania, 89 S.W.3d 239 (Tex. App. Texarkana 2002) (affirming denial of benefits to worker who claimed head injury caused brain tumor; citing to epidemiological studies that failed to show an association between trauma and brain tumors).

[3] See, e.g., George R. Parsons, “Sufficiency of Proof in Traumatic Cancer Cases,” 2 Tort & Med. Year Book 335 (1962); Stoll & Crissey, “Epithelioma from Single Trauma,” 62 N.Y. St. J. Med. 496 (Feb. 15, 1962); Wilhelm C. Hueper, Trauma and Cancer (1959); Arden R. Hedge, “Can a Single Injury Cause Cancer?” 90 Calif. Med. 55 (1959); R. Crane, “The Relationship of a Single Act of Trauma to Subsequent Malignancy,” in Alan R. Moritz & David S. Helberg, eds., Trauma and Disease 147 (1959); Shields Warren, M.D., “Minimal criteria required to prove causation of traumatic or occupational neoplasms,” Ann. Surgery 585 (1943); Bishop, “Cancer, Trauma, and Compensation,” 32 So. Med. J. 302 (1939); Knox, “Trauma and Malignant Tumors, 26 Am. J. Surg. 66, 69-70 (1934); William B. Coley & Norman L. Higinbotham, “Injury as a causative factor in the development of malignant tumors,” 98 Ann. Surg. 991 (1933); Wainwright, “Single Trauma, Carcinoma and Workman’s Compensation,” 5 Am. J. Surg. 433 (1928); Alson R. Kilgore & Curtis E. Smith, “Industrial liability for cancer,” 25 Calif. & Western Med. 70 (1926); Charles Phelps, “The relation of trauma to cancer formation,” 51 Ann. Surgery 609 (1910).

[4] James Ewing, “Modern Attitudes Toward Traumatic Cancer,” 19 Arch. Path. 690, 692 (1935); James Ewing, “The Relation of Trauma to Malignant Tumors,” Am. J. Surg. 30, 31-34 (Feb. 1926).

[5] See, e.g., James A. Tobey, Public Health Law 321 (3ed 1947) (“Although there is little, if any, scientific evidence to prove conclusively that malignant growths such as carcinoma, sarcoma, and other forms of cancer are ever caused by single blows, wounds, injuries, or other forms of trauma, the courts have awarded damages in a number of instances to persons who have developed cancers following single injuries.”) (internal citations omitted).

[6] George R. Monkman, Gregg Orwoll & John C. Ivins, “Trauma and Oncogenesis,” 49 Mayo Clinic Proc. 157 (1974).

[7] The trauma theory of carcinogenesis was discussed and questioned in several law review articles.  See, e.g., Orrin E. Tilevitz, “Judicial Attitudes Towards Legal and Scientific Proof of Cancer Causation,” 3 Colum. J. Envt’l L. 344 (1977); Donald J. Ladanyi, “Impact Trauma As ‘Legal Cause’ of Cancer,” 20 Cleveland State L. Rev. 409 (1971); Theodore Dyke, “Traumatic Cancer?” 15 Clev.-Marshall L. Rev. 472 (1966); Jerry G. Elliott, “Traumatic cancer and ‘an old misunderstanding between doctors and lawyers’,” 13 U. Kan. L. Rev. 79 (1964); Comment, Sufficiency of Proof in Traumatic Cancer: A Medico-Legal Quandary, 16 Ark. L. Rev. 243 (1962); Comment, “Sufficiency of Proof in Traumatic Cancer Cases,” 46 Cornell L.Q. 581 (1961); Adelson, Injury and Cancer, 5 Western Res. L. Rev. 150 (1954).

[8] State Compensation Ins. Fund v. Kindig, 445 P.2d 72 (Colo. 1968) (head injury held not to have caused leukemia 68 days later); Slack v. C.L. Percival Co., 198 Iowa 54, 199 N.W. 323, 326 (1924) (anticipating Daubert by rejecting expert witness opinion that was “wholly in the realm of conjecture, speculation, and surmise”); Ortner v. Zenith Carburetor Co., 207 Mich. 610, 175 N .W. 122 (1919) (holding that 30 months was too long for a claim that accident that crushed worker’s fingers caused blood poisoning and penile cancer); Stordahl v. Rush Implement Co., 417 P.2d 95 (Mont. 1966) (rejecting traumatic causation of malignant tumor); Tonkovich v. Dep’t of Lab. & Indus., 31 Wash. 2d 220, 195 P.2d 638 (1948) (injury to foot held not to have caused abdominal cancer)

[9] See Dennison v. Wing, 279 App. Div. 494, 110 N.Y.S.2d 811, 813 (1952) (rejecting cancer claim when latency was two months on grounds that cancer took longer to develop); Sikora v. Apex Beverage Corp., 282 App. Div. 193, 196-97 (1953) (reversing judgment for plaintiff based upon jury’s finding that slip and fall accelerated breast cancer based upon lack of evidentiary support), aff’d, 306 N.Y. 917, 119 N.E.2d 601 (1954); Frankenheim v. B. Altman & Co., 13 Misc. 2d 1079, 1080-81, 177 N.Y.S.2d 2 (Bronx Cty. S.Ct. 1958) (granting motion to set aside verdict for plaintiff based upon traumatic cancer claim on grounds of insufficient evidence), app. dism’d, 8 App. Div. 2d 809 (First Dep’t 1959). But see McGrath v. Irving, 24 App. Div. 2d 236, 265 N.Y.S.2d 376 (1965) (affirming jury verdict based upon claim that plaintiff’s swallowing glass in car accident caused or accelerated development of laryngeal cancer); Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 221 N.Y.S.2d 224, 224 (1st Dep’t 1961) (affirming award for traumatic cancer based upon the “usual” conflicting expert witness testimony) Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 942 (1961) (affirming workman’s compensation award for “aggravation” of cancer, which resulted after “the usual conflict of medical opinion”); Pezzolanti v. Green Bus Lines, 114 App. Div. 2d 553, 553-54, 494 N.Y.S.2d 168, 169 (1985) (affirming workman’s compensation award for disability to wrist, which resulted from “trauma” of hitting pothole, which in turn injured asymptomatic wrist destabilized by pre-existing cancer).

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Recrudescence of Traumatic Cancer Claims

Intellectual Due Process in West Virginia and Beyond

June 1st, 2014

Harris v. CSX Transportation

I have borrowed and modified the phrase “Intellectual Due Process” from earlier writers because of its obvious implications for the presentation, interpretation, synthesis, and evaluation of scientific evidence in court. See Scott Brewer, “Scientific Expert Testimony and Intellectual Due Process,” 107 Yale L. J. 1535 (1998). The major reason courts write opinions is to explain and justify their decisions to litigants, present and future, and to a wider audience of lawyers, scholars, and the general public. Judicial opinions involving scientific evidence, whether in legislation, regulation, or litigation must satisfy the societal need to explain and justify the acceptance and rejection of scientific claims. Despite a great deal of hand waving that law and science are somehow different, in the end, when courts describe their acceptance or rejection of scientific claims, they are addressing the same epistemic warrant that scientists themselves employ. Even a cursory review of the judicial output reveals an unsatisfactory state of affairs in which many courts mangle scientific and statistical evidence and inference.  There is much that is needed to correct the problem.

One proposal would be to require that the parties file proposed findings of facts in connection with Rule 702 gatekeeping challenges.  Courts should file detailed findings of facts that underlie their decisions to admit or to exclude expert witness opinion testimony.  Another proposal would require courts to cite properly the scientific studies that they discuss in reaching a legal conclusion about sufficiency or admissibility.  These are small steps, but ones that would help reduce the gross inaccuracies and the glib generalizations, while increasing the opportunity for public scrutiny and criticism.

We do not think anything is amiss with special courts for tax, patent, family law, national security, equity, or commercial matters.  There is an even greater need for scientific skill, knowledge, and aptitude in a specialized science court.  The time has come for special courts to hear cases involving scientific claims in health effects and other litigation.

*   *   *   *   *   *   *

A decision of the West Virginia Supreme Court, late last year, illustrates the need for substantial reform of how claiming based upon “scientific evidence” is permitted and evaluated in court.  Mrs. Harris sued the railroad for the wrongful death of her husband, who died of multiple myeloma. Mr. Harris had been exposed, in his railroad workplace, to diesel exhaust, which Mrs. Harris claimed caused his cancer. See Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013). The trial court excluded Mrs. Harris’s expert witnesses. Harris v. CSX Transportation, Inc., No. 12-1135, 2012 WL 8899119 (Cir. Ct. Marshall Cty., W.Va. Aug. 21, 2012).

1. The West Virginia Supreme Court reversed the trial court’s exclusion of witnesses on the basis of an asymmetrical standard of review, which would allow de novo review of trial court decisions to exclude expert witness opinions, but which would privilege trial court decisions to admit opinions by limiting appellate review to abuse of discretion. This asymmetry was, of course, the same dodge that the Third and Eleventh Circuits had used to keep the “gates open,” regardless of validity or reliability concerns, and the same dodge that the Supreme Court shut down in General Electric v. Joiner. A single judge dissented in Harris, Justice Loughry, who took the majority to task for twisting facts and law to get to a desired result.

2. The Harris Court cited a federal court case for dicta that “Rule 702 reflects an attempt to liberalize the rules governing the admissibility of expert testimony.” See Harris, 753 S.E.2d at 279 (citing and quoting from Weisgram v. Marley Co., 169 F.3d 514, 523 (8th Cir.1999). Remarkably, the Harris Court omitted reference to the United States Supreme Court’s unanimous affirmance of Weisgram, which saw Justice Ginsburg write that “[s]ince Daubert, moreover, parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.” Weisgram v. Marley Co., 528 U.S. 440, 442 (2000).  The Harris Court’s lack of scholarship is telling.

3. Meta-analysis appeared to play a role in the case, but the judicial decisions in Harris fail to describe the proffered evidence. The majority in Harris noted that one of plaintiff’s expert witnesses, Dr. Infante, relied upon a meta-analysis referred to as “Sonoda 2001.” Harris, 753 S.E.2d at 309. Neither the Court nor the dissent cited the published meta-analysis in a way that would help an interested reader in finding the paper.  One could imagine the hue and cry if courts cited judicial cases or statutes by short-hand names without providing enough information to access the relied upon source.  In this case, a PubMed search reveals the source so perhaps the error is harmless. Tomoko Sonoda, Yoshie Nagata, Mitsuru Mori, Tadao Ishida & Kohzoh Imai, “Meta-analysis of multiple myeloma and benzene exposure,” 11. J. Epidemiol. 249 (2001).  Still, the time has come for courts to describe and report the scientific evidence with the same care and detail that they would use in a car collision case.

4. A quick read shows that the Sonoda meta-analysis supports the dissent’s assessment:

“‘Dr. Infante testified on direct examination that Sonoda 2001 considered 8 case-control studies specific to engine exhaust and stated it concluded that diesel and non-diesel engine exhaust causes multiple myeloma.’ Yet, as the trial court found, ‘[o]n cross examination Dr. Infante acknowledged that none of the 8 papers included in the Sonoda meta-analysis mention diesel exhaust’.”

Harris, 753 S.E.2d at 309.  The dissent would have been considerably more powerful had it actually adverted to the language of Sonoda 2001:

“These results suggested that benzene exposure itself was not likely to be a risk factor of MM [multiple myeloma]. It is thought that several harmful chemical agents in engine exhaust, other than benzene, could be etiologically related to the risk of MM. Further case-control studies on MM are needed to obtain more information about detailed occupational exposure to toxic substances.”

Sonoda at 249 (2001) (emphasis added).  Contrary to Infante’s asseveration, Sonoda and colleagues never concluded that diesel exhaust causes multiple myeloma.  The state of scholarship and “intellectual due process” makes it impossible to tell whether or not Dr. Infante was telling the truth or the Harris Court badly misunderstood the record. Either way, something must give.

The dissent went on to note that Dr. Infante conducted his own meta-analysis, which included studies that did not mention diesel exhaust. Harris, 753 S.E.2d at 309.  The railroad complained that some of the studies were small and had limited power, but that is exactly why a meta-analysis would be appropriate.  The more disturbing complaints were that the meta-analysis left out important studies, and that it included irrelevant studies of benzene exposure and myeloma, which raised insuperable problems of external validity.

5. A half empty glass that is always full.  According to the Harris Court, the West Virginia shadow of Rule 702 is a rule of “admissibility rather than exclusion.” Harris, 753 S.E.2d at 279 (citing and quoting from In re Flood Litig. Coal River Watershed, 222 W.Va. 574, 581, 668 S.E.2d 203, 210 (2008), which in turn quoted a federal case, Arcoren v. United States, 929 F.2d 1235, 1239 (8th Cir. 1991), decided before the Supreme Court decided Daubert.)  This is just silly hand waving and blatant partisanship.  A rule that sets out criteria or bases for admissibility also demarcates the inadmissible.

6. Cherry Picking. Dr. Infante was permitted by the Harris Court to aggregate data from studies that did not observe diesel exposure, while he failed to include, or he deliberately excluded data from, a large, powerful, exonerative study conducted by scientists from the National Cancer Institute, the International Agency for Research on Cancer (IARC), and the Karolinska Institute. See Paolo Boffetta, Mustafa Dosemeci, Gloria Gridley, Heather Bath, Tahere Moradi and Debra Silverman, “Occupational exposure to diesel engine emissions and risk of cancer in Swedish men and women,” 12 Cancer Causes Control 365 (2001). Dr. Infante inexplicably excluded this study, which found a risk ratio for men exposed to diesel exhaust that was below one, 0.98, with a very narrow 95% confidence interval, 0.92-1.05. Boffetta at 368, Table 2.

7. The West Virginia articulated an incohorent definition of “reliable,” designed to give itself the ability to reject gatekeeping completely. Citing its earlier decision in Flood, the Court offered its own ipse dixit:

“The assessment of whether scientifically-based expert testimony is “reliable,” as that term is used in [Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and Wilt v. Buracker, 191 W.Va. 39, 443 S.E.2d 196 (1993)], does not mean an assessment of whether the testimony is persuasive, convincing, or well-founded. Rather, assessing ‘reliability’ is a shorthand term of art for assessing whether the testimony is to a reasonable degree based on the use of knowledge and procedures that have been arrived at using the methods of science — rather than being based on irrational and intuitive feelings, guesses, or speculation. If the former is the case, then the jury may (or may not, in its sole discretion) ‘rely upon’ the testimony. In re Flood Litig., 222 W.Va. at 582 n. 5, 668 S.E.2d at 211 n. 5.”

Harris, 753 S.E.2d at 279-80. Surely, this is circular or vacuous or both. Opinions not “well-founded” will be ones that are based upon guesses or speculation.  Opinions arrived at by the “methods of science” will be ones that have an epistemic warrant that will survive a claim that they are not “well-founded.”

8. The Harris Court evidenced its hostility to scientific evidence by dredging up one of its own decisions involving a multiple myeloma causation claim, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  Wiseman involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation:

“Some research has suggested that people in some jobs may have an increased risk of developing multiple myeloma because they are exposed to certain chemicals. But the International Agency for Research on Cancer (IARC) states that the evidence is limited overall. It has been suggested that people may have an increased risk if they work in the petrol or oil industry, farming, wood working, the leather industry, painting and decorating, hairdressing, rubber manufacturing or fire fighting. But there is no evidence to prove that any of these occupations carry an increased risk of myeloma.”

Cancer Research UK, “Myeloma risks and causes” (last visited May 28, 2014). Even the most non-progressive jurisdictions have generally eradicated specious claiming for trauma-induced cancers, but West Virginia has carved out a place second to none in its race to the bottom.

9. WOE.  Not surprisingly, the Harris Court relied heavily on the First Circuit’s “weight of the evidence” end-run around the notion of epistemic warrant for scientific claims, citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir.2011), cert. denied sub nom., U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). The Harris Court went on to conflate and confuse WOE with Bradford Hill, and cited a recent New York case that confidently saw through WOE hand waving, while ignoring its devasting critique of expert witnesses’ attempts to pass off WOE for scientific, epistemic warrant.  Reeps ex rel. Reeps v. BMW of N. Am., LLC, No. 100725/08,

2013 WL 2362566, at *3, 2012 N.Y. Misc. LEXIS 5788; 2012 NY Slip Op 33030U  (N.Y. Sup. Ct. May 10, 2013).

10.  Link.  Dr. Infante links a lot, even when his sources do not:

“Dr. Infante testified that the International Agency for Research on Cancer issued Technical Publication Number 42 in 2009, and that the publication stated that diesel exhaust exposures have been linked to multiple myeloma and leukemia.”

Harris, 753 S.E.2d at 294. The Harris Court neglected to give the title of the publication, which tells a different story.  Identification of research needs to resolve the carcinogenicity of high-priority IARC carcinogens. The dissent was willing to go behind the conclusory and false characterization that Dr. Infante and plaintiff gave to this publication.  Harris, 753 S.E.2d at 309. The trial court’s finding (and the dissent’s assertion) that the IARC Technical Publication 42 intended to express a research agenda, not to make a causation statement, seems unassailable.  Furthermore, it appears to be precisely the sort of specious claim that a court should keep from a jury.  The cited IARC source actually notes that the then current IARC classification of diesel exhaust was of inadequate evidence for human carcinogenicity, with a focus on lung cancer, and barely a mention of multiple myeloma.

11.  The Benzene Connection. Plaintiffs’ expert witnesses, including Dr. Infante, argued that benzene was a component of diesel exhaust, and benzene caused multiple myeloma.  This move ignored not only the lack of evidence to implicate benzene in the causation of multiple myeloma, but it also ignored the large quantitative differences between the benzene occupational exposure studies and the very small amounts of benzene in diesel exhaust.  The Harris Court held that the trial court acted improperly by inquiring into and finding the following facts, which were “exclusively” for the jury:

  • “There is substantially more benzene in cigarette smoke than diesel exhaust.
  • Benzene is present only in trivial doses in diesel exhaust.
  • The hypothesis that diesel exhaust causes multiple myeloma is confounded by the fact that cigarette smoking does not.”

The Harris majority further chastised the trial court for adverting to the ten or so studies that failed to find a statistically significant association between benzene exposure and multiple myeloma.  Harris, 753 S.E.2d at 305-06.  This inquiry directly calls into question, however, Dr. Infante’s methodology.

If these facts, found by the trial court, were reasonably established, then Dr. Infante’s argument was less than bogus, and a major underpinning for inclusion of benzene studies in his meta-analysis was refuted.  These are precisely the sort of foundational facts that must be part of an inquiry into the methodological grounds of an expert witness’s opinion.

12.  The Harris Court confused “proving causation” with “showing a methodology that provides an epistemic warrant for concluding.” Harris, 753 S.E.2d at 300. The Harris Court asserted that the trial court exceeded its gatekeeping function by inquiring into whether Mrs. Harris’s expert witnesses “proved” causation. Harris, 753 S.E.2d at 300. Speaking of “proof of” or “proving” causation is an affectation of lawyers, who refer to their evidence as their “proofs.”  Epidemiologic articles and meta-analyses do not end with quod erat demonstrandum. Beyond the curious diction, there is a further issue in the majority’s suggestion that the trial court set the bar too high in declaring that the plaintiff failed to “prove” causation.  Even if we were to accept the continuous nature of strength of evidence for a causal conclusion, Dr. Infante and the other plaintiff’s witnesses, would be fairly low on the curve, and their lowly position must of necessity speak to the merits of the defense motion to exclude under Rule 702.

13. Purely Matters for Jury. The Harris Court criticized the trial court for conducting a “mini-trial,” which set out to “resolve issues that were purely matters for jury consideration.” Harris, 753 S.E.2d at 305. In holding that the matters addressed in the pre-trial hearing were “exclusively grist for the jury and which had no relevancy to the limited role the trial court had under the facts of this case,” the Harris Court displayed a profound disregard for what facts would be relevant for a challenge to the plaintiff’s expert witnesses’ methodology. Many of the facts found by the trial court were directly relevant to “general acceptance,” validity (internal and external) of studies relied upon, and reliability of reasoning and inferences drawn. Aside from the lack of general acceptance and peer review of the plaintiff’s claimed causal relationship, the proffered testimony was filled with gaps and lacunae, which are very much at issue in methodological challenges to an opinion of causality.

*   *   *   *   *   *   *

The Harris case has taken its place next to Milward in the litigation industry’s arsenal of arguments for abandoning meaningful judicial supervision and gatekeeping of expert witness opinion testimony.  See Andrew S. Lipton, “Proving Toxic Harm: Getting Past Slice and Dice Tactics,” 45 McGeorge L. Rev. 707, 731 (2014) (plaintiffs’ bar cheerleading for the Harris decision as “a lengthy and thoughtful analysis”, and for the Milward case as roadmap to evade meaningful judicial oversight).  Not all was perfect with the trial court’s opinion.  The defense seemed to have misled the court by asserting that “a difference between a case group and control group is not statistically significant then there is no difference at all.”  See Respondent’s Brief at 5, Harris v. CSX Transportation, Inc., 2013 WL 4747999 (filed (Feb. 4, 2013) (citing  App. 169, 228-230 (Shields) as having explained that the p-values greater than 0.05 do not support a causal association).

This is hardly true, and indeed, the lack of statistical significance does not lead to a claim that the null hypothesis of no association between exposure and outcome is correct.  The defense, however, did not have a burden of showing the null to be correct; only that there was no reliable method deployed to reject the null in favor an alternative that the risk ratio for myeloma was raised among workers exposed to diesel exhaust.

Still, the trial court did seem to understand the importance of replication, in studies free of bias and confounding. Courts generally will have to do better at delineating what are “positive” and “negative” studies, with citations to the data and the papers, so that judicial opinions provide a satisfactory statement of reasons for judicial decisions.

Posted in Causation, Expert Witnesses, Meta-analysis, Rule 702, Scientific Evidence | Comments Off on Intellectual Due Process in West Virginia and Beyond

Biostatistics and FDA Regulation: The Convergence of Science and Law

May 29th, 2014

On May 20, 2014, the Food and Drug Law Institute (FDLI), the Drug Information Association (DIA), and the Harvard Law School’s Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics, in collaboration with the Harvard School of Public Health Department of Biostatistics and Harvard Catalyst | The Harvard Clinical and Translational Science Center, presented a symposium on“Biostatistics and FDA Regulation: The Convergence of Science and Law.”

The symposium might just as well have been described as the collision of science and law.

The Symposium agenda addressed several cutting-edge issues on statistical evidence in the law, criminal, civil, and regulatory. Names of presenters are hyperlinked to presentations slides that are available.

I. Coleen Klasmeier, of Sidley Austin LLP, introduced and moderated the first section, “Introduction to Statistics and Regulatory Law,” which focused on current biostatistical issues in regulation of drugs, devices, and foods by the Food and Drug Administration (FDA). Qi Jiang, Executive Director of Amgen, Robert T. O’Neill, retired from the FDA, and now Statistical Advisor in CDER, and Jerald S. Schindler, of Merck Research Laboratories, presented.

II. Qi Jiang moderated and introduced the second section on safety issues, and the difficulties presented by meta-analysis and other statistical assessments of safety outcomes in clinical trials and in marketing of drugs and devices. Lee-Jen Wei, of the Harvard School of Public Health, Geoffrey M. Levitt, an Associate General Counsel of Pfizer, Inc., and Janet Wittes, of the Statistics Collaborative, presented.

III. Aaron Katz, of Ropes & Gray LLP, introduced the third section, on “Statistical Disputes in Life Sciences Litigation,” which addressed recent developments in expert witness gatekeeping, the Avandia litigation, and the role of statistics in two recent cases, Matrixx, Inc. v. Siracusano, and United States v. Harkonen.  Anand Agneshwar, of Arnold & Porter LLP, Lee-Jen Wei, Christina L. Diaz, Assistant General Counsel of GlaxoSmithKline, and Nathan A. Schachtman presented.

IV. Christopher Robertson, a law professor now visiting at Harvard Law School, moderated a talk by Robert O’Neill on “Emerging Issues,” at the FDA.

V. Dr. Wittes moderated a roundtable discussion on “Can We Handle the Truth,” which explored developments in First Amendment and media issues involved in regulation and litigation. Anand Agneshwar, and Freddy A. Jimenez, Assistant General Counsel, Johnson & Johnson, presented.

Posted in Meta-analysis, Rule 702, statistical evidence | Comments Off on Biostatistics and FDA Regulation: The Convergence of Science and Law

The Outer Limits (and Beyond?) of Ex Parte Advocacy of Federal Judges

May 23rd, 2014

As every trial lawyer knows, people sometimes reveal important facts in curious ways, incorporated in their own biased narrative of events.  Recently, I heard a recorded lecture about expert witnesses, by a plaintiffs’ lawyer, who revealed a damning fact about a judge.  The lawyer clearly thought that this fact was commendatory, but in fact revealed another effort of scientific advocates and zealots to subvert the neutrality of federal judges.  See In re School Asbestos Litigation, 977 F.2d 764 (3d Cir. 1992) (describing effort by plaintiffs’ lawyers and the late Dr. Irving Selikoff to corrupt state and federal judges with one-sided ex parte presentations of their views at the so-called Third-Wave Conference).

Anthony Z. Roisman is the Managing Partner of the National Legal Scholars Law Firm.  This firm has a roster of affiliated law professors who serve as consultants for plaintiffs in environmental and tort cases. (Some other participants in this law firm include Jay M. Feinman, Lucinda M. Finley, Neil Vidmar, and Richard W. Wright.) Roisman has been active in various plaintiff organizations, including serving as the head of the ATLA Section on Toxic, Environmental & Pharmaceutical Torts (STEP). 

Roisman lectures frequently for the American Law Institute on expert witness issues. Recently, I was listening to an mp3 recording of one of Roisman’s lectures on expert witnesses in environmental litigation.  Given Roisman’s practice and politics, I was not surprised to hear him praise Judge Rothstein’s opinion that refused to exclude plaintiffs’ expert witnesses’ causation opinions in the PPA litigation.  See In re Phenylpropanolamine Prod. Liab. Litig., 289 F. 2d 1230 (2003).  What stunned me, however, was his statement that Judge Rothstein issued her opinion “fresh from a seminar at the Tellus Institute,” which he described as “organization set up by scientist trying to bring common sense to interpretation of science.”

Post hoc; ergo propter hoc?

Judge Rothstein’s PPA decision stands as a landmark of judicial gullibility.  Judge Rothstein conducted hearings and entertaining extensive briefings on the reliability of plaintiffs’ expert witnesses’ opinions, which were based largely upon one epidemiologic study, known as the “Yale Hemorrhagic Stroke Project (HSP).”  In the end, publication in a prestigious peer-reviewed journal proved to be a proxy for independent review: “The prestigious NEJM published the HSP results, further substantiating that the research bears the indicia of good science.” Id. at 1239 (citing Daubert II for the proposition that peer review shows the research meets the minimal criteria for good science). The admissibility challenges were refused.

Ultimately, the HSP study received much more careful analysis before juries, which uniformly returned verdicts for the defense. After one of the early defense verdicts, plaintiffs’ counsel challenged the defendant’s reliance upon underlying data in the HSP, which went behind the peer-reviewed publication, and which showed that the peer review failed to prevent serious errors.  The trial court rejected the plaintiffs’ request for a new trial, and spoke to the significance of challenging the superficial significance of peer review of the key study relied upon by plaintiffs in the PPA litigation:

“I mean, you could almost say that there was some unethical activity with that Yale Study.  It’s real close.  I mean, I — I am very, very concerned at the integrity of those researchers.”

“Yale gets — Yale gets a big black eye on this.”

O’Neill v. Novartis AG, California Superior Court, Los Angeles Cty., Transcript of Oral Argument on Post-Trial Motions, at 46 -47 (March 18, 2004) (Hon. Anthony J. Mohr)

Roisman’s endorsement of the PPA decision may have been purely result-oriented jurisprudence, but what of his enthusiasm for the “learning” that Judge Rothstein received at the Tellus Institute.  Tell us, what is this Tellus Institute?

In 2003, roughly contemporaneously with Judge Rothstein’s PPA decision, SKAPP published a jeremiad against the Daubert decision, with support from none other than the Tellus Group. See Daubert: The Most Influential Supreme Court Ruling You’ve Never Heard Of;  A Publication of the Project on Scientific Knowledge and Public Policy, coordinated by the Tellus Institute (2003). The Tellus Institute website tells us very little specific detail about the Institute’s projects, other than stating some vague and pious goals.  The alignment, however, of the Tellus Institute with David Michael’s SKAPP, which was created with plaintiffs’ lawyers’ funding, certainly seems like a dubious indicator of neutrality and scientific commitment.  See “Skapp a Lot” (April 30, 2010).

We might get a better idea of the organization from the Tellus membership.

Richard Clapp and David Ozonoff are both regular testifiers for plaintiffs in so-called toxic tort and environmental litigation. In an article published about the time of the PPA decision, Clapp and Ozonoff acknowledged having benefited from discussions with colleagues at the Tellus Institute.  See Richard W. Clapp & David Ozonoff, “Environment and Health: Vital Intersection or Contested Territory?” 30 Am. J. L. & Med. 189, 189 (2004) (“This Article also benefited from discussions with colleagues in the project on Scientific Knowledge and Public Policy at Tellus Institute, in Boston, Massachusetts.”).

In the infamous case of Selikoff and Motley and their effort to subvert the neutrality of Judge James M. Kelly in the school district asbestos litigation, the conspiracy was detected in time for a successful recusal effort. In re School Asbestos Litigation, 977 F.2d 764 (3d Cir. 1992).  Unfortunately, in the PPA litigation, there was no disclosure of the efforts by the advocacy group, Tellus Institute, to undermine the neutrality of a federal judge. 

Outside observers will draw their own inferences about whether Tellus was an “honest broker” of scientific advice to Judge Rothstein. One piece of evidence may be SKAPP’s website, which contains a page about Richard Clapp’s courtroom advocacy in the PPA litigation. Additional evidence comes from Clapp’s leadership role in Physicians for Social Responsibility, and his own characterization of himself as a healthcare professional advocate. Clapp, a member of Tellus, was an expert witness for plaintiffs in PPA cases.

Was Clapp present at the Tellus Institute meeting attended by Judge Rothstein? History will judge whether the Tellus Institute participated in corrupting the administration of justice.

Posted in Conflicts of Interest, Expert Witnesses, Rule 702 | Comments Off on The Outer Limits (and Beyond?) of Ex Parte Advocacy of Federal Judges

The Fallacy of Cherry Picking As Seen in American Courtrooms

May 3rd, 2014

After a long winter, the cherry trees are finally managing to blossom.  Before we know it, it will be cherry-picking time.

Cherry picking is a good thing; right?  Cherry picking yields cherries, and cherries are good.  Selective cherry picking yields the best, ripest, sweetest, tastiest cherries. Cherry picking data no doubt yields the best, unbiased, unconfounded, most probative data to be had.  Well, maybe not.

What could be wrong with picking cherries?  At the end of the process you have cherries, and if you do it right, you have all ripe, and no rotten, cherries.  Your collection of ripe cherries, however, will be unrepresentative of the universe of cherries, but at least we understand how and why your cherries were selected.

Elite colleges cherry pick the best high school students; leading law schools cherry pick the top college students; and top law firms and federal judges cherry pick the best graduates from the best law schools.  Lawyers are all-too-comfortable with “cherry picking.”  Of course, the cherry-picking process here has at least some objective criteria, which can be stated in advance of the selection.

In litigation, each side is expected to “cherry pick” the favorable evidence, and ignore or flyblow the contrary evidence.  Perhaps this aspect of the adversarial system induces complacency in judges about selectivity in the presentation of evidence by parties and their witnesses.  In science, this kind of adversarial selectivity is a sure way to inject bias and subjectivity into claims of knowledge.  And even in law, there are limits to this adversarial system. Undue selectivity in citing precedent can land a lawyer in a heap of trouble. See Thul v. OneWest Bank, FSB, No. 12 C 6380, 2013 WL 212926 (N.D. Ill. Jan. 18, 2013) (failure to cite relevant judicial precedent constitutes an ethical offense)

In science, the development of the systematic review, in large measure, has been supported by the widespread recognition that studies cannot be evaluated with post hoc, subjective evaluative criteria. See generally Matthias Egger, George Davey Smith, and Douglas Altman, Systematic Reviews in Health Care: Meta-Analysis in Context (2001).

Farmers pick the cherries they want to go to market, to make money and satisfy customers. The harvesters’ virtue lies in knowing what to pick to obtain the best crop.  The scientist’s virtue lies in the disinterested acquisition of data pursuant to a plan, and the evaluation of the data pursuant to pre-specified criteria.

The scientist’s virtue is threatened by motivations that are all-too human, and all-too common. The vice in science is wanting data that yields marketable publications, grants, promotions, awards, prizes, and perhaps a touch of fame. Picking data based upon a desired outcome is at the very least scientific fallacy if not scientific fraud. Cherry picking does not necessarily imply scienter, but in science, it is a strict liability offense.

The metaphor of cherry picking, mixed as it may be, thus gives us a label for fallacy and error.  Cherry picking incorporates sampling bias, selection bias,  confirmation bias, hasty generalization, and perhaps others as well. As explained recently, in Nature:

“Data can be dredged or cherry picked. Evidence can be arranged to support one point of view. * * * The question to ask is: ‘What am I not being told?’”

William J. Sutherland, David Spiegelhalter & Mark Burgman, “Policy: Twenty tips for interpreting scientific claims,” 503 Nature 335, 337 (2013).

Cherry picking in the orchard may be a good thing, but in the scientific world, it refers to the selection of studies or data within studies to yield results desired results, however misleading or counterfactual.  See Ben Goldacre, Bad Science 97-99 (2008). The selective use of evidence is not a fallacy unique to science. Cherry picking is widely acknowledged to seriously undermine the quality of public debate See Gary Klass, “Just Plain Data Analysis: Common Statistical Fallacies in Analyses of Social Indicator Data” (2008).  See generally Bradley Dowden, “Fallacies,” in James Fieser & Bradley Dowden, eds., Internet Encyclopedia of Philosophy.

The International Encyclopedia of Philosophy describes “cherry picking” as a fallacy, “a kind of error in reasoning.”  Cherry-picking the evidence, also known as “suppressed evidence,” is:

“[i]ntentionally failing to use information suspected of being relevant and significant is committing the fallacy of suppressed evidence. This fallacy usually occurs when the information counts against one’s own conclusion. * * * If the relevant information is not intentionally suppressed but rather inadvertently overlooked, the fallacy of suppressed evidence also is said to occur, although the fallacy’s name is misleading in this case.”

Bradley Dowden, “Suppressed Evidence,” International Encyclopedia of Philosophy (Last updated: December 31, 2010). See also “Cherry picking (fallacy),” Wikipedia (describing cherry picking as the pointing to data that appears to confirm one’s opinion, while ignoring contradictory data).

In 1965, in his landmark paper, Sir Austin Bradford Hill described some important factors to consider in determining whether a clear-cut association, beyond that which we would attribute to chance, was a causal association. Hill, Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965).

One of the key Hill factors is, of course, consistent, replicated results.  Surely, an expert witness should not be permitted to manufacture a faux consistency by conducting a partial review.  In birth defects litigation, the problem of  “cherry picking” is so severe that one of the leading professional societies concerned with birth defects has issued a position paper to remind its members, other scientists, and the public that “[c]ausation determinations are made using all the scientific evidence”:

“Causation determinations are made using all the scientific evidence. This evidence is derived from correctly interpreted papers that have been published in the peer-reviewed literature. Unpublished data may be useful if available in sufficient detail for an evaluation and if derived from a source that is known to use reliable internal or external review standards. A National Toxicology program report would be an example of an unpublished source that is typically reliable. All available papers are considered in a scientific deliberation; selective consideration of the literature is not a scientific procedure.”

The Public Affairs Committee of the Teratology Society, “Teratology Society Public Affairs Committee Position Paper Causation in Teratology-Related Litigation,” 73 Birth Defects Research (Part A) 421, 422 (2005) (emphasis added).

* * * * * *

Cherry picking is a main rhetorical device for the litigator. Given the pejorative connotations of “cherry picking,” no one should be very surprised that lawyers and judges couch their Rule 702 arguments and opinions in terms of whether expert witnesses engaged in this fulsome fruitful behavior.

The judicial approach to cherry picking is a just a little schizophrenic. Generally, in the context of exercising its gatekeeping function for expert witnesses, the elimination of cherry picking is an important goal. Lust v. Merrell Dow Pharmaceuticals, Inc., 89 F.3d 594, 596-98 (9th Cir. 1996) (affirming exclusion of Dr. Done in a Chlomid birth defects case; district court found that “Dr. Done has seen fit to ‘pick and chose’ [sic] from the scientific landscape and present the Court with what he believes the final picture looks like. This is hardly scientific.”) (internal citation omitted); Barber v. United Airlines, Inc., 17 Fed. Appx. 433, 437 (7th Cir. 2001) (holding that a “selective use of facts fails to satisfy the scientific method and Daubert”). See also Crawford v. Indiana Harbor Belt Railroad Co., 461 F.3d 844 (7th Cir. 2006) (affirming summary judgment in disparate treatment discharge case, and noting judicial tendency to require “comparability” between plaintiffs and comparison group as a “natural response to cherry-picking by plaintiffs”); Miller v. Pfizer, Inc., 196 F. Supp. 2d 1062, (D. Kan. 2002) (excluding, with aid of independent, court-appointed expert witnesses, a party expert witness, David Healy, who failed to reconcile the fact that other research is contrary to his conclusion), aff’d, 356 F.3d 1326 (10th Cir.), cert denied, 125 S. Ct. 40 (2004).

In Ellis v. Barnhart, the Eighth Circuit affirmed a district court’s reversal of an Administrative Law Judge for “cherry picking” the record in a disability case.  392 F.3d 988 (8th Cir. 2005).  Clearly cherry picking was a bad thing for a judicial officer to do when charged with the administration of justice. Several years later, however, the Eighth Circuit held that a trial court erred in excluding an expert witness for having offered an opinion that ignored the witness’s own prior, contrary opinions, a key National Institutes of Health clinical trial, and multiple other studies.  The adversary’s charges of  “cherry picking” were to no avail. Kuhn v. Wyeth, Inc., 686 F.3d 618, 633 (8th Cir. 2012) (“There may be several studies supporting Wyeth’s contrary position, but it is not the province of the court to choose between the competing theories when both are supported by reliable scientific evidence.”), rev’g Beylin v. Wyeth, 738 F.Supp. 2d 887, 892 (E.D.Ark. 2010) (MDL court) (Wilson, J. & Montgomery, J.) (excluding proffered testimony of Dr. Jasenka Demirovic who appeared to have “selected study data that best supported her opinion, while downplaying contrary findings or conclusions.”).

But wait, the court in Kuhn did not cite its own published opinion on cherry picking in Ellis.  Some might say that the Circuit cherry picked its own precedents to get to a desired result. Anthony Niblett, “Do Judges Cherry Pick Precedents to Justify Extralegal Decisions?: A Statistical Examination,” 70 Maryland L. Rev. 234 (2010) (reviewing charges of cherry picking, and examining data [cherry picked?] from California).

The situation in the federal trial courts is chaotic. Most of the caselaw recognizes the fallacy of an expert witness’s engaging in ad hoc selection of studies upon which to rely.  Federal courts, clear on their gatekeeping responsibilities and aware of the selection fallacy, have condemned cherry-picking expert witnesses. Judge Lewis Kaplan, in the Southern District of New York, expressed the proper judicial antipathy to cherry picking:

“[A]ny theory that fails to explain information that otherwise would tend to cast doubt on that theory is inherently suspect,” and “courts have excluded expert testimony ‘where the expert selectively chose his support from the scientific landscape.’”

In re Rezulin Prod. Liab. Litig., 369 F. Supp. 2d 398, 425 & n.164 (S.D.N.Y. 2005) (citation omitted).

Judge Breyer, of the Northern District of California, expressed similar sentiments in ruling on Rule 702 motions in the Celebrex personal injury litigation:

“these experts ignore the great weight of the observational studies that contradict their conclusion and rely on the handful that appear to support their litigation-created opinion.”

In re Bextra & Celebrex Mktg. Sales Pracs. & Prods. Liab. Litig., 524 F. Supp. 2d 1166, 1181 (N.D. Cal. 2007).  The “cherry-picking” of favorable data “does not reflect scientific knowledge, is not derived by the scientific method, and is not ‘good science.’” Id. at 1176.

Other illustrative federal cases include:

In re Bausch & Lomb, Inc., 2009 WL 2750462 at *13-14 (D.S.C. 2009) (“Dr. Cohen did not address [four contradictory] studies in her expert reports or affidavit, and did not include them on her literature reviewed list [. . .] This failure to address this contrary data renders plaintiffs’ theory inherently unreliable.”)

Rimbert v. Eli Lilly & Co., No. 06-0874, 2009 WL 2208570, *19 (D.N.M. July 21, 2009) )(“Even more damaging . . . is her failure to grapple with any of the myriad epidemiological studies that refute her conclusion.”), aff’d, 647 F.3d 1247 (10th Cir. 2011) (affirming exclusion but remanding to permit plaintiff to find a new expert witness)

LeClercq v. The Lockformer Co., No. 00C7164, 2005 WL 1162979, at *4, 2005 U.S. Dist. LEXIS 7602, at *15 (N.D. Ill. Apr. 28, 2005) (“failure to discuss the import of, or even mention … material facts in [expert] reports amounts to ‘cherry-pick[ing]’ … and such selective use of facts fail[s] to satisfy the scientific method and Daubert.”) (internal citations and quotations omitted)

Contractors Ass’n of E. Pa. Inc. v. City of Philadelphia, 893 F. Supp. 419, 436 (E.D. Pa., 1995) (holding that expert witness opinion was unreliable when witness’s conclusions rested on incomplete factual data)

Galaxy Computer Servs. Inc. v. Baker, 325 B.R. 544 (E.D. Va. 2005) (excluding expert witness when witness relied upon incomplete data in reaching a valuation assessment).

Dwyer v. Sec’y of Health & Human Servs., No. 03-1202V, 2010 WL 892250, at *14 (Fed. Cl. Spec. Mstr. Mar. 12, 2010)(recommending rejection of thimerosal autism claim)(“In general, respondent’s experts provided more responsive answers to such questions.  Respondent’s experts were generally more careful and nuanced in their expert reports and testimony. In contrast, petitioners’ experts were more likely to offer opinions that exceeded their areas of expertise, to “cherry-pick” data from articles that were otherwise unsupportive of their position, or to draw conclusions unsupported by the data cited… .”)

Holden Metal & Aluminum Works, Ltd. v. Wismarq Corp., No. 00C0191, 2003 WL 1797844, at *2 (N.D. Ill. Apr. 3, 2003) (“Essentially, the expert ‘cherrypicked’ the facts he considered to render his opinion, and such selective use of facts failed to satisfy the scientific method and Daubert.”) (internal citation omitted).

Flue-Cured Tobacco Cooperative Stabilization Corp. v. EPA, 4 F. Supp. 2d 435, 459 – 60  (M.D.N.C. 1998) (finding that  EPA’s selection of studies for inclusion in a meta-analysis to be “disturbing,” and that agency’s selective, incomplete inclusion of studies violated its own guidelines for conducting risk assessments), rev’d on other grounds, 313 F.3d 852, 862 (4th Cir. 2002) (Widener, J.) (holding that the issuance of the report was not “final agency action”)

Fail-Safe, LLC v. AO Smith Corp., 744 F. Supp. 2d 870, 889 (E.D. Wis. 2010) (“the court also finds the witness’s methodology unreliable because of how Dr. Keegan uniformly treated all evidence that undermined his underlying conclusion: unwarranted dismissal of the evidence or outright blindness to contrary evidence. … In fact, it is readily apparent that Dr. Keegan all but ‘cherry picked’ the data he wanted to use, providing the court with another strong reason to conclude that the witness utilized an unreliable methodology. * * * Dr. Keegan’s two reports are rich with examples of his ‘cherry picking’ of the evidence.”)

As noted, however, there are federal trial courts that are all too willing to suspend judgment and kick the case to the jury.  Here is a sampler of cases that found cherry picking to be an acceptable methodology, or at least a methodology sufficient to require that the case be submitted to the finder of fact.

In Berg v. Johnson & Johnson, the district court noted the defendants’ argument that proffered testimony is unreliable because witness “cherry-picked” data in order to form an opinion solely for purposes of litigation. 940 F.Supp. 2d 983, 991-92 (D.S.D. 2013). The trial judge, however, was not willing to look particularly closely at what was excluded or why:

“The only difference between his past and present research seems to exist in how he categorized his data. Defendants label this ‘cherry-picking’. The court views it as simply looking at the existing data from a different perspective.”

Id.  Of course, expert witnesses on opposite sides look at the case from different perspectives, but the question begged was whether the challenged expert witness had categorized data in an unprincipled way. Other cases of this ilk include:

United States v. Paracha, 2006 WL 12768, at *20 (S.D. N.Y. Jan. 3, 2006) (rejecting challenge to terrorism expert witness on grounds that he cherry picked evidence in conspiracy prosecution involving al Queda)

In re Chantix (Varenicline) Products Liab. Litig., 889 F. Supp. 2d 1272, 1288 (N.D. Ala. 2012) (“Why Dr. Kramer chose to include or exclude data from specific clinical trials is a matter for cross-examination, not exclusion under Daubert.“)

Bouchard v. Am. Home Prods. Corp., 2002 WL 32597992 at *7 (N.D. Ohio May 24, 2002) (“If Bouchard believes that [the expert]… ignored evidence that would have required him to substantially change his opinion, that is a fit subject for cross-examination, not a grounds for wholesale rejection of an expert opinion.”)

In re Celexa & Lexapro Prods. Liab. Litig., 927 F. Supp. 2d 758, 2013 WL 791780, at *5, *7, *8 (E.D. Mo. 2013) (Sippel, J.) (rejecting challenge to David Healy in antidepressant suicide case)

Allen v. Takeda Pharms., MDL No. 6:11-md-2299, No. 12-cv-00064, 2013 WL 6825953, at *11 (W.D. La. Dec. 20, 2013) (challenged expert witness in Actos litigation sufficiently explained his choices to be exonerated from charges of cherry picking)

In re NuvaRing Prods. Liab. Litig., No. 4:08–MD–1964 RWS, 2013 WL 791787 (E.D. Mo. Mar. 4, 2013) (“As to cherry picking data, the Eighth Circuit has recently made clear that such allegations should be left for crossexamination.”)

McClellan v. I-Flow Corp., 710 F. Supp. 2d 1092, 1114 (D. Ore. 2010) (“Defendants are correct that plaintiffs’ experts must elucidate how the relevant evidence lends support to their opinions by explaining…..”) (rejecting cherry picking but denying Rule 702 challenge based in part upon alleged cherry picking)

Rich v. Taser Internat’l, Inc., No. 2:09–cv–02450–ECR–RJJ, 2012 WL 1080281, at *6 (D. Nev. March 30, 2012) (noting the objection to cherry picking but holding that it was an issue for cross-examination)

In re Urethane Antitrust Litig., No. 04-1313-JWL, MDL No. 1616, 2012 WL 6681783, at *3 (D. Kan. Dec. 21, 2012) (allowing expert testimony that “certain events are consistent with collusion”; “the extent to which [an expert] considered the entirety of the evidence in the case is a matter for cross-examination.”)

In re Titanium Dioxide Antitrust Litig., No. RDB-10-0318, 2013 WL 1855980, 2013 U.S. Dist. LEXIS 62394 (D. Md. May 1, 2013) (rejecting Rule 702 cherry-picking challenge to an expert who cherry picked; witness’s selection of documents upon which to rely from a record that exceeded 14 million pages was not unreliable. “ If important portions of the record were overlooked, then the Defendants may address that issue at trial.”)

STATE COURTS

The situation in state courts is similarly chaotic and fragmented.

In Lakey v. Puget Sound Energy, Inc., the Washington Supreme Court resoundingly rejected “cherry picking” by expert witnesses in a public and private nuisance case against a local utility for fear of future illnesses from exposure to electro-magnetic frequency radiation (EMF).  Lakey v. Puget Sound Energy, Inc., 176 Wn.2d 909 (2013). The court held that the plaintiffs’ expert witnesses’ cherry-picking approach to data and studies was properly excluded under Rule 702. Their selective approach vitiated the reliability of his opinion with the consequence of :

“seriously tainting his conclusions because epidemiology is an iterative science relying on later studies to refine earlier studies in order to reach better and more accurate conclusions. Carpenter refused to account for the data from the toxicological studies, which epidemiological methodology requires unless the evidence for the link between exposure and disease is unequivocal and strong, which is not the case here. Carpenter also selectively sampled data within one of the studies he used, taking data indicating an EMF-illness link and ignoring the larger pool of data within the study that showed no such link, Carpenter’s treatment of this data created an improper false impression about what the study actually showed.”

Id.; see also “Washington Supreme Court Illustrates the Difference Between Frye and Rule 702” (April 15, 2013).

Other state Supreme Courts have recognized and rejected the gerrymandering of scientific evidence.  Betz v. Pneumo Abex LLC, 2012 WL 1860853, *16 (May 23, 2012 Pa. S. Ct.)(“According to Appellants, moreover, the pathologist’s self-admitted selectivity in his approach to the literature is decidedly inconsistent with the scientific method. Accord Brief for Amici Scientists at 17 n.2 (“‘Cherry picking’ the literature is also a departure from ‘accepted procedure’.”)); George v. Vermont League of Cities and Towns, 2010 Vt. 1, 993 A.2d 367, 398 (Vt. 2010)(expressing concern about how and why plaintiff’s expert witnesses selected some studies to include in their “weight of evidence” methodology.  Without an adequate explanation of selection and weighting criteria, the choices seemed “arbitrary” “cherry picking.”); Bowen v. E.I. DuPont de Nemours & Co., 906 A.2d 787, 797 (Del. 2006) (noting that expert witnesses cannot ignore studies contrary to their opinions).

Lower state courts have also quashed the cherry-picking harvest. Scaife v. AstraZeneca LP, 2009 WL 1610575, at *8 (Del. Super. June 9, 2009) (“Simply stated, the expert cannot accept some but reject other data from the medical literature without explaining the bases for her acceptance or rejection.”); see also In re Bextra & Celebrex Prod. Liab. Litig., No. 762000/2006, 2008 N.Y. Misc. LEXIS 720, at *47 (Sup. Ct. N.Y. Co. Jan 7, 2008) (stating that plaintiffs must show that their experts “do not ignore contrary data”).

The Nebraska Supreme Court appears to recognize the validity of considering the existence of cherry-picking in expert witness gatekeeping.  In practice, however, that Court has shown an unwillingness to tolerate close scrutiny into what was included and excluded from the expert witness’s consideration.  King v. Burlington No. Santa Fe Ry, ___N.W.2d___, 277 Neb. Reports 203, 234 (2009)(noting that the law does “not preclude a trial court from considering as part of its reliability inquiry whether an expert has cherry-picked a couple of supporting studies from an overwhelming contrary body of literature,” but ignoring the force of the fallacious expert witness testimony by noting that the questionable expert witness (Frank) had some studies that showed associations between exposure to diesel exhaust or benzene and multiple myeloma).

“Of all the offspring of time, Error is the most ancient, and is so old and familiar an acquaintance, that Truth, when discovered, comes upon most of us like an intruder, and meets the intruder’s welcome.”

Charles MacKay, Extraordinary Popular Delusions and the Madness of Crowds (1841)

Posted in Causation, Rule 702 | Comments Off on The Fallacy of Cherry Picking As Seen in American Courtrooms

On The Quaint Notion That Gatekeeping Rules Do Not Apply to Judges

April 27th, 2014

In In re Zurn Pex Plumbing Prods. Liab. Litig., 644 F.3d 604 (8th Cir. 2011), the United States Court of Appeals for the Eighth Circuit rejected the defendant’s argument that a “full and conclusive” Rule 702 gatekeeping procedure was required before a trial court could certify a class action under the Federal Rules. The Circuit remarked that “[t]he main purpose of Daubert exclusion is to protect juries from being swayed by dubious scientific testimony,” an interest “not implicated at the class certification stage where the judge is the decision maker.”  Id. at 613.

Surely, one important purpose of Rule 702 is to protect juries against dubious scientific testimony, but judges are not universally less susceptible to dubious testimony.  There are many examples of judges being misled by fallacious scientific evidence, especially when tendentiously presented by advocates in court.  No jury need be present for dubious science testimony + “zealous” advocacy to combine to create major errors and injustice.  See, e.g., Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985)(rendering verdict for plaintiffs after bench trial), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986); Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation § 6.5 n.3, at 271 (1997) (characterizing Wells as “notorious,” and noting that the case became a “lightning rod for the legal system’s ability to handle expert evidence.”).  Clearly Rule 702 does not exist only to protect juries.

Nemo iudex in causa sua! Perhaps others should judge the competence of judges’ efforts at evaluating scientific evidence.  At the very least, within the institutional framework of our rules of civil procedure and evidence, Rule 702 creates a requirement of structured inquiry into expert opinion testimony before the court.  That gatekeeping inquiry, and its requirement of a finding, subject to later appellate review and to public and professional scrutiny, are crucial to the rendering of intellectual due process in cases that involve scientific and technical issues.  The Eighth Circuit was unduly narrow in its statement of the policy bases for Rule 702, and their applicability to class certification.

The case of Obrey v. Johnson, 400 F.3d 691 (9th Cir. 2005) provides another cautionary tale about the inadequacies of judges in the evaluation of scientific and statistical evidence.  The plaintiff, Mr. Obrey, sued the Navy on a claim of race discrimination in promoting managers at the Pearl Harbor Naval Shipyard.  The district court refused plaintiff’s motion to admit the testimony of a statistician, Mr. James Dannemiller, President of the SMS Research & Marketing Services, Inc. The district court also excluded much of plaintiff’s anecdotal evidence, and entered summary judgment.  Id. at 691 – 93.

On appeal, Obrey claimed that Dannemiller’s report showed “a correlation between race and promotion.” Id. at 693. This vague claim seemed good enough for the Ninth Circuit, which reversed the district court’s grant of summary judgment and remanded for trial.

The Ninth Circuit’s opinion does not tell us what sort of correlation was supposedly shown by Mr. Dannemiller. Was it Pearson’s r?  Or Jaspen’s multi-serial coefficient? Spearman’s ρ?  Perhaps Kendall’s τ? Maybe the appellate court was using correlation loosely, and Mr. Dannemiller had conducted some other sort of statistical analysis. The district court’s opinion is not published and is not available on Westlaw.  It is all a mystery. More process is due the litigants and the public.

Even more distressing than the uncertainty as to the nature of the correlation is that the Ninth Circuit does not tell us what the correlation “effect size” was, or whether the correlation was statistically significant.  If the Circuit did not follow strict hypothesis testing, perhaps it might have told us the extent of random error in the so-called correlation.  The Circuit did not provide any information about the extent or the precision of the claim of a “correlation”; nor did the Circuit assess the potential for bias or confounding in Mr. Dannemiller’s analysis.

Indeed, the Ninth Circuit seemed to suggest that Mr. Dannemiller never even showed a correlation; rather the court described Mr. Dannemiller as having opined that there was “no statistical evidence in these data that the selection process for GS-13 through GS-15 positions between 1999 and 2002 was unbiased with respect to race.” Id. at 694. Reading between the lines, it seems that the statistical evidence was simply inconclusive, and Mr. Dannemiller surreptitiously shifted the burden of proof and offered an opinion that the Navy had not ruled out bias. The burden, of course, was on Mr. Obrey to establish a prima facie case, but the appellate court glossed over this fatal gap in plaintiff’s evidence.

On appeal, the Navy pressed its objections to the relevance and reliability of Mr. Dannemiller’s opinions. Brief of the Navy, 2004 WL 1080083, at *1 (April 7, 2004).  There seemed to be no dispute that Mr. Dannemiller’s “study” was based entirely upon “statistical disparities,” which failed to take into account education, experience, and training.  Mr. Dannemiller appeared to have simplistically compared race make up of the promoted workers, ignoring the Navy’s showing of the relevancy of education, experience, and training.  Id. at *13, 18.

The Ninth Circuit not only ignored the facts of the case, it ignored its own precedents.  See Obrey v. Johnson, 400 F.3d at 696 (citing and quoting from Coleman v. Quaker Oats Co., 232 F.3d 1271, 1283 (9th Cir. 2000) (“Because [the statistics] failed to account for many factors pertinent to [the plaintiff], we conclude that the statistics are not enough to take this case to trial.”). The court, in Obrey, made no effort to distinguish its treatment of the parties in Coleman, or to justify its decision as to why the unspecified, unquantified, mysterious statistical analysis of Mr. Dannemiller sufficed under Rule 702. The Circuit cryptically announced that “Obrey’s evidence was not rendered irrelevant under Rule 402 simply because it failed to account for the relative qualifications of the applicant pool.”  Obrey, 400 F.3d at 695.  Citing pre-Daubert decisions for the most part (such as Bazemore), the Ninth Circuit persuaded itself that Rule 702 requires nothing more than simple relevancy. Had the Circuit taken even a cursory look at Bazemore, it would have seen that the case involved a much more involved multiple regression than whatever statistical analysis Mr. Dannemiller propounded.  And the Ninth Circuit would have seen that even the Bazemore decision acknowledged that there may be

“some regressions so incomplete as to be inadmissible as irrelevant… .”

478 U.S. 385, 400 n.10 (1986). It is difficult to imagine a discrimination claim analysis more incomplete than one that did not address education, training, and experience.

Sadly, neither the Navy’s nor Mr. Obrey’s brief, 2004 WL 545873 (Feb. 4, 2004) provided any discussion of the nature, quality, findings, or limits of Mr. Dannemiller’s statistical analysis.  The Navy’s brief referred to Mr. Dannemiller as a “purported” expert.  His resume, available online, shows that Mr. Dannemiller studied history as an undergraduate, and has a master’s degree in sociology. He is the president of SMS Research, a consulting company.

The taxpayers deserved better advocacy from the Department of Justice, and greater attention to statistical methodology from its appellate judges.  See ATA Airlines, Inc. v. Federal Exp. Corp., 665 F.3d 882, 888-96 (2011) (Posner, J.) (calling for lawyers and judges to do better in understanding and explaining, in plain English, the statistical analyses that are essential to their cases). Judges at level need to pay greater attention to the precepts of Rule 702, even when there is no jury around to be snuckered.

Posted in Expert Witnesses, Rule 702, statistical evidence | Comments Off on On The Quaint Notion That Gatekeeping Rules Do Not Apply to Judges

« Older Entries
Newer Entries »