TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

David Bernstein on the Daubert Counterrevolution

April 19th, 2013

David Bernstein has posted a draft of an important new article on the law of expert witnesses, in which he documents the widespread resistance to judicial gatekeeping of expert witness opinion testimony among federal judges.  Bernstein, “The Daubert Counterrevolution” (Mar. 11, 2013).  Professor Bernstein has posted his draft article, set to be published in the Notre Dame Law Review, both on the Social Science Research Network, and on his law school’s website.

Professor Bernstein correctly notes that the Daubert case, and the subsequent revision to Federal Rule of Evidence 702, marked important changes in the law of expert witnesses.  These changes constituted important reforms, which in my view were as much evolutionary, as revolutionary.  Even before the Daubert case, the law was working to find ways to improve expert witness testimony, and to downplay “authoritative” opining in favor of well-documented ways of knowing.  After all, Rule 702, with its emphasis on “knowledge,” was part of the Federal Rules of Evidence, as adopted in 1975.  Pub. L. 93–595, § 1, Jan. 2, 1975, 88 Stat. 1926 (effective July 1, 1975).  Since their first adoption, Rule 702 required that expert witnesses’ knowledge be helpful to the trier of fact.  By implication, the rule has always suggested that hunches, speculation, and flights of fancy did not belong in the court room.

Professor Bernstein certainly acknowledges that Daubert did not spring out of a vacuum.  Critics of judicial decisions on expert witnesses had agitated for decades to limit expert witness conduct by standards and guidances that operate in the scientific community itself.  The Supreme Court’s serial opinions on Rule 702 (Daubert, Joiner, Kumho Tire, and Weisgram) reflect the need for top-down enforcement of a rule, on the books since 1975, while many lower courts were allowing “anything goes.”

What is perhaps surprising, but well documented by Professor Bernstein, is that after four opinions from the Supreme Court, and a revision in the operative statute itself (Rule 702), some lower federal courts have engaged in a rearguard action against expert witness gatekeeping.  Professor Bernstein rightfully settles on the First Circuit’s decision in Milward as exemplifying a trend to disregard the statutory language and mandate for gatekeeping.  For Bernstein, Milward represents the most recent high-water mark of counterrevolution, with its embrace of errors and fallacies in the name of liberal, if not libertine, admissibility.

I suppose that I would go a step further than Professor Bernstein and label the trend he identifies as “reactionary.”  What is clear is that many courts have been willing to flout the statutory language of Rule 702, in favor of old case law, and evasive reasoning on expert witness admissibility.  Indeed, the Supreme Court itself joined the trend in Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011), when it unanimously affirmed the reversal of a trial court’s Rule 12(b)(6) dismissal of a securities fraud class action.  The corporate defendant objected that the plaintiffs failed to plead statistical significance in alleging causation between Zicam and the loss of the sense of smell.  The Supreme Court, however, made clear that causation was not required to make out a claim of securities fraud.  It was, and would be, sufficient for the company’s product to have raised sufficient regulatory concerns, which in turn would bring regulatory scrutiny and action that would affect the product’s marketability.

Not content to resolve a relatively simple issue of materiality, for which causation and statistical significance were irrelevant, the Supreme Court waxed on, in obiter dicta, about causation and statistical significance, perhaps unwittingly planting seeds for those who would eviscerate Rule 702.  See Matrixx Unloaded (Mar. 29, 2011).  Although the Supreme Court disclaimed any intention to address expert witness admissibility in a case that was solely about the sufficiency of pleading allegations, it cited three cases for the proposition that statistical significance was not necessary for assessing biological causation:

“We note that courts frequently permit expert testimony on causation based on evidence other than statistical significance. See, e.g., Best v. Lowe’s Home Centers, Inc., 563 F. 3d 171, 178 (6th Cir 2009); Westberry v. Gislaved Gummi AB, 178 F. 3d 257, 263–264 (4th Cir. 1999) (citing cases); Wells v. Ortho Pharmaceutical Corp., 788 F. 2d 741, 744–745 (11th Cir. 1986). We need not consider whether the expert testimony was properly admitted in those cases, and we do not attempt to define here what constitutes reliable evidence of causation.”

Id. at 1319.

Remarkably, two of the three cases were about specific causation, arrived at using so-called “differential etiology,” which presupposed the establishment of general causation.  These cases never involved general causation or statistical reasoning, but rather simply the process of elimination (iterative disjunctive syllogism).  The citation to the third case, Wells, was a notorious pre-Daubert, pre-Rule 702 revision case, revealed disappointing scholarship.  Wells involved at least one study that purported to find a statistically significant association.  What was problematic about Wells was its failure to consider the complete evidentiary picture, and to evaluate study validity, bias, and confounding, as well as significance probability.  See Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 1 (Nov. 12, 2012).

Wells was an important precursor to Daubert in that it brought notoriety and disrepute to how federal courts (and state courts as well) were handling expert witness evidence.  Significantly, Wells was a bench trial, where the trial judge opined that plaintiffs’ expert witnesses seemed more credible based upon atmospherics rather than on their engagement with the actual evidence. See Marc S. Klein, “After Daubert:  Going Forward with Lessons from the Past,” 15 Cardozo L. Rev. 2219, 2225-26 (1994) (quoting trial testimony of trial testimony of Dr. Bruce Buehler: “I am sorry sir, I am not a statistician . . . I don’t understand confidence levels. I never use them. I have to use the author’s conclusions.” Transcript of Jan. 9, 1985, at 358, Wells v. Ortho Pharm. Corp., 615 F. Supp. 262 (N.D. Ga. 1985).

Given the Supreme Court’s opinion in Matrixx, the reactionary movement among lower courts is unsurprising.  Lower courts have now cited and follow the Matrixx dicta on statistical significance in expert witness gatekeeping, despite the Supreme Court’s clear pronouncement that it did not intend to address Rule 702.  See In re Chantix (Varenicline) Prods. Liab. Litig., 2012 U.S. Dist. LEXIS 130144, at *22 (N.D. Ala. 2012); Cheek v. Wyeth Pharm. Inc., 2012 U.S. Dist. LEXIS 123485 (E.D. Pa. Aug. 30, 2012).

Professor Bernstein’s article goes a long way towards documenting the disregard for law and science in this movement.  The examples of reactionary decisions could easily be multiplied. Take for instance, the recent Rule 702 gatekeeping decision in litigation over Celexa and Lexapro, two antidepressant medications.  Judge Rodney W. Sippel denied the defense motions to exclude plaintiffs’ principal expert witness, Dr. David Healy.  In re Celexa & Lexapro Prods. Liab. Litig.,  ___ F.3d ___, 2013 WL 791780 (E.D. Mo. 2013).  In attempting to support its decision, the court announced that:

1. Cherry picking of studies, and data within studies, is acceptable for expert witnesses.  Id. at *5, *7, *8.

2. Outdated law applies, regardless of being superseded by later Supreme Court decisions, and the statutory revision in Rule 702.  Id. at *2 (citing pre-Joiner case:   “The exclusion of an expert’s opinion is proper only if it is so fundamentally unsupported that it can offer no assistance to the jury.” Wood v. Minn. Mining & Mfg. Co., 112 F.3d 306, 309 (8th Cir.1997) (internal quotation marks and citation omitted).”

3.  The Bradford Hill factors can be disregarded.  Id. at *6 (citing In re Neurontin Mktg., Sales Practices, and Prod. Liab. Litig., 612 F. Supp. 2d 116, 133 (D. Mass. 2009) (MDL 1629), and In re Viagra, 572 F.2d 1071 (D. Minn. 2008)).

These features of the Celexa decision are hardly novel.  As Professor Bernstein shows in his draft article, disregard of Rule 702’s actual language, and of the post-Daubert Supreme Court decisions, is prevalent.  See, e.g., In re Avandia Marketing, Sales Practices & Prod. Liab. Litig., 2011 WL 13576 (E.D. Pa. 2011)(announcing that MDL district judge was bound to apply a “Third Circuit” approach to expert witness gatekeeping, which focused on the challenged expert witnesses’ methodology, not their conclusions, in contravention of Joiner, and of Rule 702 itself).

The Celexa decision pushes the envelope on Bradford Hill.  The two decisions cited downplayed Bradford Hill’s considerations, but did not dismiss them.  In re Neurontin Mktg., Sales Practices, and Prod. Liab. Litig., 612 F. Supp. 2d 116, 133 (D. Mass. 2009) (MDL 1629)(“Although courts have not embraced the Bradford Hill criteria as a litmus test of general causation, both parties repeatedly refer to the criteria, seemingly agreeing that it is a useful launching point and guide. Accordingly, this Court will begin its inquiry by evaluating Plaintiffs’ evidence of an association between Neurontin and suicide-related events, the starting point for an investigation under the criteria.”);  In re Viagra Prods. Liab. Litig., 572 F.Supp.2d at 1081 (“The Court agrees that the Bradford Hill criteria are helpful for determining reliability but rejects Pfizer’s suggestion that any failure to satisfy those criteria provides independent grounds for granting its Daubert Motion.”).

Of course, Sir Austin’s considerations were merely those that he identified in a speech to a medical society.  They were not put forward in a scholarly article; nor are his considerations the last word on the subject.  Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295 (1965).

Even as a précis, given almost 50 years ago, Hill’s factors warrant some consideration rather than waving them off as not a litmus test (whatever that means), followed by complete disregard for any of the important considerations in evaluating the causality of an association.

There was brief bright spot in this fairly dim judicial decision.  The district judge refused to exclude Dr. Healy on grounds that his opinion about particular studies differed from the authors’ own interpretations.  In re Celexa & Lexapro Prods. Liab. Litig.,  ___ F.3d ___, 2013 WL 791780, at *5 (E.D. Mo. 2013) (Sippel, J.).

That is the correct approach, even though there is language in Joiner that suggests that the authors’ views are dominant.  See Follow the Data Not the Discussion.  But the refusal to discount Healy’s opinions on this ground was done without any real inquiry whether Healy had offered a valid, competing interpretation of the data in the published studies.

At the core of the reactionary movement identified by Professor Bernstein is an unwillingness, or an inability, to engage with the scientific evidence that is at issue in various Rule 702 challenges.  Let’s hope that Bernstein’s article induces closer attention to the law and the science in future judicial gatekeeping.

Posted in Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on David Bernstein on the Daubert Counterrevolution

Washington Supreme Court Illustrates the Difference Between Frye and Rule 702

April 15th, 2013

Last month, the Washington Supreme Court decided a public and private nuisance case against a local utility for fear of future illnesses from exposure to electro-magnetic frequency radiation (EMF).  Lakey v. Puget Sound Energy, Inc., ___ Wn. 2d ___ , 2013 WL 865468 (Mar. 6, 2013).

The defendant utility and local municipality had moved for the exclusion of plaintiffs’ expert witnesses, under Frye v. United States, 54 App. D.C. 46, 293 F. 1013 (1923), and the Washington state version of Rule 702.  The plaintiff homeowners submitted declarations from two expert witnesses, Dr. De Kun Li and and Dr. David Carpenter.

Dr. Carpenter was the Director of the Institute for Health and the Environment at the University at Albany and a Professor of Environmental Health Sciences within the School of Public Health. He also has held the position of Dean of the School of Public Health at the University of Albany and the Director of the Wadsworth Center for Laboratories and Research of the New York State Department of Health.  Dr. Carpenter apparently has been active in testifying about EMF health issues.  See, e.g., Amended Declaration of David Carpenter (2011).

The trial court conducted a three-day Frye hearing, at which both sides offered expert witness testimony.  Plaintiffs called Carpenter, and the defendants called Dr. Nancy Lee, an epidemiologist, and Dr. Mark Israel, a professor at Dartmouth’s Geisel School of Medicine, and a specialist in the molecular and cellular biology of brain tumors.

At the hearing, Dr. Lee testified that Carpenter had failed to follow generally accepted epidemiologic methodology of considering all pertinent data.  Carpenter had “cherry picked”; his opinion selectively ignored studies that contradicted his conclusions.  Even among the studies that Carpenter had relied upon, he ignored data that undermined or contradicted his conclusion.  Slip op. at 6.  Furthermore, Carpenter ignored the toxicologic data and studies available. Id.  Among the studies ignored by Carpenter were the most recent, and most carefully conducted, studies. Id.  The trial court excluded Li and Carpenter, and the plaintiffs appealed.

Affirming the exclusion, the Supreme Court of Washington engaged in a curious two step.  The Court reversed the trial court’s exclusion on Frye grounds.  In Supreme Court’s view, the plaintiffs’ witnesses sufficiently deployed a generally accepted method when they waded into the field of epidemiology because epidemiology is generally accepted.  The Frye doctrine does not require that expert witnesses apply the established methodology well, reliably, or soundly.  The expert witnesses are immunized from exclusion by relying upon studies from a generally accepted discipline, no matter how poorly or selectively they have analyzed and interpreted these studies.  An expert witness’s errors in engaging with a particular scientific discipline that is in general considered generally accepted “go to the weight, not the admissibility, of the evidence unless the error renders the evidence unreliable.” Slip op. at 11.

Carpenter’s cherry-picking approach to data and studies, however, was properly excluded under Rule 702.  Carpenter’s approach vitiated the reliability of his opinion with the consequence of :

“seriously tainting his conclusions because epidemiology is an iterative science relying on later studies to refine earlier studies in order to reach better and more accurate conclusions. Carpenter refused to account for the data from the toxicological studies, which epidemiological methodology requires unless the evidence for the link between exposure and disease is unequivocal and strong, which is not the case here. Carpenter also selectively sampled data within one of the studies he used, taking data indicating an EMF-illness link and ignoring the larger pool of data within the study that showed no such link, Carpenter’s treatment of this data created an improper false impression about what the study actually showed.”

Slip op. at 12.

So that’s the difference between Frye and Rule 702!

Posted in Expert Witnesses, Frye, Rule 702 | Comments Off on Washington Supreme Court Illustrates the Difference Between Frye and Rule 702

Further Musings on U.S. v. Harkonen

April 15th, 2013

Epistemic Crimes

In U.S. v. Harkonen, the government prosecuted a physician, company CEO, for issuing a press release that stated a clinical trial “demonstrated” benefit when the government believed that the clinical trial was inconclusive.  No doubt the government was intent upon punishing what it thought was off-label promotion in the same press release, but the jury acquitted on the charge of misbranding, and convicted on the wire fraud count.  The trial court denied post-trial motions, and recently, the United States Court of Appeals, for the Ninth Circuit, affirmed, in an unpublished per curiam opinion.  United States v. Harkonen, No. 11-10209, No. 11-10242, 2013 WL 782354, 2013 U.S. App. LEXIS 4472 (9th Cir. March 4, 2013).

A Gedanken Experiment

An expert witness writes a report that X, a drug therapy, causes Y, a benefit in survival, for a disease, Z.

The expert witness sent his report by email, and regular mail, to counsel, who then served it upon his adversary.  The report set out some of the support for the opinion, as follows.

The expert witness relied upon a randomized clinical trial, conducted with one primary and nine secondary endpoints.  The multiple endpoints were chosen because of uncertainty of how the anticipated benefit would manifest.  Mortality (survival), although obviously a very important endpoint, was not made primary endpoint because the scientists who conducted the trial did not anticipate sufficient deaths over the course of the trials to see a statistically significant benefit.

This clinical trial had surprising results. Although the trial did not show a difference on the primary endpoint, a composite defined in terms of various pulmonary functional changes or death, the trial did “demonstrate,” according to the witness, a survival benefit.  Indeed, the survival benefit was clinically significant.  Patients randomized to therapy experienced a 40% decrease in mortality, compared to those randomized to placebo. (p = 0.084).  The expert witness pointed out, in his report, that the survival benefit was even stronger in a subgroup of the clinical trial, which consisted of the patients who had mild- to moderate-disease at the time of randomization.  For this subgroup, the decrease in mortality was even more dramatic, 70%, p = 0.004.  The witness’s report did not clearly label this subgroup as “post-hoc,” although a discerning reader might well have assessed it as such.

The expert witness was not relying upon only one clinical trial.  His report identified an earlier trial, published in a leading clinical medical journal, which reported benefit from the drug, p < 0.001.  This trial was extended, with continuing strong evidence of differential survival.  In terms of survival at five years, the earlier trial showed survival in the therapy group at 77.8%, compared to 16.7% in the control groups, p = 0.009.

The expert witness’s report did not explicitly reference clinical experience, or the in vitro and in vivo mechanistic evidence that the therapy, X, plays a role in inhibiting processes that are clearly involved in producing the disease, Z.  The expert witness could have written a stronger expert witness report with these references, but did not expect that this level of completeness was required.  The expert witness did note that he would marshal the data in more detail at a later time. The expert witness further relied upon the assessment of the principal investigator of the later clinical, who had written that the benefit against mortality of X was “compelling,” and that the finding was “a major breakthrough.”  The principal investigator of the trial noted that X was “the first treatment ever to show any meaningful clinical impact in this disease in rigorous clinical trials, and these results would indicate that [X] should be used early in the course of this disease in order to realize the most favorable long-term survival benefit.”  The report went on to note, accurately, that there are no FDA-approved therapies for Z.

Adversary counsel, receiving this report, moved pursuant to Federal Rules of Evidence 702 and 703, to exclude the expert witness’s report and his opinions.  The motion to exclude was made in advance of the deposition, and without a preliminary motion for more detail about the supporting data.  In particular, the motion to exclude claimed that the expert witness was unjustified in concluding that a benefit had been “demonstrated,” as opposed to being merely suggested.

What would be the challenger’s chances of success on the Rule 702 motion?  The outcome, Y, was not “statistically significant” at the conventional two-tail 5% (but would have been on a one-tail test).  The subgroup that sported a p-value of 0.004 was not clearly marked as a post-hoc subgroup, although the challenger could discern that it was likely exploratory, and challenged it as uncorrected for multiple testing.  The challenger, however, did not attempt to offer a modified p-value that took into account of multiple testing.  The essence of this challenge was that the expert witness’s statement that a benefit had been “demonstrated” was not supported by sufficient evidence, and that the low p-value of 0.004 was not truly “significant” because the result emerged from an analysis that was not pre-planned.

My hunch, based upon published judicial opinions on both state and federal Rule 702 motions, is that many judges would allow the challenged expert witness to testify.  There would be the usual judicial hand waving about the challenge’s going to the weight not the admissibility of the expert witness’s opinion.  Perhaps an occasional judge might order additional discovery.  I believe that most judges would not find that this expert witness had engaged in pathologically bad science such that the party proponent should be denied its expert witness.

Transmuting Disputed Causal Inferences Into Criminal Fraud

Instead of moving to exclude the expert witness’s opinion, why not turn the report over to the U.S. Attorney’s office to prosecute for wire or mail fraud?  Even if a trial court were to brand the opinion “inadmissible,” that outcome would hardly suggest that the opinion was the kind of speech that could qualify as fraudulent under federal wire or mail fraud statutes. Branding a scientist as a fraudfeasor, however, was exactly the result reached in U.S. v. Harkonen, where the Ninth Circuit upheld a wire fraud conviction of a physician whose written statements would likely have been admissible in most federal courtrooms, under Federal Rule 702.  As much as I would like to see more stringent gatekeeping of expert witness opinions, there is something unseemly about the government’s efforts here to criminalize scientific opinions with which it disagrees.

Dr. Harkonen has petitioned the Ninth Circuit for reconsideration, in a brief filed by attorneys, Mark Haddad and colleagues, of Sidley Austin.  Petition for Rehearing En Banc (filed 29, 2013).  The case raises important First Amendment and due process issues, which were addressed by the party and amici briefs before the Panel.

The case also raises the specter of prosecutions of scientists for speech in various contexts, including grant applications and reports, under the False Claims Act, for witness perjury for testimony in judicial, administrative, or legislative proceedings, or for wire or mail fraud for manuscript submissions to journals. On April 8th, Professor Robert Makuch, of Yale University, Professor Timothy Lash, of Emory University, and I filed an amicus brief, which addresses the government’s controversial branding statements “false as a matter of statistics.” The government has gone from one extreme of painting, broad brush, that statistical significance is not important or necessary (in Matrixx Initiatives Inc. v. Siracusano), to the other extreme that statistical significance is so important that a scientist who his opinion on causality on evidence the government believes is not statistically significant has committed fraud (in Harkonen).  Both extreme positions are untenable.

Posted in Causation, Rule 702, statistical evidence | Comments Off on Further Musings on U.S. v. Harkonen

Gatekeeping in Allen v. Martin Surfacing — Postscript

April 11th, 2013

Back in November 2012, I wrote a review and analysis of a district court’s handling of Rule 702 challenges to expert witness opinions, in a case involving a tragic ALS death, Allen v. Martin Surfacing, 263 F.R.D. 47 (D. Mass. 2009).  See Bad Gatekeeping or Missed Opportunity – Allen v. Martin Surfacing (Nov. 30, 2012).

I received correspondence from one of the plaintiffs’ expert witnesses, Dr. Marcia Ratner, who was not entirely happy with my discussion of the Allen case.  Suffice it to say, on the medico-legal issues, we do not have much common ground for agreement.  Dr. Ratner, however, asked me to update my post by noting two facts:

1.  First, plaintiffs’ counsel had asked her to testify that solvent fumes, including toluene, had caused Mr. Allen’s ALS, and she refused.  She was unwilling to acquiesce in their request, and she testified only about acceleration of onset.

2.  Second, Dr. Ratner objected to my including the colorful comments about her brush with the law.  My report was factual and documented, but I am sympathetic.  Dr. Ratner would have me note that she is a registered Republican and a staunch defender of Second Amendment rights; her arrest result when she “inadvertently stepped into a liberal trap when [she] came down [to Massachusetts] from Vermont.”

As a lawyer, I am indeed sympathetic to anyone who has truly stepped into a trap.  As for the Allen case, the matter settled, and so there never was a chance to see how a jury would react to the various theories in the case.  More important, there never was appellate judicial review of the gatekeeping efforts.

Posted in Expert Witnesses, Rule 702 | Comments Off on Gatekeeping in Allen v. Martin Surfacing — Postscript

UC Davis Daubert Symposium

March 28th, 2013

Earlier this month, I wrote about a Symposium on Daubert at the University of California Davis School of Law.  The UC Davis Law Review has now published the proceedings of the Symposium, including a transcript of the direct and cross-examinations of the mock expert witnesses:

Symposium — The Daubert Hearing: From All the Critical Perspectives

 

Posted in Expert Witnesses, Rule 702, Trial Procedure | Comments Off on UC Davis Daubert Symposium

U.C. Davis Symposium on the “Daubert” Hearing

March 11th, 2013

Professor David Faigman’s recent article, “The Daubert Revolution and the Birth of Modernity:  Managing Scientific Evidence in the Age of Science,” 102 U.C. Davis Law Rev. 101 (2013), notes that the paper grew out of informal remarks given at a recent symposium.  A little Googling quickly turned up the Symposium Site on the University of California, Davis, website.  Like Professor Faigman’s paper, this symposium is a valuable contribution to the art and learning of what Rule 702 hearings should, and should not, be.

The symposium, The Daubert Hearing — From All the Critical Perspectives (March 2, 2012) described itself:

Pretrial practice has long been the center of gravity in modern litigation. The vast majority of cases never go to trial. Instead, after pretrial discovery and in limine motions, the cases settle. The Supreme Court’s celebrated 1993 decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, has solidified that trend. In Daubert, the Court abandoned the traditional general acceptance standard for the admissibility of scientific testimony and announced a new empirical validation test. Throughout the country counsel began basing pretrial in limine motions on Daubert to target opposition expert testimony. In criminal cases, defense counsel started challenging the prosecution’s forensic evidence identifying the accused as the perpetrator. In civil tort cases, defense counsel filed motions attacking the plaintiff’s evidence on general causation. When counsel won these motions, the opposition lacked sufficient evidence to go to trial. The hearing on the pretrial Daubert motion became the centerpiece of the litigation.

This symposium will begin with a demonstration Daubert hearing. After the demonstration, all the participants will deliver remarks, giving their perspective on the law and tactics of Daubert hearings. In addition, there will be expert academic commentary by Professor David Faigman of U.C. Hastings School of Law, the lead author of the popular treatise, MODERN SCIENTIFIC EVIDENCE.

The symposium featured a list of distinguished speakers:

Hon. James M. Rosenbaum

Robert G. Smith

Bert Black

Professor David L. Faigman

Professor Edward Imwinkelried

Dr. William A. Toscano, Jr.

Dr. Sander Greenland

The symposium’s hypothetical is available on line, and the symposium itself, which was video recorded, is available for viewing at the UC Davis website.

The scientists who role-played as expert witnesses, Drs. Toscano and Greenland,  were obviously pushed to articulate certain positions that they did not personally subscribe to.  Still, their true colors managed to show, and to influence the mock hearing.  For instance, Dr. Toscano stated several times that causation is very difficult to prove, and in so stating, he managed to convey the impression that he had a personal, subjective higher bar for causal claims than the rest of the scientific community.  This approach is a common rookie mistake for defense counsel and their expert witnesses, and it should be avoided.  There are plenty of good examples of causal relationship that have been established with epidemiology, and the defense expert should be prepared to identify them, and to explain why in some cases, the causal relationships required more exacting evidence.  The other glaring error in the defense presentation was that the exact methodological error was not made clear through Dr. Toscano’s testimony although the defense lawyer, Mr. Smith, explored the gaps and leaps of faith in his cross-examination of Dr. Greenland.  In this setting, the defense expert witness’s focus is on the methodological inadequacies of the plaintiffs’ witness, not on why he rejected the causal claim.

Dr. Greenland was his inimitable self, even going so far as to talk into his magic marker under the impression that it was a microphone.  Who knows; perhaps it was, but it also wrote on the white board.  More telling was that Dr. Greenland embraced a probabilistic conception of causation, which he explained was essentially a bet on the correct result.  This metaphor seems fatally defective.  A bet is a bet, but you cannot call the bet until you have actual evidence of who won.  It may be lovely that Dr. Greenland, or some other expert witness, is willing to place the bet, perhaps with odds, but this metaphor fails to take causal inference out of the subjective realm.  Along with his betting metaphor, Greenland emphasized that the causation decision is driven by a cost-benefit analysis of Type I and II errors.  The slippery slide into substituting the precautionary principle for causal analysis was obvious.

On the plaintiffs’ side, Bert Black, an apostate defense lawyer, did a very good job of portraying the shenanigans used by plaintiffs’ lawyers to avoid and evade gatekeeping.  Statistical significance is not necessary; epidemiology is not necessary; Bradford Hill factors are not necessary; therefore, I can show causation without much of anything.  Black illustrated nicely how the focus is redirected to other cases, such as when someone from a drug company wrote an improvident article that concludes causation from case reports alone.  Or cases involving signature diseases, or acute outbreaks, for which causal relations were discerned and embraced by scientists on the basis of very informal epidemiologic studies or even case series (which someone characterized as anecdata).

Former federal judge James Rosenbaum presided magisterially, and cowardly denied the cross-motion Rule 702 challenges.  In his comments after the mock, Judge Rosenbaum revealed his conception of the gatekeeping process as essentially a determination that the witness is competent.  Of course this is not the law, and much more is required than to determine that the witness is minimally qualified.  Professor Faigman respectfully chastised the judge for ignoring the statute and the caselaw.

One of the more interesting dialogues in the discussions after the mock centered on the harm to an expert witness’s reputational interests from the gatekeeping process.  To be sure there can be such harm, but as Professional Faigman pointed out, the potential for such harm cannot intimidate judges from ruling on the facts and law before them.  I believe though that expert witnesses should be aware of the potential for this sort of harm from their testimonial adventures, and should require certain contractual assurances from the lawyers who engage them.  For instance, expert witnesses should insist upon whether such challenges are possible in the jurisdiction, what the standards are, and whether they will have an opportunity to speak to the challenges.  The expert witnesses should insist upon prompt notification of all such challenges, and upon prompt receipt of all briefs and affidavits that challenge the validity or reliability of their opinions, as well as an opportunity to be heard on their responses.

 

 

 

 

Posted in Expert Witnesses, Frye, Rule 702, Uncategorized | Comments Off on U.C. Davis Symposium on the “Daubert” Hearing

Professor Faigman on the Dual Goals of the Daubert Revolution

March 9th, 2013

Academic commentators on Daubert and its progeny tend to fall into two camps:  acolytes and heretics.  The acolytes have generally supported the changes brought about by Daubert and the ultimate statutory embrace of active expert witness gatekeeping.  The heretics have maintained a rearguard action against Daubert, and Rule 702; they have tried to undermine gatekeeping at every turn.

Among the chief acolytes is David Faigman, whose books and articles have contributed substantially to the discussions and debates about the law of scientific evidence and expert witnesses.  Professor Faigman’s recent article is an important contribution to the law review literature on Daubert.  David L. Faigman, “The Daubert Revolution and the Birth of Modernity:  Managing Scientific Evidence in the Age of Science,” 102 U.C. Davis Law Rev. 101 (2013) [“Revolution”].  It is well worth reading.

Professor Faigman declares himself “a fan” of Daubert, and embraces the revolution in expert witness law heralded by the Supreme Court’s 1993 decision.  Id. at 103.  He emphasizes that the decision, quickly approaching its 20th anniversary, was truly revolutionary in how the federal courts engaged with expert witness opinion testimony, and that the consequences of the revolution are still taking shape.  Id.

Faigman acknowledges that Daubert and its progeny, and the statutory embrace of gatekeeping in Rule 702, at the end of the last millennium, were important developments in ensuring the epistemic warrant of federal courts’ judgments.  Some authors, hostile to the gatekeeping enterprise, have suggested that this aspect of Daubert resulted from persistent pressures from the defense bar and industry to limit plaintiffs’ access to the courts.  Faigman does not address such suggestions, and I believe that they are cynical and incorrect.  The federal courts, by the mid-1980’s, were deeply embarrassed by the scientific community’s opprobrium, meted out over notorious decisions, such as Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).  See also In re Air Crash Disaster at New Orleans, 795 F.2d 1230, 1234 (5th Cir. 1986) (“Our message to our able trial colleagues: it is time to take hold of expert testimony in federal trials.”).  Daubert and its progeny were, in my view, the judicial response to the scientific community’s criticisms.

Faigman’s thesis in this paper, however, lies elsewhere.  He argues that the Supreme Court’s excursions into expert evidence law, in Daubert and in the later cases, were intended primarily to give trial courts greater control over their dockets by being able to excluding dubious testimony and to grant summary dispositions.  Revolution at 104  Scientific verisimilitude was secondary to docket control.  Id. at 105.

Faigman’s thesis is plausible and should be taken seriously.  The first three cases in the “Revolution,” Daubert, Joiner, and Kumho Tire, were all tort cases with “mass tort” overtones.  Daubert was one of many Bendectin cases.  Joiner was a case involving occupational PCB exposures.  If the tenuous scientific opinions were deemed “admissible,” there were sure to be many more such cases.  And Kumho Tire was a case involving dubious allegations of a defect in a tire, the sort of allegations that plague American industry because they are so easy to manufacture, and so costly to defend.

Faigman builds an impressive case for the proposition that the justices really were trying to give trial courts managerial power to control their own dockets by filtering out essential, but deficient, expert witness testimony. Id. at 118.  After all, if the Supreme Court were really interested in improving judicial use of scientific evidence, why would it have created an abuse of discretion standard for reviewing Rule 702 determinations? The abuse standard signals that decisions either way are tolerable if they are accompanied by the right verbiage and procedural steps.

Faigman also points out that the abuse-of-discretion standard deprives the appellate courts of any meaningful review of the validity of scientific opinion testimony. The claims and conclusions advanced by expert witnesses in individual cases will often be of interest and importance to scientists, policy makers, plaintiffs, defendants, beyond the confines of the individual case.  The appellate courts are in a better position to ascertain validity questions, and maintain consistency in them, as a matter of law.  Freed of the pressures of trial courts, and with input from amici curiae, the appellate court can evaluate validity issues more deliberately with a view to harmonizing competing factors across many cases.  The scientific issues are, in any event, often non-case specific, or they have the tendency to recur in many cases of the same type.  Id. at 131.

Faigman’s thesis sheds light upon who the heretics are, and why they have worked so hard to undermine expert witness gatekeeping.  At stake is not only greater scientific validity, but also summary disposition of litigation rent-seeking. Rule 702 gatekeeping challenges judges and commentators to identify their priorities:  commitment to scientific principles or to litigation as an alternative to regulation and legislation on behalf of a special constituency.

There are some ironies inherent in Faigman’s thesis.  The trial bench has been reluctant to exercise its gatekeeping function as a method of docket control.  Instead, it has moved towards greater use of pre-trial consolidations in multi-district litigations to achieve economies of scale.  The MDL trend, however, has its problems.  Placing responsibility for expert witness gatekeeping in the MDL court may be counter to its “pre-trial” rationale of the MDL statute.  Furthermore, exercising gatekeeping across hundreds or thousands of cases heightens and highlights the anxieties, fears, distaste, and institutional incompetence for deciding scientific issues. The move toward MDL handling has had the apparent result of diluting the gatekeeping mandate and reducing the use of summary dispositions.

The procedural and the validity goals of Daubert are quite independent.  Validity may have been, as Faigman argues, a secondary goal for the Justices, but it was a worthy goal in and of itself.  I believe Professor Faigman would agree.  In describing the Supreme Court’s path on validity, Faigman notes that there were two competing models of expert witness admissibility determinations that vied for acceptance:  Frye, and then Daubert.  Id. at 105. He likens Frye to nose counting among the “relevant” scientific community for support of the witness’s methodology.  All a trial judge need do is identify the relevant community and then to count the noses.  Daubert represented a possible alternative:

“to charge judges with the responsibility to consider the methods and principles underlying proffered expert opinion and have them make the validity determination.”

Id. at 105.  Making trial judges responsible for warranting the validity of scientific evidence, and ultimately all expert witness opinion testimony, was one of the important changes that resulted in the Revolution and its embrace of “good grounds” or epistemic validity:

“[p]roposed testimony must be supported by appropriate validation.”

Daubert v. Merrell Dow Pharm., 509 U.S. 579, 590 (1993)

Professor Faigman correctly observes that, although lawyers and lower court judges have obsessed over the so-called Daubert factors, the actual holding of Daubert was “the requirement that an expert’s testimony pertain to ‘scientific knowledge’ establishes a standard of evidentiary reliability.” Revolution at 111 (quoting Daubert, 509 U.S. at 590).  Despite the improvident dictum about focus on methodology and not on conclusions, the Supreme Court, in Daubert, had made clear that there are necessary implications of Rule 702’s requirement that expert witness testimony relate to specialized “knowledge”:

“This entails a preliminary assessment of whether the reasoning or methodology underlying the testimony is scientifically valid and of whether that reasoning or methodology properly can be applied to the facts in issue.”

Daubert, 509 U.S. at 592-93.

Professor Faigman writes to point out the erroneous interpretations and distortions of Daubert, its progeny, and Rule 702:

“The holding of Daubert is the requirement that judges find as a preliminary fact that the methods and principles underlying proffered expert testimony are sufficiently valid to support that testimony. The four ‘Daubert factors’ were offered as guidelines to help courts assess expert testimony.”

Revolution at 114.

Faigman’s writing is a useful reminder to those judges and commentators who would simplify and abridge the entire gatekeeping project into one or another dictum found in Daubert (or Joiner or Kumho Tire), and who ignore the actual holding of the cases, or the mandate of the subsequent statute. For those writers who try to evade the difficult scientific determinations and discriminations inherent in evaluating causal claims and other scientific opinions, Faigman reminds us that Justice Breyer, in his concurrence in Joiner, was not shy about pointing out that gatekeeping:

“will sometimes ask judges to make subtle and sophisticated determinations about scientific methodology and its relation to the conclusions an expert witness seeks to offer.”

General Electric Company v. Joiner, 522 U.S. 136, 118 S. Ct. 512, 520 (1997) (Breyer, J., concurring).  I take Faigman’s essay as an eloquent importuning of the judiciary to heed Justice Breyer, to stop whining, and to start learning.

It is a measure of Professor Faigman’s concern for the accuracy and validity of scientific testimony that he cannot bring himself to address a third way:  ignore validity, reliability, sufficiency, and simply allow expert witnesses to battle out.

This third way was what really prevailed before Daubert in much of civil litigation over health effects.  The Frye rule was rarely if ever applied to such cases, and most states excepted the opinion testimony of physicians, in any event.  Before Frye, we had whatever was dished up by ready, willing, able (and sufficiently glib) testifiers.  To be sure, expert witnesses had to be qualified, but the threshold was astonishingly low.  In Pennsylvania, for instance, the standard is that the putative “expert” must have “a reasonable pretense of expertise.” See, e.g., Ruzzi v. Butler Petroleum Co., 527 Pa. 1, 9-10 (1991); Kuisis v. Baldwin-Lima-Hamilton Corp.,457 Pa. 321, 319 A.2d 914 (1974)(“the witness must have a reasonable pretension to specialized knowledge on the subject under investigation”).  The federal courts were not far behind. Ferebee v. Chevron Chem. Co., 552 F. Supp. 1297 (D.D.C. 1982), aff’d, 736 F.2d 1529 (D.C. Cir.), cert. denied, 469 U.S. 1062 (1984).

Indeed, there is a pervasive, reactionary movement afoot, among judges and academic commentators to return to the wild, woolly days, celebrated in Ferebee’s famous dictum:

“On questions … which stand at the frontier of current medical and epidemiological inquiry, if experts are willing to testify that such a link exists, it is for the jury to decide whether to credit such testimony.”

Ferebee, 736 F.2d at 1534.  This third way then is simply to delegate to the expert witnesses themselves to assess the “weight of the evidence,” and offer up their opinions, without any scrutiny from the courts as to the validity or sufficiency of the bases for those opinions. This retrograde step is not just the stuff of naive law student musings. See, e.g., Note, “Admitting Doubt: A New Standard for Scientific Evidence,” 123 Harv. L. Rev. 2021 (2010). Reactionaries in the Academy and in the judiciary are intent to reduce gatekeeping to a weak test of relevancy, without any determination of content validity.

Posted in Expert Witnesses, Rule 702, Scientific Evidence, Trial Procedure | Comments Off on Professor Faigman on the Dual Goals of the Daubert Revolution

Milward Symposium Organized By Plaintiffs’ Counsel and Witnesses

February 16th, 2013

The criticisms of corporate free speech are motivated, at bottom, over hostility to the views that would likely flow from corporate speech. In this age, it is a marvel that there is such hostility to free expression of ideas.  We should be much more focused on validity and factual accuracy of arguments than on sponsorship.  Sometimes, even the most biased sources manage to stumble upon the truth.

Still, sponsorship remains a major debating point for those who cannot or will not take the time to evaluate the merits of an issue.  The Center for Progressive Reform (CPR) is, like many American corporations, a nonprofit organization, but it aspires to be a “research and educational organization.” The CPR’s principal aims deal with protecting health and safety against occupational and environmental harms.  These are laudable goals even if the CPR is predictably a voice of entrenched interest groups, such as the litigation industry, also known as the plaintiffs’ tort bar.

One of the CPR’s key activities is “[d]efending clean science from political or corporate interference.” The CPR raises interesting questions about what is “clean,” and what is “dirty” science, and whether it is willing to defend science from all political and corporate interference, or only that interference with which it disagrees.

The American litigation industry is represented by a highly politicized “corporation,” the American Association for Justice (AAJ), previously known by the more revealing name, Association of Trial Lawyers of America (ATLA®).   The AAJ describes itself as a corporation, or a “collective,” that supports plaintiff trial lawyers as their “collective voice … on Capitol Hill and in courthouses across the nation … .” The Robert A. Habush Foundation is endowed by the AAJ, and serves as an educational mission.  Through the Habush Foundation, the AAJ funds educational programs, “think tanks,” and writing projects designed to influence judges, law professors, lawyers, and the public, on issues of importance to the AAJ:  “the civil justice system and individual rights” for bigger, better, and more profitable litigation outcomes.

Of the two organizations, the CPR, and the AAJ/ATLA, the CPR has the more disinterested stance, in theory. The AAJ may be a “not-for-profit,” but it represents the interests of one of the most powerful, and wealthiest, interest groups in American society — the plaintiffs’ bar.

Last May, the CPR sponsored a symposium in Washington D.C. on one of the most controversial, and reactionary decisions involving federal gatekeeping of expert witness testimony, Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009), rev’d, 639 F.3d 11 (1st Cir. 2011), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012).  The CPR’s interest in the Milward decision is clear.  One of CPR’s member “scholars,” Carl Cranor, was a partisan expert witness in Milward.  The trial court had excluded Cranor’s testimony; a panel of the First Circuit of the Court of Appeals reversed and ordered that Cranor and the plaintiffs’ other expert witnesses be heard at trial.  See Milward — Unhinging the Courthouse Door to Dubious Scientific Evidence (Sept. 2, 2011); WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name (May 2, 2012).  The Milward decision embraced a vacuous methodology sometimes called “weight of the evidence” (WOE) or “inference to the best explanation,” which had been previously rejected by other Circuits, as well as by the United States Supreme Court, in General Electric Co. v. Joiner, 522 U.S. 136 (1997).

The agenda for the symposium, “Toxic Tort Litigation after Milward v. Acuity Specialty Products,” reflects the CPR’s role, in conjunction with the Wake Forest Journal of Law and Public Policy, in sponsoring the event.  The connection between the CPR and Wake Forest Law School may not be obvious.  CPR board member, Sid Shapiro, is a law professor at Wake Forest.  Shapiro and CPR member Thomas McGarity presented at the symposium. So did Professor Steve Gold, who has been an enthusiastic cheerleader for the Milward decision.  Law professors Michael Green and Joseph Sanders also presented.  There was only one practicing lawyer involved in the symposium, Texas plaintiffs’ lawyer, Steve Jensen, of Allen Stewart, P.C.  Mr. Jensen is a past chair of the AAJ’s Section on Toxic, Environmental, and Pharmaceutical Torts.

No defense counsel participated.

The proceedings of the Milward symposium will be published in an upcoming issue (volume 3, no. 1) of the Wake Forest Journal of Law and Public Policy.  This issue is scheduled to include papers from the presenters, along with one additional author, Carl Cranor.

The website of the Wake Forest Journal of Law & Policy describes the symposium:

In Milward v. Acuity Products, 639 F.3d 11 (1st Cir. 2011), the First Circuit became the first court – either federal or state – to allow a “weight of the evidence” methodology for assessing causation in a toxic tort case. The plaintiff had alleged that exposure to defendant’s benzene-containing products caused his rare leukemia (Acute Promyelocytic Leukemia (“APL”)).  His expert witness, a leading toxicologist and expert on benzene, surveyed five lines of scientific evidence from the peer-reviewed literature, and concluded the available evidence, taken as a whole, supported the inference that benzene exposure can cause APL.  The lower court, following a common post-Daubert approach, excluded the testimony “because no one line of evidence supported a reliable inference of causation, [and] an inference of causation based on the totality of the evidence was unreliable.”  The First Circuit rejected this “atomistic” approach, noting that the district court did not have the authority to exclude evidence because reasonable experts may disagree about what it means.

This symposium will explore the implications of Milward for toxic tort litigation in the federal and state courts, including whether it correctly applies Daubert v. Merrell Dow Pharmaceuticals, Inc., 509, U.S. 579 (1993).  * * * Speakers are invited to comment on any aspect on Milward that they find interesting and important, including the following issues:

  1. What role does the weight of the evidence methodology play in scientific risk assessment, and what are the implications of this role for tort litigation?
  2. Should well-founded testimony based on a weight of the evidence methodology be admissible in toxic tort litigation?  Does the Restatement of Torts (Third) § 28 cmt. c concerning the role of scientific judgment in adjudicating general causation support the admissibility of such testimony?
  3. Does the reliance of regulatory agencies, such as EPA, on a weight of the evidence methodology for purposes of regulating toxic chemicals support the result in Milward?  Or are there differences in the legal and policy judgments being made by regulators and judges that distinguish regulatory agencies from courts?
  4. What are the legal and policy implications of Milward for the future of toxic tort litigation?

The Journal also notes the sponsorship of the CPR, and, in a cryptic paragraph, reports that

“CPR thanks the Robert L. Habush Foundation for its support of the symposium.”

Most casual readers will not likely recognize the Habush foundation for what it is: an arm of AAJ/ATLA.  I suppose it was too painful for Wake Forest or the CPR to acknowledge openly that the litigation industry itself supported this symposium.  Some may find irony in the CPR’s past criticism of Citizens United v. Federal Election Commission, 558 U.S. 310 (2010), when its current status as a conduit for litigation industry money to support scholarship in that industry’s interests.  See Daniel Farber, “Of the Corporations, By the Corporations, For the Corporations? The Meaning of the Citizens United Decision” (Jan. 21, 2010).

Professor Steve Gold’s paper, “When Certainty Dissolves into Probability: A Legal Vision of Toxic Causation for the Post-Genomic Era,” has been posted at the Social Science Research Network.  Steve Jensen’s contribution to the symposium has been published as well, in the AAJ’s trade journal.  Steve Baughman Jensen, “Reframing the Daubert Issue in Toxic Tort Cases,” Trial (Feb. 2013).

Posted in Conflicts of Interest, Expert Witnesses, Rule 702 | Comments Off on Milward Symposium Organized By Plaintiffs’ Counsel and Witnesses

Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

December 27th, 2012

A recent student law review article discusses reanalyses of epidemiologic studies, an important, and overlooked topic in the jurisprudence of scientific evidence.  Alexander J. Bandza, “Epidemiological-Study Reanalyses and Daubert: A Modest Proposal to Level the Playing Field in Toxic Tort Litigation,” 39 Ecology L. Q. 247 (2012).

In the Daubert case itself, the Ninth Circuit, speaking through Judge Kozinksi, avoided the methodological issues raised by Shanna Swan’s reanalysis of Bendectin epidemiologic studies, by assuming arguendo its validity, and holding that the small relative risk yielded by the reanalysis would not support a jury verdict of specific causation. Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1317–18 (9th Cir. 1995).

There is much that can, and should, be said about reanalyses in litigation and in the scientific process, but Bandza never really gets down to the business at hand. His 36 page article curiously does not begin to address reanalysis until the bottom of the 20th page. The first half of the article, and then some, reviews some time-worn insights and factoids about scientific evidence. Finally, at page 266, the author introduces and defines reanalysis:

“Reanalysis occurs ‘when a person other than the original investigator obtains an epidemiologic data set and conducts analyses to evaluate the quality, reliability or validity of the dataset, methods, results or conclusions reported by the original investigator’.”

Bandza at 266 (quoting Raymond Neutra et al., “Toward Guidelines for the Ethical Reanalysis and Reinterpretation of Another’s Research,” 17 Epidemiology 335, 335 (2006).

Bandza correctly identifies some of the bases for judicial hostility to re-analyses. For instance, some courts are troubled or confused when expert witnesses disagree with, or reevaluate, the conclusions of a published article. The witnesses’ conclusions may not be published or peer reviewed, and thus the proffered testimony fails one of the Daubert factors.  Bandza correctly notes that peer review is greatly overrated by judges. Bandza at 270. I would add that peer review is an inappropriate proxy for validity, a “test,” which reflects a distrust of the unpublished.  Unfortunately, this judicial factor ignores the poor quality of much of what is published, and the extreme variability in the peer review process. Judges overrate peer review because they are desperate for a proxy for validity of the studies relied upon, which will allow them to pass their gatekeeping responsibility on to the jury. Furthermore, the authors’ own conclusions are hearsay, and their qualifications are often not fully before the court.  What is important is the opinion of the expert witness who can be cross-examined and challenged.  See “FOLLOW THE DATA, NOT THE DISCUSSION.” What counts is the validity of the expert witness’s reasoning and inferences.

Bandza’s article, which by title advertises itself to be about re-analyses, gives only a few examples of re-analyses without much detail.  He notes concerns that reanalyses may impugn the reputation of published scientists, and burden them with defending their data.  Who would have it any other way? After this short discussion, the article careens into a discussion of “weight of the evidence” (WOE) methodology. Bandza tells us that the rejection of re-analyses in judicial proceedings “implicitly rules out using the weight-of-the-evidence methodology often appropriate for, or even necessary to, scientific analysis of potentially toxic substances.” Bandza at 270.  This argument, however, is one sustained non-sequitur.  WOE is defined in several ways, but none of the definitions require or suggest the incorporation of re-analyses. Re-analyses raise reliability and validity issues regardless whether an expert witness incorporates them into a WOE assessment. Yet Bandza tells us that the rejection of re-analyses “Implicitly Ignores the Weight-of-the-Evidence Methodology Appropriate for the Scientific Analysis of Potentially Toxic Substances.” Bandza at 274. This conclusion simply does not follow from the nature of WOE methodology or reanalyses.

Bandza’s ipse dixit raises the independent issue whether WOE methodology is appropriate for scientific analysis. WOE is described as embraced or used by regulatory agencies, but that description hardly recommends the methodology as the basis for a scientific, as opposed to a regulatory, conclusion.  Furthermore, Bandza ignores the ambiguity and variability of WOE by referring to it as a methodology, when in reality, WOE is used to describe a wide variety of methods of reasoning to a conclusion. Bandza cites Douglas Weed’s article on WOE, but fails to come to grips with the serious objections raised by Weed in his article to the use of WOE methodologies.  Douglas Weed, “Weight of Evidence: A Review of Concept and Methods,” 25 Risk Analysis 1545, 1546–52 (2005) (describing the vagueness and imprecision of WOE methodologies). See also “WOE-fully Inadequate Methodology – An Ipse Dixit By Another Name.”

Bandza concludes his article with a hymn to the First Circuit’s decision in Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir. 2011). Plaintiffs’ expert witness, Dr. Martyn Smith claimed to have performed a WOE analysis, which in turn was based upon a re-analysis of several epidemiologic studies. True, true, and immaterial.  The re-analyses were not inherently a part of a WOE approach. Presumably, Smith re-analyzed some of the epidemiologic studies because he felt that the data as presented did not support his desired conclusion.  Given the motivations at work, the district court in Milward was correct to look skeptically and critically at the re-analyses.

Bandza notes that there are procedural and evidentiary safeguards in federal court against unreliable or invalid re-analyses of epidemiologic studies.  Bandza at 277. Yes, there are safeguards but they help only when they are actually used. The First Circuit in Milward reversed the district court for looking too closely at the re-analyses, spouting the chestnut that the objections went to the weight not the admissibility of the evidence.  Bandza embraces the rhetoric of the Circuit, but he offers no description or analysis of the liberties that Martyn Smith took with the data, or the reasonableness of Smith’s reliance upon the re-analyzed data.

There is no necessary connection between WOE methodologies and re-analyses of epidemiologic studies.  Re-analyses can be done properly to support or deconstruct the conclusions of published papers.  As Bandza points out, some re-analyses may go on to be peer reviewed and published themselves.  Validity is the key, and WOE methodologies have little to do with the process of evaluating the original or the re-analyzed study.

 

 

Posted in Causation, Data Sharing, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Reanalysis of Epidemiologic Studies – Not Intrinsically WOEful

Litmus Tests

December 27th, 2012

Rule 702 is, or is not, a litmus test for expert witness opinion admissibility.  Relative risk is, or is not, a litmus test for specific causation.  Statistical significance is, or is not, a litmus test for reasonable reliance upon the results of a study.  It is relatively easy to find judicial opinions on either side of the litmus divide.  Compare National Judicial College, Resource Guide for Managing Complex Litigation at 57 (2010) (Daubert is not a litmus test) with Cryer v. Werner Enterprises, Inc., Civ. Action No. 05-S-696-NE, Mem. Op. & Order at 16 n. 63 (N.D. Ala. Dec. 28, 2007) (describing the Eleventh Circuit’s restatement of Rule 702’s “litmus test” for the methodological reliability of proffered expert witness opinion testimony).

The “litmus test“ is one sorry, overworked metaphor.  Perhaps its appeal has to do with a vague collective memory that litmus paper is one of those “things of science,” which we used in high school chemistry, and never had occasion to use again. Perhaps, litmus tests have the appeal of “proofiness.”

The reality is different. The litmus test is a semi-quantitative test for acidity or alkalinity.  Neutral litmus is purple.  Under acidic conditions, litmus turns red; under basic conditions, it turns blue.  For some time, scientists have used pH meters when they want a precise quantification of acidity or alkalinity.  Litmus paper is a fairly crude test, which easily discriminates  moderate acidity from alkalinity (say pH 4 from pH 11), but is relatively useless for detecting an acidity at pH or 6.95, or alkalinity at 7.05.

So what exactly are legal authors trying to say when they say that some feature of a test is, or is not, a “litmus test”? The litmus test is accurate, but not precise at the important boundary at neutrality.  The litmus test color can be interpreted for degree of acidity or alkalinity, but it is not the preferred method to obtain a precise measurement. Saying that a judicial candidate’s views on abortion are a litmus test for the Senate’s evaluation of the candidate makes sense, given the relative binary nature of the outcome of a litmus test, and the polarization of political views on abortion. Apparently, neutral views or views close to neutrality on abortion are not a desideratum for judicial candidates.  A cruder, binary test is exactly what is desired by politicians.

The litmus test that is used for judicial candidates does not seem to work so well when used to describe scientific or statistical inference.  The litmus test is well understood, but fairly obsolete in modern laboratory practice.  When courts say things, such as statistical significance is not a litmus test for acceptability of a study’s results, clearly they are correct because measure of random error is only one aspect of judging a body of evidence for, or against, an association.  Yet courts seem to imply something else, at least at times:

statistical significance is not an important showing in making a case that an exposure is reliably associated with a particular outcome.

Here courts are trading in half truths.  Statistical significance is quantitative, and the choice of a level of significance is not based upon immutable law. So like the slight difference between a pH of 6.95 and 7.05, statistical significance tests have a boundary issue.  Nonetheless, a consideration of random error cannot be dismissed or overlooked on the theory that significance level is not a “litmus test.”  This metaphor obscures and attempts to excuse sloppy thinking.  It is time to move beyond this metaphor.

Posted in Rule 702, Scientific Evidence, statistical evidence | Comments Off on Litmus Tests

« Older Entries
Newer Entries »