TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

The Lie Detector and Wonder Woman – Quirks and Quacks of Legal History

January 27th, 2015

From 1923, until the United States Supreme Court decided the Daubert case in 1993, Frye was cited as “controlling authority” on questions of the admissibility of scientific opinion testimony and test results. The decision is infuriatingly cryptic and unhelpful as to background or context of the specific case, as well as how it might be applied to future controversies. Of the 669 words, these are typically cited as the guiding “rule” with respect to expert witness opinion testimony:

“Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while the courts will go a long way in admitting expert testimony deduced from a well recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs.”

Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).

As most scholars of evidence realize, the back story of the Frye case is rich and bizarre. The expert witness involved, William Marston, was a lawyer and scientist, who had made advances in a systolic blood pressure cuff to be used as a “lie detector.” Marston was also an advocate of free love and, with his wife and his mistress, the inventor of Wonder Woman and her lasso of truth.

Jill Lepore, a professor of history in Harvard University, has written an historical account of Marston and his colleagues. Jill Lepore, The Secret History of Wonder Woman (N.Y. 2014). More recently, Lepore has written an important law review on the historical and legal record of the Frye case, which is concealed in the terse 669 words of the Court of Appeals’ opinion. Jill Lepore, “On Evidence: Proving Frye as a Matter of Law, Science, and History,” 124 Yale L.J. 1092 (2015).

Lepore’s history is an important gloss on the Frye case, but her paper points to a larger, more prevalent, chronic problem in the law, which especially afflicts judicial decisions of scientific or technical issues. As an historian, Lepore is troubled, as we all should be, by the censoring, selecting, suppressing, and distorting of facts that go into judicial decisions. From cases and their holdings, lawyers are taught to infer rules that guide their conduct at the bar, and their clients’ conduct and expectations, but everyone requires fair access to the evidence to determine what facts are material to decision.

As Professor Lepore puts it:

“Marston is missing from Frye because the law of evidence, case law, the case method, and the conventions of legal scholarship — together, and relentlessly — hide facts.”

Id. at 1097. Generalizing from Marston and the Frye case, Lepore notes that:

“Case law is like that, too, except that it doesn’t only fail to notice details; it conceals them.”

Id. at 1099.

Lepore documents that Marston’s psychological research was rife with cherry picking and data dredging. Id. at 1113-14. Despite his degree magna cum laude in philosophy from Harvard College, his L.L.B from Harvard Law School (with no particular distinction), and his Ph.D. from Harvard University, Marston was not a rigorous scientist. In exploring the historical and legal record, not recounted in the Frye decision, Lepore’s history provides a wonderful early example, of what has become a familiar phenomenon of modern litigation: an expert witness who seeks to achieve acceptance for a dubious opinion or device in the courtroom rather than in the court of scientific opinion. Id. at 1122. The trial judge in Frye’s murder case, Justice McCoy, was an astute judge, and quite modest in his ability to evaluate the validity of Marston’s opinions, but he had more than sufficient perspicacity to discern that Marston’s investigation was “wildly unscientific,” with no control groups. Id. at 1135. The trial record of defense counsel’s proffer, and Justice McCoy’s rulings and comments from the bench, reproduced in Lepore’s article, anticipate and predict much of the scholarship surrounding both Frye and Daubert cases.

Lepore complains that the important historical record, including Marston’s correspondence with Professor Wigmore, the criminal fraud charges against Marston, and the correspondence of Frye’s lawyers, lies “filed, undigitized” in various archives. Id. at 1150. Although Professor Lepore tirelessly cites to internet URL sources when available, she could have easily made the primary historical materials available for all, using readily available internet technology. Lepore’s main thesis should encourage lawyers and law scholars to look beyond appellate decisions as the data for legal analysis.

Posted in Expert Witnesses, Frye, Scientific Evidence | Comments Off on The Lie Detector and Wonder Woman – Quirks and Quacks of Legal History

Pennsylvania Superior Court Takes The Bite Out of Fixodent Claims

December 12th, 2013

In the spring of 2012, Judge Sandra Mazer Moss granted summary judgment to Proctor & Gamble, after excluding, on Frye grounds, plaintiff’s expert witnesses who opined that plaintiff suffered zinc neurotoxicity from his use of Fixodent.  Jacoby v. Rite Aid Corp., 2012 Phila. Ct. Com. Pl. LEXIS 208 (2012).  See “Philadelphia Plaintiff’s Claims Against Fixodent Prove Toothless” (May 2, 2012).  Judge Moss’s exclusion of plaintiff’s expert witnesses involved a careful analysis of the evasive, hand-waving tactics of the witnesses.  Among the plaintiff’s team of expert witnesses was Dr. Martyn Smith, the chief hand waver and obscurantist in Milward v. Acuity Specialty Products Group, Inc., 664 F.Supp. 2d 137 (D. Mass. 2009), rev’d, 639 F.3d 11 (1st Cir. 2011), cert. denied, U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012).

On Monday, December 9, 2013, after a careful review, the Pennsylvania Superior Court affirmed summary judgment for Proctor & Gamble in the Jacoby case. Jacoby v. Rite Aid Corp., Pa. Super. Ct. No. 1508 EDA 2012 (Dec. 9, 2013) [Slip op.]  The Superior Court panel, consisting of Judges Stevens, Lazarus, and Colville, largely adopted Judge Moss’s analysis and affirmed in a signed, but unpublished, opinion by Judge Lazarus. 

Like Judge Moss before them, the Panel saw through the attempt to pass off “Weight of the Evidence” (WOE) and “Totality of the Evidence” (TOE) as scientific methodologies.  The witnesses, Martyn Smith and others, failed to specify what evidence they weighed, how they weighed the evidence, and what the weights supposedly were.  Another expert witness vaguely pointed to the “Naranjo scale” in support of interpreting case reports to show causal association, but this scale was similarly incompetent other than as a crude “plausibility” scale for assessing case reports.

The Superior Court’s decision in Jacoby is noteworthy on several important issues. There was no material issue as to whether zinc at some dose and duration of exposure can cause neuropathies.  The Court saw, however, that the important issue was whether zinc in the form, dose, and duration ingested by plaintiff can cause the outcome he experienced, and whether his exposure to zinc in Fix-o-dent actually caused his alleged injury.  The Superior Court re-affirmed Pennsylvania’s case law that makes extrapolation from different doses, different durations, and different biological circumstances, a “novel” claim that is subject to the gatekeeping by the so-called Frye standard. Slip op. at 11.

The Superior Court’s opinion astutely observed that the issue was not whether WOE and TOE are accepted scientific methodologies, but whether expert witness Martyn Smith can “evade a reasoned Frye inquiry merely by making reference to accepted methods in the abstract.”  Id. at 12 (citing Betz, at 58).  When pressed, Martyn Smith’s invocation of WOE amounted to little more than a distortion and abridgment of the Bradford Hill factors.  The Superior Court recognized, however, that the Bradford Hill guidelines provide an evaluative process to consider whether an association, after first being shown to be clear cut and not attributable to chance, is causal or spurious.  Id. at 13.  As Bradford Hill postulated the question that arises before his famous nine factors come into the analysis:

“Disregarding then any such problem in semantics we have this situation. Our observations reveal an association between two variables, perfectly clear-cut and beyond what we would care to attribute to the play of chance. What aspects of that association should we especially consider before deciding that the most likely interpretation of it is causation?”

Austin Bradford Hill, “The Environment and Disease: Association or Causation?” 58 Proc. Royal Soc’y Med. 295, 295 (1965) (emphasis added).

In this case, Smith never got off the dime with his evasive tactics.  He did not identify a quantified association that could be assessed for the play of random variation, or bias and confounding.  Smith tried to suggest that a mere possibility of an association was sufficient to invoke the Bradford Hill guidelines, but the Superior Court rejected this attempt to invent a new-age scientific method. Slip op. at 13.

The Superior Court acknowledged that the scientific literature describes WOE as varying from nothing more than “seat-of-the-pants qualitative assessment” to “aggregating diverse modalities.”  Id. at 14.  In Smith’s hands, WOE was little more than a personal, subjective opinion, an ipse dixit dressed as a scientific opinion.  Smith never defined his WOE approach, and the other witnesses never defined their TOE approach.  The plaintiffs’ witnesses in Jacoby failed to offer an accepted methodology when they failed to identify the forms of evidence they considered, and how they went about weighing the evidence upon which they had relied.  Id. at 15.

In a brief discussion, the Panel also embraced another basic evidentiary principle to dismiss a common tactic in specious claiming. The plaintiffs’ challenged defendants’ pharmokinetic study and tried to suggest that their deconstruction counted as affirmative evidence to support their own theory of biological fate and distribution. Id. at 16.  The Superior Court saw through the ruse; the plaintiffs had not created affirmative evidence for their theory by arguing that the defendants’ study was flawed. 

The Superior Court squarely confronted the limitations and inadequacy of relying upon descriptive, anecdotal case reports. Case reports provide a narrative and temporal history of events with respect to exposure and outcome, but they cannot fully account for confounding by known and unknown factors.  Case reports represent post hoc assessments that were not planned, and therefore lack data that would permit distinguishing coincidence from causality.  Id. at 17 (citing Dr. Lorene Nelson’s report).  See In re Denture Cream Prods. Liab. Litig., 795 F. Supp. 2d 1345 (S.D. Fla.2011).

Proctor & Gamble had the good fortunate to have obtained a good ruling in the MDL litigation in the Southern District of Florida, which no doubt helped focus the gatekeeping process in Pennsylvania state court. Unfortunately, the Superior Court Panel chose not to publish its decision.  This decision is regrettable for its inconsistency with the transparency and due process expected of all courts.  See Erica Weisgerber, “Unpublished Opinions: A Convenient Means to an Unconstitutional End,” 97 Georgetown L.J. 621 (2009).

Posted in Causation, Expert Witnesses, Frye, Scientific Evidence | Comments Off on Pennsylvania Superior Court Takes The Bite Out of Fixodent Claims

As They WOE, So No Recovery Have the Reeps

May 22nd, 2013

Late last year, Justice York excluded Dr. Shira Kramer’s WOE-ful opinion that gasoline fumes from an alleged fuel-line leak caused Sean Reep to be born with cerebral palsy.  Reeps v. BMW of North America, LLC, 2012 NY Slip Op 33030(U), N.Y.S.Ct., Index No. 100725/08 (New York Cty. Dec. 21, 2012) (York, J.).  Kramer’s opinion was a parody of science, pieced together from case reports, animal studies, and epidemiologic studies that looked at exposures utterly unlike that of Mrs. Reep’s exposure.

Justice York saw through the charade.  The animal studies were largely exonerative. The case reports were of birth defects quite different from those sustained by Sean Reeps.  The epidemiologic studies were of different chemicals or chemicals at levels very different from those experienced by Mrs. Reeps. Plaintiffs’ expert witnesses ignored established principles of teratology in claiming late-term birth defects to have been causally related to early term exposures. Plaintiffs’ expert witnesses gave a convincing presentation of how not to do science, and why judicial gatekeeping is necessary.  See “New York Breathes Life into Frye Standard – Reeps v. BMW” (Mar. 5, 2013).

Justice York clearly articulated that the “plaintiff’s burden to prove the methodology applied to reach the conclusions will not be rejected by specialists in the field.”  Reeps, slip op. at 11.  The trial court recognized that under the New York state version of Frye, the court must determine whether plaintiffs’ expert witnesses are faithfully applying a methodology, such as the Bradford Hill criteria, or whether they are they are “pay[ing] lip service to them while pursuing a completely different enterprise.”  Id.  Justice York recognized that the court must examine a proffered opinion to determine whether it “properly relates existing data, studies or literature to the plaintiff’s situation, or whether, instead, it is connected to existing data only by the ipse dixit of the expert.” Id. (internal quotations omitted).

Plaintiffs were unhappy with Justice York’s decision, and their counsel moved for reconsideration, positing only 15 supposed errors or misunderstandings in the opinion. On May 10, the trial court denied the motion for reconsideration and further explicated the scientific deficiencies of plaintiffs’ witnesses’ opinions.

The trial court was unimpressed:

“In general, attorney for plaintiffs misrepresents the substance of this court’s Decision. The court did not prefer conclusions of defendants’ experts to that of plaintiffs – disagreement among experts is to be expected, since causation analysis involves professional judgment in interpreting data and literature. An expert opinion is precluded when it is reached in violation of generally accepted scientific principles. The court determined that Drs. Kramer and Frazier did not follow generally accepted scientific methodology.”

Reeps, 2013 NY Slip Op 31055(U) at 2 (Opinion on Motions to Reargue, to Renew, and for Oral Hearing) (May 10, 2013).

The court noted that the plaintiffs’ witnesses’ novel claim that low-level gasoline vapor inhalation causes birth defects, a claim that had escaped the attention of all other scientists and regulatory agencies, cried out for judicial intervention.  Id. at 3.

The court also rebuffed the claim that plaintiffs’ witnesses, Shira Kramer and Linda Frazier, had followed the Bradford Hill guidelines:

 “These guidelines are employed only after a study finds an association to determine whether that association reflects a true causal relationship.”

Id. at 5 (quoting Federal Judicial Center, National Research Council, Reference Manual on Scientific Evidence at 598-599 (3d ed. 2011)) (emphasis in the original). Kramer and Frazier never got off the dime with the Bradford Hill guidelines.

In considering the plaintiffs’ motions, the trial court also had occasion to revisit the assertion that “weight of the evidence” (WOE) substituted for, or counted as, a scientific basis for a conclusion of causality:

“The metaphorical use of the term is, if nothing else, ‘a colorful way to say the body of evidence we have examined and judged using a method we have not described but could be more or less inferred from a careful between-the-lines reading of our paper’.”

Id. at 5 (quoting Douglas Weed, “Weight of Evidence: A Review of Concept and Methods,” 25 Risk Analysis 1545, 1546-47 (2005).

Unmoved by the sophistical hand waving, the court emphasized that Kramer and Frazier had confused “suggestive” evidence with “conclusions,” and they had misrepresented the meaning and significance of threshold limit values.  All in all, a convincing demonstration of the need for, and the judicial competence to carry out, gatekeeping of expert witness opinion testimony.

Posted in Causation, Expert Witnesses, Frye, Scientific Evidence | Comments Off on As They WOE, So No Recovery Have the Reeps

Washington Supreme Court Illustrates the Difference Between Frye and Rule 702

April 15th, 2013

Last month, the Washington Supreme Court decided a public and private nuisance case against a local utility for fear of future illnesses from exposure to electro-magnetic frequency radiation (EMF).  Lakey v. Puget Sound Energy, Inc., ___ Wn. 2d ___ , 2013 WL 865468 (Mar. 6, 2013).

The defendant utility and local municipality had moved for the exclusion of plaintiffs’ expert witnesses, under Frye v. United States, 54 App. D.C. 46, 293 F. 1013 (1923), and the Washington state version of Rule 702.  The plaintiff homeowners submitted declarations from two expert witnesses, Dr. De Kun Li and and Dr. David Carpenter.

Dr. Carpenter was the Director of the Institute for Health and the Environment at the University at Albany and a Professor of Environmental Health Sciences within the School of Public Health. He also has held the position of Dean of the School of Public Health at the University of Albany and the Director of the Wadsworth Center for Laboratories and Research of the New York State Department of Health.  Dr. Carpenter apparently has been active in testifying about EMF health issues.  See, e.g., Amended Declaration of David Carpenter (2011).

The trial court conducted a three-day Frye hearing, at which both sides offered expert witness testimony.  Plaintiffs called Carpenter, and the defendants called Dr. Nancy Lee, an epidemiologist, and Dr. Mark Israel, a professor at Dartmouth’s Geisel School of Medicine, and a specialist in the molecular and cellular biology of brain tumors.

At the hearing, Dr. Lee testified that Carpenter had failed to follow generally accepted epidemiologic methodology of considering all pertinent data.  Carpenter had “cherry picked”; his opinion selectively ignored studies that contradicted his conclusions.  Even among the studies that Carpenter had relied upon, he ignored data that undermined or contradicted his conclusion.  Slip op. at 6.  Furthermore, Carpenter ignored the toxicologic data and studies available. Id.  Among the studies ignored by Carpenter were the most recent, and most carefully conducted, studies. Id.  The trial court excluded Li and Carpenter, and the plaintiffs appealed.

Affirming the exclusion, the Supreme Court of Washington engaged in a curious two step.  The Court reversed the trial court’s exclusion on Frye grounds.  In Supreme Court’s view, the plaintiffs’ witnesses sufficiently deployed a generally accepted method when they waded into the field of epidemiology because epidemiology is generally accepted.  The Frye doctrine does not require that expert witnesses apply the established methodology well, reliably, or soundly.  The expert witnesses are immunized from exclusion by relying upon studies from a generally accepted discipline, no matter how poorly or selectively they have analyzed and interpreted these studies.  An expert witness’s errors in engaging with a particular scientific discipline that is in general considered generally accepted “go to the weight, not the admissibility, of the evidence unless the error renders the evidence unreliable.” Slip op. at 11.

Carpenter’s cherry-picking approach to data and studies, however, was properly excluded under Rule 702.  Carpenter’s approach vitiated the reliability of his opinion with the consequence of :

“seriously tainting his conclusions because epidemiology is an iterative science relying on later studies to refine earlier studies in order to reach better and more accurate conclusions. Carpenter refused to account for the data from the toxicological studies, which epidemiological methodology requires unless the evidence for the link between exposure and disease is unequivocal and strong, which is not the case here. Carpenter also selectively sampled data within one of the studies he used, taking data indicating an EMF-illness link and ignoring the larger pool of data within the study that showed no such link, Carpenter’s treatment of this data created an improper false impression about what the study actually showed.”

Slip op. at 12.

So that’s the difference between Frye and Rule 702!

Posted in Expert Witnesses, Frye, Rule 702 | Comments Off on Washington Supreme Court Illustrates the Difference Between Frye and Rule 702

U.C. Davis Symposium on the “Daubert” Hearing

March 11th, 2013

Professor David Faigman’s recent article, “The Daubert Revolution and the Birth of Modernity:  Managing Scientific Evidence in the Age of Science,” 102 U.C. Davis Law Rev. 101 (2013), notes that the paper grew out of informal remarks given at a recent symposium.  A little Googling quickly turned up the Symposium Site on the University of California, Davis, website.  Like Professor Faigman’s paper, this symposium is a valuable contribution to the art and learning of what Rule 702 hearings should, and should not, be.

The symposium, The Daubert Hearing — From All the Critical Perspectives (March 2, 2012) described itself:

Pretrial practice has long been the center of gravity in modern litigation. The vast majority of cases never go to trial. Instead, after pretrial discovery and in limine motions, the cases settle. The Supreme Court’s celebrated 1993 decision in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, has solidified that trend. In Daubert, the Court abandoned the traditional general acceptance standard for the admissibility of scientific testimony and announced a new empirical validation test. Throughout the country counsel began basing pretrial in limine motions on Daubert to target opposition expert testimony. In criminal cases, defense counsel started challenging the prosecution’s forensic evidence identifying the accused as the perpetrator. In civil tort cases, defense counsel filed motions attacking the plaintiff’s evidence on general causation. When counsel won these motions, the opposition lacked sufficient evidence to go to trial. The hearing on the pretrial Daubert motion became the centerpiece of the litigation.

This symposium will begin with a demonstration Daubert hearing. After the demonstration, all the participants will deliver remarks, giving their perspective on the law and tactics of Daubert hearings. In addition, there will be expert academic commentary by Professor David Faigman of U.C. Hastings School of Law, the lead author of the popular treatise, MODERN SCIENTIFIC EVIDENCE.

The symposium featured a list of distinguished speakers:

Hon. James M. Rosenbaum

Robert G. Smith

Bert Black

Professor David L. Faigman

Professor Edward Imwinkelried

Dr. William A. Toscano, Jr.

Dr. Sander Greenland

The symposium’s hypothetical is available on line, and the symposium itself, which was video recorded, is available for viewing at the UC Davis website.

The scientists who role-played as expert witnesses, Drs. Toscano and Greenland,  were obviously pushed to articulate certain positions that they did not personally subscribe to.  Still, their true colors managed to show, and to influence the mock hearing.  For instance, Dr. Toscano stated several times that causation is very difficult to prove, and in so stating, he managed to convey the impression that he had a personal, subjective higher bar for causal claims than the rest of the scientific community.  This approach is a common rookie mistake for defense counsel and their expert witnesses, and it should be avoided.  There are plenty of good examples of causal relationship that have been established with epidemiology, and the defense expert should be prepared to identify them, and to explain why in some cases, the causal relationships required more exacting evidence.  The other glaring error in the defense presentation was that the exact methodological error was not made clear through Dr. Toscano’s testimony although the defense lawyer, Mr. Smith, explored the gaps and leaps of faith in his cross-examination of Dr. Greenland.  In this setting, the defense expert witness’s focus is on the methodological inadequacies of the plaintiffs’ witness, not on why he rejected the causal claim.

Dr. Greenland was his inimitable self, even going so far as to talk into his magic marker under the impression that it was a microphone.  Who knows; perhaps it was, but it also wrote on the white board.  More telling was that Dr. Greenland embraced a probabilistic conception of causation, which he explained was essentially a bet on the correct result.  This metaphor seems fatally defective.  A bet is a bet, but you cannot call the bet until you have actual evidence of who won.  It may be lovely that Dr. Greenland, or some other expert witness, is willing to place the bet, perhaps with odds, but this metaphor fails to take causal inference out of the subjective realm.  Along with his betting metaphor, Greenland emphasized that the causation decision is driven by a cost-benefit analysis of Type I and II errors.  The slippery slide into substituting the precautionary principle for causal analysis was obvious.

On the plaintiffs’ side, Bert Black, an apostate defense lawyer, did a very good job of portraying the shenanigans used by plaintiffs’ lawyers to avoid and evade gatekeeping.  Statistical significance is not necessary; epidemiology is not necessary; Bradford Hill factors are not necessary; therefore, I can show causation without much of anything.  Black illustrated nicely how the focus is redirected to other cases, such as when someone from a drug company wrote an improvident article that concludes causation from case reports alone.  Or cases involving signature diseases, or acute outbreaks, for which causal relations were discerned and embraced by scientists on the basis of very informal epidemiologic studies or even case series (which someone characterized as anecdata).

Former federal judge James Rosenbaum presided magisterially, and cowardly denied the cross-motion Rule 702 challenges.  In his comments after the mock, Judge Rosenbaum revealed his conception of the gatekeeping process as essentially a determination that the witness is competent.  Of course this is not the law, and much more is required than to determine that the witness is minimally qualified.  Professor Faigman respectfully chastised the judge for ignoring the statute and the caselaw.

One of the more interesting dialogues in the discussions after the mock centered on the harm to an expert witness’s reputational interests from the gatekeeping process.  To be sure there can be such harm, but as Professional Faigman pointed out, the potential for such harm cannot intimidate judges from ruling on the facts and law before them.  I believe though that expert witnesses should be aware of the potential for this sort of harm from their testimonial adventures, and should require certain contractual assurances from the lawyers who engage them.  For instance, expert witnesses should insist upon whether such challenges are possible in the jurisdiction, what the standards are, and whether they will have an opportunity to speak to the challenges.  The expert witnesses should insist upon prompt notification of all such challenges, and upon prompt receipt of all briefs and affidavits that challenge the validity or reliability of their opinions, as well as an opportunity to be heard on their responses.

 

 

 

 

Posted in Expert Witnesses, Frye, Rule 702, Uncategorized | Comments Off on U.C. Davis Symposium on the “Daubert” Hearing

New York Breathes Life Into Frye Standard – Reeps v. BMW

March 5th, 2013

In Lumpenepidemiology, I detailed how one federal judge, the Hon. Helen Berrigan, was willing to “just say no” to bad epidemiology and bad science, and to shut an expert witness’s attempt to distort and subvert scientific methodology.  Judge Berrigan closely examined the plaintiffs’ claim that a mother’s ingestion of Paxil caused a child’s heart defect, and found the proffered expert witness testimony to fail legal and scientific standards. Frischhertz v. SmithKline Beecham Corp., 2012 U.S. Dist. LEXIS 181507 (E.D.La. 2012).  The plaintiffs’ key expert witness, Dr. Shira Kramer, attempted to provide plaintiffs with a necessary association by “lumping” all birth defects together in her analysis of epidemiologic data of birth defects among children of women who had ingested Paxil (or other SSRIs).  Given the clear evidence that different birth defects arise at different times, based upon interference with different embryological processes, the trial court discerned this “lumping” of end points to be methodologically inappropriate.  Id. at *13 (citing Chamber v. Exxon Corp., 81 F. Supp. 2d 661 (M.D. La. 2000), aff’d, 247 F.3d 240 (5th Cir. 2001).

Frischhertz was decided in December 2012, the same month that another trial judge, right here in New York City, caught Dr. Shira Kramer in the commission of similar lumpenepidemiology, in Reeps v. BMW of North America, LLC, New York S.Ct., Index No. 100725/08 (New York Cty. Dec. 21, 2012) (York, J.).   See William Ruskin, “Frye Decision in BMW Case Results in Exclusion of Plaintiff’s Experts” (Jan. 17, 2013). Reeps was also a birth defects case.  Debra Reeps claimed that during the first trimester of her pregnancy, she was exposed to gasoline fumes from a fuel-line leak in her BMS 525i. She also claimed that her son’s adverse birth outcomes (which included severe mental retardation, severe cerebral palsy, and a congenital heart defect) were caused by her inhalation of gasoline fumes.  Heading the plaintiffs’ team of expert witnesses in support of these claims, Epidemiologist Shira Kramer opined that all of the boy’s problems were caused by the mother’s exposure to unleaded gasoline fumes.

Kramer’s opinions read like the Berenstain Bears’ guide to epidemiology.  She asserted that gasoline vapors and  its constituents (toluene, benzene, solvents, etc.), individually or collectively, cause “birth defects” generally, and Sean Reeps’ defects specifically.  Kramer also asserted that she used a “weight-of-evidence assessment,” which included a consideration of Bradford Hill’s criteria for judging causality. BMW moved to exclude plaintiffs’ witnesses, including Kramer, on grounds that the witnesses’ evidence and methods were “novel, unorthodox, unreliable and not generally accepted in the relevant scientific communities.”  Reeps slip op. at 5.  The number of ways that Kramer’s opinions ran afoul of New York law of expert witness opinions is remarkable.

Animal Studies

The animal studies found no relevant adverse birth effects, even at high gasoline fume exposure levels.  Kramer and her posse nonetheless cited animal studies involving cancer, miscarriage, and anemia for the general claim that gasoline fumes causes birth defects, as though such defects could all be lumped together.

Case Reports

Kramer relied upon two published papers of case reports in which women were exposed to leaded gasoline and then gave birth to children with malformations.  Given that the exposures reported were to leaded gasoline, the case reports were dubious in the first instance.  Furthermore, the reported defects were not even the same as those experienced by Sean Reeps.  Although the court seemed willing to engage in a discussion of what these case reports might offer towards a synthesis of all evidence, it ultimately recognized that, pace Raymond Wolfinger, plural of anecdote is not data.  “Courts have recognized that … case reports are not generally accepted in the scientific community on questions of causation.” Slip op. at 17 (quoting from Heckstall v. Pincus, 19 A.D.3d 203, 205, 797 N.Y.S.2d 445 (1st Dept. 2005)).

Exposure Assessment

The chemical components in gasoline, blamed by Kramer, make up no more than two percent of gasoline vapor. Reeps, slip at 6. One of plaintiffs’ expert witnesses asserted, without measurements, that Debra Reeps experienced atmospheric concentrations of gasoline at least 1,000 p.p.m.  Plaintiffs claimed that this level of exposure was tantamount to recreational solvent abuse, in an attempt to rely upon studies of solvent exposure at very high levels. BMW showed that the witnesses’ speculation was unfounded and implausible.  The fuel-line leak would have to leak about a gallon per mile driven to generate 1,000 p.p.m. in the passenger compartment.  Id. at 8.

Teratology Principles

Because certain structures, organs, and tissues in a developing embryo or fetus form at predictable stages of pregnancy, the science of teratology plays close attention to when the exposure to the putative teratogen occurred in the time course of a pregnancy.  Late exposures to known teratogens cannot very well explain harms that can result only from exposure early in pregnancy.  Similarly, early exposures cannot explain harms that arise only out of teratogenic exposures.  Debra Reeps’ claimed gasoline exposure occurred in her first trimester.  Despite Dr. Shira Kramer’s efforts, the neurological deficits and injuries in Sean Reeps thus cannot be explained by his mother’s early term exposures, even if gasoline fumes had the claimed teratogenic properties.

Ipse Dixit

Debra Reeps had a history of herpes simplex infection, which could explain her son’s cerebral palsy.  Slip op. at 7.  Dr. Kramer asserted that there were no alternative causes.

Epidemiology

Apparently no analytical epidemiologic study (either cohort or case-control) found an association between gasoline fume exposure in pregnancy and Sean Reeps’ birth defects. Kramer attempted to claim that the “[f]ailure to detect a statistical association does not establish that there is no association between an exposure and an outcome.”  Slip op. at 15.  Absence of evidence may not show evidence of absence, but it also does not show evidence of harm.

In one episode of Seinfeld, Jerry Seinfeld chides a rental car clerk for not honoring a reservation.  “You know how to take the reservation; you just don’t know how to hold the reservation.  And that’s really the most important part.”  Scientific methodology is similar to making reservations.  Anyone can claim to be following Sir Austin Bradford Hill’s causal criteria, but actually applying the criteria faithfully is really what the “methodology” is all about.  The abridged form favored by Dr. Kramer is indeed unorthodox, novel, unreliable, invalid, and unacceptable, scientifically and legally, as Justice York found in Reeps.  In essence, the plaintiffs argued that all their expert witnesses need show is that they are aware of proper methodologies, not that they actually used the methodologies properly.  Shira Kramer, and the other plaintiffs’ expert witnesses, offered a pastiche of a method, in the hopes that this would be sufficient.  Absent was a systematic review, and a proper analysis of the evidence. The Reeps case rejoined:  the law requires the real thing.

New York’s adherence to a Frye standard creates a potential roadblock to meaningful gatekeeping.  If an expert witness could evade gatekeeping by simply claiming to be following epidemiologic methods, regardless of how badly, that witness could undermine the interests of the justice system in weeding out speculative, unreliable, or invalid opinions.  The New York Court of Appeals demonstrated its unwillingness to tolerate such evasions.  See, e.g., Parker v. Mobil Oil Corp., 7 N.Y.3d 434, 857 N.E.2d 1114, 824 N.Y.S.2d 584 (2006) (excluding testimony of Dr. Bernard Goldstein, and dismissing leukemia (AML) claim based upon claimed low-level benzene exposure from gasoline) , aff’g 16 A.D.3d 648 (App. Div. 2d Dep’t 2005).

In Reeps, Justice York makes clear that it is the “plaintiff’s burden to prove the methodology applied to reach the conclusions will not be rejected by specialists in the field.”  Slip op. at 11.  The trial court recognized that a Frye hearing in New York must determine whether plaintiffs’ expert witnesses are faithfully applying a methodology, such as the Bradford Hill criteria, or whether they are they are “pay[ing] lip service to them while pursuing a completely different enterprise.”  Id.  To be sure, litigants might not welcome this level of scrutiny for their expert witnesses.  Justice York’s recognition that the court must examine a proffered opinion to determine whether it “properly relates existing data, studies or literature to the plaintiff’s situation, or whether, instead, it is connected to existing data only by the ipse dixit of the expert,” carries with it, an acknowledgment that New York law, like federal Rule 702, requires an assessment of the validity and sufficiency of the evidence and inferences that make up an expert witness’s opinions.  Id. (internal quotations omitted).

Posted in Causation, Expert Witnesses, Frye | Comments Off on New York Breathes Life Into Frye Standard – Reeps v. BMW

Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

November 21st, 2012

In 1984, before Judge Shoob gave his verdict in the Wells case, another firm filed a birth defects case against Ortho for failure to warn in connection with its non-ionic surfactant spermicides, in the same federal district court, the Northern District of Georgia. The mother in Smith used Ortho’s product about the same time as the mother in Wells (in 1980).  The case was assigned to Judge Shoob, who recused himself.  Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1562 n.1 (N.D. Ga. 1991) (no reasons for the recusal provided).  The Smith case was reassigned to Judge Horace Ward, who entertained Ortho’s motion for summary judgment in July 1988.  Two and one-half years later, Judge Ward granted summary judgment to Ortho on grounds that the plaintiffs’ expert witnesses’ testimony was not based upon the type of data reasonably relied upon by experts in the field, and was thus inadmissible under Federal Rule of Evidence 703. 770 F. Supp. at 1681.

A prevalent interpretation of the split between Wells and Smith is that the scientific evidence developed with new studies, and that the scientific community’s views matured in the five years between the two district court opinions. The discussion in Modern Scientific Evidence is typical:

“As epidemiological evidence develops over time, courts may change their view as to whether testimony based on other evidence is admissible. In this regard it is worth comparing Wells v. Ortho Pharmaceutical Corp., 788 F.2d 741 (11th Cir. 1986), with Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561 (N.D. Ga. 1991). Both involve allegations that the use of spermicide caused a birth defect. At the time of the Wells case there was limited epidemiological evidence and this type of claim was relatively novel.  In a bench trial the court found for the plaintiff.  *** The Smith court, writing five years later, noted that, ‘The issue of causation with respect to spermicide and birth defects has been extensively researched since the Wells decision.’ Smith v. Ortho Pharmaceutical Corp., 770 F. Supp. 1561, 1563 (N.D. Ga. 1991).”

1 David L. Faigman, Michael J. Saks, Joseph Sanders, and Edward K. Cheng, Modern Scientific Evidence:  The Law and Science of Expert Testimony, “Chapter 23 – Epidemiology,” § 23:4, at 213 n.12 (West 2011) (internal citations omitted).

Although Judge Ward was being charitable to his judicial colleague, this attempt to reconcile Wells and Smith does a disservice to Judge Ward’s hard work in Smith, and Judge Shoob’s errors in Wells.

Even a casual reading of Smith and Wells reveals that the injuries were completely differently.  Plaintiff Crystal Smith was born with a chromosomal defect known as Trisomy-18; Plaintiff Katie Wells was born with limb reduction deficits.   Some studies relevant to one injury had no information about the other.  Other studies, which addressed both injuries, yielded different results for the different injuries.  Although some additional studies were available to Judge Ward in 1988, this difference is hardly the compelling difference between the two cases.

Perhaps the most important difference between the cases is that in Smith, the biologically plausibility that spermicides could cause a Trisomy-18 was completely absent.  The chromosomal defect arises from a meiotic disjunction, an error in meiosis that is part of the process in which germ cells are formed.  Simply put, spermicides arrive on the scene too late to cause a Trisomy-18.  Notwithstanding the profound differences between the injuries involved in Wells and Smith, the Smith plaintiffs sought the application of collateral estoppel.  Judge Ward refused this motion, on the basis of the factual differences in the cases, as well as the availability of new evidence.  770 F.Supp. at 1562.

The difference in injuries, however, was not the only important difference between these two cases.  Wells was actually tried, apparently without any challenge under Frye, or Rules 702 or 703, to the admissibility of expert witness testimony.  There is little to no discussion of scientific validity of studies, or analysis of the requisites for evaluating associations for causality.  It is difficult to escape the conclusion that Judge Shoob decided the Wells case on the basis of superficial appearances, and that he frequently ignored validity concerns in drawing invidious distinctions between plaintiffs’ and defendant’s expert witnesses and their “credibility.”  Smith, on the other hand, was never tried.  Judge Ward entertained and granted dispositive motions for summary judgment, on grounds that the plaintiffs’ expert witnesses’ testimony was inadmissible. Legally, the cases are light years apart.

In Smith, Judge Ward evaluated the same FDA reports and decisions seen by Judge Shoob.  Judge Ward did not, however, dismiss these agency materials simply because one or two of dozens of independent scientists involved had some fleeting connection with industry. 770 F.Supp. at 1563-64.

Judge Ward engaged with the structure and bases of the expert witnesses’ opinions, under Rules 702 and 703.  The Smith case thus turned on whether expert witness opinions were admissible, an issue not considered or discussed in Wells.  As was often the case before the Supreme Court decided Daubert in 1993, Judge Ward paid little attention to Rule 702’s requirement of helpfulness or knowledge.  The court’s 702 analysis was limited to qualifications.  Id. at 1566-67.  The qualifications of the plaintiffs’ witnesses were rather marginal.  They relied upon genetic and epidemiologic studies, but they had little training or experience in these disciplines. Finding the plaintiffs’ expert witnesses to meet the low threshold for qualification to offer an opinion in court, Judge Ward focused on Rule 703’s requirement that expert witnesses reasonably rely upon facts and data that are not otherwise admissible.

The trial court in Smith struggled with how it should analyze the underpinnings of plaintiffs’ witnesses’ proffered testimony.  The court acknowledged that conflicts between expert witnesses typically raise questions of weight, not admissibility.  Id. at 1569.  Ortho had, however, challenged plaintiffs’ witnesses for having given opinions that lacked a “sound underlying methodology.” Id.  The trial court found at least one Fifth Circuit case that suggested that Rule 703 requires trial courts to evaluate the reliability of expert witnesses’ sources.  Id. (citing Soden v. Freightliner Corp., 714 F.2d 498, 505 (5th Cir. 1983). Elsewhere, the trial court also found precedent from Judge Weinstein’s opinion in Agent Orange, as well as Court of Appeals decisions involving Bendectin, all of which turned to Rule 703 as the legal basis for reviewing, and in some cases limiting or excluding expert witness opinion testimony.  Id.

The defendant’s argument under Rule 703 was strained; Ortho argued that the plaintiffs’

“experts’ selection and use of the epidemiological data is faulty and thus provides an insufficient basis upon which experts in the field of diagnosing the source of birth defects normally form their opinions. The defendant also contends that the plaintiffs’ experts’ data on genetics is not of the kind reasonably relied upon by experts in field of determining causation of birth defects.”

Id. at 1572.  Nothing in Rule 703 addresses the completeness or thoroughness of expert witnesses in their consideration of facts and data; nor does Rule 703 address the sufficiency of data or the validity vel non of inferences drawn from facts and data considered.  Nonetheless, the trial court in Smith took Rule 703 as its legal basis for exploring the epistemic warrant for plaintiffs’ witnesses’ causation opinions.

Although plaintiffs’ expert witnesses stated that they had relied upon epidemiologic studies and method, the trial court in Smith went beyond their asseverations.  The Smith trial court explored the credibility of these witnesses at a whole other level.  The court reviewed and discussed the basic structure of epidemiologic studies, and noted that the objective of such studies is to provide a statistical analysis:

“The objective of both case-control and cohort studies is to determine whether the difference observed in the two groups, if any, is ‘statistically significant’, (that is whether the difference found in the particular study did not occur by chance alone).40 However, statistical methods alone, or the finding of a statistically significant association in one study, do not establish a causal relationship.41 As one authority states:

‘Statistical methods alone cannot establish proof of a causal relationship in an association’.42

As a result, once a statistical association is found in an epidemiological study, that data must then be evaluated in a systematic manner to determine causation. If such an association is present, then the researcher looks for ‘bias’ in the study.  Bias refers to the existence of factors in the design of a study or in the manner in which the study was carried out which might distort the result.43

If a statistically significant association is found and there is no apparent ‘bias’, an inference is created that there may be a cause-and-effect relationship between the agent and the medical effect. To confirm or rebut that inference, an epidemiologist must apply five criteria in making judgments as to whether the associations found reflect a cause-and-effect relationship.44 The five criteria are:

1. The consistency of the association;

2. The strength of the association;

3. The specificity of the association;

4. The temporal relationship of the association; and,

5. The coherence of the association.

Assuming there is some statistical association, it is these five criteria that provide the generally accepted method of establishing causation between drugs or chemicals and birth defects.45 ”

The Smith court acknowledged that there were differences of opinion in weighting these five factors, but that some of them were very important to drawing a reliable inference of causality.  Id. at 1775.

A major paradigm shift thus separates Wells and Smith.  The trial court in Wells contented itself with superficial and subjective indicia of witnesses’ personal credibility; the trial in Smith delved into the methodology of drawing an appropriate scientific conclusion about causation.  Telling was the Smith court’s citation to Moultrie v. Martin, 690 F.2d 1078, 1082 (4th Cir. 1982) (“In borrowing from another discipline. a litigant cannot be selective in which principles are applied.”).  770 F.Supp. at 1575 & n.45.  Gone is the Wells retreat from engagement with science, and the dodge that the court must make a legal, not a scientific decision.

Applying the relevant principles, the Smith court found that the plaintiffs’ expert witnesses had deviated from the scientific standards of reasoning and analysis:

“It is apparent to the court that the testimony of Doctors Bussey and Holbrook is insufficiently grounded in any reliable evidence. * * * The conclusions Doctors Bussey and Holbrook reach are also insufficient as a basis for a finding of causality because they fail to consider critical information, such as the most relevant epidemiologic studies and the other possible causes of disease.81

The court finds that the opinions of plaintiffs’ experts are not based upon the type of data reasonably relied upon by experts in determining the cause of birth defects. Experts in determining birth defects rely upon a consensus in genetic or epidemiological investigations or specific generally accepted studies in these fields. While a consensus in genetics or epidemiology is not a prerequisite to a finding of causation in any and all birth defect cases, Rule 703 requires some reliable evidence for the basis of an expert’s opinion.

Experts in determining birth defects also utilize methodologies and protocols not followed by plaintiffs’ experts. Without a well-founded methodology, opinions which run contrary to the consensus of the scientific community and are not supported by any reliable data are necessarily speculative and lacking in the type of foundation necessary to be admissible.

For the foregoing reasons, the court finds that plaintiffs have failed to produce admissible evidence sufficient to show that defendant’s product caused Crystal’s birth defects.”

Id. at 1581.  Rule 703 was forced into a service to filter out methodologically specious opinions.

Not all was smooth sailing for Judge Ward.  Like Judge Shoob, Judge Ward seemed to think that a physical examination of the plaintiff provided helpful, relevant evidence, but he never articulated what the basis for this opinion was. (His Honor did note that the parties agreed that the physical examination offered no probative evidence about causation.  Id. at 1572 n.32.) No harm came of this opinion.  Judge Ward wrestled with the lack of peer review in some unpublished studies, and the existence of a study only in abstract form.  See, e.g., id. at 1579 (“a scientific study not subject to peer review has little probative value”); id. at 1578 (insightfully noting that an abstract had insufficient data to permit a reader to evaluate its conclusions).  The Smith court recognized the importance of statistical analysis, but it confused Bayesian posterior probabilities with significance probabilities:

“Because epidemiology involves evidence on causation derived from group based information, rather than specific conclusions regarding causation in an individual case, epidemiology will not conclusively prove or disprove that an agent or chemical causes a particular birth defect. Instead, its probative value lies in the statistical likelihood of a specific agent causing a specific defect. If the statistical likelihood is negligible, it establishes a reasonable degree of medical certainty that there is no cause-and-effect relationship absent some other evidence.”

The confusion here is hardly unique, but ultimately it did not prevent Judge Ward from reaching a sound result in Smith.

What intervened between Wells and Smith was not any major change in the scientific evidence on spermicides and birth defects; the sea change came in the form of judicial attitudes toward the judge’s role in evaluating expert witness opinion testimony.  In 1986, for instance, after the Court of Appeals affirmed the judgment in Wells, Judge Higginbotham, speaking for a panel of the Fifth Circuit, declared:

“Our message to our able trial colleagues: it is time to take hold of expert testimony in federal trials.”

 In re Air Crash Disaster at New Orleans, 795 F.2d 1230, 1234 (5th Cir. 1986).  By the time the motion for summary judgment in Smith was decided, that time had come.

Posted in Causation, Data Sharing, Frye, Rule 702, Rule 703, Scientific Evidence, statistical evidence | Comments Off on Wells v. Ortho Pharmaceutical Corp. Reconsidered – Part 6

Meta-Meta-Analysis – Celebrex Litigation – The Claims – Part 2

June 25th, 2012

IMPUTATION

As I noted in part one, the tables were turned on imputation, with plaintiffs making the same accusation that G.E. made in the gadolinium litigation:  imputation involves adding “phantom events” or “imaginary events to each arm of ‘zero event’ trials.”  See Plaintiffs’ Reply Mem. of Law in Further Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 8, 9 (May 5, 2010), in Securities Litig.

The plaintiffs claimed that Wei “created” an artifact of a risk ratio of 1.0 by using imputation in each of the zero-event trials.  The reality, however, is that each of those trials had zero risk difference, and the rates of event in drug and placebo arms were both low and equal to one another.  The plaintiffs’ claim that Wei “diluted” the risk is little more than saying that he failed to inflate the risk by excluding zero-event trials.  But zero-event trials represent a test in which the risk of events in both arms is equal, and relatively low.

The plaintiffs seemed to make their point half-heartedly.  They admitted that “imputation in and of itself is a commonly used methodology,” id. at 10, but they claimed that “adding zero-event trials to a meta-analysis is debated among scientists.”  Id.  A debate over methodology in the realm of meta-analysis procedures hardly makes any one of the debated procedures “not generally accepted,” especially in the context of meta-analysis of uncommon adverse events arising in clinical trials designed for other outcomes.  After all, investigators do not design trials to assess a suspected causal association between a medication and an adverse outcome as their primary outcome.  The debate over the ethics of such a trial would be much greater than any gentle debate over whether to include zero-event trials by using either the risk difference or imputation procedures.

The gravamen of the plaintiffs’ complaint against Wei seems to be that he included too many zero-event trials, “skewing the numbers greatly, and notably cites to no publications in which the dominant portion of the meta-analysis was comprised of studies with no events.”  Id. The plaintiffs further argue that Wei could have minimized the “distortion” created by imputation by using a fractional event, ” a smaller number like .000000001 to each trial.”  Id. The plaintiffs notably cited no texts or articles for this strategy.  In any event, if the zero-event trials are small, as they typically are, then they will have large study variances.  Because meta-analyses weight each trial by the inverse of the variance, studies with large variances have little weight in the summary estimate of association.  Including small studies with imputation methods will generally not affect the outcome very much, and their contribution may well reflect the reality of lower or non-differential risk from the medication.

Eliminating trials on the grounds that they had zero events has also been criticized for throwing away important data.  Charles H. Hennekens, David L. DeMets, C. Noel Bairey Merz, Steven L. Borzak, Jeffrey S. Borer,  “Doing More Harm Than Good,” 122 Am. J. Med. 315 (2009) (criticizing Nissen’s meta-analysis of rosiglitazone in which he excluded zero event trials for as biased towards overestimating the magnitude of the summary estimate of association). George A. Diamond, L. Bax, S. Kaul, “Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death,” 147 Ann. Intern. Med. 578 (2007) (conducting sensitivity analyses on Nissen’s meta-analysis of rosiglitazone to show that Nissen’s findings lost statistical significance when continuity corrections were made for zero-event trials).

 

RISK DIFFERENCE

The plaintiffs are correct that the risk difference is not the predominant risk measure used in meta-analysis or in clinical trials for that matter.  Researchers prefer risk ratios because they reflect base rates in the ratio.  As one textbook explains:

“the limitation of the [risk difference] statistic is its insensitivity to base rates. For example, a risk that increases from 50% to 52% may be less important than one that increases from 2% to 4%, although in both instances RD = 0.02.”

Julia Littell, Jacqueline Corcoran, and Vijayan Pillai, Systematic Reviews and Meta-Analysis 85 (Oxford 2008).  This feature of the risk difference hardly makes its use unreliable, however.

Pfizer pointed out that at least one other case addressed the circumstances in which the risk difference would be superior to risk ratios in meta-analyses:

“The risk difference method is often used in meta-analyses where many of the individual studies (which are all being pooled together in one, larger analysis) do not contain any individuals who developed the investigated side effect.FN17  whereas such studies would have to be excluded from an odds ratio calculation, they can be included in a risk difference calculation. FN18

FN17. This scenario is more likely to occur when studying a particularly rare event, such as suicide.

FN18. Studies where no individuals experienced the effect must be excluded from an odds ratio calculation because their inclusion would necessitate dividing by zero, which, as perplexed middle school math students come to learn, is impossible. The risk difference’s reliance on subtraction, rather than division, enables studies with zero incidences to remain in a meta-analysis. (Hr’g Tr. 310-11, June 20, 2008 (Gibbons.)).”

In re Neurontin Marketing, Sales Practices, and Products Liab. Litig.,  612 F.Supp. 2d 116, 126 (D. Mass. 2009) (MDL 1629).  See Pfizer’s Defendants’ Mem. of Law in Opp. to Plaintiffs’ Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei (Sept. 8, 2009), in Securities Litig. (citing In re Neurontin).

Pfizer also pointed out that Wei had employed both the risk ratio and the risk difference in conducting his meta-analyses, and that none of his summary estimates of association were statistically significant.  Id. at 19, 24.


EXACT CONFIDENCE INTERVALS

The plaintiffs argued that the use of “exact confidence” intervals was not scientifically reliable and could not have been used by Pfizer at the time period covered by the securities class’s allegations.  See Plaintiffs’ Reply Mem. of Law in Further Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 15 (May 5, 2010).  Exact intervals, however, are hardly a novelty, and there is often no single way to calculate a confidence interval.  See E. B. Wilson,  “Probable inference, the law of succession, and statistical inference,” 22 J. Am. Stat. Ass’n 209 (1927); C. Clopper, E. S. Pearson, “The use of confidence or fiducial limits illustrated in the case of the binomial,” 26 Biometrika 404 (1934).  Approximation methods are often used, despite their lack of precision, because of their ease in calculation.

Plaintiffs further claimed that the combination of risk difference and exact intervals is novel, not reliable, and not in existence during the class period.  Plaintiffs’ Reply Mem at 15.  The plaintiffs’ argument traded on Wei’s having published on the use of exact intervals in conjunction with the risk difference for heart attacks in clinical trials of Avandia.  See L. Tian, T. Cai, M.A. Pfeffer, N. Piankov, P.Y. Cremieux, and L.J. Wei, “Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2 x 2 tables with all available data but without artificial continuity correction,” 10 Biostatistics 275 (2009).  Their argument ignored that Wei combined two well-understood statistical techniques, in a transparent way, with empirical testing of the validity of his approach.  Contrary to plaintiffs’ innuendo, Wei did not develop his approach as an expert witness for GlaxoSmithKline; a version of the manuscript describing his approach was posted on line well before he was ever contacted by GSK counsel. (L.J. Wei, personal communication)  Plaintiffs also claimed that Wei’s use of exact intervals for risk difference showed no increased risk of heart attack for Avandia, contrary to a well-known meta-analysis by Dr. Steven Nissen.  See Steven E. Nissen, M.D., and Kathy Wolski, M.P.H., “Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes,” 356 New Engl. J. Med. 2457, 2457 (2007).  This claim, however, is a crude distortion of Wei’s paper, which showed that there was a positive risk difference for heart attacks in the same dataset used by Nissen, but the confidence intervals included zero (no risk difference), and thus chance could not be excluded as explaining Nissen’s result.

 

DURATION OF TRIALS

Pfizer was ultimately successful in defending the Celebrex litigation on the basis of lack of risk associated with 200 mg/day use.  Pfizer also attempted to argue a duration effect on grounds that in one large trial that saw a statistically significant hazard ratio associated with higher doses, the result occurred for the first time among trial participants on medication, at 33 months into the trial.  Judge Bryer rejected this challenge, without explanation.  In re Bextra & Celebrex Marketing Celebrex Sales Practices & Prod. Liab. Litig., 524 F.Supp. 2d 1166, 1183 (2007).  The reasonable inference, however, is that the meta-analyses showed statistically significant results across trials with less duration of use, for 400 mg and 800 mg/day use.

Clearly duration of use is a potential consideration unless the mechanism of causation is such that a causally related adverse event would occur from the first use or very short-term use of the medication.  See In re Vioxx Prods. Liab. Litig., MDL No. 1657, 414 F. Supp. 2d. 574, 579 (E.D. La. 2006) (“A trial court may consider additional factors in assessing the scientific reliability of expert testimony . . . includ[ing] whether the expert’s opinion is based on incomplete or inaccurate dosage or duration data.”).  In the Celebrex litigation, plaintiffs’ counsel appeared to want to have duration effects both ways; they did not want to disenfrancise plaintiffs whose claims turned on short-term use, but at the same time, they criticized Professor Wei for including short-term trials of Celebrex.

One form that the plaintiffs’ criticism of Wei took was his failure to weight the trials included in his meta-analyses by duration.  In the plaintiffs’ words:

“Wei failed to utilize important information regarding the duration of the clinical trials that he analyzed, information that is critical to interpreting and understanding the Celebrex and Bextra safety information that is contained within those clinical trials.3 Because the types of cardiovascular events that are at issue in this case occur relatively rarely and are more likely to be observed after an extended period of exposure, the scientific community is in agreement that they would not be expected to appear in trials of very short duration.”

Plaintiffs’ Mem. of Law in Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 2 (July 23, 2009), submitted in In re Pfizer, Inc. Securities Litig., Nos. 04 Civ. 9866(LTS)(JLC), 05 md 1688(LTS) (S.D.N.Y.)[hereafter Securities Litig.]  The plaintiffs maintained that Wei’s meta-analyses were “fatally flawed” because he ignored trial duration, such as would be factored in by performing the analyses in terms of patient years.  Id. at 3

Many of the sources cited by plaintiffs do not support their argument. For instance, the plaintiffs cited articles that noted that weighted averages should be used, but virtually all methods, including Wei’s, weight studies by their variance, which takes into account sample size. Id. at 9 n.3, citing Egger, et al. “Meta-analysis: Principles and Procedures,” 315 Brit. Med. J. 1533 (1997) (an arithmetic average from all trials gives misleading results as results from small studies are more subject to the play of chance and should be given less weight. Meta-analyses use weighted results in which larger trials have more influence that smaller ones). See also id. at 22.  True, true, and immaterial.  No one in the Celebrex cases was using an arithmetic average of risk across trials or studies.

Most of the short-term studies were small, and thus contributed little to the overall summary estimate of association.  Some of the plaintiffs’ citations actually supported using “individual patient data” in the form of time-to-event analyses, which was not possible with many of the clinical trials available.  Indeed, the article the plaintiffs cited, by Dahabreh, did not use time-to-event data for rosiglitazone, because such data were not generally available.  Id. at 9 n.3, citing Dahabreh, “Meta-Analysis Of Rare Events: An Update And Sensitivity Analysis Of Cardiovascular Events In Randomized Trials Of Rosiglitazone,” 5 Clinical Trials 116 (2008).

The plaintiffs’ claim was thus a fairly weak challenge to using simple 2 x 2 tables for the included studies in Wei’s meta-analysis. Both sides failed to mention that many published meta-analyses eschew “patient years” in favor of a simple odds ratio for dichotomous count data from each included study.  See, e.g., Steven E. Nissen, M.D., and Kathy Wolski, M.P.H., “Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes,” 356 New Engl. J. Med. 2457, 2457 (2007)(using Peto method with count data, for fixed effect model).  Patient years would be a crude tool to modify the fairly common 2 x 2 table.  The analysis for large studies, with a high number of patient years, would still not reveal whether the adverse events occurred early or late in the trials.  Only a time-to-event analysis could provide the missing information about “duration,” and neither side’s expert witnesses appeared to use a time-to-event analysis.

Interestingly, plaintiffs’ expert witness, Prof. Madigan appears to have received the patient-level data from Pfizer’s clinical trials, but still did not conduct a time-to-event analysis.  Plaintiffs’ Mem. of Law in Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 12 (July 23, 2009), submitted in In re Pfizer, Inc. Securities Litig., Nos. 04 Civ. 9866(LTS)(JLC), 05 md 1688(LTS) (S.D.N.Y.)[hereafter Securities Litig] (noting that Madigan had examined all SAS data files produced by Pfizer, and that “[t]hese  files contained voluminous information on each subject in the trials, including information about duration of exposure to the drug ( or placebo), any adverse events experienced and a wide variety of other information.”).  Of course, even with time-to-event data from the Pfizer clinical trials, Madigan had the problem of whether to limit himself to just the Pfizer trials or use all the data, including non-Pfizer trials.  If he opted for completeness, he would have been forced to include trials for which he did not have underlying data.  In all likelihood, Madigan used patient-years in his analyses because he could not conduct a complete analysis with time-to-event data for all trials.

The plaintiffs’ point appears well taken if the court were to assume that there really was a duration issue, but the plaintiffs’ theories were to the contrary, and Pfizer lost its attempt to limit claims to those events that appeared 33 months (or some other fixed time) after first ingestion.  It is certainly correct that patient-year analyses, in the absence of time-to-event analyses, is generally preferred.  Pfizer had used patient-year information to analyze combined trials in its submission to the FDA’s Advisory Committee.  See Pfizer’s Submission of Advisory Committee Briefing Document at 15 (January 12, 2005).  See also  FDA Reviewer Guidance: Conducting a Clinical Safety Review of a New Product Application and Preparing a Report on the Review at 22 (2005); see also id. at 15 (“If there is a substantial difference in exposure across treatment groups, incidence rates should be calculated using person-time exposure in the denominator, rather than number of patients in the denominator.”);  R. H. Friis & T. A. Sellers, Epidemiology for Public Health Practice at 105 (2008) (“To allow for varying periods of observation of the subjects, one uses a modification of the formula for incidence in which the denominator becomes person-time of observation”).

Professor Wei chose not to do a “patient-year” analysis because such a methodological commitment would have required him to drop over a dozen Celebrex clinical trials involving thousands of patients, and dozens of heart attack and stroke events of interest.  Madigan’s approach led him to disregard a large amount of data.  Wei could, of course, stratified the summary estimates for different length clinical trials, and analyzed whether there were differences as a function of trial duration.  Pfizer claimed that Wei conducted a variety of sensitivity analyses, but it is unclear whether he ever used this technique.  Wei should have been allowed in any event to take plaintiffs at their word that thrombotic events from Celebrex occurred shortly after first ingestion.   Pfizer Mem. of Law in Opp. to Plaintiffs’ Motion to Exclude Defendants’ Expert Dr. Lee-Jen Wei at 2 (Sept. 8, 2009), in Secur. Litig.

 

MADIGAN’S META-ANALYSIS

According to Pfizer, Professor Madigan reached different results from Wei’s largely because he had used different event counts and end points.  The defendants’ challenge to Madigan turned largely upon the unreliable way he went about counting events to include in his meta-analyses.

Data concerning unexpected adverse events in clinical trials often is collected as reports of treating physicians, whose descriptions may be incomplete, inaccurate, or inadequate.  When there is a suggestion that a particular adverse event – say heart attack – occurred more frequently in the medication arm as opposed to the placebo or comparator arms, the usual course of action is to have a panel of clinical experts review all the adverse event reports, and supporting medical charts, to provide diagnoses that can be used in a more complete statistical analyses.  Obviously, the reviewers should be blinded to the patients’ assignment to medication or placebo, and the reviewers should be clinical experts in the clinical specialty of the adverse event.  Cardiologists should be making the call for heart attacks.

In addition to event definition and adjudication, clinical trial interpretation sometimes leads to the use of “composite end points,” which consist of related diagnostic categories, aggregated in some way that makes biological sense.  For instance, if the concern is that a medication causes cardiovascular thrombotic events, a suitable cardiovascular composite end point might include heart attack and ischemic stroke.  Inclusion of hemorrhagic stroke, endocarditis, and valvular disease in the composite, however, would be inappropriate, given the concern over thrombosis.

Professor Madigan is a highly qualified statistician, but, as Pfizer argued, he had no clinical expertise to reassign diagnoses or determine appropriate composite end points.  The essence of the defendants’ challenges revolved around claims of flawed outcome and endpoint ascertainment and definitions.  According to Pfizer’s briefing, the event definition process was unblinded, and conducted by inexpert, partisan reviewers.  Madigan apparently relied upon the work of another plaintiffs’ witness, cardiologist Dr. Lawrence Baruch, as well as that of Dr. Curt Furberg.  Furberg was not a cardiologist; indeed he has never been licensed to practice medicine in the United Dates, and he had not treated a patient in over 30 years. Pfizer Mem. of Law in Opp. to Plaintiffs’ Motion to Exclude Defendants’ Expert Dr. Lee-Jen Wei at 29 (Sept. 8, 2009), in Secur. Litig.  Furthermore, Furberg was not familiar with current diagnostic criteria for heart attack.  Plaintiffs’ counsel asked Furberg to rework some but not all of Baruch’s classifications, but only for fatal events.  Baruch could not explain why Furberg made these reclassifications.  Furberg acknowledged that he had never used “one-line descriptions to classify events,” which he did in the Celebrex litigation, when he received the assignment from plaintiffs’ counsel on the eve of the Court’s deadline for disclosures.  Id. According to Pfizer, if the plaintiffs’ witnesses had used appropriate end points and event counts, their meta-analyses would not have differed from Professor Wei’s work.  Id.

Pfizer pointed to Madigan’s testimony to claim that he had admitted that, based upon the impropriety of Furberg’s changing end point definitions, and his own changes, made without the assistance of a clinician, he would not submit the earlier version of his meta-analysis for peer review.  Pfizer’s [Proposed] Findings of Fact and Conclusions of Law with Respect to Motion to Exclude Certain Plaintiffs’ Experts’ Opinions Regarding Celebrex and Bextra, and Plaintiffs’ Motion to Exclude Defendants’ Expert Dr. Lee-Jen Wei, Document 175, submitted in Securities Litig. (Dec. 4, 2009). at 33,  43.  The plaintiffs countered that Furberg’s reclassifications did not change Madigan’s reports, at least for certain years. Plaintiffs’ Reply Mem. of Law in Further Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 18 (May 5, 2010), in Securities Litig.

The trial court denied Pfizer’s challenges to Madigan’s meta-analysis in the securities fraud class action.  The court attributed any weakness in the classification of fatal adverse events by Baruch and Furberg to the limitations of the underlying data created and produced by Pfizer itself.  In re Pfizer Inc. Securities Litig., 2010 WL 1047618, *4 (S.D.N.Y. 2010).

 

Composites

Pfizer also argued that Madigan put together composite outcomes that did not make biological sense in view of the plaintiffs’ causal theories.  For instance, Madigan left out strokes in his composite, although he included both heart attack and stroke in his primary end point for his Vioxx litigation analysis, and he had no reason to distinguish Vioxx and Celebrex in terms of claimed thrombotic effects.  Pfizer’s [Proposed] Findings of Fact and Conclusions of Law with Respect to Motion to Exclude Certain Plaintiffs’ Experts’ Opinions Regarding Celebrex and Bextra, and Plaintiffs’ Motion to Exclude Defendants’ Expert Dr. Lee-Jen Wei, Document 175, submitted in Securities Litig. (Dec. 4, 2009). at 13-14, 18.  According to Pfizer, Madigan’s composite was novel and unvalidated by relevant, clinical opinion.  Id. at 29, 33.

The plaintiffs’ response is obscure.  The plaintiffs seemed to claim that Madigan was justified in excluding strokes because some kinds of stroke, hemorrhagic strokes, are unrelated to thrombosis.  Plaintiffs’ Reply Memorandum of Law in Further Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 14 (May 5, 2010), in Securities Litig. at 14.  This argument is undermined by the facts:  better than 85% of strokes being ischemic in origin, and even some hemorrhagic strokes start as a result of an ischemic event.

In any event, Pfizer’s argument about Madigan’s composite end points did not gain any traction with the trial judge in the securities fraud class action:

“Dr. Madigan’s written submissions and testimony described clearly and justified cogently his statistical methods, selection of endpoints, decisions regarding event classification, sources of data, as well as the conclusions he drew from his analysis. Indeed, Dr. Madigan’s meta-analysis was based largely on data and endpoints developed by Pfizer. All four of the endpoints that Dr. Madigan used in his analysis-Hard CHD, Myocardial Thromboembolic Events, Cardiovascular Thromboembolic Events, and CV Mortality-have been employed by Pfizer in its own research and analysis. The use of Hard CHD in the relevant literature combined with the use of the other three endpoints by Pfizer in its own 2005 meta-analysis will assist the trier of fact in determining Pfizer’s knowledge and understanding of the pre-December 17, 2004, cardiovascular safety profile of Celebrex.”

In re Pfizer Inc. Securities Litig., 2010 WL 1047618, *4 (S.D.N.Y. 2010).

Posted in Causation, Frye, Meta-analysis, Rule 702 | Comments Off on Meta-Meta-Analysis – Celebrex Litigation – The Claims – Part 2

Meta-Meta-Analysis – Celebrex Litigation – The Claims – Part One

June 21st, 2012

In the Celebrex/Bextra litigation, both sides acknowledged the general acceptance and validity of meta-analysis, for both observational studies and clinical trials, but attacked the other side’s witnesses’ meta-analyses on grounds specific to how they were conducted.  See, e.g., Pfizer Defendants’ Motion to Exclude Certain Plaintiffs’ Experts’ Causation Opinion Regarding Celebrex – Memorandum of Points and Authorities in Support Thereof at 14, 16 (describing meta-analysis as “appropriate” and a “useful way to evaluate the presence and consistency of an effect,” and “a valid technique for analyzing the results of both randomized clinical trials and observational studies”)(dated July 20, 2007), submitted in MDL 1699, In re Bextra and Celebrex Marketing Sales Practices & Prod. Liab. Litig., Case No. 05-CV-01699 CRB (N.D. Calif.) [hereafter MDL 1699]; Plaintiffs’ Memorandum of Law in Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 2 (July 23, 2009) (“While use of a properly conducted meta-analysis is appropriate, there are underlying scientific principles and techniques to be used in meta-analysis that are widely accepted among biostatisticians and epidemiologists. Wei’s meta-analysis – which he acknowledges is based in part on an admittedly novel approach that is not generally recognized by the scientific community – fails to follow certain of these key principles.”), submitted in In re Pfizer, Inc. Securities Litig., Nos. 04 Civ. 9866(LTS)(JLC), 05 md 1688(LTS) (S.D.N.Y.)[hereafter Securities Litig.]

The plaintiffs and defendants expended a great deal of energy in attacking the other side’s meta-analyses as conducted.  With all the briefing in the federal MDL, the New York state cases, and the securities fraud class action, hundreds of pages were written on the suspected flaws in meta-analyses.  The courts, in both the products liability MDL cases and in the securities case, denied the challenges in a few sentences.  Indeed, it is difficult if not impossible to discern what the challenges were from reading the courts’ decisions. In re Pfizer Inc. Securities Litig., 2010 WL 1047618 (S.D.N.Y. 2010); In re Bextra and Celebrex, 2008 N.Y. Misc. LEXIS 720; 239 N.Y.L.J. 27(2008); In re Bextra and Celebrex Marketing Sales Practices and Product Liability Litig., MDL No. 1699, 524 F.Supp. 2d 1166 (N.D. Calif. 2007)

Although the issues shifted some over the course of these litigations, certain important themes recurred.  The plaintiffs focused their attack upon the meta-analyses conducted by defense expert witness, Lee-Jen Wei, a professor of biostatistics at the Harvard School of Public Health.

The plaintiffs maintained that Professor Wei’s meta-analyses should be excluded under Rule 702, or the New York case law, because of

  • inclusion of short-term clinical trials
  • failure to weight risk ratios by person years
  • inclusion of zero-event trials with use of imputation methods
  • use of risk difference instead of risk ratios
  • use of exact confidence intervals instead of estimated intervals

See generally Plaintiffs’ Memorandum of Law in Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei (July 23, 2009), in Securities Litig.

The plaintiffs advanced meta-analyses conducted by Professor David Madigan, Professor and Chair in the Department of Statistics, Columbia University.  The essence of the defendants’ challenges revolved around claims of flawed outcome and endpoint ascertainment and definitions:

  • invalid clinical endpoints
  • flawed data collection procedures
  • ad hoc changes in procedure and methods
  • novel methodologies “never used in the history of clinical research”
  • lack of documentation for classifying events
  • absence of expert clinical judgment in classifying event for inclusion in meta-analysis
  • creation of composite endpoints that included events unrelated to plaintiffs’ theory of thrombotic mechanism
  • lack of blinding to medication use when categorizing events
  • failure to adjust for multiple comparisons in meta-analyses

See generally Pfizer Defendants’ Motion to Exclude Certain Plaintiffs’ Experts’ Causation Opinion Regarding Celebrex – Memorandum of Points and Authorities in Support Thereof (dated July 20, 2007), in MDL 1699; Pfizer defendants’ [Proposed] Findings of Fact and Conclusions of Law with Respect to Motion to Exclude Certain Plaintiffs’ Experts’ Opinions Regarding Celebrex and Bextra, and Plaintiffs’ Motion to Exclude Defendants’ Expert Dr. Lee-Jen Wei, Document 175, submitted in Securities Litig. (Dec. 4, 2009).

Why did the three judges involved (Judge Breyer in the federal MDL; Justice Kornreich in the New York state cases; and Judge Swain in the federal securities putative class action) give such cursory attention to these Rule 702/Frye challenges?  The complexity of the issues, the lack of clarity in the lawyers’ briefings, and the stridency of both sides perhaps contributed to shorten judicial attention span.  Some of the claims were simply untenable, and may have obliterated more telling critiques.

ZERO-EVENT TRIALS

Many of the Celebrex parties’ claims can be traced to a broader issue of what to include or exclude in a meta-analysis.  Consider for instance the plaintiffs’ challenge to Wei’s meta-analysis.  The plaintiffs faulted Wei for including short-term clinical trials in his meta-analysis, while sponsoring their own expert witness testimony that Celebrex could induce heart attack or stroke after first ingestion of the medication.  Having made the claim, the plaintiffs were hard pressed to exclude short-term trials, other than to argue that such trials frequently had zero adverse events in either the medication or placebo arms.  Many meta-analytic methods, which treat each included study as a 2 x 2 contingency table, and calculate an odds ratio for each table, cannot accommodate zero event data.

Whether or not hard pressed, the plaintiffs made the claim. The plaintiffs’ analogized to the lack of reliability of underpowered clinical trials to provide evidence of safety.  See Plaintiffs’ Reply Memorandum of Law in Further Support of Their Motion to Exclude Expert Testimony by Defendants’ Expert Dr. Lee-Jen Wei at 6 (May 5, 2010), in Securities Litig. (citing In re Neurontin Mktg., Sales Practices, and Prod. Liab. Litig., 612 F. Supp. 2d 116, 141 (D. Mass. 2009) (noting that many of Pfizer’s studies were “underpowered” to detect the alleged connection between Neurontin and suicide).  The power argument, however, does not make sense in the context of a meta-analysis, which is aggregating data across studies to overcome the alleged lack of power in a single study.

Not surprisingly, clinical trials of a non-cardiac medication will often report no event of the outcome of interest, such as heart attack.  These trials are referred to as a “zero event”, which can happen in one or both arms of a given trial.  Some searchers exclude these studies from a meta-analysis because of the impossibility of calculating an odds ratio without using imputation in the zero cells of the 2 x 2 tables. Although there are methods to address zero-event trials, some researchers believe that the existence of several zero-event trials essentially means that the sparse data from rare outcomes deprives statistical tests of their usual meaning.  Traditional statistical standards of significance (p < 0.05) are described as “tenuous,” and too high, in this situation. A.V. Hernandez, E. Walker, J. P. Ioannidis, M.W. Kattan, “Challenges in meta-analysis of randomized clinical trials for rare harmful cardiovascular events: the case of rosiglitazone,” 156 Am. Heart J. 23, 28 (2008).

The exclusion of zero-event trials from meta-analyses of rare outcomes can yield biased results. See generally M.J. Bradburn, J.J Deeks, J.A. Berlin, and A. Russell Localio,” Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events,” 26 Statistics in Med. 53 (2007); M.J. Sweeting, A.J. Sutton, and P.C. Lambert, “What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data,” 23 Statistics in Med. 1351 (2004)(erratum at 25 Statistics in Med. 2700 (2006) (“Many routinely used summary methods provide widely ranging estimates when applied to sparse data with high imbalance between the size of the studies’ arms. A sensitivity analysis using several methods and continuity correction factors is advocated for routine practice.”).

Others researchers include zero-event trials as providing helpful information about the absence of risk. Zero-event trials:

“provide relevant data by showing that event rates for both the intervention and control groups are low and relatively equal. Excluding such trial data potentially increases the risk of inflating the magnitude of the pooled treatment effect.”

J.O. Friedrich, N.K. Adhikari, J. Beyene, “Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data,” 5 BMC Med. Res. Methodol. 2 (2007)[cited as Friedrich].  Zero event trials can be included in meta-analyses by using something called a standard “continuity correction,” which involves imputing events, or fractional events, in all cells of the 2 x 2 table. One approach, the zero is replaced with 0.5 and all other numbers are increased by 0.5. Friedrich at 7.

After examining the bias in several meta-analyses from excluding zero-event trials, Friedrich and colleagues recommended:

“We believe these trials [with zero events] should also be included if RR [relative risks] or OR [odds ratios] are the effect measures to provide a more conservative estimate of effect size(even if this change in effect size is very small for RR and OR), and to provide analytic consistency and include the same number of trials in the meta-analysis, regardless of the summary effect measure used. Inclusion of zero total event trials would enable the inclusion of all available randomized controlled data in a meta-analysis, thereby providing the most generalizable estimate of treatment effect.”

Friedrich at 5-6.

Wei addressed the problem of zero-event trials by using common imputation methods, not so different from what plaintiffs’ expert witness Dr. Ix used in the gadolinium litigation. See Meta-Meta-Analysis — The Gadolinium MDL — More Than Ix’se Dixit.  Given that plaintiffs advanced a mechanistic theory, which would explain cardiovascular thrombotic events almost immediately upon first ingestion of Celebrex, Professor Wei’s attempt to save the data inherent in zero-event trials by “continuity correction” or imputation methods seems reasonable and well within meta-analytic procedures.

 

RISK DIFFERENCE

Professor Wei did not limit himself to a single method or approach.  In addition to using imputation methods, Wei used risk difference, rather than risk ratios, as the parameter of interest.  The risk difference is simply the difference between two risks: the risk or probability of an event in one group less the risk or probability of that event in another group.  Contrary to the plaintiffs’ claims, there is nothing novel or subversive about conducting a meta-analysis with the risk difference as the parameter of interest, rather than a risk ratio.  In the context of randomized clinical trials, the risk difference is expected as a measure of absolute effect.  See generally, Michael Borenstein, L. V. Hedges, J. P. T. Higgins, and H. R. Rothstein, Introduction to Meta-Analysis (2009); Julian PT Higgins and Sally Green, eds., Cochrane Handbook for Systematic Reviews of Interventions (2008)

Like risk ratios, the risk difference yield a calculated confidence interval at any desired coefficient of confidence.  Confidence intervals for dichotomous events are often based upon approximate methods that build upon the normal approximation to the binomial distribution.  These approximate methods require assumptions of sample size that may not be met in cases involving sparse data.  With modern computers, calculating exact confidence intervals is not particularly difficult, and Professor Wei has published a methods paper in which he explains the desirability of using the risk difference with exact intervals in addressing meta-analyses of sparse data, such as was involved in the Celebrex litigation.  See L. Tian, T. Cai, M.A. Pfeffer, N. Piankov, P.Y. Cremieux, and L.J. Wei, “Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2 x 2 tables with all available data but without artificial continuity correction,” 10 Biostatistics 275 (2009).

Plaintiffs attacked Wei’s approach as “novel” and not generally accepted.  Judge Swain appropriately dismissed this attack:

“Dr. Wei’s methodology, the validity of which Plaintiffs contest and the novelty of which Plaintiffs seek to highlight, appears to have survived the rigors of peer review at least once, and is subject to critique by virtue of its transparency. Dr. Wei’s report, supplemented by his declaration, is sufficient to meet Defendants’ burden of demonstrating that his testimony is the product of reliable principles and methods. He has explained his methods, which can be tested. Plaintiffs’ critiques of Dr. Wei’s choices regarding which trials to include in his own meta-analysis, the origins of the data he used, the date at which he undertook his meta-analysis, and at whose behest he performed his analysis all go to the weight of Dr. Wei’s testimony.”

In re Pfizer Inc. Securities Litig., 2010 WL 1047618, *7 (S.D.N.Y. 2010).  The approach taken by Wei is novel only in the sense that researchers have not previously tried to push the methodological envelope of meta-analysis to deploy the technique for rare outcomes and sparse data, with many zero-event trials.  The risk difference approach is well suited to the situation, and the use of exact confidence intervals is hardly novel or dubious.

Posted in Causation, Frye, Meta-analysis, Rule 702 | Comments Off on Meta-Meta-Analysis – Celebrex Litigation – The Claims – Part One

The Pennsylvania Supreme Court Rejects “Every Exposure is Substantial” Mantra

May 23rd, 2012

Over two years ago, I wrote about a curious decision by the Pennsylvania Superior Court, in Betz v. Pneumo Abex LLC, 998 A.2d 962 (Pa. Super. 2010) (en banc). In Betz, the Superior Court reversed an Alleghany County Court of Common Pleas judge’s Frye ruling, in an asbestos mesothelioma case, that an expert witness’s opinion that each exposure had been a “substantial contributing factor” was both novel and not generally accepted. In re Toxic Substance Cases, No. A.D. 03-319, slip op., 2006 WL 2404008 (C.P. Allegheny, Aug. 17, 2006). What was remarkable was that the majority of the en banc Superior Court treated the science and the record so cavalierly, and treated the law even more so. See “Betz v. Pneumo Abex: the Recrudescence of Ferebee in Pennsylvania” (May 5th, 2010); and “The Betz Evidence Rule” (May 6th, 2010).

Today, mirabile dictu, the Pennsylvania Supreme Court unanimously reversed the Superior Court’s errant opinion.  (Justice Melvin did not, of course, participate.) The Supreme Court held that the trial judge, Judge Colville, did not abuse his discretion in conducting a Frye hearing or in ruling that the plaintiffs’ expert witness’s opinion, that every fiber contributes substantially to plaintiff’s mesothelioma, was both novel and not “generally accepted.”

The Supreme Court remanded to the Superior Court for a decision on unspecified, remaining issues. The Court’s 53 page opinion carefully dissects the ipse dixit nature of plaintiffs’ expert witness’s specific causation opinion, and essentially concludes that there was no science in it all.

Posted in Causation, Frye, Scientific Evidence | Comments Off on The Pennsylvania Supreme Court Rejects “Every Exposure is Substantial” Mantra

Newer Entries »