TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Transparency, Confusion, and Obscurantism

October 31st, 2014

In NIEHS Transparency? We Can See Right Through You (July 10, 2014), I chastised authors Kevin C. Elliott and David B. Resnik for their confusing and confused arguments about standards of proof, the definition of risk, and conflicts of interest (COIs). See Kevin C. Elliott and David B. Resnik, “Science, Policy, and the Transparency of Values,” 122 Envt’l Health Persp. 647 (2014) [Elliott & Resnik]. In their focus on environmentalism and environmental policy, Elliott and Resnik seem intent upon substituting various presumptions, leaps of faith, and unproven extrapolations for actual evidence and valid inference, in the hope of improving the environment and reducing risk to life. But to get to their goal, Elliott and Resnik engage in various equivocations and ambiguities in their use of “risk,” and they compound the muddle by introducing a sliding scale of “standards of evidence,” for legal, regulatory, and scientific conclusions.

Dr. David H. Schwartz is a scientist, who received his doctoral degree in Neuroscience from Princeton University, and his postdoctoral training in Neuropharmacology and Neurophysiology at the Center for Molecular and Behavioral Neuroscience, in Rutgers University. Dr. Schwartz has since gone to found one of the leading scientific consulting firms, Innovative Science Solutions (ISS), which supports both regulatory and litigation claims and defenses, as may scientifically appropriate. Given his experience, Dr. Schwartz is well positioned to address the standards of scientific evidentiary conclusions across regulatory, litigation, and scientific communities.

In this month’s issue of Environmental Health Perspectives (EHP), Dr. David Schwartz adds to the criticism of Elliott and Resnik’s tendentious editorial. David H. Schwartz, “Policy and the Transparency of Values in Science,” 122 Envt’l Health Persp. A291 (2014). Schwartz points out that “[a]lthough … different venues or contexts require different standards of evidence, it is important to emphasize that the actual scientific evidence remains constant.” Id.

Dr. Schwartz points out transparency is needed in how standards and evidence are represented in scientific and legal discourse, and he takes Elliott and Resnik to task for arguing, from ignorance, that litigation burdens are different from scientific standards. At times some writers misrepresent the nature of their evidence, or its weakness, and when challenged, attempt to excuse the laxness in standards by adverting to the regulatory or litigation contexts in which they are speaking. In some regulatory contexts, the burdens of proof are deliberately reduced, or shifted to the regulated industry. In litigation, the standard or burden of proof is rarely different from the scientific enterprise itself. As the United States Supreme Court made clear, trial courts must inquire whether an expert witness ‘‘employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.’’ Kumho Tire Co. v. Carmichael, 526 U.S. 137, 152 (1999). Expert witnesses who fail to exercise the same intellectual rigor in the courtroom as in the laboratory, are eminently disposable or excludable from the legal process.

Schwartz also points out, as I had in my blog post, that “[w]hen using science to inform policy, transparency is critical. However, this transparency should include not only financial ties to industry but also ties to advocacy organizations and other strongly held points of view.”

In their Reply to Dr. Schwartz, Elliott and Resnik concede the importance of non-financial conflicts of interest, but they dig in on the supposed lower standard for scientific claims:

“we caution against equating the standards of evidence expected in tort law with those expected in more traditional scientific contexts. The tort system requires only a preponderance of evidence (> 50% likelihood) to win a case; this is much weaker evidence than scientists typically demand when presenting or publishing results, and confusion about these differing standards has led to significant legal controversies (Cranor 2006).”

Rather than citing any pertinent or persuasive legal authority, Elliott and Resnik cite an expert witness, Carl Cranor, neither a lawyer nor a scientist, who has worked steadfastly for the litigation industry (the plaintiffs’ bar) on various matters. The “caution” of Elliott and Resnik is directly contradicted by the Supreme Court’s pronouncement in Kumho Tire, and is fueled by a ignoratio elenchi that is based upon a confusion between the probability of repeated sampling with confidence intervals (usually 95%) and the posterior probability of a claim: namely, the probability of the claim given the admissible evidence. As the Reference Manual for Scientific Evidence makes clear, these are very different probabilities, which Cranor and others have consistently confused. Elliott and Resnik ought to know better.

The Current Crisis – Ebola Comes to the Land of Litigation

October 29th, 2014

Lying About

President Obama has appointed a political operative, a lawyer, to be the “Ebola czar,” while the Surgeon General and Secretary of Health and Human Resources remain in hiding. Dr. Craig Spencer, who lies in a Bellevue Hospital isolation ward, lied about his travels about New York City when talking to the New York City authorities. He claimed to have been in voluntary quarantine and isolation at his Manhattan home upon returning from West Africa. Jamie Schram & Bruce Golding, “Ebola doctor ‛lied’ about NYC travelsNY Post (Oct. 29, 2014) (“The city’s first Ebola patient initially lied to authorities about his travels around the city following his return from treating disease victims in Africa, law-enforcement sources said.”) We now know he used the subways, ate at public restaurants, and generally cavorted about town.

Foolish Consistencies and Some Inconsistency, Too

President Obama has pressured Governors Christie and Cuomo to back off their stricter quarantine rules, and demonstrated that Cuomo is politically soft in the center. At the same time that the Obama’s administration has bullied critics of its voluntary quarantine protocol, they have imposed mandatory quarantine on military personnel, returning from West Africa. Secretary of War Defense has announced a mandatory quarantine. See Starr, “Hagel announces mandatory Ebola quarantineCNN (Oct. 29, 2014). Ah, our leaders would follow Ralph Waldo Emerson, on Self-Reliance and self-quarantine: “[a] foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall.”

Australia has banned travel with Ebola affected countries, which should now include the United States. Michelle Nichols and Umaru Fofana, “Australia bans travel from Ebola-hit countries; U.S. isolates troopsReuters (Oct. 28, 2014). Of course, Australia was settled by criminals, as we all know.

Wild Nurse Hickox

Voluntary quarantine is a quaint notion. A healthcare worker takes his or her temperature twice a day, but fevers come on, when they come on. Nurse Kaci Hickox, whose “human rights” were supposedly violated by Order of Governor Christie, has been removed to Maine, whence she has announced her attention to violate Maine’s lax rule that requires voluntary quarantine. Jennifer Levitz, “Nurse in Ebola Quarantine Flap Says She Won’t Obey Maine’s Isolation Rules: Kaci Hickox Says She Will Go to Court if Restrictions Aren’t Removed by ThursdayWall Street Journal (Oct. 29, 2014). So much for the human rights of Maine’s good citizens, not to mention the rights of the moose, and other innocent species.

The litigation industry is, I am sure, gearing up to meet the crisis. And Nurse Hickox is now be free to litigate her voluntary quarantine in Maine.

Gad-zooks – Expert Witness Dishonesty

October 18th, 2014

This is the first in what I hope will be a continuing series, tagged as the Expert Witness Hall of Shame.

*     *     *     *     *

Shayne Cox Gad is a toxicologist and a principal in the firm, Gad Consulting Services, in Cary, North Carolina. In 1977, Gad was awarded his doctorate in pharmacology and toxicology by the University of Texas (Austin). Some years later, Gad apparently awarded himself a Silver Star, and three Purple Hearts.

The Stolen Valor Act[1], effective in December 2006, made false representations of having received military decorations or awards a federal crime. Gad was charged with violating the Stolen Valor Act, and in February 2009, he pleaded guilty to dishonesty and specious claiming prohibited by the Act.

Before and after his conviction by guilty plea, Gad testified as an expert witness in litigation. He was an expert witness for plaintiff in an Oklahoma state court case, Helton v. Allergan, Inc., in which Dr. Sharla Helton complained that Botox caused her neurologic problems and pain that prevented her from working as an obstetrician/gynecologist.

Whatever the merits of the claims about Botox, Allergan might well have resisted settling a case in which plaintiff’s claim rested upon the testimony of an expert witness, convicted for dishonesty. Trial counsel for Allergan, Vaughn Crawford, cross-examined Gad, on April 27, 2010. Vaughn’s examination went immediately to prior conviction. “Allergan unmasks anti-Botox expert” (April 28, 2010; updated Aug. 21, 2013). Vaughn sprung the impeachment:

Q You are the same Shayne Cox Gad who has been adjudged guilty by the Eastern Federal District Court in North Carolina for crimes involving false statements and dishonesty, aren’t you, sir.

A Yes, sir.

Q Yes. Specifically in February of 2009, you were adjudged guilty by that Court of falsely representing that you had been awarded military decorations and medals including the Navy cross, aren’t you, sir.

A Yes, sir.

Helton v. Allergan, Inc., Notes of Testimony by Shayne Cox Gad at 48-49 (April 27, 2010).

Crawford pressed. Not only had Gad confessed to the crime, he had made various acts of contrition in his Pre-sentencing Report, in which Gad asked that he be placed on probation rather than incarcerated. One of the representations Gad made in the Report was that he would no longer testify as an expert witness in litigation. Gad’s plea was accepted and he was placed on probation as he and his lawyer requested.

As dramatic as Crawford’s impeachment of Gad must have been, the jury shrugged it off and awarded Dr. Helton 15 million dollars, which came to 18 million with pre-judgment interest. Helton v. Allergan Inc., No. CJ-2009-2171 (Okla. Dist. Ct., Oklahoma Cty.) (jury voted 10 to 2 to award actual but no punitive damages). The Oklahoma intermediate appellate court affirmed in an unpublished opinion, and the Oklahoma Supreme Court refused to grant discretionary review. Helton v. Allergan Inc., No. 2009-2171 (Okla. Civ. App. Sept. 6, 2013); “Okla. Appeals Court Backs $15M Award In Botox Injury,” Law 360 (Sept. 10, 2013). See PR Newswire, “Botox Victim Wins $18 Million: Oklahoma Supreme Court Affirms Botulism Verdict for McGinnis Lochridge Client Against Allergan, Inc.,” (May 9, 2014) (law firm press release misleadingly claiming that the Oklahoma Supreme Court had affirmed, when in fact, the Court had declined discretionary review).

Having pled guilty in federal court, Gad would have recited the facts of his crime in court before the imposition of sentencing, as required under Federal Rule of Criminal Procedure 11. Furthermore, even if Gad’s criminal defense lawyer drafted the Pre-sentencing Report, Gad was the principal responsible for his agent attorney’s representation that he, Gad, would not testify again as an expert witness.

Gad tried to resist the gist of the cross-examination by suggesting that others, not he, had made the representation. And on redirect, plaintiff’s counsel Chester elicited an apology, not to the court, or to the defendants, but to Dr. Helton, the plaintiff:

Q Would you, at least, apologize to my client for me because she hired me and I hired you.

A I do apologize for that.

Q Have you lied about anything in this case?

A No, sir.

Q You put five kids through college; is that right?

A Yes, sir.

Q You’ve had this career. Why would you do something like this?

A Well, that, of course, was discussed in a lot more detail in the documents having to do with it, but it was something that got out there 25 years ago and I thought it was put away. I did my best to expunge it from the record, and I was unsuccessful. Twenty-five years ago I was a very different person, a lot younger than I am now.

Helton v. Allergan, Inc., Notes of Testimony by Shayne Cox Gad at 141 (April 27, 2010).

The internet is, however, unforgiving and unforgetting. A curriculum vitae for Gad, labeled August 2005, states the following for military service:

June 1970 to April 1974 (Active):

Served on riverine craft in Mekong Delta of Vietnam and as O.I.C. of Armory, Quonset Point, Rhode Island. Served on USS Intrepid (CVS-11) as special 7 weapons officer, deck division officer and as First Lieutenant. Qualified as O.D. underway on Intrepid. Made several deployments overseas – mainly to Europe and the Mediterranean. Released from active duty in the permanent grade of LT(jg). Received Silver Star, 3 Bronze Stars, 3 Purple Hearts.  * * *

Holds current (2003) top secret clearance.”

C.V. for Shayne Cox Gad, Ph.D., D.A.B.T., A.T.S. (emphasis added).

The charging document against Gad, from United States v. Gad, also refutes the notion that Gad’s false statements occurred in the distant past, but rather that they were made “[o]n or about November 2004, and continuing up to and including March 29, 2007 … .” Immunity from prosecution for perjury in another case, United States v. Caputo appeared to be part of the consideration for the plea deal in U.S. v. Gad. Thus the inclusion of a representation, in the pre-sentencing report, that “[a]dditionally, Dr. Gad has agreed to no longer testify as an expert witness in the future.”

The mischief Gad created by his dishonesty was thus not limited to the Helton case. Gad’s testimony looks even more dubious in view of the Caputo case, a criminal case in which Gad testified for a federal prosecutor. In Caputo, the prosecutor informed the defendants, executives of AbTox Inc., that Gad “had committed perjury by falsely claiming military experience and decorations.” United States v. Caputo, Case No. 10-1964, 397 Fed. Appx. 216, (7th Cir. Oct. 12, 2010) (unpublished). See alsoAbTox Execs Seek New Trial Over Witness ‘Perjury’ – Law360” (Sept. 16, 2010).

The Caputo defendants had been charged with lying to the FDA and selling a misbranded medical sterilization devices. United States v. Caputo, 517 F.3d 935 (7th Cir. 2008) (affirming convictions). In rebuttal, Gad testified that defendants could not reasonably have held the beliefs they claimed to have held in good faith. Because of how the issue of good faith arose, the Circuit held that Gad’s perjurious testimony was harmless error that could not support the grant of a new trial. United States v. Caputo, Case No. 10-1964, 397 Fed. Appx. 216, (7th Cir. Oct. 12, 2010).

When the government informed the defendants that Gad had committed perjury, the Caputo defendants moved for a new trial on grounds of newly discovered evidence. The defendants went beyond Gad’s perjury disclosed by the prosecutors, and charged that Gad’s resume was a sham and that Gad had lied about other credentials as well.

According to the defendants’ motion in Caputo, Gad had misrepresented several credentials and misleadingly claimed to have had professional experience in medicine and toxicology, which experience Gad, in fact, lacked. The defendants, in Caputo, alleged other misrepresentations. Gad had testified in their case, and in the Helton case, that he had taught a course at the Duke University Medical School in the early 2000s and had lectured at the school since. Gad’s resume listed a professorship of toxicology at the College of St. Elizabeth, where he purportedly developed the school’s bachelor of science toxicology program, according to the motion. In their motion for a new trial, the AbTox executives, Caputo and Riley, provided an offer of proof that neither Duke University Medical School nor the College of St. Elizabeth had any record of Gad’s faculty status, and St. Elizabeth lacked a bachelor’s program in toxicology. Caputo and Riley also adverted to the federal prosecutors’ own earlier finding that Gad had lied about his military record during their trial.

I don’t know whether Gad testified again.  Some of Gad’s dubious views on toxicology are cited with approval by legal commentators who would dilute the scientific standard for causation[2].

“Falsus in uno, falsus in omnibus.” The essence of the crime is specious claiming.


[1] United States v. Alvarez, 132 S. Ct. 1421 (2012) (holding that the Stolen Valor Act was unconstitutional).

[2] See Shayne C. Gad, “Model Selection and Scaling,” in Shayne C. Gad & Christopher P. Chengelis eds., Animal Models in Toxicology 813 (1992), cited by Carl F. Cranor & David A. Eastmond, “Scientific Ignorance and Reliable Patterns of Evidence in Toxic Tort Causation: Is There a Need for Liability Reform? 64 Law & Contemporary Problems 5, 27 & n.120 (2001), and by Erica Beecher-Monas, Evaluating Scientific Evidence An Interdisciplinary Framework for Intellectual Due Process at 74 & n.63 (2007) (citing Gad’s book at page 826, for the argument that humans may be more sensitive to chemical effects than smaller species).

 

Courts Can and Must Acknowledge Multiple Comparisons in Statistical Analyses

October 14th, 2014

In excluding the proffered testimony of Dr. Anick Bérard, a Canadian perinatal epidemiologist in the Université de Montréal, the Zoloft MDL trial court discussed several methodological shortcomings and failures, including Bérard’s reliance upon claims of statistical significance from studies that conducted dozens and hundreds of multiple comparisons. See In re Zoloft (Sertraline Hydrochloride) Prods. Liab. Litig., MDL No. 2342; 12-md-2342, 2014 U.S. Dist. LEXIS 87592; 2014 WL 2921648 (E.D. Pa. June 27, 2014) (Rufe, J.). The Zoloft MDL court was not the first court to recognize the problem of over-interpreting the putative statistical significance of results that were one among many statistical tests in a single study. The court was, however, among a fairly small group of judges who have shown the needed statistical acumen in looking beyond the reported p-value or confidence interval to the actual methods used in a study[1].

A complete and fair evaluation of the evidence in situations as occurred in the Zoloft birth defects epidemiology required more than the presentation of the size of the random error, or the width of the 95 percent confidence interval.  When the sample estimate arises from a study with multiple testing, presenting the sample estimate with the confidence interval, or p-value, can be highly misleading if the p-value is used for hypothesis testing.  The fact of multiple testing will inflate the false-positive error rate. Dr. Bérard ignored the context of the studies she relied upon. What was noteworthy is that Bérard encountered a federal judge who adhered to the assigned task of evaluating methodology and its relationship with conclusions.

*   *   *   *   *   *   *

There is no unique solution to the problem of multiple comparisons. Some researchers use Bonferroni or other quantitative adjustments to p-values or confidence intervals, whereas others reject adjustments in favor of qualitative assessments of the data in the full context of the study and its methods. See, e.g., Kenneth J. Rothman, “No Adjustments Are Needed For Multiple Comparisons,” 1 Epidemiology 43 (1990) (arguing that adjustments mechanize and trivialize the problem of interpreting multiple comparisons). Two things are clear from Professor Rothman’s analysis. First for someone intent upon strict statistical significance testing, the presence of multiple comparisons means that the rejection of the null hypothesis cannot be done without further consideration of the nature and extent of both the disclosed and undisclosed statistical testing. Rothman, of course, has inveighed against strict significance testing under any circumstance, but the multiple testing would only compound the problem. Second, although failure to adjust p-values or intervals quantitatively may be acceptable, failure to acknowledge the multiple testing is poor statistical practice. The practice is, alas, too prevalent for anyone to say that ignoring multiple testing is fraudulent, and the Zoloft MDL court certainly did not condemn Dr. Bérard as a fraudfeasor[2].

In one case, a pharmaceutical company described a p-value of 0.058 as statistical significant in a “Dear Doctor” letter, no doubt to avoid a claim of under-warning physicians. Vanderwerf v. SmithKline Beecham Corp., 529 F.Supp. 2d 1294, 1301 & n.9 (D. Kan. 2008), appeal dism’d, 603 F.3d 842 (10th Cir. 2010). The trial court[3], quoting the FDA clinical review, reported that a finding of “significance” at the 0.05 level “must be discounted for the large number of comparisons made. Id. at 1303, 1308.

Previous cases have also acknowledged the multiple testing problem. In litigation claims for compensation for brain tumors for cell phone use, plaintiffs’ expert witness relied upon subgroup analysis, which added to the number of tests conducted within the epidemiologic study at issue. Newman v. Motorola, Inc., 218 F. Supp. 2d 769, 779 (D. Md. 2002), aff’d, 78 Fed. App’x 292 (4th Cir. 2003). The trial court explained:

“[Plaintiff’s expert] puts overdue emphasis on the positive findings for isolated subgroups of tumors. As Dr. Stampfer explained, it is not good scientific methodology to highlight certain elevated subgroups as significant findings without having earlier enunciated a hypothesis to look for or explain particular patterns, such as dose-response effect. In addition, when there is a high number of subgroup comparisons, at least some will show a statistical significance by chance alone.”

Id. And shortly after the Supreme Court decided Daubert, the Tenth Circuit faced the reality of data dredging in litigation, and its effect on the meaning of “significance”:

“Even if the elevated levels of lung cancer for men had been statistically significant a court might well take account of the statistical “Texas Sharpshooter” fallacy in which a person shoots bullets at the side of a barn, then, after the fact, finds a cluster of holes and draws a circle around it to show how accurate his aim was. With eight kinds of cancer for each sex there would be sixteen potential categories here around which to “draw a circle” to show a statistically significant level of cancer. With independent variables one would expect one statistically significant reading in every twenty categories at a 95% confidence level purely by random chance.”

Boughton v. Cotter Corp., 65 F.3d 823, 835 n. 20 (10th Cir. 1995). See also Novo Nordisk A/S v. Caraco Pharm. Labs., 775 F.Supp. 2d 985, 1019-20 & n.21 (2011) (describing the Bonferroni correction, and noting that expert witness biostatistician Marcello Pagano had criticized the use of post-hoc, “cherry-picked” data that were not part of the prespecified protocol analysis, and the failure to use a “correction factor,” and that another biostatistician expert witness, Howard Tzvi Thaler, had described a “strict set of well-accepted guidelines for correcting or adjusting analysis obtained from the `post hoc’ analysis”).

The notorious Wells[4] case was cited by the Supreme Court in Matrixx Initiatives[5] for the proposition that statistical significance was unnecessary. Ironically, at least one of the studies relied upon by the plaintiffs’ expert witnesses in Wells had some outcomes with p-values below five percent. The problem, addressed by defense expert witnesses and ignored by the plaintiffs’ witnesses and Judge Shoob, was that there were over 20 reported outcomes, and probably many more outcomes analyzed but not reported. Accordingly, some qualitative or quantitative adjustment was required in Wells. See Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation 93 (1997)[6].

Reference Manual on Scientific Evidence

David Kaye’s and the late David Freedman’s chapter on statistics in the third, most recent, edition of Reference Manual, offers some helpful insights into the problem of multiple testing:

4. How many tests have been done?

Repeated testing complicates the interpretation of significance levels. If enough comparisons are made, random error almost guarantees that some will yield ‘significant’ findings, even when there is no real effect. To illustrate the point, consider the problem of deciding whether a coin is biased. The probability that a fair coin will produce 10 heads when tossed 10 times is (1/2)10 = 1/1024. Observing 10 heads in the first 10 tosses, therefore, would be strong evidence that the coin is biased. Nonetheless, if a fair coin is tossed a few thousand times, it is likely that at least one string of ten consecutive heads will appear. Ten heads in the first ten tosses means one thing; a run of ten heads somewhere along the way to a few thousand tosses of a coin means quite another. A test—looking for a run of ten heads—can be repeated too often.

Artifacts from multiple testing are commonplace. Because research that fails to uncover significance often is not published, reviews of the literature may produce an unduly large number of studies finding statistical significance.111 Even a single researcher may examine so many different relationships that a few will achieve statistical significance by mere happenstance. Almost any large dataset—even pages from a table of random digits—will contain some unusual pattern that can be uncovered by diligent search. Having detected the pattern, the analyst can perform a statistical test for it, blandly ignoring the search effort. Statistical significance is bound to follow.

There are statistical methods for dealing with multiple looks at the data, which permit the calculation of meaningful p-values in certain cases.112 However, no general solution is available… . In these situations, courts should not be overly impressed with claims that estimates are significant. …”

Reference Manual on Scientific Evidence at 256-57 (3d ed. 2011).

When a lawyer asks a witness whether a sample statistic is “statistically significant,” there is the danger that the answer will be interpreted or argued as a Type I error rate, or worse yet, as a posterior probability for the null hypothesis.  When the sample statistic has a p-value below 0.05, in the context of multiple testing, completeness requires the presentation of the information about the number of tests and the distorting effect of multiple testing on preserving a pre-specified Type I error rate.  Even a nominally statistically significant finding must be understood in the full context of the study.

Some texts and journals recommend that the Type I error rate not be modified in the paper, as long as readers can observe the number of multiple comparisons that took place and make the adjustment for themselves.  Most jurors and judges are not sufficiently knowledgeable to make the adjustment without expert assistance, and so the fact of multiple testing, and its implication, are additional examples of how the rule of completeness may require the presentation of appropriate qualifications and explanations at the same time as the information about “statistical significance.”

*     *     *     *     *

Despite the guidance provided by the Reference Manual, some courts have remained resistant to the need to consider multiple comparison issues. Statistical issues arise frequently in securities fraud cases against pharmaceutical cases, involving the need to evaluate and interpret clinical trial data for the benefit of shareholders. In a typical case, joint venturers Aeterna Zentaris Inc. and Keryx Biopharmaceuticals, Inc., were both targeted by investors for alleged Rule 10(b)(5) violations involving statements of clinical trial results, made in SEC filings, press releases, investor presentations and investor conference calls from 2009 to 2012. Abely v. Aeterna Zentaris Inc., No. 12 Civ. 4711(PKC), 2013 WL 2399869 (S.D.N.Y. May 29, 2013); In re Keryx Biopharms, Inc., Sec. Litig., 1307(KBF), 2014 WL 585658 (S.D.N.Y. Feb. 14, 2014).

The clinical trial at issue tested perifosine in conjunction with, and without, other therapies, in multiple arms, which examined efficacy for seven different types of cancer. After a preliminary phase II trial yielded promising results for metastatic colon cancer, the colon cancer arm proceeded. According to plaintiffs, the defendants repeatedly claimed that perifosine had demonstrated “statistically significant positive results.” In re Keryx at *2, 3.

The plaintiffs alleged that defendants’ statements omitted material facts, including the full extent of multiple testing in the design and conduct of the phase II trial, without adjustments supposedly “required” by regulatory guidance and generally accepted statistical principles. The plaintiffs asserted that the multiple comparisons involved in testing perifosine in so many different kinds of cancer patients, at various doses, with and against so many different types of other cancer therapies, compounded by multiple interim analyses, inflated the risk of Type I errors such that some statistical adjustment should have been applied before claiming that a statistically significant survival benefit had been found in one arm, with colorectal cancer patients. In re Keryx at *2-3, *10.

The trial court dismissed these allegation given that the trial protocol had been published, although over two years after the initial press release, which started the class period, and which failed to disclose the full extent of multiple testing and lack of statistical correction, which omitted this disclosure. In re Keryx at *4, *11. The trial court emphatically rejected the plaintiffs’ efforts to dictate methodology and interpretative strategy. The trial court was loathe to allow securities fraud claims over allegations of improper statistical methodology, which:

“would be equivalent to a determination that if a researcher leaves any of its methodology out of its public statements — how it did what it did or was planning to do — it could amount to an actionable false statement or omission. This is not what the law anticipates or requires.”

In re Keryx at *10[7]. According to the trial court, providing p-values for comparisons between therapies, without disclosing the extent of unplanned interim analyses or the number of multiple comparisons is “not falsity; it is less disclosure than plaintiffs would have liked.” Id. at *11.

“It would indeed be unjust—and could lead to unfortunate consequences beyond a single lawsuit—if the securities laws become a tool to second guess how clinical trials are designed and managed. The law prevents such a result; the Court applies that law here, and thus dismisses these actions.”

Id. at *1.

The court’s characterization of the fraud claims as a challenge to trial methodology rather than data interpretation and communication decidedly evaded the thrust of the plaintiffs’ fraud complaint. Data interpretation will often be part of the methodology outlined in a protocol. The Keryx case also confused criticism of the design and execution of a clinical trial with criticism of the communication of the trial results.


[1] Predictably, some plaintiffs’ counsel accused the MDL trial judge of acting as a statistician and second-guessing the statistical inferences drawn by the party expert witness. See, e.g., Max Kennerly, “Daubert Doesn’t Ask Judges To Become Experts On Statistics” (July 22, 2014). Federal Rule of Evidence 702 requires trial judges to evaluate the methodology used to determine whether it is valid. Kennerly would limit the trial judge to a simple determination of whether the expert witness used statistics, and whether statistics generally are appropriately used. In his words, “[t]o go with the baseball metaphors so often (and wrongly) used in the law, when it comes to Daubert, the judge isn’t an umpire calling balls and strikes, they’re [sic] more like a league official checking to make sure the players are using regulation equipment. Mere disagreements about the science itself, and about the expert’s conclusions, are to be made by the jury in the courtroom.” This position is rejected by the explicit wording of the statute, as well as the Supreme Court opinions leading up to the revision in the statute. To extend Kennerly’s overextended metaphor even further, the trial court must not only make sure that the players are using regulation equipment, but also that pitchers, expert witnesses, aren’t throwing spitballs or balking in their pitching of opinions. Judge Rufe, in the Zoloft MDL, did no more than asked of her by Rule 702 and the Reference Manual.

[2] Perhaps the prosecutor, jury, and trial and appellate judges in United States v. Harkonen would be willing to brand Dr. Bérard a fraudfeasor. U.S. v. Harkonen, 2009 WL 1578712, 2010 WL 2985257 (N.D. Cal.), aff’d, 2013 WL 782354 (9th Cir. Mar. 4, 2013), cert. denied, ___ U.S. ___ (2013).

[3] The trial court also acknowledged the Reference Manual on Scientific Evidence 127-28 (2d ed. 2000). Unfortunately, the court erred in interpreting the meaning of a 95 percent confidence interval as showing “the true relative risk value will be between the high and low ends of the confidence interval 95 percent of the time.” Vanderwerf v. SmithKlineBeecham Corp., 529 F.Supp. 2d at 1302 n.10.

[4] Wells v. Ortho Pharm. Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff ’d, and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S. 950 (1986).

[5] Matrixx Initiatives, Inc. v. Siracusano, 131 S.Ct. 1309 (2011)

[6] Zeisel and Kaye contrast the lack of appreciation for statistical methodology in Wells with the handling of the multiple comparison issue in an English case, Reay v. British Nuclear Fuels (Q.B. Oct. 8, 1993). In Reay, children of fathers who worked in nuclear power plants and who developed leukemia, sued. Their expert witnesses relied upon a study that reported 50 or so hypotheses. Zeisel and Kaye quote the trial judge as acknowledging that the number of hypotheses considered inflates the nominal value of the p-value and reduces confidence in the study’s result. Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation 93 (1997) (discussing Reay case as published in The Independent, Nov. 22, 1993).

[7] Of course, this is exactly what happened to Dr. Scott Harkonen, who was indicted and convicted under the Wire Fraud Act, despite issuing a press release that included a notice of an investor conference call within a couple of weeks, when investors and others could inquire fully about the clinical trial results.