TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

The Role of the “Science Lawyer” in Modern Litigation

July 27th, 2014

“For the rational study of the law the blackletter man may be the man of the present, but the man of the future is the man of statistics and the master of economics.”[1]

“Judges commonly are elderly men, and are more likely to hate at sight any analysis to which they are not accustomed, and which disturbs repose of mind … .”[2]

The emergence of complex scientific issues in post-World War II American litigation has challenged state and federal legal systems, both civil and criminal. Various issues, such as the validity of forensic science and biological causation, have tested the competency of lawyers, both judges and counsel. This scientific complexity also questions whether lay juries can or should be continued in their traditional role as fact finders[3].  Some commentators have suggested that complexity should be a basis for reallocating fact finding to judges.[4] Other commentators remain ideologically or constitutionally committed to the jury as fact finder.

The superiority of judges as fact finders in complex scientific cases remains to be shown. Clearly juries often perform better than individual lay judges as scientific fact finders, but their time commitment is inadequate.  Furthermore, the jury decision process is typically a binary decision without any articulation of reasoning.  The jury’s hidden reasoning process on important scientific issues violates basic tenets of transparency and due process.  Many modern cases present such difficult scientific issues that some authors have argued that we should establish scientific courts or institute procedures for blue-ribbon juries[5].  The constitutionality of having scientists, or specially qualified judges, serve as fact finders has never been clearly addressed.[6]  Other commentators have argued in favor of the existing set of judicial tools, such as appointment of testifying “neutral” expert witnesses and scientific advisors for trial judges.[7] These approaches have been generally available in federal and some state trial courts, but they have rarely been used.  Some examples of their deployment include the silicone gel breast implant litigation[8], and cases involving Parlodel[9] and Bendectin[10].

Over 20 years ago, in 1993, the United States Supreme Court handed down its Daubert decision.  In large measure, the Court’s insistence upon trial court gatekeeping of expert witnesses has obscured the discussion and debate about science courts and alternative procedures for addressing complex scientific issues in litigation.  With fear and trembling, and sometimes sickness not quite unto death, federal and state judges, and lawyers on both sides of the “v,” must now do more than attack, defend, and evaluate expert witnesses on simplistic surrogates for the truth, such as personal bias or qualifications.  Lord have mercy, judges and lawyers must now actually read and analyze the bases of expert witnesses’ opinions, assess validity of studies and conclusions, and present their challenges and evaluations in clear, non-technical language.

The notion that “law is an empty vessel” is an imperfect metaphor, but it serves to emphasize that lawyers may have to get their hands wet with vessel’s contents, now and then.  In litigating scientific issues, lawyers and judges will necessarily have to engage with substantive matters.  Lawyers without scientific training or aptitude are not likely to serve clients, whether plaintiffs or defendants, well in the post-Daubert litigation world.  Untrained lawyers will choose the wrong theory, emphasize the wrong evidence, and advance the wrong conclusions.  The stakes are higher now for all the players.  An improvident claim or defense will become a blot on a lawyer’s escutcheon.  A poor gatekeeping effort or judicial decision[11] can embarrass the entire judicial system[12].

Over a decade ago, Professor David Faigman asked whether science was different for lawyers.  Professor Faigman emphatically answered his own question in the negative, and urged lawyers and courts to evolve from their pre-scientific world view.[13] Both lawyers and judges must learn about the culture, process, and content of science.  Science is often idealized as a cooperative endeavor, when in fact, much scientific work can be quite adversarial.[14]Some judges and commentators have argued that the scientific enterprise should be immune from the rough and tumble of legal discovery because the essential collaborative nature of science is threatened by the adversarial interests at play in litigation.  In becoming better judges of science, judges (and lawyers) need to develop a sense of the history of science, with its findings, fanaticisms, feuds, and fraud. The good, bad, and the ugly are all the proper subject for study. Professor George Olah, in accepting his Nobel Prize in Chemistry, rebutted the lofty sentiments about scientific collegiality and collaboration[15]:

“Intensive, critical studies of a controversial topic always help to eliminate the possibility of any errors. One of my favorite quotation is that by George von Bekessy (Nobel Prize in Medicine, 1961).

‘[One] way of dealing with errors is to have friends who are willing to spend the time necessary to carry out a critical examination of the experimental design beforehand and the results after the experiments have been completed. An even better way is to have an enemy. An enemy is willing to devote a vast amount of time and brain power to ferreting out errors both large and small, and this without any compensation. The trouble is that really capable enemies are scarce; most of them are only ordinary. Another trouble with enemies is that they sometimes develop into friends and lose a good deal of their zeal. It was in this way the writer lost his three best enemies. Everyone, not just scientists, needs a few good enemies!’”

In other words, peer review is a shabby substitute for cross-examination and an adversarial process.[16]  That adversarial process cannot always unfold fully and fairly in front of a jury.

Chief Justice Rehnquist no doubt spoke for most judges and lawyers in expressing his discomfort with the notion that courts would have to actually look at science (rather than qualifications, demeanor, and credibility)[17]:

“I defer to no one in my confidence in federal judges; but I am at a loss to know what is meant when it is said that the scientific status of a theory depends on its ‘falsifiability’, and I suspect some of them will be, too.”

Before the Daubert decision, some commentators opined that judges and lawyers were simply too innumerate and too dull to be involved in litigating scientific issues[18]. Today, most federal judges at least would not wear their ignorance so proudly, largely because of the efforts of the Federal Judicial Center, and its many educational efforts.  Judges (and lawyers) cannot and should not be scientific or statistically illiterate[19].  If generalist judges think this is beyond their ken, then they should have the intellectual integrity to say so, and get out of the way.  If the scientific tasks are beyond the ken of judges, then they are likely beyond the abilities of ordinary jurors as well, and we should start to think seriously once again about science courts and blue-ribbon juries.

Astute and diligent gatekeeping judges and qualified juries can get the job done[20]. Certainly, some judges have seen through expert witnesses’ evasions and errors in reasoning, and lawyers’ shystering on both sides of litigation.  Evaluating scientific evidence and drawing inferences from scientific studies do require understanding of basic statistical concepts, study design, and scientific apparatus, but at the bottom, there is no esoteric “scientific method” that is not careful and skeptical reasoning about evidence in the context of all the facts.[21]  Surely, judges with university educations, “20 years of schooling,” and at least 10 years of exemplary professional practice should, in theory, be able to get the job done, even if a few judges are unqualified or ill suited by aptitude or training. To take an example from the breast implant litigation, understanding a claim about immunology may well require some depth in what the immune system is, how it works, and how it may be altered.  This understanding will require time, diligence, intellectual energy, and acumen.  Surely, ordinary lay juries often do not have the time to devote to such a case to do justice to the issues. If judges do not have the interest or the time to render the required service, they should say so.

And what about lawyers?  Twenty years into the Daubert era is time enough for law schools to implement curriculum to train lawyers to understand and to litigate scientific issues, in civil, criminal, and regulatory contexts.  Many law schools still fail to turn out graduates with basic competence to understand and advocate about scientific issues.[22]  Lawyers who want to litigate scientific issues owe their clients and themselves the commitment to understand the issues.  It has been a long time since C.P. Snow complained of the “two cultures,” science and the humanities, and the hostility he faced when he challenged colleagues about their ignorance of science.[23]  Law schools have a role in building a bridge between the two cultures of science and the humanities. Just as tax programs in law schools require basic accounting, law schools should ensure that their graduates, destined to work on scientific litigation, legislation, and regulation, have had some education in statistics, probability, and scientific method.

Becoming or remaining scientific literate is a daunting but essential task for busy lawyers who hope to serve clients well in technical litigation.  There are many resources available for the “scientific legal counselor.”  Dr. David Schwartz at Innovative Science Solutions LLC has published a resource guide, The Litigator’s Guide to Combating Junk Science, available at his firm’s website.[24] Sense About Science, a charitable trust, has worked hard to advance an evidence-based world view, and to help people, including lawyers, make sense of scientific and medical claims.  Its website offers some helpful publications,[25] one of the more interesting being its recent pamphlet, Making Sense of Uncertainty.[26]

The Sense About Science group’s annual lectures are important events, which have featured lectures by:

  • Dr Fiona Godlee, the editor of the British Medical Journal, on “It’s time to stand up for science once more,” in 2010.[27]
  • Dr Olivia Judson, scientist and science journalist, on “Why Experiment?” in 2009.[28]
  • Professor Alan Sokal, quantum physicist and famous perpetrator of the Sokal hoax[29] that helped deconstruct the post-modernist deconstructionists, on “What is Science and Why Should We Care?” in 2008.[30]
  • Sir John Krebs FRS, discoverer of the Krebs cycle, on “Science Advice, Impartiality and Policy,” in 2006.[31]

In addition to keeping abreast of scientific developments, science lawyers need to understand both how science is “made,” and how it goes bad.  The National Institutes of Health website has many resources about the grant process.  Science lawyers should understand the grant process and how to use the Freedom of Information Act to obtain information about research at all stages of development.  Contrary to the fairy-tale accounts of idealized science, the scientific research process goes astray with some frequency, so much so that there is a federal agency, the Office of Research Integrity (ORI) tasked with investigating research misconduct. News of the ORI’s investigations, and findings, is readily available through its website and its informative blog.[32] Research misconduct has resulted in an ever-increasing rate of retractions in peer-reviewed journals.  The Retraction Watch[33] blog offers a fascinating window into the inadequacies of peer review, and the attempts, some successful, some not, of the scientific community to police itself.

Before Daubert was decided, the legal system emphasized credentials and qualifications of witnesses, “authoritative texts,” and general acceptance.  Science, however, is built upon an evidence-based foundation.  Since 1663, the Royal Society has sported the motto:  “Nullius in verba”: on no person’s authority.  When confronted with a pamphlet entitled “100 Authors against Einstein,” Albert Einstein quipped “if I were wrong, one would have been enough.”[34]  Disputes in science are resolved with data, from high-quality, reproducible experimental or observational studies, not with appeals to the prestige of the speaker.  The shift from authority-based decision making to evidence-based inferences and conclusions has strained the legal system in the United States, where people are free to propagate cults and superstitions.  The notion that lawyers who spend most of their time litigating leveraged boxcar leases can appreciate the nuances, strengths, weaknesses, and flaws of scientific studies and causal inference is misguided.  Litigants, whether those who make claims or those who defend claims, deserve legal counsel who are conversant with the language of their issues.  Law schools, courts, and bar associations must rise to the occasion to meet the social need.


[1] Oliver Wendell Holmes, Jr., “The Path of the Law,” 10 Harv. L. Rev. 457, 469 (1897)

[2] Oliver Wendell Holmes, Jr., Collected Legal Papers 230 (1921).

[3] An earlier version of this post appeared as a paper in David M. Cohen & Nathan A. Schachtman, eds., Daubert Practice 2013: How to Avoid Getting Sliced by Cutting-Edge Developments in Expert Witness Challenges (PLI 2013).

[4] See Ross v. Bernhard, 396 U.S. 531, 538 n.l0 (1970) (suggesting that the right to a jury trial may be limited by considerations of complexity, and that practical abilities and limitations of juries are a factor in determining whether an issue is to be decided by the judge or the jury).  See also Note, “The Right to a Jury Trial in Complex Civil litigation,” 92 Harv. L. Rev. 898 (1979) (considering the implications of footnote 10 in Ross for complex litigation; noting that the Ross test has been applied infrequently, and in the limited contexts of antitrust or securities cases); Peter Huber, “Comment, A Comment on Toward Incentive-Based Procedure: Three Approaches for Regulating Scientific Evidence by E. Donald Elliott,” 69 Boston Univ. L. Rev. 513, 514 (1989) (arguing that judges cannot make difficult scientific judgments).  See also George K. Chamberlin, “Annotation: Complexity of Civil Action as Affecting Seventh Amendment Right to Trial by Jury,” 54 A.L.R. FED. 733, 737-44 (1981).

[5] James A. Martin, “The Proposed Science Court,” 75 Mich. L. Rev. 1058, 1058-91 (1977) (evaluating the need and feasibility of various proposals for establishing science courts); Troyen A. Brennan, “Helping Courts with Toxic Torts,” 51 U. Pitt. L. Rev. 1, 5 (1989) (arguing for administrative panels or boards to assist courts with scientific issues); Donald Elliott, “Toward Incentive-Based Procedure: Three Approaches for Regulating Scientific Evidence,” 69 Boston Univ. L. Rev. 487, 501-07 (1989) (discussing various models of ancilliary peer review for courts); Paul K. Sidorenko, “Comment, Evidentiary Dilemmas in Establishing Causation: Are Courts Capable of Adjudicating Toxic Torts?” 7 Cooley L. Rev. 441, 442 (1990) (recommending administrative panels of scientific experts in toxic tort litigation); John W. Osborne, “Judicial Technical Assessment of Novel Scientific Evidence,” 1990 U. Ill. L. Rev. 497, 540-46 (arguing for procedures with specially qualified judges); Edward V. DiLello, “Note, Fighting Fire with Firefighters: A Proposal for Expert Judges at the Trial Level,” 93 Colum. L. Rev. 473, 473 (1993) (advocating establishment of specialist judicial assistants for judges).  For a bibliography of publications on the science court concept, see Jon R. Cavicchi, “The Science Court: A Bibliography” (1994) <http://ipmall.info/risk/vol4/spring/bibliography.htm>, last visited on July 27, 2014.

[6] See Ross v. Bernhard, 396 U.S. at 538 n.l0.

[7] John W. Wesley, “Scientific Evidence and the Question of Judicial Capacity,” 25 William & Mary L. Rev. 675, 702-03 (1984).

[8] See Karen Butler Reisinger, “Expert Panels: A Comparison of Two Models,” 32 Indiana L.J. 225 (1998); Laural L. Hooper, Joe S. Cecil & Thomas E. Willging, Neutral Science Panels: Two Examples of Panels of Court-Appointed Experts in the Breast Implants Product Liability Litigation, (Federal Judicial Center 2001), available at http://www.fjc.gov/public/pdf.nsf/lookup/neuscipa.pdf/$file/neuscipa.pdf

[9] Soldo v. Sandoz Pharms. Corp., 244 F. Supp. 2d 434 (W.D. Pa. 2003); see also Joe S. Cecil, “Construing Science in the Quest for Ipse Dixit,” 33 Seton Hall L. Rev. 967 (2003).

[10] DePyper v. Navarro, No. 191949, 1998 WL 1988927 (Mich. Ct. App. Nov. 6, 1998)

[11] See, e.g., Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).

[12] See, e.g., James L. Mills and Duane Alexander, “Teratogens and ‘Litogens’,” 15 New Engl. J. Med. 1234 (1986); Samuel R. Gross, “Expert Evidence,” 1991 Wis. L. Rev. 1113, 1121-24 (1991) (“Unfortunately, Judge Shoob’s decision is absolutely wrong. There is no scientifically credible evidence that Ortho-Gynol Contraceptive Jelly ever causes birth defects.”). See also Editorial, “Federal Judges v. Science,” N.Y. Times, December 27, 1986, at A22 (unsigned editorial);  David E. Bernstein, “Junk Science in the Courtroom,” Wall St. J. at A 15 (Mar. 24,1993) (pointing to Wells as a prominent example of how the federal judiciary had embarrassed American judicial system with its careless, non-evidence based approach to scientific evidence); Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, “Science and the Law in the Wake of Daubert: A New Search for Scientific Knowledge,” 72 Texas L. Rev. 715, 733-34 (1994) (lawyers and leading scientist noting that the district judge “found that the scientific studies relied upon by the plaintiffs’ expert were inconclusive, but nonetheless held his testimony sufficient to support a plaintiffs’ verdict. *** [T]he court explicitly based its decision on the demeanor, tone, motives, biases, and interests that might have influenced each expert’s opinion. Scientific validity apparently did not matter at all.”) (internal citations omitted); Troyen A. Brennan, “Untangling Causation Issues in Law and Medicine: Hazardous Substance Litigation,” 107 Ann. Intern. Med. 741, 744-45 (1987) (describing the result in Wells as arising from the difficulties created by the Ferebee case; “[t]he Wells case can be characterized as the court embracing the hypothesis when the epidemiologic study fails to show any effect”).  Kenneth R. Foster, David E. Bernstein, and Peter W. Huber, eds., Phantom Risk: Scientific Inference and the Law 28-29, 138-39 (MIT Press 1993) (criticizing Wells decision); Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation § 6.5, at 93 (1997) (noting the multiple comparisons in studies of birth defects among women who used spermicides, based upon the many reported categories of birth malformations, and the large potential for even more unreported categories); id. at § 6.5 n.3, at 271 (characterizing Wells as “notorious,” and noting that the case became a “lightning rod for the legal system’s ability to handle expert evidence.”).

[13] David L. Faigman, “Is Science Different for Lawyers?” 297 Science 339, 340 (2002) (“some courts are still in a prescientific age”).

[14] McMillan v. Togus Reg’l Office, 294 F. Supp. 2d 305, 317 (E.D.N.Y. 2003) (“As in political controversy, ‘science is, above all, an adversary process’.”) (internal citation omitted).

[15] George A. Olah, “My Search for Carbocations and Their Role in Chemistry,” Nobel Lecture (Dec. 8, 1994), quoting George von Békésy, Experiments in Hearing 8 (N.Y. 1960).

[16] Nietzsche expressed Olah’s sentiment more succinctly.  Friedrich Nietzsche, The Twilight of the Idols Maxim 8 (1899) (“Out of life’s school of war: What does not destroy me, makes me stronger.”).

[17] Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579, 598 (1993) (Justice Rehnquist, C.J., concurring and dissenting).

[18] Joseph Nicol, “Symposium on Science and the Rules of Evidence,” 99 F.R.D. 188, 221 (1983) (claiming that lawyers and judges are “simply are incapable by education, and all too often by inclination, to become sufficiently familiar with scientific evidence to discharge their responsibilities toward the administration of justice.”)

[19] Robert P. Merges, “The Nature and Necessity of Law and Science,” 381 Leg. Educ. 315, 324-26 (1988) (arguing that lawyers can and should understand scientific concepts and issues)

[20] David L. Faigman, “To Have and Have Not: Assessing the Value of Social Science to the Law as Science and Policy,” 38 Emory L.J. 1005, 1014, 1030 (1989) (contending that judges can understand science and that lawyers must probe beyond conclusions into methods by which the conclusions were reached).

[21] David H. Kaye, “Proof in Law and Science,” 32 Jurimetrics J. 313, 318 (1992) (arguing that science and law share identical methods of establishing factual conclusions); Lee Loevinger, “Standards of Proof in Science and Law,” 32 Jurimetrics J. 323, 328 (1992) (“[T]he basic principles of reasoning or logic are no different in the field of law than science.”).

[22] Howell E. Jackson, “Analytical Methods for Lawyers,” 53 J. Legal Educ. 321 (2003); Steven B. Dow, “There’s Madness in the Method:  A Commentary on Law, Statistics, and the Nature of Legal Education,” 57 Okla. L. Rev. 579 (2004).

[23] C.P. Snow, The Rede Lecture 1959.

[24] Available with registration at http://www.innovativescience.net/complimentary-copy-of-our-junk-science-ebook/

[25]http://www.senseaboutscience.org/

[26] http://www.senseaboutscience.org/resources.php/127/making-sense-of-uncertainty

[27] http://www.senseaboutscience.org/pages/annual-lecture-2010.html

[28] http://www.senseaboutscience.org/pages/annual-lecture-2009.html

[29] Sokal’s famous parody of postmodern writers, the so-called “‘Sokal Hoax,” was his publication in one of the postmodernists’ own journals on how gravity was nothing more than a social construct.  Alan D. Sokal, “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity,” Social Text 217 (Nos. 46/47 1996) (“It has thus become increasingly apparent that physical ‘reality’, no less than social ‘reality’, is at bottom a social and linguistic construct; that scientific ‘knowledge’, far from being objective, reflects and encodes the dominant ideologies and power relations of the culture that produced it; that the truth claims of science are inherently theory-laden and self-referential; and consequently, that the discourse of the scientific community, for all its undeniable value, cannot assert a privileged epistemological status with respect to counter-hegemonic narratives emanating from dissident or marginalized communities.”)

[30] http://www.senseaboutscience.org/pages/annual-lecture-2008.html

[31] http://www.senseaboutscience.org/pages/annual-lecture-2006.htm

[32] http://ori.hhs.gov/blog/

[33] http://retractionwatch.wordpress.com/

[34] See Remigio Russo, 18 Mathematical Problems in Elasticity 125 (1996) (quoting Einstein).

Conflict of Interest Regulations Apply Symmetrically

July 25th, 2014

Last week, a federal judge ruled that working as a plaintiffs’ expert witness in tobacco litigation or as consultants for pharmaceutical companies with tobacco-cessation medications was a conflict of interest, which invalidated a FDA report on menthol cigarettes. Lorillard Inc. et al. v. U.S. Food and Drug Administration, 1:11-cv-00440, D.D.C. (July 21, 2014).

The report was issued in 2011 by the agency’s Tobacco Products Scientific Advisory Committee (TPSAC).  Two tobacco companies sought an injunction against the FDA’s report because of the improper memberships on the TPSAC of Drs. Jonathan Samet and Neil Benowitz. Lorillard’s General Counsel also challenged agency pronouncements under the Data Quality Act.  Letter Request for Correction of Information Disseminated to the Public and the Tobacco Products Scientific Advisory Committee (March 16, 2011) (petition lodged under the Data Quality Act (DQA), 44 U.S.C. § 3516, and the related agency regulations of the Office of Management and Budget, Health and Human Services, and FDA).

Although the TPSAC report concluded that menthol cigarettes imparted no greater risk of lung cancer than the already deadly non-menthol cigarettes, the report claimed that studies show menthol flavoring increased usage among young people and African Americans.  The TPSAC recommended a ban on menthol cigarettes in the interest of public health. The district court held that the FDA’s decision that the disputed members had no conflict of interest violated the Administrative Procedures Act, and that the violation required the agency to reconstitute the TPSAC in compliance with the applicable ethics regulations.

What is remarkable about the case is its rejection of the delusion that advocacy for plaintiffs is not a potential conflict of interest, whereas advocacy for a company is.  An amicus brief filed on behalf of several medical and public health groups, including Public Citizen, Inc., American Cancer Society, American Medical Association, American Thoracic Society, and others, supported the agency’s motion to dismiss. Amici argued that balancing scientific opinions of experts on the committee would impair the integrity of advisory committees and would be unmanageable for courts. Given the obvious adversarial bias from service as an expert witness in tobacco litigation, the district court rejected amici’s arguments. Although amici’s position is understandable as a reaction to the callousness of tobacco marketing, the brief’s indifference to the ethical implications about conflicts of interest is surprising.  One can only imagine the hue and cry if there had been committee members who had been engaged as expert witnesses for the tobacco companies.

The implications of the Lorillard decision are considerable, especially for FDA advisory committees. See Glenn Lammi, “FDA Advisory Committee Not Rife with Conflicts of Interest? — ‛Please!’ Quips Federal Judge” (July 24, 2014) (discussing Lorillard case and detailing efforts to obtain FDA compliance with the Federal Advisory Committee Act). Industry representatives are typically non-voting members, but members who have been retained by the litigation industry have served in voting positions. In other contexts, expert witnesses for plaintiffs accuse scientists who testify for a defendant of “conflicts of interest,” but conveniently ignore and fail to disclose their own. SeeMore Hypocrisy Over Conflicts of Interest” (Dec. 4, 2010) (Arthur Frank, Richard Lemen, and Barry Castleman); James Coyne, “Lessons in Conflict of Interest: The Construction of the Martyrdom of David Healy and The Dilemma of Bioethics,” 5 Am. J. Bioethics W3 (2005). The Lorillard case teaches that “white hat” bias is as disqualifying as “black hat” bias.

Subgroups — Subpar Statistical Practice versus Fraud

July 24th, 2014

Several people have asked me why I do not enable comments on this blog.  Although some bloggers (e.g., Deborah Mayo’s Error Statistics site) have had great success in generating interesting and important discussions, I have seen too much spam on other websites, and I want to avoid having to police the untoward posts.  Still, I welcome comments and I try to respond to helpful criticism.  If and when I am wrong, I will gladly eat my words, which usually have been quite digestible.

Probably none of the posts here have generated more comments and criticisms than those written about the prosecution of Dr. Harkonen.  In general, critics have argued that defending Harkonen and his press release was tantamount to condoning bad statistical practice.  I have tried to show that Dr. Harkonen’s press release was much more revealing than it was portrayed in abbreviated accounts of his case, and the evidentiary support for his claim of efficacy in a subgroup was deeper and broader than acknowledged. The criticism and condemnation of Dr. Harkonen’s press release in the face of prevalent statistical practice, among leading journals and practitioners, is nothing short of hypocrisy and bad faith. If Dr. Harkonen deserves prison time for a press release, which promised a full analysis and discussion in upcoming conference calls and presentations at scientific meetings, then we can only imagine what criminal sanction awaits the scientists and journal editors who publish purportedly definitive accounts of clinical trials and epidemiologic studies, with subgroup analyses not prespecified and not labeled as post-hoc.

The prevalence of the practice does not transform Dr. Harkonen’s press release into “best practice,” but some allowance must be made for offering a causal opinion in the informal context of a press release rather than in a manuscript for submission to a journal.  And those critics, with prosecutorial temperaments, must recognize that, when the study was presented at conferences, and when manuscript was written up and submitted to the New England Journal of Medicine, the authors did reveal the ad hoc nature of the subgroup.

The Harkonen case will remain important for several reasons. There is an important distinction in the Harkonen case, ignored and violated by the government’s position, between opinion and fact.  If Harkonen is guilty of Wire Fraud, then so are virtually every cleric, minister, priest, rabbi, imam, mullah, and other religious person who makes supernatural claims and predictions.  Add in all politicians, homeopaths, vaccine deniers, and others who reject evidence for superstition, who are much more culpable than a scientist who accurately reports the actual data and p-value.

Then there is the disconnect between what expert witnesses are permitted to say and what resulted in Dr. Harkonen’s conviction. If any good could come from the government’s win, it would be the insistence upon “best practice” for gatekeeping of expert witness opinion testimony.

For better or worse, scientists often describe post-hoc subgroup findings as “demonstrated” effects. Although some scientists would disagree with this reporting, the practice is prevalent.  Some scientists would go further and contest the claim that pre-specified hypotheses are inherently more reliable than post-hoc hypotheses. See Timothy Lash & Jan Vandenbroucke, “Should Preregistration of Epidemiologic Study Protocols Become Compulsory?,” 23 Epidemiology 184 (2012).

One survey compared grant applications with later published papers and found that subgroup analyses were pre-specified in only a minority of cases; in a substantial majority (77%) of the subgroup analyses in the published papers, the analyses were not characterized as either pre-specified or post hoc. Chantal W. B. Boonacker, Arno W. Hoes, Karen van Liere-Visser, Anne G. M. Schilder, and Maroeska M. Rovers, “A Comparison of Subgroup Analyses in Grant

Applications and Publications,” 174 Am. J. Epidem. 291, 291 (2011).  Indeed, this survey’s comparison between grant applications and published papers revealed that most of the published subgroup analyses were post hoc, and that the authors of the published papers rarely reported justifications for their post-hoc subgroup. Id.

Again, for better or worse, the practice of presenting unplanned subgroup analyses, is common in the biomedical literature. Several years ago, the New England Journal of Medicine reported a survey of publication practice in its own pages, with findings similar to those of Boonacker and colleagues. Rui Wang, Stephen W. Lagakos, James H. Ware, David J. Hunter, and Jeffrey M. Drazen, “Statistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials,” 357 New Eng. J. Med. 2189 (2007).  In general, Wang, et al.,  were unable to determine the total number of subgroup analyses performed; and in the majority (68%) of trials discussed, Wang could not determine whether the subgroup analyses were prespecified. Id. at 2912. Although Wang proposed guidelines for identifying subgroup analyses as prespecified or post-hoc, she emphasized that the proposals were not “rules” that could be rigidly prescribed. Id. at 2194.

The Wang study is hardly unique; the Journal of the American Medical Association reported a similar set of results. An-Wen Chan, Asbjørn Hrobjartsson, Mette T. Haahr, Peter C. Gøtzsche, and Douglas G. Altman, “Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials Comparison of Protocols to Published Articles,” 291 J. Am. Med. Ass’n 2457 (2004).  Chan and colleagues set out to document and analyze “outcome reporting bias” in studies; that is, the extent to which publications fail to report accurately the pre-specified outcomes in published studies of randomized clinical trials.  The authors compared and analyzed protocols and published reports of randomized clinical trials conducted in Denmark in 1994 and 1995. Their findings document a large discrepancy between idealized notion of pre-specification of study design, outcomes, and analyses, and the actual practice revealed by later publication.

Chan identified 102 clinical trials, with 3,736 outcomes, and found that 50% of efficacy, and 65% of harm outcomes were incompletely reported. There was a statistically significant risk of statistically significant outcomes to be fully reported compared with statistically insignificant results. (pooled odds ratio for efficacy outcomes = 2.4; 95% confidence interval, 1.4 – 4.0, and pooled odds ratio for harm outcomes = 4.7; 95% confidence interval, 1.8 -12.0. Their comparison of protocols with later published articles revealed that a majority of trials (62%) had at least one primary outcome that was changed, omitted, or innovated in the published version. The authors concluded that published accounts of clinical trials were frequently incomplete, biased, and inconsistent with protocols.

This week, an international group of scientists published their analysis of agreement vel non between protocols and corresponding later publications of randomized clinical trials. Matthias Briel, DISCO study group, “Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications,” 349 Brit. Med. J. g4539 (Published 16 July 2014). Predictably, the authors found a good deal of sloppy practice, or worse.  Of the 515 journal articles identified, about half (246 or 47.8%) reported one or more subgroup analysis. Of the articles that reported subgroup analyses, 81 (32.9%) publications stated that the subgroup analyses were prespecified, but in 28 of these articles (34.6%), the corresponding protocols did not identify the subgroup analysis.

In 86 of the publications surveyed, the authors found that the articles claimed a subgroup “effect,” but only 36 of the corresponding protocols reported a planned subgroup analysis.  Briel and the DISCO study group concluded that protocols of randomized clinical trials insufficiently describe subgroup analyses. In over one-third of publications, the articles reported subgroup analyses not pre-specified in earlier protocols. The DISCO study group called for access to protocols and statistical analysis plans for all randomized clinical trials.

In view of these empirical data, the government’s claims against Dr. Harkonen stand out, at best, as vindictive, selective prosecution.

Careless Scholarship About Silica History

July 21st, 2014

David Egilman is the Editor in Chief of the International Journal of Occupational and Environmental Health (IJOEH). A YouTube “selfie” interview provides some insight into Dr. Egilman’s motivations and editorial agenda.  Previous posts have chronicled Egilman’s testimonial adventures because of his propensity to surface in litigations of interest. See, e.g., “David Egilman’s Methodology for Divining Causation” (Sept. 6, 2012); “Egilman Petitions the Supreme Court for Review of His Own Exclusion in Newkirk v. Conagra Foods” (Dec. 13, 2012).

Dr. Egilman has used his editorial role at the IJOEH to disseminate his litigation positions.  Several of his articles are little more than his litigation reports, filed in various cases, ranging from occupational dust disease claims to pharmaceutical off-target effect claims. A recent issue of the IJOEH has yet another article of this ilk, which scatters invective across several litigations. David Egilman, Tess Bird[1], and Caroline Lee[2], “Dust diseases and the legacy of corporate manipulation of science and law, 20 Internat’l J. Occup. & Envt’l Health 115 (2014).

The article mostly concerns Egilman’s allegations that companies influenced the scientific, medical, and governmental understanding and perception of asbestos hazards.  I will defer to others to address his allegations with respect to asbestos. The article, however, in its Abstract, takes broader aim at other exposures, in particular, silica:

“Knowledge that asbestos and silica were hazardous to health became public several decades after the industry knew of the health concerns. This delay was largely influenced by the interests of Metropolitan Life Insurance Company (MetLife) and other asbestos mining and product manufacturing companies.”

Egilman at 115, Abstract (emphasis added).

In their Abstract, the authors further proclaim their purpose

“To understand the ongoing corporate influence on the science and politics of asbestos and silica exposure, including litigation defense strategies related to historical manipulation of science.”

Egilman at 115. I demur for the time being with respect to asbestos, but the authors’ claims about silica are never supported in their article. A brief review of two monographs by Frederick L. Hoffman should be sufficient to condemn the authors’ carelessness to the dustbin of occupational history. Frederick L. Hoffman, Mortality from Respiratory Diseases in the Dusty Trades; Dep’t of Labor, Bureau of Labor Statistics (1918); The Problem of Dust Phthisis in the Granite Stone Industry; Dep’t of Labor, Bureau of Labor Statistics (1922).  The bibliographies in both these monographs documents the widespread interest in, and awareness of, the occupational hazards of silica dusts, going back into the 19th century, among the media, the labor movement, and the non-industrial scientific community.

Not surprisingly, the authors’ conclusions are stated only in terms of asbestos hazards, knowledge, and company conduct:

“Conclusions: Asbestos product companies would like the public to believe that there was a legitimate debate surrounding the dangers of asbestos during the twentieth century, particularly regarding the link to cancer, which delayed adequate regulation. The asbestos–cancer link was not a legitimate contestation of science; rather the companies directly manipulated the scientific literature. There is evidence that industry manipulation of scientific literature remains a continuing problem today, resulting in inadequate regulation and compensation and perpetuating otherwise preventable worker and consumer injuries and deaths.”

The authors note that Rutherford Johnstone’s 1960 “seminal” textbook relied upon a study (Braun and Truan), which study Egilman attacks as corrupted by industry influence. Rutherford Johnstone & Seward E. Miller, Occupational Diseases and Industrial Medicine 328 (Philadelphia 1960). According to the Egilman, Rutherford Johnstone was the official American Medical Asociation’s consultant for occupational disease questions, which explains why he was providing answers to questions submitted to the Journal of the American Medical Association, on silica and asbestos issues. The authors note that Johnstone, in 1961, asserted that there was no epidemiologic evidence that asbestos causes lung cancer among American workers, which view reflects Johnstone’s reliance upon the Braun-Truan study. The authors fail, however, to note that Johnstone also opined that

“There is no epidemiological evidence that silicosis, resulting from undue exposure to free silica produces cancer of the lung.”

Rutherford T. Johnstone, “Silicosis and Cancer,” 176 J. Am. Med. Ass’n 81, 81 (1961). Neither the authors nor anyone else has ever shown that Johnstone was misled by any industry group with respect to his silica/lung cancer opinion.

Some of the Egilman’s scholarship is quite careless.  For instance, he, along with his employees, assert that

“By the mid 1940s, the international scientific community had recognized the link between asbestos and cancer.10–18”

Readers should review all the endnotes, 10 – 18, but endnote 12 is especially interesting:

“12 Macklin MT, Macklin CC. Does chronic irritation cause primary carcinoma of the human lung? Arch Path. 1940;30:924–55.”

As I have noted before, the Macklins, and especially Dr. Madge Macklin, brought a great deal of rigor and skepticism to broad claims about the causation of lung cancer. SeeSilicosis, Lung Cancer, and Evidence-Based Medicine in North America” (July 4, 2014).  This citation and others do not appear to support the sweep of Egilman and his student authors’ claim.

The next mention of silica occurs in the context of an allegation that corporations (presumably not plaintiffs’ lawyers’ law firm corporations) have worked to “disguise” health concerns and influence governmental policy about several products, materials, including silica:

“During the last several decades, researchers in a wide spectrum of fields have documented the direct and purposeful efforts of corporations to disguise public health concerns and affect government policies, particularly in the tobacco, alcohol, silica, and asbestos industries, and more recently, the pharmaceutical, chemical, and ultra-processed food and drink industries.79,73

Egilman at 121.

The authors’ citations, however, do not support any such allegation about silica. Endnote 73[3] is an article by Egilman, and others, on Vioxx; and endnote 79[4] is an article about alcohol, tobacco, and foods. In the very next sentence, the authors further claim that:

“Corporate-funded ‘objective science’ leading to the corruption of scientific literature remains a major problem.65,68,69,71,73,75,80–86

Once again, none of the endnotes (65, 68, 69, 71, 75, and 80-86) supports the authors’ claim that anyone in the mining, milling, or marketing of crystalline silica has funded science in a way that led to the corruption of the scientific literature. Not surprisingly, the authors ignore the frauds perpetuated by litigation industry players. See, e.g., In re Silica Products Liab. Lit., 398 F. Supp. 2d 563 (S.D. Tex. 2005) (federal trial judge rebukes the litigation industry for fraudulent claiming in MDL 1553).


[1] The article acknowledges that Ms. Bird and Ms. Lee were employees of Dr. Egilman.  Ms. Bird appears now to be a student in the U.K., studying medical anthropology.  Ms. Bird, and Ms. Lee, appeared on earlier works by Egilman.  See, e.g., David S Egilman, Tess Bird, and Caroline Lee, “MetLife and its corporate allies: dust diseases and the manipulation of science,” 19 Internat’l J. Occup. & Envt’l Health 287 (2013); David Steven Egilman, Emily Laura Ardolino, Samantha Howe, and Tess Bird, “Deconstructing a state-of-the-art review of the asbestos brake industry,” 21 New Solutions 545 (2011).

[2] Ms. Lee appears to have been employed for Egilman’s litigation consulting firm, Never Again Consulting, from 2011 until August 2013, when she entered the University of Maryland law school.

[3] Krumholz HM, Ross JS, Presler AH, Egilman DS. What have we learnt from Vioxx. Br Med J. 2007;334(7585):120–3.

[4] Moodie R, Stuckler D, Montiero C, Sheron N, Neal B, Thamarangsi T, et al. Profits and pandemics: prevention of harmful effects of tobacco, alcohol, and ultra-processed food and drink industries. Lancet. 2013;381:670–79.

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.