TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Archives »

The Illinois Medical Studies Act and Access to Underlying Research Data

August 2nd, 2014

In the course of the welding fume litigation, defense counsel sought underlying data and documentation from a study published by Dr. Brad Racette and others, on their observations of Alabama welders[1]. The Alabama study, with its duplicate publication, aroused counsel’s suspicion for several reasons, not the least of which was that plaintiffs’ counsel had participated in creating the study by providing their clients as participants at screenings to be conducted by Dr. Racette and his colleagues.  According to the disclosure in the Neurology article, the study was

“[s]upported by NIH grants K23NS43351 and NS41509, the Greater St. Louis Chapter of the American Parkinson’s Disease Association, and the Welder Health Fund. The Welder Health Fund was created by Gulf States Trial Attorneys to support medical–legal screening of welders for Parkinson disease. No author has taken any money personally related to this research.”

A little research showed that the referenced NIH grants were not for this specific study, which was a creation of the litigation industry d/b/a the Welder Health Fund.

Defense counsel directed subpoenas to both Dr. Racette and his institution, Washington University St. Louis, for the study protocol, underlying data, data codes, and statistical analyses.  After a long discovery fight, the MDL court largely enforced the subpoenas.  See, e.g., In re Welding Fume Prods. Liab. Litig., MDL 1535, 2005 WL 5417815 (N.D. Ohio Oct. 18, 2005) (upholding defendants’ subpoena for protocol, data, data codes, statistical analyses, and other things from Dr. Racette author of study on welding and parkinsonism).

Dr. Racette continued the controversy with a one-sided editorial in Neurology, which he wrote along with his legal counsel from the subpoena contest. Brad A. Racette, Ann Bradley, Carrie A. Wrisberg, and Joel S. Perlmutter, “The Impact of Litigation on Neurologic Research,” 67 Neurology 2124 (2006) [“Litigation Impact”]. Defense counsel wrote a response, which was limited due to the nature of the 400-word cap on letters to the editor. Nathan A. Schachtman, “The Impact of Litigation on Neurologic Research,” 69 Neurology 495 (2007).  A slightly more expansive response was published on the TortsProf blog. Schachtman, “Letter: Litigation and Research” (April 15, 2007).

The debate continued in October 2007, when Dr. Brad Racette and I were invited by the Committee on Science, Technology, and Law of the National Academies of Science to discuss litigation and compelled access to underlying research data.  Dr. Racette complained about how disruptive subpoenas were into the operation of his research.  The Committee members appeared unimpressed by the alleged burdens when they realized that Racette’s research grew out of medico-legal screenings organized, conducted, and paid for by plaintiffs’ lawyers, and that it had been over 30 years since the National Research Council urged scientists to plan proactively for sharing data from their research, and that current National Institutes of Health guidelines require such a plan[2]. Most members of the Committee expressed the view that nothing untoward had occurred with the subpoena of Dr. Racette’s underlying data.

The fight over Dr. Racette’s data had consequences in the courtroom.  Defense expert witnesses were able to see how Dr. Racette was able to generate the appearance of large increased prevalence of “parkinsonism” (idiosyncratically defined and ascertained), which appearance was at odds with the vast data from analytical epidemiologic studies from around the world. James Mortimer, Amy Borenstein, and Laurene Nelson, “Associations of welding and manganese exposure with Parkinson disease: Review and meta-analysis,” 79 Neurology 1174 (2012) (reporting a statistically significant decreased risk of Parkinson’s disease among welding tradesmen).  Remarkably, the plaintiffs’ counsel, who had sponsored the Alabama study, did not show the underlying Racette data and materials to their expert witnesses.  Within a short time after the subpoena was enforced, plaintiffs’ counsel mostly withdrew the epidemiologic component of their cases. Racette’s study ceased to play a major role in the plaintiffs’ claims. The debate over access to underlying data continues, both for clinical trials and for observational studies[3].

In “Litigation Impact,” Racette and colleagues offered specific remedial suggestions concerning the access of data.  One suggestion is to prohibit all data access categorically. Ironically, Racette’s suggestion run contrary to the widespread view that clinical trial data must be shared. Here is how Racette presents his suggestion:

“Stronger, more comprehensive federal legislation recognizing a research scholar privilege and ensuring that research data are uniformly protected against disclosure in all states is essential to advancing medical science.

Protections for study participants and researchers against disclosure of study data could be modeled after provisions such as those set forth in the Illinois Medical Studies Act. This Act specifically designates medical research study data and protocols as protected and prohibits the disclosure of research data obtained as part of a medical study.40,41 Enforcement includes a potential misdemeanor conviction for any person disclosing medical study data. Such legislation must strike a critical balance between the rights of industry to defend itself and the rights of researchers.”

Litigation Impact at 2128.

Is Racette correct that Illinois Medical Studies Act prevents disclosure of underlying research study data?

The claim seems dubious, but the Act is hardly a model of clarity:

Sec. 8-2101. Information obtained. All information, interviews, reports, statements, memoranda, recommendations, letters of reference or other third party confidential assessments of a health care practitioner’s professional competence, or other data of the Illinois Department of Public Health, local health departments, the Department of Human Services (as successor to the Department of Mental Health and Developmental Disabilities), the Mental Health and Developmental Disabilities Medical Review Board, Illinois State Medical Society, allied medical societies, health maintenance organizations, medical organizations under contract with health maintenance organizations or with insurance or other health care delivery entities or facilities, tissue banks, organ procurement agencies, physician-owned insurance companies and their agents, committees of ambulatory surgical treatment centers or post-surgical recovery centers or their medical staffs, or committees of licensed or accredited hospitals or their medical staffs, including Patient Care Audit Committees, Medical Care Evaluation Committees, Utilization Review Committees, Credential Committees and Executive Committees, or their designees (but not the medical records pertaining to the patient), used in the course of internal quality control or of medical study for the purpose of reducing morbidity or mortality, or for improving patient care or increasing organ and tissue donation, shall be privileged, strictly confidential and shall be used only for medical research, increasing organ and tissue donation, the evaluation and improvement of quality care, or granting, limiting or revoking staff privileges or agreements for services, except that in any health maintenance organization proceeding to decide upon a physician’s services or any hospital or ambulatory surgical treatment center proceeding to decide upon a physician’s staff privileges, or in any judicial review of either, the claim of confidentiality shall not be invoked to deny such physician access to or use of data upon which such a decision was based.
(Source: P.A. 92-644, eff. 1-1-03.)

Ill.Ann.Stat. ch. 735 § 5 (Code of Civil Procedure)/8–2101, et seq. (Medical Studies).

Racette’s suggestion that the Act protects the underlying data from research studies is not supported by reliable precedent.  Plaintiffs’ lawyers, including lawyers who worked with Dr. Racette in the welding litigation, have objected to discovery on the basis of the Act, but the objections were apparently not further litigated. See Videotape Deposition of Paul A. Nausieda, M.D. in Boren v. A.O. Smith Corp., Circuit Court of Illinois, Third Judicial Circuit (Dec. 22, 2004).

The Act itself has no ostensible connection to protecting the underlying data in an epidemiologic study.  Niven v. Siquiera, 109 Ill. 2d 357, 487 N.E.2d 937 (1985) (“the purpose of the Act is to encourage candid and voluntary studies and programs used to improve hospital conditions and patient care or to reduce the rates of death and disease”); Matviuw v. Johnson, 70 Ill. App.3d 481, 486, 388 N.E.2d 795, 799 (1st Dist. 1979). The Act protects peer-review or quality-control committees that supervise overall medical care with a view to improving the quality and standard of care, and reducing morbidity and mortality of patients. The privilege is designed to permit persons to speak freely to committees, and to protect their self-evaluative deliberations. Id.; Walker v. Alton Memorial Hospital Ass’n, 91 Ill. App.3d 310, 314, 414 N.E.2d 850, 853 (5th Dist. 1980). For a document to fall within the scope of the Act, it must be “generated specifically for the use of a peer-review committee.” Webb v. Mt. Sinai Hospital and Medical Center of Chicago, Inc., 807 N.E.2d 1026, 1033 (Ill. App. Ct. 2004)(internal citation omitted).

Given the language of the Act, and its interpretation in the Illinois courts, it is not surprising that attempts to assert the statutory privilege to defeat research subpoenas have failed.

In Andrews v. Eli Lilly & Co., Inc., 97 F.R.D. 494 (N.D. Ill. 1983), the district judge quashed a subpoena that sought production of records from research that was exploring the association between maternal use of diethylstilbestrol and daughters’ development of adenocarcinoma of the vagina. In granting the motion to quash, the district judge acknowledged that the Medical Studies Act protected the records from disclosure. Id. at 500. The party seeking discovery, argued that the Illinois statute was limited to hospital peer review.  The district court acknowledged that the purpose of the Act was to ensure confidentiality of the peer-review process, but that the “plain language” of the statute did not limit protected material to that used for peer review. Id. at 500 n.19. In any event, Andrews was stripped of any precedential force when it was vacated by the United States Court of Appeals.   Deitchman v. E.R. Squibb & Sons, Inc., 740 F.2d 556 (7th Cir. 1984).

In the Orthopedic Bone Screw MDL, plaintiffs sought research study documents from the Scoliosis Research Society. Invoking the Illinois Medical Studies Act, the Society moved for a protective order to prevent disclosure of the names of physicians and patients who participated in “The Pedicle Screw Cohort Study,” and the disclosure of documents from that study. In re Orthopedic Bone Screw Prods. Liab. Litig., MDL No. 1014, 1996 WL 34394351 (E.D. Pa. Feb. 14, 1996), app. dism’d, 94 F.3d 110 (3d Cir. 1996). The Society claimed that the Act provided an unconditional confidentiality for all data and related information underlying a “a medical study.” Id. at *2.

The MDL court denied the plaintiffs’ motion to enforce, but rejected the claim that the Act created a privilege that applied in the case. Id. at *3 (citing the predominant federal interest in determining whether a state court privilege applied, and the lack of Illinois interests in a choice of law analysis).

In another multi-district litigation, Pfizer sought peer reviewer and editorial comments from the Journal of the American Medical Association, on articles and manuscripts of studies involving its COX-2 inhibitor medications, Bextra and Celebrex.  In re Bextra & Celebrex Mktg. Sales Practices & Prod. Liab. Litig., Case No. 08 C 402, 2008 U.S. Dist. LEXIS 21098; 2008 WL 4345158 (N.D. Ill. Mar. 14, 2008). The Journals asserted privileges, including a statutory privilege under the Illinois Medical Studies Act. Id. at *10-12. The federal district court ruled that the privilege appeared not to apply to manuscripts, articles, peer reviewer and editorial comments:

“The Medical Studies Act, 735 ILCS 5/8-2101, provides that [a]ll information, interviews, reports, statements, memoranda, recommendations, letters of reference or third party confidential assessments of a health care practitioner’s professional competence, or other data …  of allied medical societies . . . used in the course of internal quality control or of medical study for the purpose of reducing morbidity or mortality, or for improving patient care or increasing organ and tissue donation, shall be privileged, strictly confidential and shall be used only for medical research, increasing organ and tissue donation, the evaluation and improvement of quality care, or granting, limiting or revoking staff privileges or agreements for services . . . . 735 ILCS 5/8-2101. The purpose of the Act is to ‛ensure that members of the medical profession can maintain effective professional self-evaluation and to improve the quality of healthcare’.

To that end, the Act clearly protects from disclosure information gathered in the course of a peer review, but only if it is used in connection with a program or study designed to improve internal quality control or patient care, or to reduce morbidity and mortality. Putting aside the question of whether the Journals would qualify as ‛allied medical societies’, the Court is not persuaded that the specific information identified in the subpoenas fits within this definition. In her declaration, Dr. DeAngelis represented that JAMA ‛publishes scientific articles, commentaries, and news involving all fields of medicine, including medical research, significant clinical observations, diagnostic and therapeutic developments, legal and social matters of interest to  physicians, and issues of medical ethics’; JAMA’s motto, she said, is ‛to promote the science and art of medicine and the betterment of the public health’. DeAngelis Declaration, PP12-13. A lofty goal – but one that far transcends the provisions of the Medical Studies Act. The Act protects from disclosure documents only when ‛used in the course of internal quality control or of medical study for the purpose of reducing morbidity or mortality, or for improving patient care or increasing organ and tissue donation’; surely not everything published in JAMA and AIM – or everything considered for publication – can fairly be described this way. And, on the limited record before it, the Court cannot say that all of the documents requested in the subpoenas can be either.”

Id. at *11-14.

Racette’s suggestion that the Illinois Medical Studies Act can provide guidance with respect to research subpoenas is wrong on the law, facts, and policy of the need for data sharing. The public, and its members, remain entitled to every man’s evidence. See Lord Hardwicke’s Speech on Lord Orford’s Bill (May 25, 1742), quoted in George Harris, The Life of Lord Chancellor Hardwicke; with Selections from His Correspondence, Diaries, Speeches, and Judgments, Vol. 2, at 8 (1847).

[1] See Brad A. Racette, S.D. Tabbal, D. Jennings, L. Good, J.S. Perlmutter, and Brad Evanoff, “Prevalence of parkinsonism and relationship to exposure in a large sample of Alabama welders,” 64 Neurology 230 (2005); Brad A. Racette, et al., “A rapid method for mass screening for parkinsonism,” 27 Neurotoxicology 357 (2006).

[2] See National Institutes of Health, Final Statement on Sharing Research Data (Feb. 26, 2003); Stephen E. Fienberg, et al., eds. Committee on National Statistics, National Research Council, Sharing Research Data (1985); Eleanor Singer, chair, Panel on Data Access for Research Purposes, National Research Council, Expanding Access to Research Data: Reconciling Risks and Opportunities (2005).  See also National Academy of Sciences Committee on Ensuring the Utility and Integrity of Research Data in a Digital Age, Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age (2009).

[3] See, e.g., Jesse A Berlin, Sandra Morris, Frank Rockhold, Lisa Askie, Davina Ghersi, and Joanne Waldstreicher, “Bumps and bridges on the road to responsible sharing of clinical trial data,” 11 Clinical Trials 7 (2014); Rebecca Kush & Michel Goldman, “Fostering Responsible Data Sharing through Standards,” 370 New Engl. J. Med. 2163 (June 5, 2014); Jerome P. Reiter & Satkartar K. Kinney, “Sharing Confidential Data for Research Purposes: A Primer,” 22 Epidemiology 632 (2011).

Posted in Data Sharing, Scientific Evidence | Comments Off on The Illinois Medical Studies Act and Access to Underlying Research Data

The Role of the “Science Lawyer” in Modern Litigation

July 27th, 2014

“For the rational study of the law the blackletter man may be the man of the present, but the man of the future is the man of statistics and the master of economics.”[1]

“Judges commonly are elderly men, and are more likely to hate at sight any analysis to which they are not accustomed, and which disturbs repose of mind … .”[2]

The emergence of complex scientific issues in post-World War II American litigation has challenged state and federal legal systems, both civil and criminal. Various issues, such as the validity of forensic science and biological causation, have tested the competency of lawyers, both judges and counsel. This scientific complexity also questions whether lay juries can or should be continued in their traditional role as fact finders[3].  Some commentators have suggested that complexity should be a basis for reallocating fact finding to judges.[4] Other commentators remain ideologically or constitutionally committed to the jury as fact finder.

The superiority of judges as fact finders in complex scientific cases remains to be shown. Clearly juries often perform better than individual lay judges as scientific fact finders, but their time commitment is inadequate.  Furthermore, the jury decision process is typically a binary decision without any articulation of reasoning.  The jury’s hidden reasoning process on important scientific issues violates basic tenets of transparency and due process.  Many modern cases present such difficult scientific issues that some authors have argued that we should establish scientific courts or institute procedures for blue-ribbon juries[5].  The constitutionality of having scientists, or specially qualified judges, serve as fact finders has never been clearly addressed.[6]  Other commentators have argued in favor of the existing set of judicial tools, such as appointment of testifying “neutral” expert witnesses and scientific advisors for trial judges.[7] These approaches have been generally available in federal and some state trial courts, but they have rarely been used.  Some examples of their deployment include the silicone gel breast implant litigation[8], and cases involving Parlodel[9] and Bendectin[10].

Over 20 years ago, in 1993, the United States Supreme Court handed down its Daubert decision.  In large measure, the Court’s insistence upon trial court gatekeeping of expert witnesses has obscured the discussion and debate about science courts and alternative procedures for addressing complex scientific issues in litigation.  With fear and trembling, and sometimes sickness not quite unto death, federal and state judges, and lawyers on both sides of the “v,” must now do more than attack, defend, and evaluate expert witnesses on simplistic surrogates for the truth, such as personal bias or qualifications.  Lord have mercy, judges and lawyers must now actually read and analyze the bases of expert witnesses’ opinions, assess validity of studies and conclusions, and present their challenges and evaluations in clear, non-technical language.

The notion that “law is an empty vessel” is an imperfect metaphor, but it serves to emphasize that lawyers may have to get their hands wet with vessel’s contents, now and then.  In litigating scientific issues, lawyers and judges will necessarily have to engage with substantive matters.  Lawyers without scientific training or aptitude are not likely to serve clients, whether plaintiffs or defendants, well in the post-Daubert litigation world.  Untrained lawyers will choose the wrong theory, emphasize the wrong evidence, and advance the wrong conclusions.  The stakes are higher now for all the players.  An improvident claim or defense will become a blot on a lawyer’s escutcheon.  A poor gatekeeping effort or judicial decision[11] can embarrass the entire judicial system[12].

Over a decade ago, Professor David Faigman asked whether science was different for lawyers.  Professor Faigman emphatically answered his own question in the negative, and urged lawyers and courts to evolve from their pre-scientific world view.[13] Both lawyers and judges must learn about the culture, process, and content of science.  Science is often idealized as a cooperative endeavor, when in fact, much scientific work can be quite adversarial.[14]Some judges and commentators have argued that the scientific enterprise should be immune from the rough and tumble of legal discovery because the essential collaborative nature of science is threatened by the adversarial interests at play in litigation.  In becoming better judges of science, judges (and lawyers) need to develop a sense of the history of science, with its findings, fanaticisms, feuds, and fraud. The good, bad, and the ugly are all the proper subject for study. Professor George Olah, in accepting his Nobel Prize in Chemistry, rebutted the lofty sentiments about scientific collegiality and collaboration[15]:

“Intensive, critical studies of a controversial topic always help to eliminate the possibility of any errors. One of my favorite quotation is that by George von Bekessy (Nobel Prize in Medicine, 1961).

‘[One] way of dealing with errors is to have friends who are willing to spend the time necessary to carry out a critical examination of the experimental design beforehand and the results after the experiments have been completed. An even better way is to have an enemy. An enemy is willing to devote a vast amount of time and brain power to ferreting out errors both large and small, and this without any compensation. The trouble is that really capable enemies are scarce; most of them are only ordinary. Another trouble with enemies is that they sometimes develop into friends and lose a good deal of their zeal. It was in this way the writer lost his three best enemies. Everyone, not just scientists, needs a few good enemies!’”

In other words, peer review is a shabby substitute for cross-examination and an adversarial process.[16]  That adversarial process cannot always unfold fully and fairly in front of a jury.

Chief Justice Rehnquist no doubt spoke for most judges and lawyers in expressing his discomfort with the notion that courts would have to actually look at science (rather than qualifications, demeanor, and credibility)[17]:

“I defer to no one in my confidence in federal judges; but I am at a loss to know what is meant when it is said that the scientific status of a theory depends on its ‘falsifiability’, and I suspect some of them will be, too.”

Before the Daubert decision, some commentators opined that judges and lawyers were simply too innumerate and too dull to be involved in litigating scientific issues[18]. Today, most federal judges at least would not wear their ignorance so proudly, largely because of the efforts of the Federal Judicial Center, and its many educational efforts.  Judges (and lawyers) cannot and should not be scientific or statistically illiterate[19].  If generalist judges think this is beyond their ken, then they should have the intellectual integrity to say so, and get out of the way.  If the scientific tasks are beyond the ken of judges, then they are likely beyond the abilities of ordinary jurors as well, and we should start to think seriously once again about science courts and blue-ribbon juries.

Astute and diligent gatekeeping judges and qualified juries can get the job done[20]. Certainly, some judges have seen through expert witnesses’ evasions and errors in reasoning, and lawyers’ shystering on both sides of litigation.  Evaluating scientific evidence and drawing inferences from scientific studies do require understanding of basic statistical concepts, study design, and scientific apparatus, but at the bottom, there is no esoteric “scientific method” that is not careful and skeptical reasoning about evidence in the context of all the facts.[21]  Surely, judges with university educations, “20 years of schooling,” and at least 10 years of exemplary professional practice should, in theory, be able to get the job done, even if a few judges are unqualified or ill suited by aptitude or training. To take an example from the breast implant litigation, understanding a claim about immunology may well require some depth in what the immune system is, how it works, and how it may be altered.  This understanding will require time, diligence, intellectual energy, and acumen.  Surely, ordinary lay juries often do not have the time to devote to such a case to do justice to the issues. If judges do not have the interest or the time to render the required service, they should say so.

And what about lawyers?  Twenty years into the Daubert era is time enough for law schools to implement curriculum to train lawyers to understand and to litigate scientific issues, in civil, criminal, and regulatory contexts.  Many law schools still fail to turn out graduates with basic competence to understand and advocate about scientific issues.[22]  Lawyers who want to litigate scientific issues owe their clients and themselves the commitment to understand the issues.  It has been a long time since C.P. Snow complained of the “two cultures,” science and the humanities, and the hostility he faced when he challenged colleagues about their ignorance of science.[23]  Law schools have a role in building a bridge between the two cultures of science and the humanities. Just as tax programs in law schools require basic accounting, law schools should ensure that their graduates, destined to work on scientific litigation, legislation, and regulation, have had some education in statistics, probability, and scientific method.

Becoming or remaining scientific literate is a daunting but essential task for busy lawyers who hope to serve clients well in technical litigation.  There are many resources available for the “scientific legal counselor.”  Dr. David Schwartz at Innovative Science Solutions LLC has published a resource guide, The Litigator’s Guide to Combating Junk Science, available at his firm’s website.[24] Sense About Science, a charitable trust, has worked hard to advance an evidence-based world view, and to help people, including lawyers, make sense of scientific and medical claims.  Its website offers some helpful publications,[25] one of the more interesting being its recent pamphlet, Making Sense of Uncertainty.[26]

The Sense About Science group’s annual lectures are important events, which have featured lectures by:

  • Dr Fiona Godlee, the editor of the British Medical Journal, on “It’s time to stand up for science once more,” in 2010.[27]
  • Dr Olivia Judson, scientist and science journalist, on “Why Experiment?” in 2009.[28]
  • Professor Alan Sokal, quantum physicist and famous perpetrator of the Sokal hoax[29] that helped deconstruct the post-modernist deconstructionists, on “What is Science and Why Should We Care?” in 2008.[30]
  • Sir John Krebs FRS, discoverer of the Krebs cycle, on “Science Advice, Impartiality and Policy,” in 2006.[31]

In addition to keeping abreast of scientific developments, science lawyers need to understand both how science is “made,” and how it goes bad.  The National Institutes of Health website has many resources about the grant process.  Science lawyers should understand the grant process and how to use the Freedom of Information Act to obtain information about research at all stages of development.  Contrary to the fairy-tale accounts of idealized science, the scientific research process goes astray with some frequency, so much so that there is a federal agency, the Office of Research Integrity (ORI) tasked with investigating research misconduct. News of the ORI’s investigations, and findings, is readily available through its website and its informative blog.[32] Research misconduct has resulted in an ever-increasing rate of retractions in peer-reviewed journals.  The Retraction Watch[33] blog offers a fascinating window into the inadequacies of peer review, and the attempts, some successful, some not, of the scientific community to police itself.

Before Daubert was decided, the legal system emphasized credentials and qualifications of witnesses, “authoritative texts,” and general acceptance.  Science, however, is built upon an evidence-based foundation.  Since 1663, the Royal Society has sported the motto:  “Nullius in verba”: on no person’s authority.  When confronted with a pamphlet entitled “100 Authors against Einstein,” Albert Einstein quipped “if I were wrong, one would have been enough.”[34]  Disputes in science are resolved with data, from high-quality, reproducible experimental or observational studies, not with appeals to the prestige of the speaker.  The shift from authority-based decision making to evidence-based inferences and conclusions has strained the legal system in the United States, where people are free to propagate cults and superstitions.  The notion that lawyers who spend most of their time litigating leveraged boxcar leases can appreciate the nuances, strengths, weaknesses, and flaws of scientific studies and causal inference is misguided.  Litigants, whether those who make claims or those who defend claims, deserve legal counsel who are conversant with the language of their issues.  Law schools, courts, and bar associations must rise to the occasion to meet the social need.

[1] Oliver Wendell Holmes, Jr., “The Path of the Law,” 10 Harv. L. Rev. 457, 469 (1897)

[2] Oliver Wendell Holmes, Jr., Collected Legal Papers 230 (1921).

[3] An earlier version of this post appeared as a paper in David M. Cohen & Nathan A. Schachtman, eds., Daubert Practice 2013: How to Avoid Getting Sliced by Cutting-Edge Developments in Expert Witness Challenges (PLI 2013).

[4] See Ross v. Bernhard, 396 U.S. 531, 538 n.l0 (1970) (suggesting that the right to a jury trial may be limited by considerations of complexity, and that practical abilities and limitations of juries are a factor in determining whether an issue is to be decided by the judge or the jury).  See also Note, “The Right to a Jury Trial in Complex Civil litigation,” 92 Harv. L. Rev. 898 (1979) (considering the implications of footnote 10 in Ross for complex litigation; noting that the Ross test has been applied infrequently, and in the limited contexts of antitrust or securities cases); Peter Huber, “Comment, A Comment on Toward Incentive-Based Procedure: Three Approaches for Regulating Scientific Evidence by E. Donald Elliott,” 69 Boston Univ. L. Rev. 513, 514 (1989) (arguing that judges cannot make difficult scientific judgments).  See also George K. Chamberlin, “Annotation: Complexity of Civil Action as Affecting Seventh Amendment Right to Trial by Jury,” 54 A.L.R. FED. 733, 737-44 (1981).

[5] James A. Martin, “The Proposed Science Court,” 75 Mich. L. Rev. 1058, 1058-91 (1977) (evaluating the need and feasibility of various proposals for establishing science courts); Troyen A. Brennan, “Helping Courts with Toxic Torts,” 51 U. Pitt. L. Rev. 1, 5 (1989) (arguing for administrative panels or boards to assist courts with scientific issues); Donald Elliott, “Toward Incentive-Based Procedure: Three Approaches for Regulating Scientific Evidence,” 69 Boston Univ. L. Rev. 487, 501-07 (1989) (discussing various models of ancilliary peer review for courts); Paul K. Sidorenko, “Comment, Evidentiary Dilemmas in Establishing Causation: Are Courts Capable of Adjudicating Toxic Torts?” 7 Cooley L. Rev. 441, 442 (1990) (recommending administrative panels of scientific experts in toxic tort litigation); John W. Osborne, “Judicial Technical Assessment of Novel Scientific Evidence,” 1990 U. Ill. L. Rev. 497, 540-46 (arguing for procedures with specially qualified judges); Edward V. DiLello, “Note, Fighting Fire with Firefighters: A Proposal for Expert Judges at the Trial Level,” 93 Colum. L. Rev. 473, 473 (1993) (advocating establishment of specialist judicial assistants for judges).  For a bibliography of publications on the science court concept, see Jon R. Cavicchi, “The Science Court: A Bibliography” (1994) <http://ipmall.info/risk/vol4/spring/bibliography.htm>, last visited on July 27, 2014.

[6] See Ross v. Bernhard, 396 U.S. at 538 n.l0.

[7] John W. Wesley, “Scientific Evidence and the Question of Judicial Capacity,” 25 William & Mary L. Rev. 675, 702-03 (1984).

[8] See Karen Butler Reisinger, “Expert Panels: A Comparison of Two Models,” 32 Indiana L.J. 225 (1998); Laural L. Hooper, Joe S. Cecil & Thomas E. Willging, Neutral Science Panels: Two Examples of Panels of Court-Appointed Experts in the Breast Implants Product Liability Litigation, (Federal Judicial Center 2001), available at http://www.fjc.gov/public/pdf.nsf/lookup/neuscipa.pdf/$file/neuscipa.pdf

[9] Soldo v. Sandoz Pharms. Corp., 244 F. Supp. 2d 434 (W.D. Pa. 2003); see also Joe S. Cecil, “Construing Science in the Quest for Ipse Dixit,” 33 Seton Hall L. Rev. 967 (2003).

[10] DePyper v. Navarro, No. 191949, 1998 WL 1988927 (Mich. Ct. App. Nov. 6, 1998)

[11] See, e.g., Wells v. Ortho Pharmaceutical Corp., 615 F. Supp. 262 (N.D. Ga. 1985), aff’d and rev’d in part on other grounds, 788 F.2d 741 (11th Cir.), cert. denied, 479 U.S.950 (1986).

[12] See, e.g., James L. Mills and Duane Alexander, “Teratogens and ‘Litogens’,” 15 New Engl. J. Med. 1234 (1986); Samuel R. Gross, “Expert Evidence,” 1991 Wis. L. Rev. 1113, 1121-24 (1991) (“Unfortunately, Judge Shoob’s decision is absolutely wrong. There is no scientifically credible evidence that Ortho-Gynol Contraceptive Jelly ever causes birth defects.”). See also Editorial, “Federal Judges v. Science,” N.Y. Times, December 27, 1986, at A22 (unsigned editorial);  David E. Bernstein, “Junk Science in the Courtroom,” Wall St. J. at A 15 (Mar. 24,1993) (pointing to Wells as a prominent example of how the federal judiciary had embarrassed American judicial system with its careless, non-evidence based approach to scientific evidence); Bert Black, Francisco J. Ayala & Carol Saffran-Brinks, “Science and the Law in the Wake of Daubert: A New Search for Scientific Knowledge,” 72 Texas L. Rev. 715, 733-34 (1994) (lawyers and leading scientist noting that the district judge “found that the scientific studies relied upon by the plaintiffs’ expert were inconclusive, but nonetheless held his testimony sufficient to support a plaintiffs’ verdict. *** [T]he court explicitly based its decision on the demeanor, tone, motives, biases, and interests that might have influenced each expert’s opinion. Scientific validity apparently did not matter at all.”) (internal citations omitted); Troyen A. Brennan, “Untangling Causation Issues in Law and Medicine: Hazardous Substance Litigation,” 107 Ann. Intern. Med. 741, 744-45 (1987) (describing the result in Wells as arising from the difficulties created by the Ferebee case; “[t]he Wells case can be characterized as the court embracing the hypothesis when the epidemiologic study fails to show any effect”).  Kenneth R. Foster, David E. Bernstein, and Peter W. Huber, eds., Phantom Risk: Scientific Inference and the Law 28-29, 138-39 (MIT Press 1993) (criticizing Wells decision); Hans Zeisel & David Kaye, Prove It With Figures: Empirical Methods in Law and Litigation § 6.5, at 93 (1997) (noting the multiple comparisons in studies of birth defects among women who used spermicides, based upon the many reported categories of birth malformations, and the large potential for even more unreported categories); id. at § 6.5 n.3, at 271 (characterizing Wells as “notorious,” and noting that the case became a “lightning rod for the legal system’s ability to handle expert evidence.”).

[13] David L. Faigman, “Is Science Different for Lawyers?” 297 Science 339, 340 (2002) (“some courts are still in a prescientific age”).

[14] McMillan v. Togus Reg’l Office, 294 F. Supp. 2d 305, 317 (E.D.N.Y. 2003) (“As in political controversy, ‘science is, above all, an adversary process’.”) (internal citation omitted).

[15] George A. Olah, “My Search for Carbocations and Their Role in Chemistry,” Nobel Lecture (Dec. 8, 1994), quoting George von Békésy, Experiments in Hearing 8 (N.Y. 1960).

[16] Nietzsche expressed Olah’s sentiment more succinctly.  Friedrich Nietzsche, The Twilight of the Idols Maxim 8 (1899) (“Out of life’s school of war: What does not destroy me, makes me stronger.”).

[17] Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579, 598 (1993) (Justice Rehnquist, C.J., concurring and dissenting).

[18] Joseph Nicol, “Symposium on Science and the Rules of Evidence,” 99 F.R.D. 188, 221 (1983) (claiming that lawyers and judges are “simply are incapable by education, and all too often by inclination, to become sufficiently familiar with scientific evidence to discharge their responsibilities toward the administration of justice.”)

[19] Robert P. Merges, “The Nature and Necessity of Law and Science,” 381 Leg. Educ. 315, 324-26 (1988) (arguing that lawyers can and should understand scientific concepts and issues)

[20] David L. Faigman, “To Have and Have Not: Assessing the Value of Social Science to the Law as Science and Policy,” 38 Emory L.J. 1005, 1014, 1030 (1989) (contending that judges can understand science and that lawyers must probe beyond conclusions into methods by which the conclusions were reached).

[21] David H. Kaye, “Proof in Law and Science,” 32 Jurimetrics J. 313, 318 (1992) (arguing that science and law share identical methods of establishing factual conclusions); Lee Loevinger, “Standards of Proof in Science and Law,” 32 Jurimetrics J. 323, 328 (1992) (“[T]he basic principles of reasoning or logic are no different in the field of law than science.”).

[22] Howell E. Jackson, “Analytical Methods for Lawyers,” 53 J. Legal Educ. 321 (2003); Steven B. Dow, “There’s Madness in the Method:  A Commentary on Law, Statistics, and the Nature of Legal Education,” 57 Okla. L. Rev. 579 (2004).

[23] C.P. Snow, The Rede Lecture 1959.

[24] Available with registration at http://www.innovativescience.net/complimentary-copy-of-our-junk-science-ebook/

[25]http://www.senseaboutscience.org/

[26] http://www.senseaboutscience.org/resources.php/127/making-sense-of-uncertainty

[27] http://www.senseaboutscience.org/pages/annual-lecture-2010.html

[28] http://www.senseaboutscience.org/pages/annual-lecture-2009.html

[29] Sokal’s famous parody of postmodern writers, the so-called “‘Sokal Hoax,” was his publication in one of the postmodernists’ own journals on how gravity was nothing more than a social construct.  Alan D. Sokal, “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity,” Social Text 217 (Nos. 46/47 1996) (“It has thus become increasingly apparent that physical ‘reality’, no less than social ‘reality’, is at bottom a social and linguistic construct; that scientific ‘knowledge’, far from being objective, reflects and encodes the dominant ideologies and power relations of the culture that produced it; that the truth claims of science are inherently theory-laden and self-referential; and consequently, that the discourse of the scientific community, for all its undeniable value, cannot assert a privileged epistemological status with respect to counter-hegemonic narratives emanating from dissident or marginalized communities.”)

[30] http://www.senseaboutscience.org/pages/annual-lecture-2008.html

[31] http://www.senseaboutscience.org/pages/annual-lecture-2006.htm

[32] http://ori.hhs.gov/blog/

[33] http://retractionwatch.wordpress.com/

[34] See Remigio Russo, 18 Mathematical Problems in Elasticity 125 (1996) (quoting Einstein).

Posted in Scientific Evidence | Comments Off on The Role of the “Science Lawyer” in Modern Litigation

NIEHS Transparency? We Can See Right Through You

July 10th, 2014

The recent issue of Environmental Health Perspectives contains several interesting articles on scientific methodology of interest to lawyers who litigate claimed health effects.[1] The issue also contains a commentary that argues for greater transparency in science and science policy, which should be a good thing, but yet the commentary has the potential to obscure and confuse. Kevin C. Elliott and David B. Resnik, “Science, Policy, and the Transparency of Values,” 122 Envt’l Health Persp. 647 (2014) [Elliott & Resnik].

David B. Resnik has a Ph.D., in philosophy from University of North Carolina, and his law degree from the on-line Concord University School of Law.  He is currently a bioethicist and the chairman of the NIEHS Institutional Review Board. Kevin Elliott received his doctorate in the History and Philosophy of Science (Notre Dame), and he is currently an Associate Professor in Michigan State University. Elliott and Resnik advance a plea for transparency that superficially is as appealing as motherhood and apple pie. The authors argue

“that society is better served when scientists strive to be as transparent as possible about the ways in which interests or values may influence their reasoning.”

The argument appears superficially innocuous.  Indeed, in addition to the usual calls for great disclosure of conflicts of interest, the authors call for more data sharing and less tendentious data interpretation:

“When scientists are aware of important background assumptions or values that inform their work, it is valuable for them to make these considerations explicit. They can also make their data publicly available and strive to acknowledge the range of plausible interpretations of available scientific information, the limitations of their own conclusions, the prevalence of various interpretations across the scientific community, and the policy options supported by these different interpretations.”

Alas, we may as well wish for the Kingdom of Heaven on Earth!  An ethos or a requirement of publicly sharing data would indeed advance the most important transparency, the transparency that would allow full exploration of the inferences and conclusions claimed in a particular study.  Despite their high-mindedness, the authors’ argument becomes muddled when it comes to conflating scientific objectivity with subjective values:

“In the past, scientists and philosophers have argued that the best way to maintain science’s objectivity and the public’s trust is to draw a sharp line between science and human values or policy (Longino 1990). However, it is not possible to maintain this distinction, both because values are crucial for assessing what counts as sufficient evidence and because ethical, political, economic, cultural, and religious factors unavoidably affect scientific judgment (Douglas 2009; Elliott 2011; Longino 1990; Resnik 2007, 2009).”

This argument confuses pathology of science with what actually makes science valuable and enduring.  The Nazis invoked cultural arguments, explicitly or implicitly to reject “Jewish” science; religious groups in the United States invoke religious and political considerations to place creationism on an equal or superior footing with evolution; anti-vaccine advocacy groups embrace case reports over rigorous epidemiologic analyses. To be sure, these and other examples show that “ethical, political, economic, cultural, and religious factors unavoidably affect scientific judgment,” but yet science can and does transcend them.  There is no Jewish or Nazi science; indeed, there is no science worthy of its name that comes from any revealed religion or cult.  As Tim Minchin has pointed out, alternative medicine is either known not to work or not known to work because if alternative medicine is known to work, then we call it “medicine.” The authors are correct that these subjective influences require awareness and understanding of prevalent beliefs, prejudices, and corrupting influences, but they do not, and they should not, upset our commitment to an evidence-based world view.

Elliott and Resnik are focused on environmentalism and environmental policy, and they seem to want to substitute various presumptions, leaps of faith, and unproven extrapolations for actual evidence  and valid inference, in the hope of improving the environment and reducing risk to life.  The authors avoid the obvious resolution: value the environment, but acknowledge ignorance and uncertainty.  Rather than allow precautionary policies to advance with a confession of ignorance, the authors want to retain their ability to claim knowledge even when they simply do not know, just because the potential stakes are high. The circularity becomes manifest in their ambiguous use of “risk,” which strictly means a known causal relationship between the “risk” and some deleterious outcome.  There is a much weaker usage, popularized by journalists and environmentalists, in which “risk” refers to something that might cause a deleterious outcome.  The might in “risk” here does not refer to a known probabilistic or stochastic relationship between the ex ante risk and the outcome, but rather to an uncertainty whether or not the relationship exists at all. We can see the equivocation in how the authors attempt to defend the precautionary principle:

“Insisting that chemicals should be regulated only in response to evidence from human studies would help to prevent false positive conclusions about chemical toxicity, but it would also prevent society from taking effective action to minimize the risks of chemicals before they produce measurable adverse effects in humans. Moreover, insisting on human studies would result in failure to identify some human health risks because the diseases are rare, or the induction and latency periods are long, or the effects are subtle (Cranor 2011).”

Elliott & Resnik at 648.

If there is uncertainty about the causal relationship, then by calling some exposures a “risk,” the authors prejudge whether there will be “adverse effects” at all. This is just muddled.  If the relationship is uncertain, and false positive conclusions are possible, then we simply cannot claim to know that there will be such adverse effects, without assuming what we wish to prove.

The authors compound the muddle by introducing a sliding scale of “standards of evidence,” which appears to involve both variable posterior probabilities that the causal claim is correct, as well as variable weighting of types of evidence.  It is difficult to see how this will aid transparency and reduce confusion. Indeed, we can see how manipulative the authors’ so-called transparency becomes in the context of evaluating causal claims in pharmaceutical approvals versus tort claims:

“Very high standards of evidence are typically expected in order to infer causal relationships or to approve the marketing of new drugs. In other social contexts, such as tort law and chemical regulation, weaker standards of evidence are sometimes acceptable to protect the public (Cranor 2008).”

Remarkably, the authors cite no statute, no case law, no legal treatise writer for the proposition that the tort law standard for causation is somehow lower than for a claim of drug efficacy before the Food and Drug Administration.  The one author they cite, Carl Cranor, is neither a scientist nor a lawyer, but a philosophy professor who has served as an expert witness for plaintiffs in tort litigation (usually without transparently disclosing his litigation work). As for the erroneous identification of tort and regulatory standards, there is of course, much real legal authority to the contrary[2].

The authors go on to suggest that demanding

“the very highest standards of evidence for chemical regulation—including, for example, human evidence, accompanying animal data, mechanistic evidence, and clear exposure data—would take very long periods of time and leave the public’s health at risk.”

Elliott & Resnik at 648.

Of course, the point is that until such data are developed, we really do not know whether the public’s health is at risk.  Transparency would be aided not by some sliding and slippery scale of evidence, but by frank admissions that we do not know whether the public’s health is at risk, but we choose to act anyway, and to impose whatever costs, inconvenience, and further uncertainty by promoting alternatives that are accompanied by even greater risk or uncertainty.  Environmentalists rarely want to advance such wishy-washy proposals, devoid of claims of scientific knowledge that their regulations will avoid harm, and promote health, but honesty and transparency require such admissions.

The authors advance another claim in their Commentary:  transparency in the form of more extensive disclosure of conflicts of interest will aid sound policy formulation.  To their credit, the authors do not limit the need for disclosure to financial benefits; rather they take an appropriately expansive view:

“Disclosures of competing financial interests and nonfinancial interests (such as professional or political allegiances) also provide opportunities for more transparent discussions of the impact of potentially implicit and subconscious values (Resnik and Elliott 2013).”

Elliott & Resnik at 649.  Problematically, however, when the authors discuss some specific instances of apparent conflicts, they note industry “ties,” of the authors of an opinion piece on endocrine disruptors[3], but they are insensate to the ties of critics, such as David Ozonoff and Carl Cranor, to the litigation industry, and of others to advocacy groups that might exert much more substantial positional bias and control over those critics.

The authors go further in suggesting that women have greater perceptions of risk than men, and presumably we must know whether we are being presented with a feminist or a masculinist risk assessment. Will self-reported gender suffice or must we have a karyotype? Perhaps we should have tax returns and a family pedigree as well? The call for transparency seems at bottom a call for radical subjectivism, infused with smug beliefs that want to be excused from real epistemic standards.


[1] In addition to the Elliott and Resnick commentary, see Andrew A. Rooney, Abee L. Boyles, Mary S. Wolfe, John R. Bucher, and Kristina A. Thayer, “Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments,” 122 Envt’l Health Persp. 711 (2014); Janet Pelley, “Science and Policy: Understanding the Role of Value Judgments,” 122 Envt’l Health Persp. A192 (2014); Kristina A. Thayer, Mary S. Wolfe, Andrew A. Rooney, Abee L. Boyles, John R. Bucher, and Linda S. Birnbaum, “Intersection of Systematic Review Methodology with the NIH Reproducibility Initiative,” 122 Envt’l Health Persp. A176 (2014).

[2] Sutera v. The Perrier Group of America, 986 F. Supp. 655, 660 (D. Mass. 1997); In re Agent Orange Product Liab. Litig., 597 F. Supp. 740, 781 (E.D.N.Y. 1984) (Weinstein, J.), aff’d, 818 F.2d 145 (2d Cir. 1987); Allen v. Pennsylvania Engineering Corp., 102 F.3d 194, 198 (5th Cir. 1996) (distinguishing regulatory pronouncements from causation in common law actions, which requires higher thresholds of proof); Glastetter v. Novartis Pharms. Corp., 107 F. Supp. 2d 1015, 1036 (E.D. Mo. 2000), aff’d, 252 F.3d 986 (8th Cir. 2001);  Wright v. Willamette Indus., Inc., 91 F.3d 1105 (8th Cir. 1996); Siharath v. Sandoz Pharms. Corp., 131 F. Supp. 2d 1347, 1366 (N.D. Ga. 2001), aff’d, 295 F.3d 1194 330 (11th Cir. 2002).

[3] Daniel R. Dietrich, Sonja von Aulock, Hans Marquardt, Bas Blaauboer, Wolfgang Dekant, Jan Hengstler, James Kehrer, Abby Collier, Gio Batta Gori, Olavi Pelkonen, Frans P. Nijkamp, Florian Lang, Kerstin Stemmer, Albert Li, KaiSavolainen, A. Wallace Hayes, Nigel Gooderham, and Alan Harvey, “Scientifically unfounded precaution drives European Commission’s recommendations on EDC regulation, while defying common sense, well-established science and risk assessment principles,” 62 Food Chem. Toxicol. A1 (2013)

 

Posted in Conflicts of Interest, Data Sharing, Scientific Evidence | Comments Off on NIEHS Transparency? We Can See Right Through You

Twerski’s Defense of Daubert

July 6th, 2014

Professor Aaron D. Twerski teaches torts and products liability at the Brooklyn Law School.  Along with a graduating student, Lior Sapir, Twerski has published an article in which the authors mistakenly asseverate that “[t]his is not another article about Daubert.” Aaron D. Twerski & Lior Sapir, “Sufficiency of the Evidence Does Not Meet Daubert Standards: A Critique of the Green-Sanders Proposal,” 23 Widener L.J. 641, 641 (2014) [Twerski & Sapir].

A few other comments.

1. The title of the article.  True, true, and immaterial. As Professor David Bernstein has pointed out many times, Daubert is no longer the law; Federal Rule of Evidence 702, a statute, is the law.  Just as the original Rule 702 superseded Frye in 1975, a revised Rule 702, in 2000, superseded Daubert in 1975. See David E. Bernstein, “The Misbegotten Judicial Resistance to the Daubert Revolution,” 89 Notre Dame L. Rev. 27 (2013).

2. Twerski and Sapir have taken aim at a draft paper by Professors Green and Sanders, who also presented similar ideas at a workshop in March 2012, in Spain. The Green-Sanders manuscript is available on line. Michael D. Green & Joseph Sanders, “Admissibility Versus Sufficiency: Controlling the Quality of Expert Witness Testimony in the United States,” (March 5, 2012) <downloaded on March 25, 2012>. This article appears to have matured since spring 2012, but it has never progressed to parturition.  Professor Green’s website suggests a mutated version is in the works:  “The Daubert Sleight of Hand: Substituting Reliability, Methodology, and Reasoning for an Old Fashioned Sufficiency of the Evidence Test.”

Indeed, the draft paper is a worthwhile target. See “Admissibility versus Sufficiency of Expert Witness Evidence” (April 18, 2012).  Green and Sanders pursue a reductionist approach to Rule 702, which is unfaithful to the letter and spirit of the law.

3. In their critique of Green and Sanders, Twerski and Sapir get some issues wrong. First they insist upon talking about Daubert criteria.  The “criteria” were never really criteria, and as Bernstein’s scholarship establishes, it is time to move past Daubert.

4. Twerski and Sapir assert that Daubert imposes a substantial or heavy burden of proof upon the proponent of expert witness opinion testimony:

“The Daubert trilogy was intended to set a formidable standard for admissibility before one entered the thicket of evaluating whether it was sufficient to serve as grounds for recovery.”

Twerski & Sapir at 648.

Daubert instituted a “high threshold of reliability”.

Twerski & Sapir at 649.

“But, the message from the Daubert trilogy is unmistakable: a court must have a high degree of confidence in the integrity of scientific evidence before it qualifies for consideration in any formal test to be utilized in litigation.”

Twerski & Sapir at 650.

“The Daubert standard is anything but minimal.”

Twerski & Sapir at 651.

Twerski and Sapir never explain whence comes “high,” “formidable,” and “anything but minimal.” To be sure, the Supreme Court noted that “[s]ince Daubert . . . parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.” Weisgram v. Marley Co., 528 U.S. 440, 455 (2000) (emphasis added). An exacting standard, however, is not necessarily a heavy burden.  It may be that the exacting standard is infrequently satisfied because the necessary evidence and inferences, of sufficiency quality and validity, are often missing. The truth is that science is often in the no-man’s land of indeterminate, inconclusive, and incomplete. Nevertheless, Twerski and Sapir play into the hands of the reductionist Green-Sanders’ thesis by talking about what appears to be a [heavy] burden of proof and the “weight of evidence” needed to sustain the burden.

5. Twerski and Sapir obviously recognize that reliability is different from sufficiency, but they miss the multi-dimensional aspect of expert witness opinion testimony.  Consider their assertion that:

“[t]he Court of Appeals for the Eleventh Circuit in Joiner had not lost its senses when it relied on animal studies to prove that PCBs cause lung cancer. If the question was whether any evidence viewed in the light most favorable to plaintiff supported liability, the answer was probably yes.”

Twerski & Sapir at 649; see Joiner v. Gen. Electric Co., 78 F.3d 524, 532 (11th Cir. 1996) rev’d, 522 U.S. 136 (1997).

The imprecision in thinking about expert witness testimony obscures what happened in Joiner, and what must happen under the structure of the evidence statutes (or case law).  The Court of Appeals never relied upon animal studies; nor did the district court below.  Expert witnesses relied upon animal studies, and other studies, and then offered an opinion that these studies “prove” PCBs cause human lung cancer, and Mr. Joiner’s lung cancer in particular.  Those opinions, which the Eleventh Circuit would have taken at face value, would be sufficient to support submitting the case to jury.  Indeed, courts that evade the gatekeeping requirements of Rule 702 routinely tout the credentials of the expert witnesses, recite that they have used science in some sense, and that criticisms of their opinions “go to the weight not the admissibility” of the opinions.  These are, of course, evasions used to dodge Daubert and Rule 702. They are evasions because the science recited is at a very high level of abstraction (“I relied upon epidemiology”), because credentials are irrelevant, and because “weight not the admissibility” is a conclusion not a reason.

Some of the issues obscured by the reductionist weight-of-the-evidence approach are the internal and external validity of the studies cited, whether the inferences drawn from the studies cited are valid and accurate, and whether the method of synthesizing  conclusion from disparate studies is appropriate. These various aspects of an evidentiary display cannot be reduced to a unidimensional “weight.” Consider how many observational studies suggested, some would say demonstrated, that beta carotene supplements reduced the risk of lung cancer, only to be pushed aside by one or two randomized clinical trials.

6. Twerski and Sapir illustrate the crucial point that gatekeeping judges must press beyond the conclusory opinions by exploring the legal controversy over Parlodel and post-partum strokes.  Twerski & Sapir at 652. Their exploration takes them into some of the same issues that confronted the Supreme Court in Joiner:  extrapolations or “leaps of faith” between different indications, different species, different study outcomes, between surrogate end points and the end point of interest, between very high to relatively low therapeutic doses. Twerski and Sapir correctly discern that these various issues cannot be simply subsumed under weight or sufficiency.

7. Professors Green and Sanders have published a brief reply, in which they continue their “weight of the evidence” reductionist argument. Michael D. Green & Joseph Sanders, “In Defense of Sufficiency: A Reply to Professor Twerski and Mr. Sapir,” 23 Widener L.J. 663 (2014). Green and Sanders restate their position that courts can, should, and do sweep all the nuances of evidence and inference validity into a single metric – weight and sufficiency – to adjudicate so-called Daubert challenges.  What Twerski and Sapir seem to have stumbled upon is that Green and Sanders are not engaged in a descriptive enterprise; they are prescribing a standard that abridges and distorts the law and best practice in order to ensure that dubious causal claims are submitted to the finder of fact.

Posted in Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Twerski’s Defense of Daubert

Goodman v Viljoen – Statistical Fallacies from Both Sides

June 8th, 2014

There was a deep irony to the Goodman[1] case.  If a drug company, in 1995, marketed antenatal corticosteroid (ACS) for the prevention of cerebral palsy (CP) in the United States, the government might well have prosecuted the company for misbranding.  The company might also be subject to a False Claims Act case as well. No clinical trial had found ACS efficacious for the prevention of CP at the significance level typically required by the FDA; no meta-analysis had found ACS statistically significantly better than placebo for this purpose.  In the Goodman case, however, failure to order a full course of ACS was malpractice with respect to the claimed causation of CP in the Goodman twins.

The Goodman case also occasioned a well-worn debate over the difference between scientific and legal evidence, inference, and standards of “proof.” The plaintiffs’ case rested upon a Cochrane review of ACS with respect to various outcomes. For CP, the Cochrane meta-analyzed only clinical trial data, and reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”[2]

The defendant, Dr. Viljoen, appeared to argue that the Cochrane meta-analysis must be disregarded because it did not provide a showing of efficacy for ACS in preventing CP, at a significance probability less than 5 percent.  Here is the trial court’s characterization of Dr. Viljoen’s argument:

“[192] The argument that the Cochrane data concerning the effects of ACS on CP must be ignored because it fails to reach statistical significance rests on the flawed premise that legal causation requires the same standard of proof as medical/scientific causation. This is of course not the case; the two standards are in fact quite different. The law is clear that scientific certainty is not required to prove causation to the legal standard of proof on a balance of probabilities (See: Snell v. Farrell, [1990] 2 S.C.R. 311, at para. 34). Accordingly, the defendant’s argument in this regard must fail and for the purposes of this court, I accept the finding of the Cochrane analysis that ACS reduces the instance [sic] of CP by 40%.”

“Disregard” seems extreme for a meta-analysis that showed a 40% reduction in risk of a serious central nervous system disorder, with p = 0.065.  Perhaps Dr. Viljoen might have tempered his challenge some by arguing that the Cochrane analysis was insufficient.  One problem with Dr. Viljoen’s strident argument about statistical significance was that it overshadowed the more difficult, qualitative arguments about threats to validity in the Cochrane finding from loss to follow up in the aggregated trial data. These threats were probably stronger arguments against accepting the Cochrane “trend” as a causal conclusion. Indeed, the validity and the individual studies and the meta-analyses, along with questions about the accuracy of data, were not reflected in Bayesian analysis.

Another problem is that Dr. Viljoen’s strident assertion that p < 0.05 was absolutely necessary fed plaintiffs’ argument that the defendant was attempting to change the burden of proof for plaintiffs from greater than 50% to 95% or greater.  Given the defendant’s position, great care was required to prevent the trial court from committing the transposition fallacy.

Justice Walters rejected the suggestion that a meta-analysis with a p-value of 6.5% should be disregarded, but the court’s discussion skirts the question whether and how the Cochrane data can be sufficient to support a conclusion of ACS efficacy. Aside from citing a legal case, however, Justice Walters provided no basis for suggesting that the scientific standard of proof was different from the legal standard. From the trial court’s opinion, the parties or their expert witnesses appeared to conflate “confidence,” a technical term when used to describe intervals or random error around sample statistics, with “level of certainty” in the obtained result.

Justice Walters is certainly not the first judge to fall prey to the fallacious argument that the scientific burden of proof is 95%.[3]  The 95% is, of course, the coefficient of confidence for the confidence interval that is based upon a p-value of 5%. No other explanation for why 95% is a “scientific” standard of proof was offered in Goodman; nor is it likely that anyone could point to an authoritative source for the claim that scientists actually adjudge facts and theories by this 95 percent probability level.

Justice Walters’ confusion was led by the transposition fallacy, which confuses posterior and significance probabilities.  Here is a sampling from Her Honor’s opinion, first from Dr. Jon Barrett, one of the plaintiffs’ expert witnesses, an obstetrician and fetal maternal medicine specialist at Sunnybrook Hospital, in Toronto, Ontario:

“[85] Dr. Barrett’s opinion was not undermined during his lengthy cross-examination. He acknowledged that the scientific standard demands 95% certainty. He is, however, prepared to accept a lower degree of certainty. To him, 85 % is not merely a chance outcome.

                                                                                        * * *

[87] He acknowledged that scientific evidence in support of the use of corticosteroids has never shown statistical significance with respect to CP. However, he explained it is very close at 93.5%. He cautioned that if you use a black and white outlook and ignore the obvious trends, you will falsely come to the conclusion that there is no effect.”

Dr. Jon (Yoseph) Barrett is a well-respected physician, who specializes in high-risk pregnancies, but his characterization of a black-white outlook on significance testing as leading to a false conclusion of no effect was statistically doubtful.[4]  Dr. Barrett may have to make divinely inspired choices in surgery, but in a courtroom, expert witnesses are permitted to say that they just do not know. Failure to achieve statistical significance, with p < 0.05, does support a conclusion that there is no effect.

Professor Andrew Willan was plaintiffs’ testifying expert witness on statistics.  Here is how Justice Walters summarized Willan’s testimony:

“[125] Dr. Willan described different statistical approaches and in particular, the frequentist or classical approach and the Bayesian approach which differ in their respective definitions of probability. Simply, the classical approach allows you to test the hypothesis that there is no difference between the treatment and a placebo. Assuming that there is no difference, allows one to make statements about the probability that the results are not due to chance alone.

To reach statistical significance, a standard of 95% is required. A new treatment will not be adopted into practice unless there is less than a 5% chance that the results are due to chance alone (rather than due to true treatment effect).

[127] * * * The P value represents the frequentist term of probability. For the CP analysis [from the Cochrane meta-analysis], the P value is 0.065. From a statistical perspective, that means that there is a 6.5% chance that the differences that are being observed between the treatment arm versus the non-treatment arm are due to chance rather than the treatment, or conversely, a 93.5% chance that they are not.”

Justice Walters did not provide transcript references for these statements, but they are clear examples of the transposition fallacy. The court’s summary may have been unfair to Professor Willan, who seems to have taken care to avoid the transposition fallacy in his testimony:

“And I just want to draw your attention to the thing in parenthesis where it says, “P = 0.065.” So, basically that is the probability of observing data this extremely, this much in favor of ACS given, if, if in fact the no [sic, null] hypothesis was true. So, if, if the no hypothesis was true, that is there was no difference, then the probability of observing this data is only 6.5 percent.”

Notes of Testimony of Andrew Willan at 26 (April , 2010). In this quote, Professor Willan might have been more careful to point out that the significance probability of 6.5%  is a cumulative probability by describing the data observed “this extremely” and more. Nevertheless, Willan certainly made clear that the probability measure was based upon assuming the correctness of the null hypothesis. The trial court, alas, erred in stating the relevant statistical concepts.

And then there was the bizarre description by Justice Walters, of the Cochrane data, as embodying a near-uniform distribution represented by the Cochrane data:

“[190] * * * The Cochrane analysis found that ACS reduced the risk of CP (in its entirety) by 40%, 93.5% of the time.”

The trial court did not give the basis for this erroneous description of the Cochrane ACS/CP data.[5] To be sure, if the Cochrane result were true, then 40% reduction might be the expected value for all trials, but it would be a remarkable occurrence for 93.5% of the trials to obtain the same risk ratio as the one observed in the meta-analysis.

The defendant’s expert witness on statistical issues, Prof. Robert Platt, similarly testified that the significance probability reported by the Cochrane was dependent upon an assumption of the null hypothesis of no association:

“What statistical significance tells us, and I mentioned at the beginning that it refers to the probability of a chance finding could occur under the null-hypothesis of no effect. Essentially, it provides evidence in favour of there being an effect.  It doesn’t tell us anything about the magnitude of that effect.”

Notes of Testimony of Robert Platt at 11 (April 19, 2010)

Perhaps part of the confusion resulted from Prof. Willan’s sponsored Bayesian analysis, which led him to opine that the Cochrane data permitted him to state that there was a 91 to 97 percent probability of an effect, which might have appeared to the trial court to be saying the same thing as interpretation of the Cochrane’s p-value of 6.5%.  Indeed, Justice Walters may have had some assistance in this confusion from the defense statistical expert witness, Prof. Platt, who testified:

“From the inference perspective the p-value of 0.065 that we observe in the Cochrane review versus a 91 to 97 percent probability that there is an effect, those amount to the same thing.”

Notes of Testimony of Robert Platt at 50 (April 19, 2010).  Now the complement of the p-value, 93.5%, may have fallen within the range of posterior probabilities asserted by Professor Willan, but these probabilities are decidedly not the same thing.

Perhaps Prof. Platt was referring only to the numerical equivalence, but his language, “the same thing,” certainly could have bred misunderstanding.  The defense apparently attacked the reliability of the Bayesian analysis before trial, only to abandon the challenge by the time of trial.  At trial, defense expert witness Prof. Platt testified that he did not challenge Willan’s Bayesian analysis, or the computation of posterior probabilities.  Platt’s acquiescence in Willan’s Bayesian analysis is unfortunate because the parties never developed testimony exactly as to how Willan arrived at his posterior probabilities, and especially as to what prior probability he employed.

Professor Platt went on to qualify his understanding of Willan’s Bayesian analysis as providing a posterior probability that there is an effect, or in other words, that the “effect size” is greater than 1.0.  At trial, the parties spent a good deal of time showing that the Cochrane risk ratio of 0.6 represented the decreased risk for CP of administering a full course of ACS, and that this statistic could be presented as an increased CP risk ratio of 1.7, for not having administered a full course of ACS.  Platt and Willan appeared to agree that the posterior probability described the cumulative posterior probabilities for increased risks above 1.0.

“[T]he 91% is a probability that the effect is greater than 1.0, not that it is 1.7 relative risk.”

Notes of Testimony of Robert Platt at 51 (April 19, 2010); see also Notes of Testimony of Andrew Willan at 34 (April 9, 2010) (concluding that ACS reduces risk of CP, with a probability of 91 to 97 percent, depending upon whether random effects or fixed effect models are used).[6]

One point on which the parties’ expert witnesses did not agree was whether the failure of the Cochrane’s meta-analysis to achieve statistical significance was due solely to the sparse data aggregated from the randomized trials. Plaintiffs’ witnesses appeared to have testified that had the Cochrane been able to aggregate additional clinical trial data, the “effect size” would have remained constant, and the p-value would have shrunk, ultimately to below the level of 5 percent.  Prof. Platt, testifying for the defense, appropriately criticized this hand-waving excuse:

“Q. and the probability factor, the P value, was 0.065, which the previous witness had suggested is an increase in probability of our reliability on the underlying data.  Is it reasonable to assume that this data that a further increase in the sample size will achieve statistical significance?

A. No, that’s not a reasonable assumption….”

Notes of Testimony of Robert Platt at 29 (April 19, 2010).

Positions on Appeal

Dr. Viljoen continued to assert the need for significance on appeal. As appellant, he challenged the trial court’s finding that the Cochrane review concluded that there was a 40% risk reduction. See Goodman v. Viljoen, 2011 ONSC 821, at ¶192 (CanLII) (“I accept the finding of the Cochrane analysis that ACS reduces the instance of CP by 40%”). Dr. Viljoen correctly pointed out that the Cochrane review never reached such a conclusion. Appellant’s Factum, 2012 CCLTFactum 20936, ¶64.  It was the plaintiffs’ expert witnesses, not the Cochrane reviewers, who reached the conclusion of causality from the Cochrane data.

On appeal, Dr. Viljoen pressed the point that his expert witnesses described statistical significance in the Cochrane analysis would have been “a basic and universally accepted standard” for showing that ACS was efficacious in preventing CP or PVL. Id. at ¶40. The appellant’s brief then commits to the very error that Dr. Barrett complained would follow from a finding that did not have statistical significance; Dr. Viljoen maintained that the “trend” of reduced CP reduced CD rates from ACS administration “is the same as a chance occurrence.” Defendant (Appellant), 2012 CCLTFactum 20936, at ¶40; see also id. at ¶14(e) (arguing that the Cochrane result for ACS/CP “should be treated as pure chance given it was not a statistically significant difference”).

Relying upon the Daubert decision from the United States, as well as Canadian cases, Dr. Viljoen framed one of his appellate issues as whether the trial court had “erred in relying upon scientific evidence that had not satisfied the benchmark of statistical significance”:

“101. Where a scientific effect is not shown to a level of statistical significance, it is not proven. No study has demonstrated a reduction in cerebral palsy with antenatal corticosteroids at a level of statistical significance.

102. The Trial Judge erred in law in accepting that antenatal corticosteroids reduce the risk of cerebral palsy based on Dr. Willan’s unpublished Bayesian probability analysis of the 48 cases of cerebral palsy reviewed by Cochrane—an analysis prepared for the specific purpose of overcoming the statistical limitations faced by the Plaintiffs on causation.”

Defendant (Appellant), 2012 CCLTFactum 20936. The use of the verb “proven” is problematic because it suggests a mathematical demonstration, which is never available for empirical propositions about the world, and especially not for the biological world.  The use of a mathematical standard begs the question whether the Cochrane data were sufficient to establish a scientific conclusion of the efficacy of ACS in preventing CP.

In opposing Dr. Viljoen’s appeal, the plaintiffs capitalized upon his assertion that science requires a very high level of posterior probability for establishing a causal claim, by simply agreeing with it. See Plaintiffs’ (Respondents’) Factum,  2012 CCLTFactum 20937, at ¶31 (“The scientific method requires statistical significance at a 95% level.”).  By accepting the idealized notion that science somehow requires 95% certainty (as opposed to 95% confidence levels as a test for assessing random error), the plaintiffs made the defendant’s legal position untenable.

In order to keep the appellate court thinking that the defendant was imposing an extra-legal, higher burden of proof upon plaintiffs, the plaintiffs went so far as to misrepresent the testimony of their own expert witness, Professor Willan, as having committed the transposition fallacy:

“49. Dr. Willan provided the frequentist explanation of the Cochrane analysis on CP:

a. The risk ratio (RR) is .060 which means that there is a 40% risk reduction in cerebral palsy where there has been administration of antenatal corticosteroids;

b. The upper limit of the confidence interval (CI) barely crosses 1 so it just barely fails to meet the rigid test of statistical significance;

c. The p value represents the frequentist term of probability;

d. In this case the p value is .065;

e. From a statistical perspective that means that there is a 6.5% chance that the difference observed in CP rates is due to chance alone;

f. Conversely there is a 93.5% chance that the result (the 40% reduction in CP) is due to a true treatment effect of ACS.”

2012 CCLTFactum 20937, at ¶49 (citing Evidence of Dr. Willan, Respondents’ Compendium, Tab 4, pgs. 43-52).

Although Justice Doherty dissented from the affirmance of the trial court’s judgment, he succumbed to the parties’ misrepresentations about scientific certainty, and their prevalent commission of the transposition fallacy. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36 (“Scientists will draw a cause and effect relationship only when a result follows at least 95 per cent of the time. The results reported in the Cochrane analysis fell just below that standard.”), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

The statistical errors on both sides redounded to the benefit of the plaintiffs.

[1] Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[2] Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews, at 8, Issue 3. Art. No. CD004454 (2006).

[3] See, e.g., In re Ephedra Prods. Liab. Litig., 393 F.Supp. 2d 181, 191, 193 (S.D.N.Y. 2005) (fallaciously arguing that the use of a critical value of less than 5% of significance probability increased the “more likely than not” burden of proof upon a civil litigant.  Id. at 188, 193.  See also Michael O. Finkelstein, Basic Concepts of Probability and Statistics in the Law 65 (2009) (criticizing the Ephedra decision for confusing posterior probability with significance probability).

[4] I do not have the complete transcript of Dr. Barrett’s testimony, but the following excerpt from April 9, 2010, at page 100, suggests that he helped lead Justice Walters into error: “When you say statistical significance, if you say that something is statistically significance, it means you’re, for the scientific notation, 95 percent sure. That’s the standard we use, 95 percent sure that that result could not have happened by chance. There’s still a 5 percent chance it could. It doesn’t mean for sure, but 95 percent you’re sure that the result you’ve got didn’t happen by chance.”

[5] On appeal, the dissenting judge erroneously accepted Justice Walters’ description of the Cochrane review as having supposedly reported a 40% reduction in CP incidence, 93.5% of the time, from use of ACS. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) at ¶36, leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

[6] The Bayesian analysis did not cure the attributability problem with respect to specific causation.

 

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Goodman v Viljoen – Statistical Fallacies from Both Sides

Goodman v Viljoen – Subterfuge to Circumvent Relative Risks Less Than 2

June 6th, 2014

Back in March, I wrote about a “Black Swan” case, in which litigants advanced a Bayesian analysis to support their claims. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII), leave appeal den’d, Supreme Court of Canada No. 35230 (July 11, 2013).

Goodman was a complex medical practice case in which Mrs. Goodman alleged that her obstetrician, Dr. Johan Viljoen, deviated from the standard of care by failing to prescribe antenatal corticosteroids (ACS) sufficiently in advance of delivery to reduce the risks attendant early delivery for her twin boys, of early delivery. Both boys developed cerebral palsy (CP). The parties and their experts agreed that the administration of ACS reduced the risks of respiratory distress and other complications of pre-term birth, but they disputed the efficacy of ACS to avoid or diminish the risk of CP.

According to the plaintiffs, ACS would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition.  Dr. Viljoen disputed both general and specific causation. Evidence of general causation came from both randomized clinical trials (RCTs) and observational studies.

Limitations Issue

There were many peculiar aspects to the Goodman case, not the least of which was that the twins sued Dr. Viljoen over a decade after they were born.  Dr. Viljoen had moved his practice in the passage of time, and he was unable to produce crucial records that supported his account of how his staff responded to Mrs. Goodman’s telephone call about signs and symptoms of labor. The prejudice to Dr. Viljoen illustrates the harshness of broad tolling statutes, the unfairness of which could be reduced by requiring infant plaintiffs to give notice of their intent to sue, even if they wait until the age of majority before filing their complaints.

State of the Art Issue

Dr. Viljoen suffered perhaps a more serious prejudice in the form of hindsight bias that resulted from the evaluation of his professional conduct by evidence that was unavailable when the twins were born in 1995. The following roughly contemporaneous statement from the New England Journal of Medicine is typical of serious thinking at the time of the alleged malpractice:

“Antenatal glucocorticoid therapy decreases the incidence of several complications among very premature infants. However, its effect on the occurrence of cystic periventricular leukomalacia, a major cause of cerebral palsy, remains unknown.”

Olivier Baud, Laurence Laurence Foix l’Hélias, et al., “Antenatal Glucocorticoid- Treatment and Cystic Periventricular Leukomalacia in Very Premature Infants,” 341 New Engl. J. Med. 1190, 1190 (1999) (emphasis added). The findings of this observational study illustrate some of the difficulties with the claim that Dr. Viljoen failed to prevent an avoidable consequence of pre-term delivery:

“Our results suggest that exposure to betamethasone but not dexamethasone is associated with a decreased risk of cystic periventricular leukomalacia.”

Id. at 1194. Results varied among various corticosteroids, among doses, among timing regimens.  There hardly seemed enough data in 1995 to dictate a standard of care.

Meta-Analysis Issues

Over ten years after the Goodman twins were born, the Cochrane collaboration published a meta-analysis that was primarily concerned with the efficacy of ACS for lung maturation. Devender Roberts & Stuart R Dalziel “Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth,” Cochrane Database of Systematic Reviews Issue 3. Art. No. CD004454 (2006). The trials included mostly post-dated the birth of the twins, and the alleged malpractice. The relevance of the trials to address the causation of CP in infants who experienced periventricular leukomalacia (PVL) was hotly disputed, but for now, I will gloss over the external validity problem of the Cochrane meta-analysis.

The Cochrane Collaboration usually limits its meta-analyses to the highest quality evidence, or RCTs, but in this instance, the RCTs did not include CP in its primary pre-specified outcomes. Furthermore, the trials were generally designed to ascertain short-term benefits from ACS, and the data in the trials were uncertain with respect to longer-term outcomes, which may have been ascertained differentially. Furthermore, the trials were generally small and were plagued by sparse data.  None of the individual trials was itself statistically significant at the 5 percent level.  The meta-analysis did not show a statistically significant decrease in CP from ACS treatment.  The authors reported:

“a trend towards fewer children having cerebral palsy (RR 0.60, 95% CI 0.34 to 1.03, five studies, 904 children, age at follow up two to six years in four studies, and unknown in one study).”

 Id. at 8 (emphasis added).

The Cochrane authors were appropriately cautious in interpreting the sparse data:

“Results suggest that antenatal corticosteroids result in less neurodevelopmental delay and possibly less cerebral palsy in childhood.”

Id. at 13-14 (emphasis added).

The quality of the trials included in the Cochrane meta-analysis varied, as did the trial methodologies.  Despite the strong clinical heterogeneity, the Cochrane authors performed their meta-analysis with a fixed-effect model. The confidence interval, which included 1.0, reflected a p-value of 0.065, but that p-value would have certainly increased if a more appropriate random-effects model had been used.

Furthermore, the RCTs were often no better than observational studies on the CP outcome. The RCTs here perhaps should not have been relied upon to the apparent exclusion of observational epidemiology.

Relative Risk Less Than Two

There is much to be said about the handling of statistical significance, the Bayesian analysis, the arguments about causal inference, but for now, let us look at one of the clearest errors in the case:  the inference of specific causation from a relative risk less than two.  To be sure, the Cochrane meta-analysis reported a non-statistically significant 40% decrease, but if we were to look at this outcome in terms of the increase in risk of CP from the physician’s failure to administer ACS timely, then the risk ratio would be 1.67, or a 67% increase.  On either interpretation, fewer than half the cases of CP can be attributed to the failure to administer ACS fully and timely in the case.

The parties tried their case before Justice Walters, in St. Catherines, Ontario. Goodman v. Viljoen, 2011 ONSC 821 (CanLII).  Justice Walters recognized that specific causation was essential and at the heart of the parties’ disagreement:

“[47] In order to succeed, the plaintiffs must establish that the failure to receive a full course of ACS materially affected the twins’ outcome. That is, they must establish that “but for” the failureto receive a full course of ACS, the twins would not have suffered from the conditions they now do, or that the severity of these afflictions would have been materially reduced.

[48] Not surprisingly, this was the most contentious issue at trial and the court heard a good deal of evidence with respect to the issue of causation.”

One of the defendant’s expert witnesses, Robert Platt, a professor of statistics at McGill University School of Medicine, testified, according to Justice Walters:

“[144] Dr. Platt also stated that the absolute risk in and of itself does not tell us anything about what might have happened in a specific case absent clinical and mechanistic explanations for that specific case.”

The plaintiffs’ expert witnesses apparently conceded the point.  Professor Andrew Willan, a statistician, testifying for the plaintiffs, attempted to brush Platt’s point aside by suggesting it would render clinical research useless, but that was hardly the point.  Platt embraced clinical research for what it could show about the “averages” in a sample of the population, even if we cannot discern causal efficacy retrospectively in a specific patient:

“[133] Dr. Willan also responded to Dr. Platt’s criticism that it was impossible to determine the distribution of the effect across the population. Professor Willan felt this issue was a red herring, and if it were valid, it would render most clinical research useless. There is really no way of knowing who will benefit from a treatment and who will not. Unless there are reasons to believe otherwise, it is best to apply the population average effect to each person.”

Although Willan labeled Platt’s point as cold-blooded and fishy, he ultimately concurred that the population average effect should be applied to each person in the absence of evidence of risk being sequestered in a subgroup.

A closer look at Willan’s testimony at trial is instructive. Willan acknowledged, on direct examination, that the plaintiffs were at increased risk, even if their mother had received a full course of ACS.  All he would commit to, on behalf of the plaintiffs, was that their risk would have been less had the ACS been given earlier:

“All we can say is that there’s a high probability that that risk would be reduced and that this is probably the best estimate of the excess risk for not being treated and I would say that puts that in the 70 percent range of excess risk and I would say the probability that the risk would have been reduced is into the 90 percentage points.”

Notes of Testimony of Andrew Willan at 62 (April 6, 2010).  The 90 percentage points reference here was Willan’s posterior probability that the claimed effect was real.

On cross-examination, the defense pressed the point:

Q. What you did not do in this, in this report, is provide any quantification for the reduction in the risk, true?

A. That’s correct.

Notes of Testimony of Andrew Willan at 35 (April 9, 2010)

Q. And you stated that there is no evidence that the benefits of steroids is restricted to any particular subgroup of patients?

A. I wasn’t given any. I haven’t seen any evidence of that.

Id. at 43.

Q. And what you’re suggesting with that statement, is that the statistics should be generally, should be considered by the court to be generally applicable, true?

A. That’s correct.

Id. at 44.

Q. But given your report, you can’t offer assistance on the clinical application to the statistics, true?

A. That’s true.

Id. at 46.

With these concessions in hand, defense counsel elicited the ultimate concession relevant to the “but for” standard of causation:

Q. And to do that by looking at an increase in risk, the risk ratio from the data must achieve 2 in order for there to be a 50 percent change in the underlying data, true?

A. Yeah, to double the risk, the risk ratio would have to be 2, to double the risk.

Id. at 63.

* * *

Q. So, none of this data achieves the threshold of a 50 percent change in the underlying data, whether you look at it as an increase in risk or …

A. Sure.

Q …. a decrease in risk …

A. Yeah.

Id. at 66.

Leaping Inferences

The legal standard for causation in Canada is the same counterfactual requirement that applies in most jurisdictions in the United States.  Goodman v. Viljoen, 2011 ONSC 821 (CanLII), at ¶14, 47. The trial court well understood that the plaintiffs’ evidence left them short of showing that their CP would not have occurred but for the delay in administering ACS. Remarkably, the court permitted the plaintiffs to use non-existing evidence to bridge the gap.

According to Dr. Max Perlman, plaintiffs’ expert witness on neonatology and pediatrics, CP is not a dichotomous condition, but a spectrum that is manifested on a continuum of signs and symptoms.  The RCTs relied upon had criteria for ascertaining CP and including it as an outcome.  The result of these criteria was that CP was analyzed as a binary outcome.  Dr. Perlman, however, held forth that “common sense and clinical experience” told him that CP is not a condition that is either present or not, but rather presented on a continuum. Id. at [74].

Without any evidence, Perlman testified that when CP is not avoided by ACS, “it is likely that it is less severe for those who do go on to develop it.” Id. [75].  Indeed, Perlman made the absence of evidence a claimed virtue; with all his experience and common sense, he “could not think of a single treatment which affects a basic biological process that has a yes or no effect; they are all on a continuum.” Id. From here, Perlman soared to his pre-specified conclusion that “that it is more likely than not that the twins would have seen a material advantage had they received the optimal course of steroids.” Id. at [76].

Perlman’s testimony is remarkable for inventing a non-existing feature of biological evidence:  everything is a continuum. Justice Walters could not resist this seductive testimony:

“[195] The statistical information is but one piece of the puzzle; one way of assessing the impact of ACS on CP. Notably, the 40% reduction in CP attributable to ACS represents an all or nothing proposal. In other words, 93.5% of the time, CP is reduced in its entirety by 40%. It was the evidence of Dr. Perlman, which I accept, that CP is not a black and white condition, and, like all biological processes, it can be scaled on a continuum of severity. It therefore follows that in those cases where CP is not reduced in its entirety, it is likely to be less severe for those who go on to develop it. Such cases are not reflected in the Cochrane figure.

[196] Since the figure of 40% represents an all or nothing proposal, it does not accurately reflect the total impact of ACS on CP. Based on this evidence, it is a logical  conclusion that if one were able to measure the total effect of ACS on CP, the statistical measure of that effect would be inflated beyond 40%.

[197] Unfortunately, this common sense conclusion has never and can never be tested by science. As Dr. Perlman testified, such a study would be impossible to conduct because it would require pre-identification of those persons who go on to develop CP.  Furthermore, because the short term benefits of ACS are now widely accepted, it would be unethical to withhold steroids to conduct further studies on long term outcomes.”

Doubly unfortunate, because Perlman’s argument was premised on a counterfactual assumption.  Many biological phenomena are dichotomous.  Pregnancy, for instance, does not admit of degrees.  Disease states are frequently dichotomous, and no evidence was presented that CP was not dichotomous. Threshold effects abound in living organisms. Perlman’s argument further falls apart when we consider that the non-experimental arm of the RCTs would also have had additional “less-severe” CP cases, with no evidence that they occurred disproportionately in the control arms of these RCTs. Furthermore, high-quality observational studies might have greater validity than post-hoc RCTs in this area, and there have been, and likely will continue to be, such studies to attempt better understanding of the efficacy of ACS, as well as differing effects among the various corticosteroids, doses, and patterns of administration.

On appeal, the Justice Walters’ verdict for plaintiffs was affirmed, but over a careful, thoughtful dissent. Goodman v. Viljoen, 2012 ONCA 896 (CanLII) (Doherty, J., dissenting). Justice Doherty caught the ultimate futility of Dr. Perlman’s opinion based upon non-existent evidence: even if there were additional sub-CP cases in the treatment arms of the RCTs, and if they occurred disporportionately more often in the treatment than in the placebo arms, we are still left guessing about the quantitative adjustment to make to the 40% decrease, doubtful as it was, which came from the Cochrane review.

Posted in Causation, Scientific Evidence, statistical evidence | Comments Off on Goodman v Viljoen – Subterfuge to Circumvent Relative Risks Less Than 2

Recrudescence of Traumatic Cancer Claims

June 4th, 2014

In 1991, Peter Huber, discussing traumatic cancer claims, wrote:

“After years of floundering in the junk science morass of traumatic cancer, judges slowly abandoned sequence-of-events logic, turned away from the sympathetic speculations of family doctors, and struggled on to the higher and firmer ground of epidemiology and medical science.  Eventually, the change of heart among appellate judges was communicated back down to trial judges and worker’s compensation boards, and traumatic cancer went into almost complete remission.”

Peter W. Huber, Galileo’s Revenge: Junk Science in the Courtroom 55-56 (1991).

With the advent of Daubert and meaningful gatekeeping of expert witness opinion testimony, the traumatic cancer claims did recede. For a while. Plaintiffs’ counsel, and stalwart opponent of epistemic standards for scientific claims in court, Kenneth Chesebro attacked Huber’s précis of the traumatic cancer law and science. Kenneth J. Chesebro, “Galileo’s Retort: Peter Huber’s Junk Scholarship,” 42 Am. Univ. L. Rev. 1637 (1993). Defenses of the dubious science continue to appear, although mostly in non-peer-reviewed publications.[1]

One of the more disturbing implications of the West Virginia Supreme Court’s decision in Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013), was the Court’s reliance upon its own, recent approval of traumatic cancer claims.  The Harris Court cited, with approval, a 2002 traumatic cancer case, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  The Wiseman case involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation.  The West Virginia Supreme Court blinked at the challenge to the physician expert witness who advanced the causal claim in Wiseman; and in Harris, the Court made clear that blinking is what trial courts should do when confronted with methodological challenges to far-fetched causal opinions.

A couple of years ago, the New York Times ran an article about traumatic cancer. C. Claiborne Ray, “Injury and Insult” (Nov. 5, 2012), responding to the question “Is it possible for cancer to develop as a result of an injury?” Here is how Times science reporter responded:

A. “It’s a common myth that injuries can cause cancer,” the American Cancer Society says on its Web site. Until the 1920s, some doctors believed trauma did cause cancer, “despite the failure of injury to cause cancer in experimental animals.” But most medical authorities, including the cancer society and the National Cancer Institute, see no such link. The more likely explanation, the society suggests, is that a visit to the doctor for an injury could lead to finding an existing cancer.

Other possibilities are that scar tissue from an old trauma could look like a cancerous lesion and that an injured breast or limb would be more closely watched for cancer to develop.

Ms. Ray went on to note a published study, in which would-be myth-busters presented observational data purportedly showing a relationship between physical injury and subsequent breast cancer.  The paper cited by Ms. Ray was a report on a small case-control study done by investigators at the Department of Geography, Lancaster University. See Jan Rigby, et al., “Can physical trauma cause breast cancer?” 11 Eur. J. Cancer. Prev. 307 (2002). The study consisted of 67 breast cancer cases and 134 controls, matched on age, family history, age of menarche, parity, age at first birth, and menopausal status.

Not surprisingly, considering its small size, the Rigby study reported no statistically significant differences for several factors known to be associated with breast cancer: social class, education, residence, smoking and alcohol consumption.  Although lacking power to detect differences of known risk factors, this study turned up a large, statistically significant association between physical trauma and breast cancer:

“Women with breast carcinoma were more likely to report physical trauma to the breast in the previous 5 years than were the controls (odds ratio (OR) 3.3, 95% confidence interval (CI) 1.3-10.8, P < 0.0001).”

* * * * *

“More likely to [self-]report” hardly implies causation, but the authors jumped not only to a causal explanation but to a causal conclusion:

* * * * *

“In conclusion, recall bias is an unlikely explanation for these results in view of the nature and severity of physical trauma. Models of epithelial cell generation indicate that a causal link between physical trauma and cancer is plausible. A latent interval between cancer onset and presentation of under 5 years is also plausible. The most likely explanation of the findings is that physical trauma can cause breast cancer.”

Rigby at 307.

The Rigby study is a valuable demonstration of how malleable researchers can be in discovering plausible explanations for their data.  The authors fail to discuss the natural history of breast carcinoma, such as tumor doubling time, which would make their five-year window decidedly implausible.  The Rigby paper also demonstrates how strident researchers can be in claiming that they have produced a study that has eliminated bias in observational research, when they have barely scratched the surface of bias or confounding. Magical thinking is not the exclusive domain of lawyers.

Until reading the Harris and Wiseman cases, I had thought that the legal system had graduated from the “mythology” of traumatic cancer cases.[2]  To be sure, in the past, any number of physicians have supported traumatic cancer claims, in print and in the courtroom.[3] Some authors attempted to put some rational limits on the extent of the traumatic cancer claims.[4] By 1947, at least, the trauma theory was criticized in leading texts.[5]  In 1974, the Mayo Clinic published a review that emphasized the lack of experimental evidence to support the claim that uncomplicated trauma causes cancer.[6] The law review literature attempted to make sense of the compensation-frenzied courts, without much success.[7]

Many cases from most jurisdictions have approved traumatic cancer claims.  Some are set out below. Some courts heroically resisted the pro-compensation Zeitgeist, usually on case-specific evidentiary issues.[8]

In New York, judges seem to be well aware that post hoc ergo propter hoc is a fallacy.  Cassano v. Hagstrom, 5 N.Y.2d 643, 159 N.E.2d 348, 187 N.Y.S.2d 1 (1959) (affirming dismissal of case based because of plaintiffs’ attempt to use fallacious reasoning in the form of  “post hoc ergo propter hoc”); Holzberg v. Flower & Fifth Ave. Hosps., 39 AD 2d 526 (N.Y. 1st Dep’t 1972). Still, the New York courts struggled with traumatic cancer claims, and appeared to oscillate wildly without clear guidance on whether or to what extent the courts could reject specious claiming supported by speculative or unreliable expert witness opinion testimony.[9] Given the current hostility to gatekeeping of expert witness opinion, a recrudescence of traumatic cancer claims is likely.

Opinions Approving Causation in Traumatic Cancer Cases

California

Santa Ana Sugar Co. v. Industrial Accid. Comm’n, 170 P. 630, 630 (Cal. Dist. Ct. App. 1917)

Colorado

Canon Reliance Coal Co. v. Indus. Comm’n, 72 Colo. 477, 211 P. 868, 869-70 (1922) (cancer caused by being hit on cheek with a lump of coal)

Georgia

National Dairy Prods. Corp. v. Durham, 154 S.E.2d 752, 753-54 (Ga. Ct. App. 1967)

Kentucky

Louisville Ry v. Steubing’s Adm’r, 136 S.W. 634, 634 (Ky. Ct. App. 1911)

Louisiana

Reed v. Mullin Wood Co., 274 So. 2d 845, 846-47 (La. Ct. App. 1972), cert. denied, 275 So. 2d 729, 791 (La. 1973);

Thompson v. New Orleans Ry. & Light Co., 83 So. 19, 20 (La. 1919)

Michigan

Wilson v. Doehler-Jarvis Div. of Nat’l Lead Co., 353 Mich. 363, 91 N.W.2d 538, 539-40 (1958) (blow to lip caused cancer)

Mooney v. Copper Range RR, 27 N.W.2d 603, 604 (Mich. 1947)

Minnesota

Daly v. Bergstedt, 267 Minn. 244, 126 N.W.2d 242, 247–48 (1964) (affirming jury finding of causation between traumatic leg fracture and breast cancer; six physicians testified against causation; one stated cancer “could” result from trauma; imagining that scientific and legal standards of causation differ)

Pittman v. Pillsbury Flour Mills, Inc., 48 N.W.2d 735, 736 (Minn. 1951)

Hertz v. Watab Pulp & Paper Co., 237 N.W. 610, 611 (Minn. 1931)

Austin v. Red Wing Sewer Pipe Co., 163 Minn. 397, 204 N.W. 323, 323-24 (Minn. 1925) (cancer developed one year after worker was hit in the face with coal)

Gaetz v. City of Melrose, 193 N.W. 691, 692 (Minn. 1923)

Missouri

Vitale v. Duerbeck, 338 Mo. 536, 92 S.W.2d 691, 695 (1936)

New Hampshire

Jewell v. Grand Trunk Ry, 55 N.H. 84 (1874) (reversing traumatic cancer verdict on other grounds)

New Mexico

White v. Valley Land Co., P.2d 707, 708-10 (N.M. 1957)

Ohio

Hanna v. Aetna Ins., 24 Ohio Misc. 27, 52 Ohio Op. 2d 316, 259 N.E.2d 177, 177-79 (Ohio Mun. Ct. Dayton 1970)(breast lump found three months after car accident)

Glenn v. National Supply, 129 N.E.2d 189, 190-91 (Ohio Ct. App. 1954)

Oregon

Devine v. Southern Pacific Co., 207 Or. 261, 295 P.2d 201 (1956) (holding that physician’s testimony as to “probable” causation between shoulder fracture and lung cancer was sufficient; jury verdict for plaintiff reversed on other grounds).

Pennsylvania

Baker v. DeRosa, 413 Pa. 164, 196 A.2d 387, 389–90 (Pa. 1964)

Menarde v. Philadelphia Transp. Co., 376 Pa. 497, 103 A.2d 681, 684(1954) (the fact that breast cancer was found in the same place as the injury-caused bruise helped establish causation);

Southern S.S. Co. v. Norton, 41 F. Supp. 103 (E.D. Pa. 1940) (trauma to skull and lower back held to have caused lung cancer)

Tennessee

Koehring-Southern & Am. Mut. Ins. Co. v. Burnette, 464 S.W.2d 820, 821 (Tenn. 1970)

Boyd v. Young, 193 Tenn. 272, 246 S.W.2d 10, 10 (Tenn. 1951)

Rhode Island

Valente v. Bourne Mills, 77 R.I. 274, 278-79, 75 A.2d 191, 193-94 (1950) (adopting house of cards position in which any rational inference suffices even if not supported by expert medical opinion)

Emma v. A.D. Julliard & Co., 75 R.I. 94, 63 A.2d 786, 787-89 (R.I. 1949)(plaintiff had malignant tumor removed from her breast seven weeks after being hit with a can of juice)

Texas

Traders & General Insur. Co. v. Turner, 149 S.W.2d 593, 597-98 (Tex. Civ. App. 1941) (testicular cancer)

Virginia

Ellis v. Commonwealth Dep’t of Highways, 28 S.E.2d 730, 731-32, 735 (Va. 1944) (accepting post-hoc reasoning “[f]acts prevail over possibilities or probabilities”)

Winchester Milling Corp. v. Sencindiver, 138 S.E. 479, 480-81 (Va. 1927)

[1] See, e.g., Melvin A. Shiffman, Can Trauma Cause or Accelerate the Growth of Cancer? Forensic Examiner 6 (Fall 2004).

[2] See Manasco v. Insurance Co. of State of Pennsylvania, 89 S.W.3d 239 (Tex. App. Texarkana 2002) (affirming denial of benefits to worker who claimed head injury caused brain tumor; citing to epidemiological studies that failed to show an association between trauma and brain tumors).

[3] See, e.g., George R. Parsons, “Sufficiency of Proof in Traumatic Cancer Cases,” 2 Tort & Med. Year Book 335 (1962); Stoll & Crissey, “Epithelioma from Single Trauma,” 62 N.Y. St. J. Med. 496 (Feb. 15, 1962); Wilhelm C. Hueper, Trauma and Cancer (1959); Arden R. Hedge, “Can a Single Injury Cause Cancer?” 90 Calif. Med. 55 (1959); R. Crane, “The Relationship of a Single Act of Trauma to Subsequent Malignancy,” in Alan R. Moritz & David S. Helberg, eds., Trauma and Disease 147 (1959); Shields Warren, M.D., “Minimal criteria required to prove causation of traumatic or occupational neoplasms,” Ann. Surgery 585 (1943); Bishop, “Cancer, Trauma, and Compensation,” 32 So. Med. J. 302 (1939); Knox, “Trauma and Malignant Tumors, 26 Am. J. Surg. 66, 69-70 (1934); William B. Coley & Norman L. Higinbotham, “Injury as a causative factor in the development of malignant tumors,” 98 Ann. Surg. 991 (1933); Wainwright, “Single Trauma, Carcinoma and Workman’s Compensation,” 5 Am. J. Surg. 433 (1928); Alson R. Kilgore & Curtis E. Smith, “Industrial liability for cancer,” 25 Calif. & Western Med. 70 (1926); Charles Phelps, “The relation of trauma to cancer formation,” 51 Ann. Surgery 609 (1910).

[4] James Ewing, “Modern Attitudes Toward Traumatic Cancer,” 19 Arch. Path. 690, 692 (1935); James Ewing, “The Relation of Trauma to Malignant Tumors,” Am. J. Surg. 30, 31-34 (Feb. 1926).

[5] See, e.g., James A. Tobey, Public Health Law 321 (3ed 1947) (“Although there is little, if any, scientific evidence to prove conclusively that malignant growths such as carcinoma, sarcoma, and other forms of cancer are ever caused by single blows, wounds, injuries, or other forms of trauma, the courts have awarded damages in a number of instances to persons who have developed cancers following single injuries.”) (internal citations omitted).

[6] George R. Monkman, Gregg Orwoll & John C. Ivins, “Trauma and Oncogenesis,” 49 Mayo Clinic Proc. 157 (1974).

[7] The trauma theory of carcinogenesis was discussed and questioned in several law review articles.  See, e.g., Orrin E. Tilevitz, “Judicial Attitudes Towards Legal and Scientific Proof of Cancer Causation,” 3 Colum. J. Envt’l L. 344 (1977); Donald J. Ladanyi, “Impact Trauma As ‘Legal Cause’ of Cancer,” 20 Cleveland State L. Rev. 409 (1971); Theodore Dyke, “Traumatic Cancer?” 15 Clev.-Marshall L. Rev. 472 (1966); Jerry G. Elliott, “Traumatic cancer and ‘an old misunderstanding between doctors and lawyers’,” 13 U. Kan. L. Rev. 79 (1964); Comment, Sufficiency of Proof in Traumatic Cancer: A Medico-Legal Quandary, 16 Ark. L. Rev. 243 (1962); Comment, “Sufficiency of Proof in Traumatic Cancer Cases,” 46 Cornell L.Q. 581 (1961); Adelson, Injury and Cancer, 5 Western Res. L. Rev. 150 (1954).

[8] State Compensation Ins. Fund v. Kindig, 445 P.2d 72 (Colo. 1968) (head injury held not to have caused leukemia 68 days later); Slack v. C.L. Percival Co., 198 Iowa 54, 199 N.W. 323, 326 (1924) (anticipating Daubert by rejecting expert witness opinion that was “wholly in the realm of conjecture, speculation, and surmise”); Ortner v. Zenith Carburetor Co., 207 Mich. 610, 175 N .W. 122 (1919) (holding that 30 months was too long for a claim that accident that crushed worker’s fingers caused blood poisoning and penile cancer); Stordahl v. Rush Implement Co., 417 P.2d 95 (Mont. 1966) (rejecting traumatic causation of malignant tumor); Tonkovich v. Dep’t of Lab. & Indus., 31 Wash. 2d 220, 195 P.2d 638 (1948) (injury to foot held not to have caused abdominal cancer)

[9] See Dennison v. Wing, 279 App. Div. 494, 110 N.Y.S.2d 811, 813 (1952) (rejecting cancer claim when latency was two months on grounds that cancer took longer to develop); Sikora v. Apex Beverage Corp., 282 App. Div. 193, 196-97 (1953) (reversing judgment for plaintiff based upon jury’s finding that slip and fall accelerated breast cancer based upon lack of evidentiary support), aff’d, 306 N.Y. 917, 119 N.E.2d 601 (1954); Frankenheim v. B. Altman & Co., 13 Misc. 2d 1079, 1080-81, 177 N.Y.S.2d 2 (Bronx Cty. S.Ct. 1958) (granting motion to set aside verdict for plaintiff based upon traumatic cancer claim on grounds of insufficient evidence), app. dism’d, 8 App. Div. 2d 809 (First Dep’t 1959). But see McGrath v. Irving, 24 App. Div. 2d 236, 265 N.Y.S.2d 376 (1965) (affirming jury verdict based upon claim that plaintiff’s swallowing glass in car accident caused or accelerated development of laryngeal cancer); Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 221 N.Y.S.2d 224, 224 (1st Dep’t 1961) (affirming award for traumatic cancer based upon the “usual” conflicting expert witness testimony) Mattfield v. Ward Baking Co., 14 App. Div. 2d 942, 942 (1961) (affirming workman’s compensation award for “aggravation” of cancer, which resulted after “the usual conflict of medical opinion”); Pezzolanti v. Green Bus Lines, 114 App. Div. 2d 553, 553-54, 494 N.Y.S.2d 168, 169 (1985) (affirming workman’s compensation award for disability to wrist, which resulted from “trauma” of hitting pothole, which in turn injured asymptomatic wrist destabilized by pre-existing cancer).

Posted in Causation, Expert Witnesses, Rule 702, Scientific Evidence | Comments Off on Recrudescence of Traumatic Cancer Claims

Intellectual Due Process in West Virginia and Beyond

June 1st, 2014

Harris v. CSX Transportation

I have borrowed and modified the phrase “Intellectual Due Process” from earlier writers because of its obvious implications for the presentation, interpretation, synthesis, and evaluation of scientific evidence in court. See Scott Brewer, “Scientific Expert Testimony and Intellectual Due Process,” 107 Yale L. J. 1535 (1998). The major reason courts write opinions is to explain and justify their decisions to litigants, present and future, and to a wider audience of lawyers, scholars, and the general public. Judicial opinions involving scientific evidence, whether in legislation, regulation, or litigation must satisfy the societal need to explain and justify the acceptance and rejection of scientific claims. Despite a great deal of hand waving that law and science are somehow different, in the end, when courts describe their acceptance or rejection of scientific claims, they are addressing the same epistemic warrant that scientists themselves employ. Even a cursory review of the judicial output reveals an unsatisfactory state of affairs in which many courts mangle scientific and statistical evidence and inference.  There is much that is needed to correct the problem.

One proposal would be to require that the parties file proposed findings of facts in connection with Rule 702 gatekeeping challenges.  Courts should file detailed findings of facts that underlie their decisions to admit or to exclude expert witness opinion testimony.  Another proposal would require courts to cite properly the scientific studies that they discuss in reaching a legal conclusion about sufficiency or admissibility.  These are small steps, but ones that would help reduce the gross inaccuracies and the glib generalizations, while increasing the opportunity for public scrutiny and criticism.

We do not think anything is amiss with special courts for tax, patent, family law, national security, equity, or commercial matters.  There is an even greater need for scientific skill, knowledge, and aptitude in a specialized science court.  The time has come for special courts to hear cases involving scientific claims in health effects and other litigation.

*   *   *   *   *   *   *

A decision of the West Virginia Supreme Court, late last year, illustrates the need for substantial reform of how claiming based upon “scientific evidence” is permitted and evaluated in court.  Mrs. Harris sued the railroad for the wrongful death of her husband, who died of multiple myeloma. Mr. Harris had been exposed, in his railroad workplace, to diesel exhaust, which Mrs. Harris claimed caused his cancer. See Harris v. CSX Transportation, Inc., 232 W.Va. 617, 753 S.E.2d 275 (2013). The trial court excluded Mrs. Harris’s expert witnesses. Harris v. CSX Transportation, Inc., No. 12-1135, 2012 WL 8899119 (Cir. Ct. Marshall Cty., W.Va. Aug. 21, 2012).

1. The West Virginia Supreme Court reversed the trial court’s exclusion of witnesses on the basis of an asymmetrical standard of review, which would allow de novo review of trial court decisions to exclude expert witness opinions, but which would privilege trial court decisions to admit opinions by limiting appellate review to abuse of discretion. This asymmetry was, of course, the same dodge that the Third and Eleventh Circuits had used to keep the “gates open,” regardless of validity or reliability concerns, and the same dodge that the Supreme Court shut down in General Electric v. Joiner. A single judge dissented in Harris, Justice Loughry, who took the majority to task for twisting facts and law to get to a desired result.

2. The Harris Court cited a federal court case for dicta that “Rule 702 reflects an attempt to liberalize the rules governing the admissibility of expert testimony.” See Harris, 753 S.E.2d at 279 (citing and quoting from Weisgram v. Marley Co., 169 F.3d 514, 523 (8th Cir.1999). Remarkably, the Harris Court omitted reference to the United States Supreme Court’s unanimous affirmance of Weisgram, which saw Justice Ginsburg write that “[s]ince Daubert, moreover, parties relying on expert evidence have had notice of the exacting standards of reliability such evidence must meet.” Weisgram v. Marley Co., 528 U.S. 440, 442 (2000).  The Harris Court’s lack of scholarship is telling.

3. Meta-analysis appeared to play a role in the case, but the judicial decisions in Harris fail to describe the proffered evidence. The majority in Harris noted that one of plaintiff’s expert witnesses, Dr. Infante, relied upon a meta-analysis referred to as “Sonoda 2001.” Harris, 753 S.E.2d at 309. Neither the Court nor the dissent cited the published meta-analysis in a way that would help an interested reader in finding the paper.  One could imagine the hue and cry if courts cited judicial cases or statutes by short-hand names without providing enough information to access the relied upon source.  In this case, a PubMed search reveals the source so perhaps the error is harmless. Tomoko Sonoda, Yoshie Nagata, Mitsuru Mori, Tadao Ishida & Kohzoh Imai, “Meta-analysis of multiple myeloma and benzene exposure,” 11. J. Epidemiol. 249 (2001).  Still, the time has come for courts to describe and report the scientific evidence with the same care and detail that they would use in a car collision case.

4. A quick read shows that the Sonoda meta-analysis supports the dissent’s assessment:

“‘Dr. Infante testified on direct examination that Sonoda 2001 considered 8 case-control studies specific to engine exhaust and stated it concluded that diesel and non-diesel engine exhaust causes multiple myeloma.’ Yet, as the trial court found, ‘[o]n cross examination Dr. Infante acknowledged that none of the 8 papers included in the Sonoda meta-analysis mention diesel exhaust’.”

Harris, 753 S.E.2d at 309.  The dissent would have been considerably more powerful had it actually adverted to the language of Sonoda 2001:

“These results suggested that benzene exposure itself was not likely to be a risk factor of MM [multiple myeloma]. It is thought that several harmful chemical agents in engine exhaust, other than benzene, could be etiologically related to the risk of MM. Further case-control studies on MM are needed to obtain more information about detailed occupational exposure to toxic substances.”

Sonoda at 249 (2001) (emphasis added).  Contrary to Infante’s asseveration, Sonoda and colleagues never concluded that diesel exhaust causes multiple myeloma.  The state of scholarship and “intellectual due process” makes it impossible to tell whether or not Dr. Infante was telling the truth or the Harris Court badly misunderstood the record. Either way, something must give.

The dissent went on to note that Dr. Infante conducted his own meta-analysis, which included studies that did not mention diesel exhaust. Harris, 753 S.E.2d at 309.  The railroad complained that some of the studies were small and had limited power, but that is exactly why a meta-analysis would be appropriate.  The more disturbing complaints were that the meta-analysis left out important studies, and that it included irrelevant studies of benzene exposure and myeloma, which raised insuperable problems of external validity.

5. A half empty glass that is always full.  According to the Harris Court, the West Virginia shadow of Rule 702 is a rule of “admissibility rather than exclusion.” Harris, 753 S.E.2d at 279 (citing and quoting from In re Flood Litig. Coal River Watershed, 222 W.Va. 574, 581, 668 S.E.2d 203, 210 (2008), which in turn quoted a federal case, Arcoren v. United States, 929 F.2d 1235, 1239 (8th Cir. 1991), decided before the Supreme Court decided Daubert.)  This is just silly hand waving and blatant partisanship.  A rule that sets out criteria or bases for admissibility also demarcates the inadmissible.

6. Cherry Picking. Dr. Infante was permitted by the Harris Court to aggregate data from studies that did not observe diesel exposure, while he failed to include, or he deliberately excluded data from, a large, powerful, exonerative study conducted by scientists from the National Cancer Institute, the International Agency for Research on Cancer (IARC), and the Karolinska Institute. See Paolo Boffetta, Mustafa Dosemeci, Gloria Gridley, Heather Bath, Tahere Moradi and Debra Silverman, “Occupational exposure to diesel engine emissions and risk of cancer in Swedish men and women,” 12 Cancer Causes Control 365 (2001). Dr. Infante inexplicably excluded this study, which found a risk ratio for men exposed to diesel exhaust that was below one, 0.98, with a very narrow 95% confidence interval, 0.92-1.05. Boffetta at 368, Table 2.

7. The West Virginia articulated an incohorent definition of “reliable,” designed to give itself the ability to reject gatekeeping completely. Citing its earlier decision in Flood, the Court offered its own ipse dixit:

“The assessment of whether scientifically-based expert testimony is “reliable,” as that term is used in [Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and Wilt v. Buracker, 191 W.Va. 39, 443 S.E.2d 196 (1993)], does not mean an assessment of whether the testimony is persuasive, convincing, or well-founded. Rather, assessing ‘reliability’ is a shorthand term of art for assessing whether the testimony is to a reasonable degree based on the use of knowledge and procedures that have been arrived at using the methods of science — rather than being based on irrational and intuitive feelings, guesses, or speculation. If the former is the case, then the jury may (or may not, in its sole discretion) ‘rely upon’ the testimony. In re Flood Litig., 222 W.Va. at 582 n. 5, 668 S.E.2d at 211 n. 5.”

Harris, 753 S.E.2d at 279-80. Surely, this is circular or vacuous or both. Opinions not “well-founded” will be ones that are based upon guesses or speculation.  Opinions arrived at by the “methods of science” will be ones that have an epistemic warrant that will survive a claim that they are not “well-founded.”

8. The Harris Court evidenced its hostility to scientific evidence by dredging up one of its own decisions involving a multiple myeloma causation claim, State ex rel. Wiseman v. Henning, 212 W.Va. 128, 569 S.E.2d 204 (2002).  Wiseman involved a specious claim that a traumatic rib injury caused multiple myeloma, a claim at odds with scientific method and observation:

“Some research has suggested that people in some jobs may have an increased risk of developing multiple myeloma because they are exposed to certain chemicals. But the International Agency for Research on Cancer (IARC) states that the evidence is limited overall. It has been suggested that people may have an increased risk if they work in the petrol or oil industry, farming, wood working, the leather industry, painting and decorating, hairdressing, rubber manufacturing or fire fighting. But there is no evidence to prove that any of these occupations carry an increased risk of myeloma.”

Cancer Research UK, “Myeloma risks and causes” (last visited May 28, 2014). Even the most non-progressive jurisdictions have generally eradicated specious claiming for trauma-induced cancers, but West Virginia has carved out a place second to none in its race to the bottom.

9. WOE.  Not surprisingly, the Harris Court relied heavily on the First Circuit’s “weight of the evidence” end-run around the notion of epistemic warrant for scientific claims, citing Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11 (1st Cir.2011), cert. denied sub nom., U.S. Steel Corp. v. Milward, ___ U.S. ___, 2012 WL 33303 (2012). The Harris Court went on to conflate and confuse WOE with Bradford Hill, and cited a recent New York case that confidently saw through WOE hand waving, while ignoring its devasting critique of expert witnesses’ attempts to pass off WOE for scientific, epistemic warrant.  Reeps ex rel. Reeps v. BMW of N. Am., LLC, No. 100725/08,

2013 WL 2362566, at *3, 2012 N.Y. Misc. LEXIS 5788; 2012 NY Slip Op 33030U  (N.Y. Sup. Ct. May 10, 2013).

10.  Link.  Dr. Infante links a lot, even when his sources do not:

“Dr. Infante testified that the International Agency for Research on Cancer issued Technical Publication Number 42 in 2009, and that the publication stated that diesel exhaust exposures have been linked to multiple myeloma and leukemia.”

Harris, 753 S.E.2d at 294. The Harris Court neglected to give the title of the publication, which tells a different story.  Identification of research needs to resolve the carcinogenicity of high-priority IARC carcinogens. The dissent was willing to go behind the conclusory and false characterization that Dr. Infante and plaintiff gave to this publication.  Harris, 753 S.E.2d at 309. The trial court’s finding (and the dissent’s assertion) that the IARC Technical Publication 42 intended to express a research agenda, not to make a causation statement, seems unassailable.  Furthermore, it appears to be precisely the sort of specious claim that a court should keep from a jury.  The cited IARC source actually notes that the then current IARC classification of diesel exhaust was of inadequate evidence for human carcinogenicity, with a focus on lung cancer, and barely a mention of multiple myeloma.

11.  The Benzene Connection. Plaintiffs’ expert witnesses, including Dr. Infante, argued that benzene was a component of diesel exhaust, and benzene caused multiple myeloma.  This move ignored not only the lack of evidence to implicate benzene in the causation of multiple myeloma, but it also ignored the large quantitative differences between the benzene occupational exposure studies and the very small amounts of benzene in diesel exhaust.  The Harris Court held that the trial court acted improperly by inquiring into and finding the following facts, which were “exclusively” for the jury:

  • “There is substantially more benzene in cigarette smoke than diesel exhaust.
  • Benzene is present only in trivial doses in diesel exhaust.
  • The hypothesis that diesel exhaust causes multiple myeloma is confounded by the fact that cigarette smoking does not.”

The Harris majority further chastised the trial court for adverting to the ten or so studies that failed to find a statistically significant association between benzene exposure and multiple myeloma.  Harris, 753 S.E.2d at 305-06.  This inquiry directly calls into question, however, Dr. Infante’s methodology.

If these facts, found by the trial court, were reasonably established, then Dr. Infante’s argument was less than bogus, and a major underpinning for inclusion of benzene studies in his meta-analysis was refuted.  These are precisely the sort of foundational facts that must be part of an inquiry into the methodological grounds of an expert witness’s opinion.

12.  The Harris Court confused “proving causation” with “showing a methodology that provides an epistemic warrant for concluding.” Harris, 753 S.E.2d at 300. The Harris Court asserted that the trial court exceeded its gatekeeping function by inquiring into whether Mrs. Harris’s expert witnesses “proved” causation. Harris, 753 S.E.2d at 300. Speaking of “proof of” or “proving” causation is an affectation of lawyers, who refer to their evidence as their “proofs.”  Epidemiologic articles and meta-analyses do not end with quod erat demonstrandum. Beyond the curious diction, there is a further issue in the majority’s suggestion that the trial court set the bar too high in declaring that the plaintiff failed to “prove” causation.  Even if we were to accept the continuous nature of strength of evidence for a causal conclusion, Dr. Infante and the other plaintiff’s witnesses, would be fairly low on the curve, and their lowly position must of necessity speak to the merits of the defense motion to exclude under Rule 702.

13. Purely Matters for Jury. The Harris Court criticized the trial court for conducting a “mini-trial,” which set out to “resolve issues that were purely matters for jury consideration.” Harris, 753 S.E.2d at 305. In holding that the matters addressed in the pre-trial hearing were “exclusively grist for the jury and which had no relevancy to the limited role the trial court had under the facts of this case,” the Harris Court displayed a profound disregard for what facts would be relevant for a challenge to the plaintiff’s expert witnesses’ methodology. Many of the facts found by the trial court were directly relevant to “general acceptance,” validity (internal and external) of studies relied upon, and reliability of reasoning and inferences drawn. Aside from the lack of general acceptance and peer review of the plaintiff’s claimed causal relationship, the proffered testimony was filled with gaps and lacunae, which are very much at issue in methodological challenges to an opinion of causality.

*   *   *   *   *   *   *

The Harris case has taken its place next to Milward in the litigation industry’s arsenal of arguments for abandoning meaningful judicial supervision and gatekeeping of expert witness opinion testimony.  See Andrew S. Lipton, “Proving Toxic Harm: Getting Past Slice and Dice Tactics,” 45 McGeorge L. Rev. 707, 731 (2014) (plaintiffs’ bar cheerleading for the Harris decision as “a lengthy and thoughtful analysis”, and for the Milward case as roadmap to evade meaningful judicial oversight).  Not all was perfect with the trial court’s opinion.  The defense seemed to have misled the court by asserting that “a difference between a case group and control group is not statistically significant then there is no difference at all.”  See Respondent’s Brief at 5, Harris v. CSX Transportation, Inc., 2013 WL 4747999 (filed (Feb. 4, 2013) (citing  App. 169, 228-230 (Shields) as having explained that the p-values greater than 0.05 do not support a causal association).

This is hardly true, and indeed, the lack of statistical significance does not lead to a claim that the null hypothesis of no association between exposure and outcome is correct.  The defense, however, did not have a burden of showing the null to be correct; only that there was no reliable method deployed to reject the null in favor an alternative that the risk ratio for myeloma was raised among workers exposed to diesel exhaust.

Still, the trial court did seem to understand the importance of replication, in studies free of bias and confounding. Courts generally will have to do better at delineating what are “positive” and “negative” studies, with citations to the data and the papers, so that judicial opinions provide a satisfactory statement of reasons for judicial decisions.

Posted in Causation, Expert Witnesses, Meta-analysis, Rule 702, Scientific Evidence | Comments Off on Intellectual Due Process in West Virginia and Beyond

A Black Swan Case – Bayesian Analysis on Medical Causation

March 15th, 2014

Last month, I posted about an article that Professor Greenland wrote several years ago about his experience as a plaintiffs’ expert witness in a fenfluramine case. “The Infrequency of Bayesian Analyses in Non-Forensic Court Decisions (Feb. 16, 2014).” Greenland chided a defense expert for having declared that Bayesian analyses are rarely or never used in analyzing clinical trials or in assessments of pharmaco-epidemiologic data.  Greenland’s accusation of ludicrousness appeared mostly to blow back on him, but his stridency for Bayesian analyses did raise the question, whether such analyses have ever moved beyond random-match probability analyses in forensic evidence (DNA, fingerprint, paternity, etc.) or in screening and profiling cases.  I searched Google Scholar and Westlaw for counter-examples and found none, but I did solicit references to “Black Swan” cases. Shortly after I posted about the infrequency of Bayesian analyses, I came across a website that was dedicated to collecting legal citations of cases in which Bayesian analyses were important, but this website appeared to confirm my initial research.

Some months ago, Professor Brian Baigrie, of the Jackman Humanities Institute, at the University of Toronto, invited me to attend a meeting of an Institute working group on The Reliability of Evidence in Science and the Law.  The Institute fosters interdisciplinary scholarship, and this particular working group has a mission statement close to my interests:

“The object of this series of workshops is to formulate a clear set of markers governing the reliability of evidence in the life sciences. The notion of evidence is a staple in epistemology and the philosophy of science; the notion of this group will be the way the notion of ‘evidence’ is understood in scientific contexts, especially in the life sciences, and in judicial form as something that ensures the objectivity of scientific results and the institutions that produce these results.”

The Reliability of Evidence in Science and the Law. The faculty on the working group represent disciplines of medicine (Andrew Baines), philosophy (James R. Brown, Brian Baigrie), and law (Helena Likwornik, Hamish Stewart), with graduate students in the environmental science (Amy Lemay), history & philosophy of science and technology (Karolyn Koestler, Gwyndaf Garbutt ), and computer science (Maya Kovats).

Coincidentally, in preparation for the meeting, Professor Baigrie sent me links to a Canadian case, Goodman v. Viljoen, which turned out to be a black swan case! The trial court’s decision, in this medical malpractice case focused mostly on a disputed claim of medical causation, in which the plaintiffs’ expert witnesses sponsored a Bayesian analysis of the available epidemiologic evidence; the defense experts maintained that causation was not shown, and they countered with the unreliability of the proffered Bayesian analysis. The trial court resolved the causation dispute in favor of the plaintiffs, and their witnesses’ Bayesian approach. Goodman v. Viljoen, 2011 ONSC 821 (CanLII), aff’d, 2012 ONCA 896 (CanLII).  The Court of Appeals’ affirmance was issued over a lengthy, thoughtful dissent. The Canadian Supreme Court denied leave to appeal.

Goodman was a medical practice case. Mrs. Goodman alleged that her obstetrician deviated from the standard of care by failing to prescribe corticosteroids sufficiently early in advance of delivery to avoid or diminish the risk of cerebral palsy in her twins.  Damages were stipulated, and the breach of duty turned on a claim that Mrs. Goodman, in distress, called her obstetrician.  Given the decade that passed between the event and the lawsuit, the obstetrician was unable to document a response.  Duty and breach were disputed, but were not the focus of the trial.

The medical causation claim, in Goodman, turned upon a claim that the phone call to the obstetrician should have led to an earlier admission to the hospital, and the administration of antenatal corticosteroids.  According to the plaintiffs, the corticosteroids would have, more probably than not, prevented the twins from developing cerebral palsy, or would have diminished the severity of their condition.  The plaintiffs’ expert witnesses relied upon studies that suggested a 40% reduction and risk, and a probabilistic argument that they could infer from this risk ratio that the plaintiffs’ condition would have been avoided.  The case thus raises the issue whether evidence of risk can substitute for evidence of causation.  The Canadian court held that risk sufficed, and it went further, contrary to the majority of courts in the United States, to hold that a 40% reduction in risk sufficed to satisfy the more-likely-than-not standard.  See, e.g., Samaan v. St. Joseph Hosp., 670 F.3d 21 (1st Cir. 2012) (excluding expert witness testimony based upon risk ratios too small to support opinion that failure to administer intravenous tissue plasminogen activator (t-PA) to a patient caused serious stroke sequelae); see also “Federal Rule of Evidence 702 Requires Perscrutations — Samaan v. St. Joseph Hospital (2012)” (Feb. 4, 2012).

The Goodman courts, including the dissenting justice on the Ontario Court of Appeals, wrestled with a range of issues that warrant further consideration.  Here are some that come to mind from my preliminary read of the opinions:

1. Does evidence of risk suffice to show causation in a particular case?

2. If evidence of risk can show causation in a particular case, are there requirements that the magnitude of risk be quantified and of a sufficient magnitude to support the inference of causation in a particular case?

3. The judges and lawyers spoke of scientific “proof.”  When, if ever, is it appropriate to speak of scientific proof of a medical causal association?

4. Did the judges incorrectly dichotomize legal and scientific standards of causation?

5. Did the judges, by rejecting the need for “conclusive proof,” fail to articulate a meaningful standard for scientific evidence in any context, including judicial contexts?

6. What exactly does the “the balance of probabilities” mean, especially in the face of non-quantitative evidence?

7. What is the relationship between “but for” and “substantial factor” standards of causation?

8. Can judges ever manage to define “statistical significance” correctly?

9. What is the role of “common sense” in drawing inferences by judges and expert witnesses in biological causal reasoning?  Is it really a matter of common sense that if a drug did not fully avert the onset of a disease, it would surely have led to a less severe case of the disease?

10. What is the difference between “effect size” and the measure of random or sampling error?

11. Is scientific certainty really a matter of being 95% certain, or is this just another manifestation of the transposition fallacy?

12. Are Bayesian analyses acceptable in judicial settings, and if so, what information about prior probabilities must be documented before posterior probabilities can be given by expert witnesses and accepted by courts?

13. Are secular or ecological trends sufficiently reliable data for expert witnesses to rely upon in court proceedings?

14. Is the ability to identify biological plausibility sufficient to excuse the lack of statistical significance and other factors that are typically needed to support the causality of a putative association?

15. What are the indicia of reliability of meta-analyses used in judicial proceedings?

16. Should courts give full citations to scientific articles that are heavily relied upon as part of the requirement that they publicly explain and justify their decisions?

These are some of the questions that come to mind from my first read of the Goodman case.  The trial judge attempted to explain her decision in a fairly lengthy opinion. Unfortunately, the two judges, of the Ontario Court of Appeals, who voted to affirm, did not write at length. Justice Doherty wrote a thoughtful dissent, but the Supreme Court denied leave to appeal.  Many of the issues are not fully understandable from the opinions, but I hope to be able to read the underlying testimony before commenting.

Thanks to Professor Baigrie for the reference to this case.

Posted in Causation, Scientific Evidence, statistical evidence | Comments Off on A Black Swan Case – Bayesian Analysis on Medical Causation

The Rise and Rise of Junk Science

March 8th, 2014

Many authors attribute the term “junk science” to Peter Huber and his use of it in the term in his book, Galileo’s Revenge: Junk Science In The Courtroom (1991). As important as Huber’s book was to raising judicial consciousness to what was going on in courtrooms around the United States, the phrase “junk science” clearly predates Huber’s book.

Lawrence Hubert and Howard Wainer note that the phrase appears to have been in use by the early 1980’s, and sugggest that the first use of the pejorative phrase occurred in a Reagan administration white paper.  Lawrence Hubert and Howard Wainer, A Statistical Guide for the Ethically Perplexed 460 (Boca Raton 2013). The document cited by Hubert and Wainer notes:

“Another way in which causation often is undermined – also an increasingly serious problem in toxic tort cases – is the reliance by  judges and juries on non-credible scientific or medical testimony, studies or opinions. It has become all too common for “experts” or “studies” on the fringes of, or even well beyond the outer parameters of mainstream scientific or medical views, to be presented to juries as valid evidence from which conclusions may be drawn. The use of such invalid scientific evidence (commonly referred to as “junk science”) has resulted in findings of causation which simply cannot be justified or understood from the standpoint of the current state of credible scientific and medical knowledge. Most importantly, this development has led to a deep and growing cynicism about the ability of tort law to deal with difficult scientific and medical concepts in a principled and rational way.”

United States Dep’t of Justice, Tort Policy Working Group, Report of the Tort Policy Working Group on the causes, extent and policy implications of the current crisis in insurance availability and affordability at 35 (Report No. 027-000-01251-5) (Wash.DC 1986).  So according to the Justice Department authors, “junk science” was already in common use by 1986.  We really would not expect linguistic creativity in such a document.

Whence comes the phrase “junk science”?  Clearly, the phrase is an analogue of “junk food,” food that fills but fails to nourish.  Here is the Google ngram of the emergence of the phrase “junk food,” which shows the phrase took off in common use shortly before 1970: junk food Google NgramWhat then about junk science?

The Rise of Junk Science - Google Ngram Viewer

With a little tweaking of Google’s smoothing function, this search can be run to reveal more about the low end of the curve.

Junk Science without smoothing

This chart suggests that there was very small flurry of usage in the first half of the 1970s, with a re-emergence around 1982 or so, and then a re-introduction in 1985, with a steady increase every since.

Here is how “junk science” compares to “junk food” (with more smoothing to the curve added):

junk science vs junk food

Junk science seems to have overtaken junk food in books, at any rate.

Of course, “junk science” is an epithet to be hurled at science that the speaker dislikes.  It has an emotive content, but its persistence reflects that it has an epistemic content as well.  “Junk science” is science that lacks an epistemic warrant, and pretends to be something that it is not.  Honest scientists, engaged in hypothesis-generating work, should not be defamed as junk scientists, but the boundary between hypothesis generation and conclusion mongering is often blurred by advocate scientists of all political persuasions.

There are many synonyms for junk science, which has been with us ever science gained prestige and persuasiveness over religious pronouncements about the real world.  To avoid the politicization of the term “junk science,” here are some alternatives:

advertising

alternative medicine

alchemy

anti-vaccination movements

aroma therapy

astrology

baloney

blackguardism

blood letting

bogus science

bullshit

bunk

bunkum

cargo cult science

clinical ecology

cold fusion

creationism

cult

denialism

dodgy data

error

faith-based science

flim-flam

flotsam and jetsam

fraud

free-energy devices

fuzzy thinking

homeopathy

ignorance

intelligent design

junk science

Lysenkoism

magical thinking

magnetic therapy

miracles

misrepresentation

New Age science

nonsense on stilts

N rays

nostrums

not even wrong

paranormal phenomena

pathological science

phrenology

political science

propaganda

pseudoscience

pseudosymmetry

quackademic

parapsychology

quackery

rubbish

shamanism

spiritualism

voodoo science

Posted in Scientific Evidence | Comments Off on The Rise and Rise of Junk Science

« Older Entries
Newer Entries »