TORTINI

For your delectation and delight, desultory dicta on the law of delicts.

Bentham’s Legacy – Quantification of Fact Finding

March 1st, 2015

Jeremy Bentham, radical philosopher, was a source of many antic proposals. Perhaps his most antic proposal was to have himself stuffed, mounted, and displayed in the halls of University College of London, where he may still be observed during normal school hours. In ethical theory, Bentham advocated for an extreme ethical reductionism, known as utilitarianism. Bentham shared Edmund Burke’s opposition to the invocation of natural rights, but unlike Burke, Bentham was an ardent foe of the American Revolution.

Bentham was also a non-practicing lawyer who had an inexhaustible capacity for rationalistic revisions of legal practice. Among his revisionary schemes, Bentham proposed to reduce or translate qualitative beliefs to a numerical a scale, like a thermometer. Jeremy Bentham, 6 The Works of Jeremy Bentham; Rationale of Evidence, Rationale of Judicial Evidence at 225 (1843); 1 Rationale of Judicial Evidence Specially Applied to Judicial Practice at 76 (1827). The legal profession, that is lawyers who actually tried or judged cases, did think much of Bentham’s proposal:

“The notions of those who have proposed that mere moral probabilities or relations could ever be represented by numbers or space, and thus be subjected to arithmetical analysis, cannot but be regarded as visionary and chimerical.”

Thomas Starkie, A Practical Treatise of the Law of Evidence 225 (2d ed. 1833). Having graduated from St. John’s College, Cambridge University, as senior wrangler, Starkie was no slouch in mathematics, and he was an accomplished lawyer and judge later in life.

Starkie’s pronouncement upon Bentham’s proposal was, in the legal profession, a final judgment. The idea of having witnesses provide a decigrade or centigrade scale of belief in facts never caught on in the law. No evidentiary code or set of rules allows for, or requires, such quantification, but on the fringes, Bentham’s ideas still resonate with some observers who would require juries or judges to quantify their findings of fact:

“Consequently statistical ideas should be used in court and have already been used in the analysis of forensic data. But there are other areas to explore. Thus I do not think a jury should be required to decide guilty or innocent; they should provide their probability of guilt. The judge can then apply MEU [maximised expected utility] by incorporating society’s utility. Hutton could usefully have used some probability. A lawyer and I wrote a paper on the evidential worth of failure to produce evidence.”

Lindley, “Bayesian Thoughts,” Significance 73, 74-75 (June 2004). Some might say that Lindley was trash picking in the dustbin of legal history.

Lies, False Memories, or Bad Biochemistry?

February 23rd, 2015

NBC suspended Brian Williams for six months for a bout of misremembering[1]. The electronic media hardly skipped a sine wave. David Brooks thinks that the public reaction was “barbaric” [sic][2], and counseled more empathy. Indeed, some commentators jumped on Williams’ apology as an admission of lying.[3] Science Times writer Parker-Pope more charitably asked whether Brian Williams was a victim of false memory[4]. The media fretted that Williams, their most trustworthy voice and face, had undermined the public’s trust in the major news services. Why worry though? There was little trust in the nightly news programs that serve up news as entertainment.

Memory is key to what trial lawyers do in presenting evidence in support of historical narrative about events that took place in the past. False memory, whether the innocent or guilty kind, is a major threat to the judicial system’s attempt to discern the truth based upon narrative testimony. Lawyers recognize that memories are fragile and subject to manipulation. The tenuous connection between past events and current recollection is an omnipresent challenge to the basic process of juridical fact finding. The most cynical members[5] of the bar, no doubt, exploit the amorphous quality of memory, but even at its best, memory can be a slender reed to support our judgments.

Consider the criminal prosecution against Adnan Syed, which turned on the dodgy testimony of Jay Wilds. As recounted in Sarah Koenig’s Serial podcasts, the State of Maryland tried Syed twice. The first trial ended in a mistrial before the case could be submitted to a jury. The murder charges turned on factual accounts of one witness, Jay Wilds, who gave a wildly different account each time he talked about his participation in the cover up of the murder of Hae Min Lee.

Koenig’s series not only breathed life into Syed’s efforts to obtain a hearing on his ineffective assistance challenge, but they demonstrated the power of hindsight and other cognitive bias in the criminal justice system, as well as in all human endeavors[6]. Witnesses, including Syed, struggled to give accurate accounts of past events, even a few weeks after the fact, whatever that was.

Every lawyer who has tried cases can recount instances in which witnesses failed to recall, or recalled erroneously, essential facts in the litigation. In one recent case I had in upstate New York, a witness, who appeared very honest to the defense counsel in the deposition room, testified that he had never seen or read any warnings concerning crystalline silica in the pottery factory, where had worked for several decades. When the employer produced documents from its safety program, the documentary record showed that the plaintiff sat on a safety committee, and that he had helped to prepare safety warning placards for the workplace. Memories are like that, especially when they are inconvenient.

In 2013, Mauro Costa-Mattioli and his research group showed that actin polymerization, fostered by the mTORC2 pathway, is essential for long-term memory in mice and flies. See Wei Huang,Ping Jun Zhu, Shixing Zhang, Hongyi Zhou, Loredana Stoica, Mauricio Galiano, Krešimir Krnjević, Gregg Roman & Mauro Costa-Mattioli, “mTORC2 controls actin polymerization required for consolidation of long term memory,” 16 Nature Neurosci. 441 (2013). Well, perhaps Brian Williams simply had a surfeit of actin polymerization working to remodel his memory in a false and misleading way? And perhaps my plaintiff had a deficit of actin remodeling of his neurons?


[1] Emily Steel & Ravi Somaiya, “Brian Williams Suspended From NBC for 6 Months Without Pay,” N.Y. Times (Feb. 10, 2015).

[2] David Brooks, “The Act of Rigorous Forgiving,” N.Y. Times (Feb. 10, 2015). Brooks would probably think this use of “sic” was barbaric rather than merely barbarous pedantry in pointing out a diction error.

[3] “NBC’S Brian Williams admits on air to lying: Apologizes for bogus story of his ‛personal heroics’.”

[4] Tara Parker-Pope, “Was Brian Williams a Victim of False Memory?” N.Y. Times (Feb. 9, 2015). (The print edition contained a different, more biting headline: “False Memory vs. Bald Faced Lie”).

[5] See “Preparing for your deposition” a.k.a. the Baron & Budd asbestos memo” Wikipedia; Walter Olson, “Thanks for the Memories: How lawyers get the testimony they want,” Reason (June 1998).

[6] Not everyone agrees. JaneAnne Murray, a lawyer with criminal defense experience, took a surprising approach in suggesting that Adnan Syed should have pleaded guilty, in part because the prosecution’s case against Syed was strong. See JaneAnne Murray, “Why Adnan Syed of ‘Serial’ Should Have Pleaded Guilty,” N.Y. Times (Jan. 22, 2015) (“In this case, the injustice may lie not in the conviction, but in the failure to negotiate the charges. The unstructured presentation of the facts in “Serial” obscured a strong case for the prosecution.”). Actually, Koenig’s Serial podcasts appeared to play down the strength of the defense, and what should and could have been done with the testimony of the chief prosecution witness, Jay Wilds. Koenig lauded Syed’s defense lawyer, Cristina Gutierrez, as having a reputation for brillance, which seemed conspicuously absent in the segments of trial tapes excerpted on Serial. In any event, the Maryland Special Court of Appeals has only recently granted Syed an appeal on his claim for post-conviction relief. Emma G. Fitzsimmons, “Appeal to Be Heard in ‘Serial’ Murder Case,” N.Y. Times (Feb. 7, 2015).

 

 

Plaintiffs Sue Friendly Physicians for Access to Opinions and to Data

February 14th, 2015

Access to underlying data continues to be a difficult and contentious issue in litigation and in the world of public health. Healthcare providers have made intemperate demands upon pharmaceutical manufacturers to produce underlying data from clinical trials, but often the same providers object the loudest when asked to share underlying data from their own observational or experimental studies.

Last month saw a judicial rejection of an effort by New York researchers to resist production of underlying data. Plaintiffs in the World Trade Center health effects litigation sought to compel their treating physicians at the Mt. Sinai Health System to testify to their opinions about general and specific causation of the plaintiffs’ alleged injuries. Previously, in discovery depositions of the plaintiffs’ treating physicians as fact witnesses, Mt. Sinai Hospital’s attorneys objected to plaintiffs’ counsel’s efforts to elicit causation opinions not recorded in the plaintiffs’ personal medical records. Plaintiffs then moved to compel the treating physicians to give opinions on causation, and to produce underlying data from their published papers on the claimed health effects of exposure to World Trade Center dust. The hospital opposed these motions on grounds that its neutrality in the litigation would be compromised by compulsory testimony and production of data.

In a short opinion, Judge Alvin K. Hellerstein refused plaintiffs’ motions to compel testimony, but granted their motion to compel production of underlying data. In re World Trade Center Lower Manhattan Disaster Site Litig., ___ F.R.D. ___, 2015 WL 220988, at *1-2 (S.D.N.Y. Jan. 15, 2015). Judge Hellerstein acknowledged that, in rare instances, federal trial courts have compelled experts to provide opinion testimony. Carter–Wallace, Inc. v. Otte, 474 F.2d 529, 536 (2d Cir.1972) (“The weight of authority holds that, although it is not the usual practice, a court does have the power to subpoena an expert witness and … require him [or her] to state whatever opinions he [or she] may have previously formed.”).

Judge Hellerstein cited the five factors articulated in Kaufman v. Edelstein, as governing the exercise of discretion in compelling expert testimony:

(1) “[T]he degree to which the expert is being called because of his knowledge of facts relevant to the case rather than in order to give opinion testimony”;

(2) “the difference between testifying to a previously formed or expressed opinion and forming a new one”;

(3) “the possibility that, for other reasons, the witness is a unique expert”;

(4) “the extent to which the calling party is able to show the unlikelihood that any comparable witness will willingly testify”; and

(5) “the degree to which the witness is able to show that he has been oppressed by having continually to testify[.]”

2015 WL 220988, at *2-3, citing and quoting Kaufman v. Edelstein, 539 F.2d 811, 822 (2d Cir.1976).

Considering these factors, Judge Hellerstein refused to enforce the subpoena ad testificandum against the Mt. Sinai physicians. Plaintiffs have their own retained expert witness, Dr. Tee Guidotti, who was prepared to testify about both specific and general causation of the plaintiffs’ claimed injuries. The underlying data gathered by Mt. Sinai physicians and scientists, however, were another matter:

“However, it is undisputed that the scope and detail of the research conducted by the Mt. Sinai WTC Health Program is unparalleled. This litigation can only benefit from a full record and the inclusion of the relevant data underlying the research conducted by the Mt. Sinai WTC Health Program is appropriate. Accordingly, Mt. Sinai is ordered to produce all Mt. Sinai WTC Health Program data pursuant to the protocol established in In re World Trade Center Disaster Site Litigation, 21–mc–100, accounting for the redaction of personal identifying information and other sensitive patient material.”

Id. at *4; see also id. at *1.

Cynical observers may wonder whether the Mt. Sinai opposition to the subpoenas and motions to compel was posturing. The hospital and many of its physicians have been outspoken advocates on many occupational and environmental issues. Perhaps like Brer Rabbit, they were protesting not to be thrown in the briar patch. Or maybe, they realized that they could not resist the subpoena for data unless they also declined to testify about their opinions. In any event, Judge Hellerstein maintained the right of expert witnesses to hold their opinions to themselves and to avoid participating in the litigation system, while ensuring that the data are available to all. The plaintiffs may well have been clever by halves in bringing their motion.

Sander Greenland on “The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics”

February 8th, 2015

Sander Greenland is one of the few academics, who has served as an expert witness, who has written post-mortems of his involvement in various litigations[1]. Although settling scores with opposing expert witnesses can be a risky business[2], the practice can provide important insights for judges and lawyers who want to avoid the errors of the past. Greenland correctly senses that many errors seem endlessly recycled, and that courts could benefit from disinterested commentary on cases. And so, there should be a resounding affirmation from federal and state courts to the proclaimed “need for critical appraisal of expert witnesses in epidemiology and statistics,” as well as in many other disciplines.

A recent exchange[3] with Professor Greenland led me to revisit his Wake Forest Law Review article. His article raises some interesting points, some mistaken, but some valuable and thoughtful considerations about how to improve the state of statistical expert witness testimony. For better and worse[4], lawyers who litigate health effects issues should read it.

Other Misunderstandings

Greenland posits criticisms of defense expert witnesses[5], who he believes have misinterpreted or misstated the appropriate inferences to be drawn from null studies. In one instance, Greenland revisits one of his own cases, without any clear acknowledgment that his views were largely rejected.[6] The State of California had declared, pursuant to Proposition 65 ( the Safe Drinking Water and Toxic Enforcement Act of 1986, Health and Safety Code sections 25249.5, et seq.), that the State “knew” that di(2-ethylhexyl)phthalate, or “DEHP” caused cancer. Baxter Healthcare challenged the classification, and according to Greenland, the defense experts erroneously interpreted inclusive studies with evidence supporting a conclusion that DEHP does not cause cancer.

Greenland argues that the Baxter expert’s reference[7] to an IARC working group’s classification of DEHP as “not classifiable as to its carcinogenicity to humans” did not support the expert’s conclusion that DEHP does not cause cancer in human. If Baxter’s expert invoked the IARC working group’s classification for complete exoneration of DEHP, then Greenland’s point is fair enough. In his single-minded attack on Baxter’s expert’s testimony, however, Greenland missed a more important point, which is that the IARC’s determination that DEHP is not classifiable as to carcinogenicity is directly contradictory of California’s epistemic claim to “know” that DEHP causes cancer. And Greenland conveniently omits any discussion that the IARC working group had reclassified DEHP from “possibly carcinogenic” to “not classifiable,” in the light of its conclusion that mechanistic evidence of carcinogenesis in rodents did not pertain to humans.[8] Greenland maintains that Baxter’s experts misrepresented the IARC working group’s conclusion[9], but that conclusion, at the very least, demonstrates that California was on very shaky ground when it declared that it “knew” that DEHP was a carcinogen. California’s semantic gamesmanship over its epistemic claims is at the root of the problem, not a misstep by defense experts in describing inconclusive evidence as exonerative.

Greenland goes on to complain that in litigation over health claims:

“A verdict of ‛uncertain’ is not allowed, yet it is the scientific verdict most often warranted. Elimination of this verdict from an expert’s options leads to the rather perverse practice (illustrated in the DEHP testimony cited above) of applying criminal law standards to risk assessments, as if chemicals were citizens to be presumed innocent until proven guilty.

39 Wake Forest Law Rev. at 303. Despite Greenland’s alignment with California in the Denton case, the fact of the matter is that a verdict of “uncertain” was allowed, and he was free to criticize California for making a grossly exaggerated epistemic claim on inconclusive evidence.

Perhaps recognizing that he may be readily be seen as an advocate for coming to the defense of California on the DEHP issue, Greenland protests that:

“I am not suggesting that judgments for plaintiffs or actions against chemicals should be taken when evidence is inconclusive.”

39 Wake Forest Law Rev. at 305. And yet, his involvement in the Denton case (as well as other cases, such as silicone gel breast implant cases, thimerosal cases, etc.) suggest that he is willing to lend aid and support to judgments for plaintiffs when the evidence is inconclusive.

Important Advice and Recommendations

These foregoing points are rather severe limitations to Greenland’s article, but lawyers and judges should also look to what is good and helpful here. Greenland is correct to call out expert witnesses, regardless of party of affiliation, who opine that inconclusive studies are “proof” of the null hypothesis. Although some of Greenland’s arguments against the use of significance probability may be overstated, his corrections to the misstatements and misunderstandings of significance probability should command greater attention in the legal community. In one strained passage, however, Greenland uses a disjunction to juxtapose null hypothesis testing with proof beyond a reasonable doubt[10]. Greenland of course understands the difference, but the context would lead some untutored readers to think he has equated the two probabilistic assessments. Writing in a law review for lawyers and judges might have led him to be more careful. Given the prevalence of plaintiffs’ counsel’s confusing the 95% confidence coefficient with a burden of proof akin to beyond a reasonable doubt, great care in this area is, indeed, required.

Despite his appearing for plaintiffs’ counsel in health effects litigation, some of Greenland’s suggestions are balanced and perhaps more truth-promoting than many plaintiffs’ counsel would abide. His article provides an important argument in favor of raising the legal criteria for witnesses who purport to have expertise to address and interpret epidemiologic and experimental evidence[11]. And beyond raising qualification requirements above mere “reasonable pretense at expertise,” Professor Greenland offers some thoughtful, helpful recommendations for improving expert witness testimony in the courts:

  • “Begin publishing projects in which controversial testimony (a matter of public record) is submitted, and as space allows, published on a regular basis in scientific or law journals, perhaps with commentary. An online version could provide extended excerpts, with additional context.
  • Give courts the resources and encouragement to hire neutral experts to peer-review expert testimony.
  • Encourage universities and established scholarly societies (such as AAAS, ASA, APHA, and SER) to conduct workshops on basic epidemiologic and statistical inference for judges and other legal professionals.”

39 Wake Forest Law Rev. at 308.

Each of these three suggestions is valuable and constructive, and worthy of an independent paper. The recommendation of neutral expert witnesses and scholarly tutorials for judges is hardly new. Many defense counsel and judges have argued for them in litigation and in commentary. The first recommendation, of publishing “controversial testimony” is part of the purpose of this blog. There would be great utility to making expert witness testimony, and analysis thereof, more available for didactic purposes. Perhaps the more egregious testimonial adventures should be republished in professional journals, as Greenland suggests. Greenland qualifies his recommendation with “as space allows,” but space is hardly the limiting consideration in the digital age.

Causation

Professor Greenland correctly points out that causal concepts and conclusions are often essentially contested[12], but his argument might well be incorrectly taken for “anything goes.” More helpfully, Greenland argues that various academic ideals should infuse expert witness testimony. He suggests that greater scholarship, with acknowledgment of all viewpoints, and all evidence, is needed in expert witnessing. 39 Wake Forest Law Rev. at 293.

Greenland’s argument provides an important corrective to the rhetoric of Oreskes, Cranor, Michaels, Egilman, and others on “manufacturing doubt”:

“Never force a choice among competing theories; always maintain the option of concluding that more research is needed before a defensible choice can be made.”

Id. Despite his position in the Denton case, and others, Greenland and all expert witnesses are free to maintain that more research is needed before a causal claim can be supported. Greenland also maintains that expert witnesses should “look past” the conclusions drawn by authors, and base their opinions on the “actual data” on which the statistical analyses are based, and from which conclusions have been drawn. Courts have generally rejected this view, but if courts were to insist upon real expertise in epidemiology and statistics, then the testifying expert witnesses should not be constrained by the hearsay opinions in the discussion sections of published studies – sections which by nature are incomplete and tendentious. See Follow the Data, Not the Discussion” (May 2, 2010).

Greenland urges expert witnesses and legal counsel to be forthcoming about their assumptions, their uncertainty about conclusions:

“Acknowledgment of controversy and uncertainty is a hallmark of good science as well as good policy, but clashes with the very time limited tasks faced by attorneys and courts”

39 Wake Forest Law Rev. at 293-4. This recommendation would be helpful in assuring courts that the data may simply not support conclusions sufficiently certain to be submitted to lay judges and jurors. Rosen v. Ciba-Geigy Corp., 78 F.3d 316, 319, 320 (7th Cir. 1996) (“But the courtroom is not the place for scientific guesswork, even of the inspired sort. Law lags science; it does not lead it.”) (internal citations omitted).

Threats to Validity

One of the serious mistakes counsel often make in health effects litigation is to invite courts to believe that statistical significance is sufficient for causal inferences. Greenland emphasizes that validity considerations often are much stronger, and more important considerations than the play of random error[13]:

“For very imperfect data (e.g., epidemiologic data), the limited conclusions offered by statistics must be further tempered by validity considerations.”

*   *   *   *   *   *

“Examples of validity problems include non-random distribution of the exposure in question, non-random selection or cooperation of subjects, and errors in assessment of exposure or disease.”

39 Wake Forest Law Rev. at 302 – 03. Greenland’s abbreviated list of threats to validity should remind courts that they cannot sniff a p-value below five percent and then safely kick the can to the jury. The literature on evaluating bias and confounding is huge, but Greenland was a co-author on an important recent paper, which needs to be added to the required reading lists of judges charged with gatekeeping expert witness opinion testimony about health effects. See Timothy L. Lash, et al., “Good practices for quantitative bias analysis,” 43 Internat’l J. Epidem. 1969 (2014).


[1] For an influential example of this sparse genre, see James T. Rosenbaum, “Lessons from litigation over silicone breast implants: A call for activism by scientists,” 276 Science 1524 (1997) (describing the exaggerations, distortions, and misrepresentations of plaintiffs’ expert witnesses in silicone gel breast implant litigation, from perspective of a highly accomplished scientist physician, who served as a defense expert witness, in proceedings before Judge Robert Jones, in Hall v. Baxter Healthcare Corp., 947 F. Supp. 1387 (D. Or. 1996). In one attempt to “correct the record” in the aftermath of a case, Greenland excoriated a defense expert witness, Professor Robert Makuch, for stating that Bayesian methods are rarely used in medicine or in the regulation of medicines. Sander Greenland, “The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics,” 39 Wake Forest Law Rev. 291, 306 (2004).  Greenland heaped adjectives upon his adversary, “ludicrous claim,” “disturbing, “misleading expert testimony,” and “demonstrably quite false.” See “The Infrequency of Bayesian Analyses in Non-Forensic Court Decisions” (Feb. 16, 2014) (debunking Prof. Greenland’s claims).

[2] One almost comical example of trying too hard to settle a score occurs in a footnote, where Greenland cites a breast implant case as having been reversed in part by another case in the same appellate court. See 39 Wake Forest Law Rev. at 309 n.68, citing Allison v. McGhan Med. Corp., 184 F.3d 1300, 1310 (11th Cir. 1999), aff’d in part & rev’d in part, United States v. Baxter Int’l, Inc., 345 F.3d 866 (11th Cir. 2003). The subsequent case was not by any stretch of the imagination a reversal of the earlier Allison case; the egregious citation is a legal fantasy. Furthermore, Allison had no connection with the procedures for court-appointed expert witnesses or technical advisors. Perhaps the most charitable interpretation of this footnote is that it was injected by the law review editors or supervisors.

[3] SeeSignificance Levels are Made a Whipping Boy on Climate Change Evidence: Is .05 Too Strict? (Schachtman on Oreskes)” (Jan. 4, 2015).

[4] In addition to the unfair attack on Professor Makuch, see supra, n.1, there is much that some will find “disturbing,” “misleading,” and even “ludicrous,” (some of Greenland’s favorite pejorative adjectives) in the article. Greenland repeats in brief his arguments against the legal system’s use of probabilities of causation[4], which I have addressed elsewhere.

[5] One of Baxter’s expert witnesses appeared to be the late Professor Patricia Buffler.

[6] See 39 Wake Forest Law Rev. at 294-95, citing Baxter Healthcare Corp. v. Denton, No. 99CS00868, 2002 WL 31600035, at *1 (Cal. App. Dep’t Super. Ct. Oct. 3, 2002) (unpublished); Baxter Healthcare Corp. v. Denton, 120 Cal. App. 4th 333 (2004)

[7] Although Greenland cites to a transcript, the citation is to a judicial opinion, and the actual transcript of testimony is not available at the citation give.

[8] See Denton, supra.

[9] 39 Wake Forest L. Rev. at 297.

[10] 39 Wake Forest L. Rev. at 305 (“If it is necessary to prove causation ‛beyond a reasonable doubt’–or be ‛compelled to give up the null’ – then action can be forestalled forever by focusing on any aspect of available evidence that fails to conform neatly with the causal (alternative) hypothesis. And in medical and social science there is almost always such evidence available, not only because of the ‛play of chance’ (the focus of ordinary statistical theory), but also because of the numerous validity problems in human research.”

[11] See Peter Green, “Letter from the President to the Lord Chancellor regarding the use of statistical evidence in court cases” (Jan. 23, 2002) (writing on behalf of The Royal Statistical Society; “Although many scientists have some familiarity with statistical methods, statistics remains a specialised area. The Society urges you to take steps to ensure that statistical evidence is presented only by appropriately qualified statistical experts, as would be the case for any other form of expert evidence.”).

[12] 39 Wake Forest Law Rev. at 291 (“In reality, there is no universally accepted method for inferring presence or absence of causation from human observational data, nor is there any universally accepted method for inferring probabilities of causation (as courts often desire); there is not even a universally accepted definition of cause or effect.”).

[13] 39 Wake Forest Law Rev. at 302-03 (“If one is more concerned with explaining associations scientifically, rather than with mechanical statistical analysis, evidence about validity can be more important than statistical results.”).

The opinions, statements, and asseverations expressed on Tortini are my own, or those of invited guests, and these writings do not necessarily represent the views of clients, friends, or family, even when supported by good and sufficient reason.