At the heart of epidemiologic studies and clinical trials is the contingency table. The term, contingency table, was introduced by Karl Pearson in the early 20th century as a way to explore the independence, vel non, in a multivariate model. The simplest version of the table is the “2 by 2” table that is at the heart of case-control and other studies:
| Cases (with outcome of interest) | Controls (without outcome of interest) | ||
| Exposure of Interest Present | A | B | A + B
Marginal total of all exposed |
| Exposure of Interest Absent | C | D | C + D
Marginal total of all non-exposed |
| A + C
Marginal total of cases |
B + D
Marginal total of controls |
A + B + C + D
Total observed in study |
A measure of association between the exposure of interest and the outcome of interest can be shown in the odds ratio (OR), which can be assessed for random error on the assumption of no association.
OR = (A/C)/(B/D) = A*D/B*C
The measurement of the OR turns on faithfully applying the same method of counting cases regardless of exposure status. When investigators expand the “A” cell by loosening their criteria for exposure, we say that they have engaged in “hacking the A cell.”
Something akin to hacking the A cell occurred in the large epidemiologic study, known as “Yale Hemorrhagic Stroke Project (HSP),” which was the center piece of the plaintiffs’ case in In re Phenylpropanolamine Products Liability Litigation. Although the HSP was sponsored by manufacturers, it was conducted independently without any manufacturer oversight beyond the protocol. The FDA reviewed the HSP results, and ultimately the HSP was published in the New England Journal of Medicine.[1]
The HSP was challenged in a Rule 702 hearing in the Multi-District Litigation (MDL). The MDL judge, Judge Rothstein, conducted hearings and entertained extensive briefings on the reliability of plaintiffs’ expert witnesses’ opinions, which were based largely upon the HSP. The hearings, however, could not go beyond doubts raised by the published paper, and Judge Rothstein permitted plaintiffs’ expert witnesses’ proffered testimony based upon the study, finding that:
“The prestigious NEJM published the HSP results, further substantiating that the research bears the indicia of good science.”[2]
The HSP study was subjected to much greater analysis in litigation. After the MDL concluded its abridged gatekeeping process, the defense successfully sought the underlying data to the HSP. These data unraveled the HSP paper by showing that the study investigators had deviated from the protocol in a way to increase the number of exposed cases (A cell), with the obvious result of increasing the OR reported by the study.
Both sides of the PPA litigation accused the other side of “hacking at the A cell,” but juries seemed to understand that the hacking had started before the paper was published. A notable string of defense verdicts ensued. After one of the early defense verdicts, plaintiffs’ counsel challenged the defendant’s reliance upon underlying data that went behind the peer-reviewed publication. The trial court rejected the request for a new trial, and spoke to the significance of challenging the superficial significance of peer review of the key study relied upon by plaintiffs in the PPA litigation:
“I mean, you could almost say that there was some unethical activity with that Yale Study. It’s real close. I mean, I — I am very, very concerned at the integrity of those researchers. Yale gets — Yale gets a big black eye on this.”[3]
Today we can see the equivalent of “A” cell hacking in a rather sleazy attempt by the Banana Republicans to steal a presidential election they lost. Cry-baby conservatives are seeking recounts where they lost, but not where they won. They are challenging individual ballots on the basis of outcome. They are raising speculative questions about the electoral processes of entire states, even where the states in question have handed them notable wins down ballot.
[1] Walter N. Kernan, Catherine M. Viscoli, Lawrence M. Brass, Joseph P. Broderick, Thomas Brott, Edward Feldmann, Lewis B. Morgenstern, Janet Lee Wilterdink, and Ralph I. Horwitz, “Phenylpropanolamine and the Risk of Hemorrhagic Stroke,” 343 New Engl. J. Med. 1826 (2000). See “Misplaced Reliance On Peer Review to Separate Valid Science From Nonsense” (Aug. 14, 2011).
[2] In re Phenylpropanolamine Prod. Liab. Litig., 289 F. 2d 1230, 1239 (2003) (citing Daubert II for the proposition that peer review shows the research meets the minimal criteria for good science). There were many layers of peer review for the HSP study, all of which proved ultimately ineffectual compared with the closer scrutiny that the HSP received in litigation where underlying data were produced.
[3] O’Neill v. Novartis AG, California Superior Court, Los Angeles Cty., Transcript of Oral Argument on Post-Trial Motions, at 46 -47 (March 18, 2004) (Hon. Anthony J. Mohr).
