The Challenge
Critics and anti-industry zealots have argued that industry-sponsored science is vitiated by conflicts of interest. The critics of industry research have supported their attack with studies that claim to show that industry studies disproportionately report outcomes favorable to their sponsors.
The potential conflicts posed by industry-sponsored research studies are fairly obvious. Industries that make or sell products, raw materials, or chemicals have an interest in having toxicological and epidemiologic studies support claims of safety. Research that suggests an industry’s product causes harm may hurt the industry’s financial interests directly by inhibiting sales, or indirectly by undermining the industry’s position in litigation, or by leading to greater regulatory scrutiny and control. If the harm evidenced by the research is sufficiently severe, the research may lead to product recall or bans, again with serious economic consequences for the industry. Evidence of harm may require heightened warnings or instructions, which may limit sales or encourage sales of competing, less hazardous products.
The Response.
The answer to the anti-industry challenge lies in a reality more complicated and nuanced that the simplistic views of the critics. Industry often must conduct or sponsor research to live up to its common law or regulatory duties to test its products. At other times, industry sponsors research in response to emerging scientific papers that claim that the industry product is harmful in ways not previously known. While on occasion, industry might be accused of “manufacturing doubt[1],” there are scientists equally intent upon “manufacturing certainty,” whether from dodgy data or iffy inferences.
The motives and interests of scientists intent on manufacturing a faux certainty are harder to discern than the potential conflicts of industry and scientists who are funded by industry. Many scientists purport to have only the lofty goals of science or the public interest at heart in pursuing their research, but a close look shows that there are several potential conflicts of interest prevalent in the research community, and that these interests help shape science into anti-industry positions.
1. White Hat Bias
Over 20 years ago, a science journalist published an account of how dire predictions of asbestos deaths had not come to pass. Tom Reynolds, “Asbestos-Linked Cancer Rates Up Less Than Predicted,” 84 J. Nat’l Cancer Instit. 560 (1992). In trying to understand why the earlier predictions had been disconfirmed by later data, Reynolds quoted one scientist as saying that:
“the government’s exaggeration of the asbestos danger reflects a 1970s’ Zeitgeist that developed partly in response to revelations of industry misdeeds. ‘It was sort of the “in” thing to exaggerate … [because] that would be good for the environmental movement’…. ‘At the time it looked like you were wearing a white hat if you made these wild estimates. But I wasn’t sure whoever did that was doing all that much good.”
Id. at 562. Reynolds captured the essence of “white-hat” bias, a form of political correctness applied to issues that really depend upon scientific method and data for their resolution.
More recently, David Allison and Mark Cope, two public health researchers described[2] white-hat bias as a prevalent cognitive bias in how research is reported and interpreted. They described white-hat bias as a “bias leading to the distortion of information in the service of what may be perceived to be righteous ends.” Perhaps the temptation to overstate the evidence against a toxic substance is unavoidable, but it diminishes the authority and credibility of regulators entrusted with promulgating and enforcing protective measures. And error is still error, regardless of its origin. Allison and Cope give examples of white-hat bias in how papers are cited, with “exonerative” studies cited less often than those than claim harmful outcomes. And when positive papers were cited, they were often interpreted misleadingly to overstate the harms previously reported.
2. Other “Interested” Actors Who Sponsor and Conduct Scientific Research
The discussion and debate surrounding industry sponsorship of scientific research often ignores the activities of other, intensely interested sponsors and researchers. Labor unions, for instance, have sponsored research on occupational hazards. Evidence of harm often becomes potent ammunition in labor-management negotiations over work conditions and wages. Environmental groups have sponsored studies with sympathetic researchers who are openly eager to embarrass industrial interests and support increasing regulatory restrictions or bans on chemicals or products. Plaintiffs’ counsel may well be the largest rent-seeking group in the United States, and they have sponsored studies with the implicit or explicit goal of supporting their claims for compensation. Disease support groups, often aligned with plaintiffs’ counsel, have sponsored research, to support their claims for compensation and fees, the more, the better. Scientists and physicians who serve as expert witnesses for plaintiffs’ counsel have undertaken research to advance and support their testimonial enterprise in the litigation brought by the counsel who engaged them to testify.
3. Regulatory Biases
Many scientists are funded by regulatory agencies intent upon showing harm from particularly exposures or industrial processes. The agencies may be motivated by their desire to expand their regulatory control under their statutory mandate by having scientific evidence that fits their political and regulatory goals. Some scientists may be politically aligned, or simply believe that the public interest requires the sought after enhanced regulation.
4. Publication Bias
“Null” studies, studies that do not find a statistical significant increased risk of harm from exposure, do not sell. More to the point, null studies do not routinely get published. Spending substantial amounts of time on a study without obtaining an outcome that will make the study publishable is a sure-fire way to sideline one’s academic career. Publications are the measurable units of success in academic circles. Publications beget future grants and funding, which further enhance academic reputation and careers.
The pressure on researchers to find some untoward outcome from research on a potential toxic material or product is great. As John Ioannides and others have shown, prevalent research methods encourage technique that will lead to some findings. For instance, an epidemiologic cohort study will compare a group with and without exposure, but will look at multiple health outcomes. Perhaps there was an a priori hunch that lung cancer rates would be increased among the exposed group, but the researchers also looked at a 100 other cancers, and perhaps another 100 other non-malignant diseases, in the study. Chance alone would have led to several statistically significant findings of increased disease outcomes among the exposed group, and researchers can publish these findings that encourages readers to believe that they were the subject of the research all along.
Similarly, case-control studies can collect cases of a disease of interest and compare them with non-diseased controls in terms of dozens if not hundreds of prior exposures and life-style variables. Again, the researchers can report the exposures and variables statistically significantly associated with the disease, and neglect to report all those not associated.[3]
5. Biased Studies with Biased Outcomes, Making Biased Claims
The critics of industry-sponsored studies have themselves conducted “studies of studies,” to assess whether sponsorship correlated with outcomes favorable to the sponsor. These studies of studies claim to find that industry studies invariably or disproportionately favor their sponsors’ interests, but these studies of studies are themselves critically flawed by selection bias.[4]
To understand the selection bias involved, we need only pay attention to the conditions that lead industry to conduct the study. In connection with potential toxicities of chemicals or products, industry often is responding to an emerging scientific literature that claims the industrial materials cause harm. If the studies that claim harm are clearly valid, there is likely to be no counter-study undertaken. In this scenario, the anti-industry critics will find no industry study for inclusion in their analysis of whether industry studies are correlated with a favorable outcome.
Studies with doubtful data or overstretched conclusions are more likely to provoke an industrial concern into sponsoring research, and given the inciting condition, we should not be surprised that outcome is more favorable to industry than the previously published studies. Studies ultimately favorable to the defense are thus counted in the “studies of studies,” but given the circumstances that led to the sponsorship, the imbalance in the results is exactly what we should expect to see. Of course, if the study somehow supports the prior findings, the critics of industry will claim that the industry-sponsored study is an “admission” of harm. Inconclusive or exonerative studies will be dismissed by these critics as uninformative or vitiated by conflicts of interest.
There are many examples of how this selection bias plays out. In the early days of the silicone gel breast implant litigation, ostensibly neutral researchers published papers that claimed an association between silicone implants and scleroderma. See Harry Spiera, “Scleroderma After Silicone Augmentation Mammoplasty,” 260 J. Am. Med. Ass’n 236 (1988). These papers led to an FDA-imposed moratorium, which in turn instigated a litigation tsunami. The then current and former manufacturers of silicone implants sponsored several epidemiologic studies. Although the companies engaged reputable researchers from the leading medical research centers in the United States, public-interest advocates such as Sidney Wolfe decried the “biased” industry-financed research that came out. Not surprisingly, the sponsored studies mostly exonerated any causal role of silicone in producing autoimmune diseases.
The biases of the physicians who supported the alleged causal association were much more difficult to ferret out. Many of the physicians adopted pro-causation views to attract patients from the many support “victim” support groups, and many of the scientists who advocated pro-causation opinions sought to market test kits that would identify biomarkers of a unique silicone-induced disease. The house of cards created by the support groups and plaintiffs’ counsel ultimately collapsed when court-appointed expert witnesses and the Institute of Medicine examined the evidence closely. Today, the entire episode can be chalked up mostly to litigation fraud. See Hon. Judge Jack B. Weinstein, “Preliminary Reflections on Administration of Complex Litigation.” Cardozo Law Review DeNovo 1, 14 (2009) (“[t]he breast implant litigation was largely based on a litigation fraud. … Claims—supported by medical charlatans—that enormous damages to women’s systems resulted could not be supported.”).
Another example is provided by the welding fume litigation. In 2001, Dr. Brad Racette published a case series that he erroneously called a case-control study, which purported to find an earlier age of onset for Parkinson’s disease among welders. In 2005, working in cooperation with plaintiffs’ counsel, Racette transmuted litigation screenings into a cross-sectional study, which purported to find an increased prevalence ratio of “parkinsonism” among welders. Both studies were seriously flawed, but the plaintiffs’ counsels’ use of Racette’s studies created an incentive for industry to sponsor several studies. The plaintiffs’ counsel blasted the industry defendants for “manufacturing doubt,” but in fact, the industry-sponsored studies were valid scientific efforts, conducted by respected epidemiologists. The industry sponsored studies were corroborated several times over by governmentally sponsored studies, conducted in the United States and abroad. A meta-analysis published last year by well-regarded neuroepidemiologists ultimately vindicated the industry position, as well as the industry-sponsored studies. See James Mortimer, Amy Borenstein, and Lorene Nelson, “Associations of welding and manganese exposure with Parkinson disease: Review and meta-analysis,” 79 Neurology 1174 (2012) (reporting a statistically significant decreased risk of Parkinson’s disease among welding tradesmen).
In the pharmaceutical industry, publications will invariably show outcomes favorable to drug sponsors. New drugs that fail to show efficacy in clinical trials will not likely be written about because the sponsor will not file an application for regulatory approval. Writing up the clinical trials for publication typically is part of the process of seeking regulatory approval, and the publication effort is not undertaken unless there is a reasonable chance of approval. Unremarkably, the published papers describing these clinical trials disproportionately show efficacy and manageable side effects.
6. Hypocrisy Over Conflicts of Interest
The persistence in ignoring non-financial conflict of interests is not innocent. By elevating financial conflicts of interest, public advocates attempt to silence industry and discount industry-sponsored studies. The single-minded focus on financial conflicts of interest misdirects attention away from the existence of other sorts of biases, and the distortions they create.[5]
Conflicts of interest have thus largely and incorrectly been reduced to financial conflicts. For instance, in 2011, the National Institutes of Health (NIH) addressed only financial issues when it promulgated rules for managing conflicts of interest in the field of medical research.[6] Several commentators advocated regulation of non-financial COI, but the agency refused to include such conflicts within its rules.[7] The Institute of Medicine (IOM), in a monograph on conflicts of interest in medicine, similarly gave almost exclusive priority to financial ties.[8]
The focus on economic conflicts of interest is dangerous because it instills complacency about non-financial interests, and provides a false sense of assurance that the most serious biases are disclosed or eliminated. One need only consider the many examples of retractions, frauds, and ethical lapses in biomedical research to understand that non-financial interests, such as friendships and alliances, institutional hierarchies, intellectual biases and commitments, beneficence, “white-hat” advocacy, as well as the drive for professional achievement, recognition, and rewards, all have the potential to complicate, distort, and sometimes undermine scientific research in myriad ways. The failure to recognize and regulate non-economic conflicts and biases endangers the validity of science. Not only are these non-financial threats ignored, but financial interests receive undue attention, resulting in the erosion of public trust in scientific research that is sound.
The Cure
1. Push Back Against the Rhetoric of Conflicts of Interest
Industry should not be cowed by the anti-industry attack on industry-sponsored studies. Twenty years ago, one of the leading epidemiologists, Professor Kenneth Rothman, referred to the anti-industry bias as “new McCarthyism in science.[9]” This intolerance toward corporate sponsorship has been going on for some time, fueled by one-sided accounts of industry-sponsored studies. As noted by one of the leading 20th century epidemiologists, “an opinion should be evaluated on the basis of its contents, not on the interests or credentials of the individuals who hold it.[10]” Courageously, some scientists have fought for science to be judged on its merits.[11]
Professor Stossel and others created an organization, ACRE – The Association of Clinical Researchers and Educators, to defend legitimate interactions between Physicians and Industry. ACRE has spoken out against the lopsided demonization of the pharmaceutical industry, and the lionizing of the industry’s critics. A few years ago, a scientist working for the pharmaceutical industry, joined Professor Rothman, to cry out against the unfairness and partiality of journals’ conflict-of-interest rules and policies.[12] Dr. Hirsch published a strongly worded commentary that journals’ concerns are often poorly disguised prejudices against industry. Many of the journals in question rarely or never fuss over the obvious conflicts of interest created by the “profit motive” of researchers who want to climb the academic ladder, increase their salaries, enlarge their budgets, extend their influence, travel to organizational conferences, bolster their prestige, win more grants, enhance their reputations, or advance their political goals or ideologies.
The medical profession, the courts, and the public are seriously misled by the obsession with conflicts of interest, on either side. The obsession allows a disclosed or undisclosed conflict of interest to substitute for the much harder work of considering the merits of a study.
[1] David Michaels, Doubt is Their Product (2008).
[2] Mark B. Cope and David B. Allison, “White hat bias: examples of its presence in obesity research and a call for renewed commitment to faithfulness in research reporting,” 34 Internat’l J. Obesity 84 (2010).
[3] The works of John Ioannides explore the problem of false discovery rate and publication practices that encourage error. See N. S. Young, J. P. Ioannidis and O. Al-Ubaydli, 2008. Why current publication practices may distort science. PLoS Medicine, 5, e20.
[4] Miquel Porta, Sander Greenland, and John M. Last, eds., A Dictionary of Epidemiology 225-26 (5th ed. 2008).
[5] Richard S. Saver, “Is It Really All About The Money? Reconsidering Non-Financial Interests In Medical Research,” 40 J. L. Med. & Ethics 467 (2012).
[6] Department of Health and Human Services, “Responsibility of Applicants for Promoting Objectivity in Research for Which Public Health Service Funding is Sought and Responsible Prospective Contractors,” 76 Fed. Reg. 53256 (Aug. 25, 2011).
[7] Id. at 53258.
[8] Institute of Medicine, Conflict of Interest in Medical Research, Education, and Practice (Washington, D.C.: The National Academies Press, 2009).
[9] Kenneth J. Rothman, “Conflict of interest: the new McCarthyism in science,” 269 J. Am. Med. Ass’n 2782 (1993).
[10] Brian MacMahon, “Epidemiology: another perspective,” 37 Internat’l J. Epidem. 1192, 1192 (2008).
[11] See Thomas P. Stossel, “Has the hunt for conflicts of interest gone too far?” 336 Brit. Med. J. 476 (2008); Nature Publishing Group, “Nothing to see here: based on one company’s past poor publishing practices, a top-tier medical journal misguidedly stigmatizes any paper from industry,” 26 Nature Biotechnol. 476 (2008); Kenneth J. Rothman & S. Evans, “Extra scrutiny for industry funded trials: JAMA’s demand for an additional hurdle is unfair–and absurd, 331 Brit. Med. J. 1350 (2005), and 332 Brit. Med. J. 151 (2006) (erratum).
[12] Laurence J. Hirsch, “Conflicts of Interest, Authorship, and Disclosures in Industry-Related Scientific Publications: The Tort Bar and Editorial Oversight of Medical Journals,” 84 Mayo Clin. Proc. 811 (2009).
