On June 8, PRIM&R hosted a webinar titled New Ethical Challenges in Experimental Political Science, presented by Scott Desposato, PhD, and Trisha Phillips, PhD.

This webinar explored controversial new trends in political science research, including increased field experimentation and research conducted overseas, and aimed to clarify the IRB’s role in addressing the ethical challenges these protocols often present. Speakers addressed several common challenge areas for IRBs when reviewing political science protocols, such as assessing risks and benefits, granting exemptions, and granting waivers of informed consent. Highlighting the need for continued ethics education in the social sciences, the webinar provided recommendations for future collaboration and communication between IRBs and political scientists to improve processes and avoid future controversies.

After the webinar, speakers answered some of the questions from webinar attendees that time didn’t permit us to answer live. We’re pleased to share those answers with the readers of Ampersand.

Would bribing a foreign official to facilitate approval of a research project make the investigator susceptible to charges under the Foreign Corrupt Practices Act (FCPA)?
Scott Desposato (SD): I’m no lawyer, but it might indeed. This is a great point that I had not considered. The same concern could also apply to studies where researchers are soliciting or paying bribes. These foreign studies might technically be illegal in the United States, and could cause legal problems for scholars and their institutions. It is troubling that scholars are willing to commit crimes in the course of research overseas, when they would not do so here in the United States.

This is a controversial issue and one that is being discussed in many different venues within the discipline. This webinar is one perspective, but there are others out there, such as concern that overly rigorous IRBs will reject ethical proposals that should be approved. The beginning of the presentation mentioned a large increase in experimental research in political science in recent years. Do you consider the unethical cases presented in the slides (Brazil, Montana, etc.) to be troubling outliers among field experiments, or do you believe they are the norm within this methodology?
SD: The most troubling cases are outliers, not the norm. The great majority of experiments are safe and don’t cause any problems. However, outliers can cause real harms, and can also do serious damage to the entire research enterprise. On this last point, it’s easy to imagine one of these outliers being a contributor to political science’s already thorny relations with funding agencies and Congress, as well as hurting public support for and trust of research.

I agree that there are other perspectives, and that important research may be rejected by overly rigorous IRBs. Political scientists are divided as to whether many of the studies presented are ethical or not, and ongoing dialogue among scholars is critical. In addition, navigating the IRB process can be frustrating and slow for safe, valuable, and legitimate research. To me, that’s an argument for better communication and dialogue with IRBs.

Trisha Phillips (TP): This question raises a good point: a lot of this research is really important. Some of these studies experimentally prove discrimination or corruption, and some of these findings are being used to craft policy and change practices.

But it is also important to note that the value of the knowledge that will be generated by the research is only one of many factors that determine whether a study is ethical. IRBs need to consider the benefits, the risks, respect for persons, and justice. Even if the benefits to society are really high, if the study presents problems in other areas, the study still might not be ethical.

This question also highlights the importance of a two-way exchange of information between researchers and IRBs, and this should be happening on both the individual and collective level. The researcher should help the IRB understand the importance of the research, and the IRB should help the researcher understand the ways in which the research might threaten the welfare of subjects, and together they should work towards a design that is appropriate. This conversation should be happening between individual researchers and their institutional IRBs, but it also needs to be happening on the professional level within and between fields. We hope to see more conversations like this in the near future.

Do you recommend IRBs ask researchers about local reviews and approvals for what we would consider exempt research under the Common Rule?
SD: Yes, I think IRBs should encourage researchers to comply with local review. Research that is exempt in the US may not be exempt in other countries.

TP: I agree with Scott. Research that is exempt under our regulations might not be exempt under foreign rules, regulations, or laws. If an IRB, institution, or funding agency decides that its researchers should comply with foreign laws requiring local review and approval, then this should apply to all research subject to these foreign laws.

What do you recommend in a situation where a scholar is seeking exemption for a field experiment because s/he is working with a political party?
SD: I think Trisha and I disagree a little bit on this one. If a political scientist is providing consulting services to a political party, and helping the party answer real questions about messaging and/or mobilization or something like that, and if the party is willing to fund, run, and take responsibility for the randomization, then I think exemption is appropriate. The political party is assuming all liability and can be sued or prosecuted for misbehavior. However, the researcher should certainly advise the party if s/he sees any real risks or ethical issues, and should not be involved in clearly immoral research (helping corrupt elections or finding the best way to buy votes with cash, for example). The researcher should also fully disclose the nature of the researcher/party relationship in any resulting research publications.

In my response above, I am thinking of contexts where the political party is a sophisticated and true third-party independent actor with real research questions and where treatments aren’t illegal or individually harmful. Presumably, in the hypothetical offered above, these conditions are all met. There is an independent political party working with a scholar to test fundraising, turnout, or persuasion messages. The party has every reason to do the study for its own benefit, and we are trusting that the legal system and electoral authorities will keep them from doing anything illegal. We are also trusting that opportunity costs and risks will keep the party from wasting time and money on research that they wouldn’t do on their own.

If the third-party were in fact the researcher’s own political party that they or a close relative had just founded, or a minor party that had been manipulated into conducting the study, then an exemption would not be appropriate. Or, if the researcher had a confidential agreement with the third party and refused to identify who conducted the research (I’ve seen this recently), then I would be concerned that no one was taking any responsibility for the experiment. An exemption here could be problematic.

I’ll confess, lastly, that I find Trisha’s answer below to be very compelling.

TP: Yes, I do see this situation differently. And maybe that’s because Scott and I interpret “responsibility” differently. Even though the party is conducting the randomization, if a researcher asked them to do it, or convinced them to do it one way rather than another, then the researcher bears some responsibility for the intervention as well, especially if the researcher then publishes the data using words like “we did this…” and “our study” and “our results.”  

If the researcher’s involvement in the research project pre-dates the collection of the data, then it does not automatically follow that the study needs to be reviewed; it only means that the study does not clearly meet the criteria for exemption. The IRB should ask for more information about the researcher’s role in the project, and then the IRB needs to wade through the murky categories of what constitutes “research,” when a researcher is “engaged” in research, and when an institution is “engaged” in research. In the end, the IRB might determine that the research is indeed exempt, or it might determine that expedited or full-board review is appropriate.

Beyond the importance of protecting subjects, this is also about institutional complicity in the research project. When a researcher is involved in a project related to his or her job (i.e. s/he will claim credit for the research and publications), then the employer is also involved in the project. This is probably why HHS says that when an institution’s employees perform commercial services for investigators, and these services merit professional recognition and publication privileges, then an institution is considered “engaged” in the research (HHS, Engagement of Institutions in Human Subjects Research, 2008).

Are there ever cultural problems with international political science research? Can you give us an example? How was it resolved?
SD: Two scholars were conducting laboratory experiments in the Middle East. The subjects were playing standard economic/political science games for money. The country had religious norms against gambling, which could have made the subjects very upset and generated social backlash against subjects, researchers, and all scholarship. The scholars had a terrific idea to adapt their design to be about traffic patterns, instead of playing a game for money. Traffic and travel times are, of course, random variables. So the calculus of the subjects was the same, but there was no sense of gambling.

In another case, scholars conducting a study in rural Africa found that compensating subjects unequally, or only compensating subjects, led to anger and divisions in a village. They had to redesign their compensation scheme, holding community meetings and explaining random sampling and compensation in a language appropriate to the context. These scholars also suggest that in some cases, a single donation to the community or to a nonprofit might be a better solution.

Both examples are explained in detail in the edited book I mentioned in the presentation.

You implied that IRBs should not grant a waiver of informed consent unless the researcher can show that informed consent will hurt the research. How could anyone show this? Isn’t this the point of the research?
SD: I don’t think this should be a single necessary condition for granting a waiver, but it is a question the IRB should ask and consider as part of the totality of the circumstances. Here are two ways the scholar might respond: One is that similar research on related questions or in other places might show whether or not the waiver affects the quality of research. Another is that this could be built into the research design. Imagine a study with a target of 10,000 subjects. Why not try the first 500 with informed consent and then 500 with a waiver, and see if there are different results? If the waiver has a dramatic impact, then continue with a waiver of informed consent. If not, then the harm of no informed consent has been limited to just 500 of 10,000 subjects.

In the webinar, you discussed ethical issues presented by role-playing field experiments. Can you give an additional example of a role-playing field experiment?
SD: Sure! Subjects have pretended to be drivers committing traffic violations, or constituents needing help from politicians. They have pretended to be graduate students needing appointments, information, or data. They have also pretended to be potential clients interested in forming corporations, or in relocating businesses. There have been many studies where scholars send fake resumes to explore the labor market. In all these cases, they are interacting with subjects who are unaware of the deception. A business study where a scholar pretended to be a restaurant patron who became sick after an expensive dinner actually led to a lawsuit against Columbia University that was settled out of court. I actually wonder whether an under-employed lawyer might be able to file a class-action lawsuit for fraud in some of these field experiment cases.

PRIM&R would like to thank our presenters for their time and expertise!

The recording of this webinar is available for individuals to purchase in PRIM&R’s online store. If you would like to purchase the webinar for group viewing, please download the order form and send it to webinars@primr.org.