peer-review

As someone who has spent his career following and upholding the scientific method as an author and journal editor responsible for determining publication acceptance versus rejection, I’ve been struck by a lack of adherence to the scientific method in animal advocacy work. This occurs in the generation of advocacy strategies, their evaluation, and the interpretation of such strategies. Here I discuss a recent widely publicized study, the “Humane Research Council Study of Current and Former Vegetarians and Vegans,” and the ways in which this investigation didn’t follow the accepted scientific method.

Several recent such studies have been put forward and I’ve observed animal advocates indicate that they’ve changed their approach as a result. I caution against relying too heavily on these studies to inform advocacy because they often don’t adhere to key tenets of the scientific method and thus the conclusions we can draw from them are limited.

Before getting into the HRC study details, here are some “basics” of the scientific method that are often neglected in animal advocacy research:

  1. Generate a set of testable hypotheses that are based on a specific theory or conceptual model. If a study does not have any expectations for what the data analysis will tell us, then hypotheses can’t be tested and the study should be considered exploratory.
  2. Operationally define and measure the constructs of interest in a clear manner using accepted definitions.
  3. Draw conclusions supported by the data analysis and avoid going beyond the evidence.
  4. Subject the research to a peer review process such that other experts in the field can offer their input and ensure that the research is scientifically sound and unbiased.

For those not familiar with the HRC study, this was an epidemiological survey of over 11,000 respondents to better understand factors that lead to the long-term adoption of a “vegan” or vegetarian diet. For purposes of statistical analyses, “vegans” (defined as those adopting a plant-based diet) and vegetarians were grouped together. A key finding was that there was more than five times the number of former vegetarians/vegans than current vegetarians/vegans, suggesting that adherence to such diets is low and problematic.

Based on a series of analyses comparing current versus former vegans/vegetarians, the authors drew several conclusions including that we should place greater emphasis on reductions in animal consumption (relative to complete cessation), and we should focus more on the “how” of vegetarianism/veganism (presumably relative to the “why” of vegetarianism/veganism).

So let’s take a look at the basics of the scientific method I listed earlier as it applies to this study.

Generate a set of testable hypotheses based on a theory or conceptual model. The authors didn’t have specific hypotheses and didn’t lay out their expectations prior to conducting the analyses. Therefore we don’t know what they were expecting when they undertook this study and the interpretation of the results must be tempered with caution.

Operationally define the constructs of interest using accepted definitions. “Veganism” in this study was not defined according to the accepted original definition of The Vegan Society: “Veganism is a way of living which seeks to exclude, as far as is possible and practicable, all forms of exploitation of, and cruelty to, animals for food, clothing or any other purpose.” The authors defined veganism according to diet, and veganism is not a diet. Moreover, vegans and vegetarians were grouped together despite the fact that these two groups are very different with respect to their motivations for rejecting animal use.

If the true definitions of “vegan” and “vegetarian” were used and analyses comparing retention of meatless diets, I’m confident that vegans would have extremely high rates of continuing their meatless diets relative to vegetarians. The conclusion for that analysis might be that we should directly promote veganism rather than incremental steps towards that goal.

Draw conclusions supported by the data analysis. Interestingly, the authors’ conclusions that we should place greater focus on reducing meat intake (relative to ending meat intake) and the “how” of making a meatless transition (relative to the “why”) seem to run counter to what the results actually show. The most important variables that determined whether one remained a vegan/vegetarian is whether they maintained their diet for reasons related to animal rights, environmental rights, or social justice. More specifically, those indicating that they followed a vegan/vegetarian diet due to “animal protection,” “feelings of disgust about meat/animal products,” “concern for the environment,” and “social justice or world hunger” were most likely to continue with their meat-free diet. A separate analysis showed that the primary challenge to maintaining a meat-free diet among study participants was that they didn’t see vegetarianism/veganism as part of their identity.

Across each of these sets of analyses, variables that refer to empathy for others were more strongly linked to continued adherence to a meat-free diet than self-interested concerns such as health, cost, taste, social influence, religious/spiritual beliefs, trendiness, or cravings. When we do it for other animals, we are less likely to revert back to consuming animal products. In other words, the study suggests that we should promote veganism according to its original definition of minimizing harm to others. And we should empower people to proudly identify with being vegan and unashamedly eliminate animal products from their lives. In sum, the results suggest that we should be placing greater focus on the “why” of veganism (ending cruelty towards animals), place more emphasis on encouraging others to end all harm done to other animals (not reducing meat intake), and work to increase identification with veganism (not weaken it).

Subject the research to a peer review process. If the HRC study had undergone scientific review, at least some of the issues I note above and others would be raised and the authors would need to respond to these concerns in a revised manuscript (if invited to resubmit) for it to be published. Peer review is what gives studies legitimacy and shows that they rise above a threshold of scientific merit, helps eliminate the impact that researcher bias may have on how results are interpreted, and assists in improving the science overall. I recognize that there may be limited publication options for a study of this nature in scientific journals, though we must find a way for more rigorous scientific review for such work to be considered impactful.

Clearly the data gathered here is potentially valuable and can help answer important questions. The value of these efforts can be enhanced enormously if the questions and interpretations undergo rigorous scrutiny and feedback from others in the scientific and vegan communities. It is also important for us who advocate for animals to consume research with a basic understanding that research is a human process and we must be aware of its limitations.

casey-180x180

In addition to his work managing Vegan Publishers, Casey Taft is Professor of Psychiatry at Boston University School of Medicine. He’s an internationally recognized researcher in the areas of trauma and the family, winning prestigious awards for his work from the International Society for Traumatic Stress Studies, the Institute on Violence, Abuse and Trauma, and the Centers for Disease Control and Prevention. He has published over 100 journal articles, book chapters, and scientific reports, is Associate Editor of the journal “Psychological Trauma: Theory, Research, Practice, and Policy,” and is on the Board of Directors of “The War Writers’ Campaign.” Casey has also chaired an American Psychological Association task force on trauma in the military and has consulted with the United Nations on preventing violence and abuse globally. He sees the prevention of violence towards animals as a natural extension of this work.