Clinical Advisor is Misadvising

Clinical Advisor claims to be a medical news and opinion site designed for nurse practitioners and physician assistants. Discerning whether that news is real or fake will require some effort from readers.

A recent article offered up an intriguing headline: “Cultural competency training changes behavior among emergency department nurses.” The headline references a poster presentation at the recent DNPs of Color Annual Meeting. The poster shares results from an intervention in which the researcher measured “cultural awareness” at baseline, then once again immediately after watching a one-hour movie called “Ending Racism is Everyone’s Responsibility,” and then a third time four to six weeks after that. The researcher observed that “cultural awareness” scores increased through the three observation periods.

According to the researcher, “the data revealed a behavioral trend and clinical significance.” That characterization is inaccurate for several reasons.

First, experiments must include intervention and control groups to parse out treatment effects from other factors that could shape the outcomes of the intervention group, such as placebo effects (i.e., benefits that arise from the participant’s belief that the intervention is effective) or Hawthorne effects (i.e., participants altering their behavior because it is being observed by researchers). It is entirely possible that the survey responses of the participants changed because they were mindful of being observed or that they hoped for the intervention to succeed.  

Second, attitudinal surveys are not indicative of “behavioral trends.” Surveys and tests are often poor predictors of human behavior, including issues related to race. The popularly utilized Implicit Association Test, which is used to measure “implicit bias,” does not demonstrate any ability to predict racist behavior. Most likely, the survey trends observed in this “study” do not equate to behavioral changes, and indeed it strains credibility and common sense that watching a one-hour video would alter patterns of human behavior.

Third, the claim the results were “clinically significant” is a matter of personal judgement, and one that is contradicted by the objective measure of statistical significance. In fact, the reported statistical significance—a measure that determines the likelihood that the results were determined by some effect rather than random chance—doesn’t even come close to reaching conventional thresholds of significance.

The study and its characterization in Clinical Advisor are emblematic of a healthcare establishment that continues to embrace the pseudoscientific claim that racism permeates American medicine, and the equally dubious idea that it can be expunged through interventions with no history of success. Evidence for these ideas is wanting, even if Clinical Advisor pretends otherwise.