Too Anecdotal to Be True? Mechanical Turk Is Not All Bots and Bad Data: Response to Webb and Tangney (2022)
In response to Webb and Tangney (2022) we call into question the conclusion that data collected on Amazon's Mechanical Turk (MTurk) was "at best-only 2.6% valid" (p. 1). We suggest that Webb and Tangney made certain choices during the study-design and data-collection process that adversely affected the quality of the data collected. As a result, the anecdotal experience of these authors provides weak evidence that MTurk provides low-quality data as implied. In our commentary we highlight best practice recommendations and make suggestions for more effectively collecting and screening online panel data.
The Burden for High-Quality Online Data Collection Lies With Researchers, Not Recruitment Platforms
A recent article in (Webb & Tangney, 2022) reported a study in which just 2.6% of participants recruited on Amazon's Mechanical Turk (MTurk) were deemed "valid." The authors highlighted some well-established limitations of MTurk, but their central claims-that MTurk is "too good to be true" and that it captured "only 14 human beings . . . [out of] = 529"-are radically misleading, yet have been repeated widely. This commentary aims to (a) correct the record (i.e., by showing that Webb and Tangney's approach to data collection led to unusually low data quality) and (b) offer a shift in perspective for running high-quality studies online. Negative attitudes toward MTurk sometimes reflect a fundamental misunderstanding of what the platform offers and how it should be used in research. Beyond pointing to research that details strategies for effective design and recruitment on MTurk, we stress that MTurk is not suitable for every study. Effective use requires specific expertise and design considerations. Like all tools used in research-from advanced hardware to specialist software-the tool itself places constraints on what one should use it for. Ultimately, high-quality data is the responsibility of the researcher, not the crowdsourcing platform.
Why DON'T We "Say Her Name"? An Intersectional Model of the Invisibility of Police Violence Against Black Women and Girls
Racialized police violence is a profound form of systemic oppression affecting Black Americans, yet the narratives surrounding police brutality have disproportionately centered on Black men and boys, overshadowing the victimization of Black women and girls. In 2014, the #SayHerName campaign emerged to bring attention to the often-overlooked instances of police brutality against Black women and girls, including incidents of both nonsexual and sexual violence. Despite these efforts, mainstream discourse and psychological scholarship on police violence continue to marginalize the experiences of Black women and girls. This raises a critical question: Why DON'T we "Say Her Name"? This article employs intersectional frameworks to demonstrate how the historic and systemic factors that render Black women and girls particularly vulnerable to police violence also deny their legitimacy as victims, perpetuate their invisibility, and increase their susceptibility to state-sanctioned violence. We extend models of intersectional invisibility by arguing that ideologies related to age, in addition to racial and gender identities, contribute to their marginalization. Finally, we reflect on how psychological researchers can play a pivotal role in dismantling the invisibility of Black women and girls through scientific efforts and advocacy.
The Cognitive Architecture of Infant Attachment
Meta-analytic evidence indicates that the quality of the attachment relationship that infants establish with their primary caregiver has enduring significance for socioemotional and cognitive outcomes. However, the mechanisms by which early attachment experiences contribute to subsequent development remain underspecified. According to attachment theory, early attachment experiences become embodied in the form of cognitive-affective representations, referred to as internal working models (IWMs), that guide future behavior. Little is known, however, about the cognitive architecture of IWMs in infancy. In this article, we discuss significant advances made in the field of infant cognitive development and propose that leveraging insights from this research has the potential to fundamentally shape our understanding of the cognitive architecture of attachment representations in infancy. We also propose that the integration of attachment research into cognitive research can shed light on the role of early experiences, individual differences, and stability and change in infant cognition, as well as open new routes of investigation in cognitive studies, which will further our understanding of human knowledge. We provide recommendations for future research throughout the article and conclude by using our collaborative research as an example.
The Psychological Science of Pandemics: Contributions to and Recommendations for Social, Educational, and Health Policy
Health Communication and Behavioral Change During the COVID-19 Pandemic
The COVID-19 pandemic challenged the public health system to respond to an emerging, difficult-to-understand pathogen through demanding behaviors, including staying at home, masking for long periods, and vaccinating multiple times. We discuss key challenges of the pandemic health communication efforts deployed in the United States from 2020 to 2022 and identify research priorities. One priority is communicating about uncertainty in ways that prepare the public for disagreement and likely changes in recommendations as scientific understanding advances: How can changes in understanding and recommendations foster a sense that "science works as intended" rather than "the experts are clueless" and prevent creating a void to be filled by misinformation? A second priority concerns creating a culturally fluent framework for asking people to engage in difficult and novel actions: How can health messages foster the perception that difficulties of behavior change signal that the change is important rather than that the change "is not for people like me?" A third priority entails a shift from communication strategies that focus on knowledge and attitudes to interventions that focus on norms, policy, communication about policy, and channel factors that impair behavior change: How can we move beyond educating and correcting misinformation to achieving desired actions?
Diversity Is Diverse: Social Justice Reparations and Science
Because the term "diversity" has two related but different meanings, what authors mean when they use the term is inherently unclear. In its broad form, it refers to vast variety. In its narrow form, it refers to human demographic categories deemed deserving of special attention by social justice-oriented activists. In this article, I review Hommel's critique of Roberts et al. (2020), which, I suggest, essentially constitutes two claims: that Roberts et al.'s (2020) call for diversity in psychological science focuses exclusively on the latter narrow form of diversity and ignores the scientific importance of diversity in the broader sense, and ignoring diversity in the broader sense is scientifically unjustified. Although Hommel's critique is mostly justified, this is not because Roberts et al. (2020) are wrong to call for greater social justice-oriented demographic diversity in psychology but because Hommel's call for the broader form of diversity subsumes that of Roberts et al. (2020) and has other aspects critical to creating a valid, generalizable, rigorous, and inclusive psychological science. In doing so, I also highlight omissions, limitations, and potential downsides to the narrow manner in which psychology and the broader academy are currently implementing diversity, equity, and inclusion.
Dealing With Diversity in Psychology: Science and Ideology
In the spirit of America's Shakespeare, August Wilson (1997), I have written this article as a testimony to the conditions under which I, and too many others, engage in scholarly discourse. I hope to make clear from the beginning that although the ideas presented here are not entirely my own-as they have been inherited from the minority of scholars who dared and managed to bring the most necessary, unpalatable, and unsettling truths about our discipline to the broader scientific community-I do not write for anyone but myself and those scholars who have felt similarly marginalized, oppressed, and silenced. And I write as a scholar, meaning simply that I believe that race-and racism-affects the sociopolitical conditions in which humans, and scholars, develop their thoughts, feelings, and actions. I believe that it is important for scholars to have a basic understanding of these conditions, as well as the landmines and pitfalls that define them, as they shape how research is conducted, reviewed, and disseminated. I also believe that to evolve one's discipline into one that is truly robust and objective, it must first become diverse and self-aware. Any effort to suggest otherwise, no matter how scholarly it might present itself, is intellectually unsound.
Toward a Psychology of Ideas Rather Than Demographics: Commentary on Hommel (2024)
The public will rightly not value a science that is more concerned with demographic population matching than with ideas. Taking further steps in the direction of identity politics will reduce public confidence in psychology's conclusions and reduce trust and respect. If psychology embraces demographic quotas, there will be self-selection out of the discipline, and that self-selection will harm our science.
The Myth of the Need for Diversity Among Subjects in Theory-Testing Research: Comments on "Racial Inequality in Psychological Research" by Roberts et al. (2020)
Roberts and colleagues focus on two aspects of racial inequality in psychological research, namely an alleged underrepresentation of racial minorities and the effects attributed to this state of affairs. My comment focuses only on one aspect, namely the assumed consequences of the lack of diversity in subject populations. Representativeness of samples is essential in survey research or applied research that examines whether a particular intervention will work for a particular population. Representativeness or diversity is not necessary in theory-testing research, where we attempt to establish laws of causality. Because theories typically apply to all of humanity, all members of humanity (even American undergraduates) are suitable for assessing the validity of theoretical hypotheses. Admittedly, the assumption that a theory applies to all of humanity is also a hypothesis that can be tested. However, to test it, we need theoretical hypotheses about specific moderating variables. Supporting a theory with a racially diverse sample does not make conclusions more valid than support from a nondiverse sample. In fact, cause-effect conclusions based on a diverse sample might not be valid for any member of that sample.
Dealing With Diversity in Psychology: Science or Ideology?
The increasing use of political activist arguments and reasoning in scientific communication about diversity is criticized. Based on an article of Roberts et al. (2020) on "racial inequality in psychological research," three hallmarks of the intrusion of activist thinking into science are described: blindness to the multidimensional nature of diversity, the failure to distinguish psychological mechanisms from the impact of moderators, and a blindness to agency as an explanation for psychological observations. It is argued that uncritically accepting and introducing political activist arguments into science is likely to damage scientific freedom and independence.
Papers Involved in the December 2022 APS Vote of No Confidence in the Editor-in-Chief of
What Happens When Payments End? Fostering Long-Term Behavior Change With Financial Incentives
Financial incentives are widely used to get people to adopt desirable behaviors. Many small landholders in developing countries, for example, receive multiyear payments to engage in conservation behaviors, and the hope is that they will continue to engage in these behaviors after the program ends. Although effective in the short term, financial incentives rarely lead to long-term behavior change because program participants tend to revert to their initial behaviors soon after the payments stop. In this article, we propose that four psychological constructs can be leveraged to increase the long-term effectiveness of financial-incentive programs: motivation, habit formation, social norms, and recursive processes. We review successful and unsuccessful behavior-change initiatives involving financial incentives in several domains: public health, education, sustainability, and conservation. We make concrete recommendations on how to implement the four above-mentioned constructs in field settings. Finally, we identify unresolved issues that future research might want to address to advance knowledge, promote theory development, and understand the psychological mechanisms that can be used to improve the effectiveness of incentive programs in the real world.
Taboos and Self-Censorship Among U.S. Psychology Professors
We identify points of conflict and consensus regarding (a) controversial empirical claims and (b) normative preferences for how controversial scholarship-and scholars-should be treated. In 2021, we conducted qualitative interviews ( = 41) to generate a quantitative survey ( = 470) of U.S. psychology professors' beliefs and values. Professors strongly disagreed on the truth status of 10 candidate taboo conclusions: For each conclusion, some professors reported 100% certainty in its veracity and others 100% certainty in its falsehood. Professors more confident in the truth of the taboo conclusions reported more self-censorship, a pattern that could bias perceived scientific consensus regarding the inaccuracy of controversial conclusions. Almost all professors worried about social sanctions if they were to express their own empirical beliefs. Tenured professors reported as much self-censorship and as much fear of consequences as untenured professors, including fear of getting fired. Most professors opposed suppressing scholarship and punishing peers on the basis of moral concerns about research conclusions and reported contempt for peers who petition to retract papers on moral grounds. Younger, more left-leaning, and female faculty were generally more opposed to controversial scholarship. These results do not resolve empirical or normative disagreements among psychology professors, but they may provide an empirical context for their discussion.
New Insights on Expert Opinion About Eyewitness Memory Research
Experimental psychologists investigating eyewitness memory have periodically gathered their thoughts on a variety of eyewitness memory phenomena. Courts and other stakeholders of eyewitness research rely on the expert opinions reflected in these surveys to make informed decisions. However, the last survey of this sort was published more than 20 years ago, and the science of eyewitness memory has developed since that time. Stakeholders need a current database of expert opinions to make informed decisions. In this article, we provide that update. We surveyed 76 scientists for their opinions on eyewitness memory phenomena. We compared these current expert opinions to expert opinions from the past several decades. We found that experts today share many of the same opinions as experts in the past and have more nuanced thoughts about two issues. Experts in the past endorsed the idea that confidence is weakly related to accuracy, but experts today acknowledge the potential diagnostic value of initial confidence collected from a properly administered lineup. In addition, experts in the past may have favored sequential over simultaneous lineup presentation, but experts today are divided on this issue. We believe this new survey will prove useful to the court and to other stakeholders of eyewitness research.
"When" Versus "Whether" Gender/Sex Differences: Insights From Psychological Research on Negotiation, Risk-Taking, and Leadership
We present a conceptual framework of situational moderators of gender/sex effects in negotiation, risk-taking, and leadership-three masculine-stereotypic domains associated with gender/sex gaps in pay and authority. We propose that greater situational ambiguity and higher relevance and salience of gender/sex increase the likelihood of gender/sex-linked behaviors in these domains. We argue that greater ambiguity increases the extent to which actors and audiences must search inwardly (e.g., mental schema, past experience) or outwardly (e.g., social norms) for cues on how to behave or evaluate a situation and thereby widens the door for gender/sex-linked influences. Correspondingly, we propose that gender/sex effects on behavior and evaluations in these domains will be more likely when gender/sex is more relevant and salient to the setting or task. We propose further that these two situational moderators may work jointly or interactively to influence the likelihood of gender/sex effects in negotiation, risk-taking, and leadership. We conclude by discussing applications of our conceptual framework to psychological science and its translation to practice, including directions for future research.
How Genetic-Conflict Theory Can Inform Studies of Human Nature
Understanding how genetics influences human psychology is something that the evolutionary sciences emphasize. However, the functions of complex genetic influences on behavior have been overlooked in favor of perspectives that posit unitary influences of genes on behavior. One such example is the belief that human growth, development, and behavior are influenced uniformly by their genes even though previous research has highlighted the genetic conflict endemic in these domains. Although much psychological research has robustly documented areas in which we see the footprints of genetic conflict in human behavior, these areas are referred to by different names that prevent researchers from making connections under a unifying framework. In this article, I outline what genetic conflict is and how genetic conflict can provide a unifying framework for psychological investigations of social relationships. I also discuss avenues for future research on genetic conflict in humans and the importance of considering cultural, ecological, and other developmental factors when researching the genetic influences on human behavior.
Happiness Maximization Is a WEIRD Way of Living
Psychological science tends to treat subjective well-being and happiness synonymously. We start from the assumption that subjective well-being is more than being happy to ask the fundamental question: What is the level of happiness? From a cross-cultural perspective, we propose that the idealization of attaining maximum levels of happiness may be especially characteristic of Western, educated, industrial, rich, and democratic (WEIRD) societies but less so for others. Searching for an explanation for why "happiness maximization" might have emerged in these societies, we turn to studies linking cultures to their eco-environmental habitat. We discuss the premise that WEIRD cultures emerged in an exceptionally benign ecological habitat (i.e., faced relatively light existential pressures compared with other regions). We review the influence of the Gulf Stream on the Northwestern European climate as a source of these comparatively benign geographical conditions. We propose that the ecological conditions in which WEIRD societies emerged afforded them a basis to endorse happiness as a value and to idealize attaining its maximum level. To provide a nomological network for happiness maximization, we also studied some of its potential side effects, namely alcohol and drug consumption and abuse and the prevalence of mania. To evaluate our hypothesis, we reanalyze data from two large-scale studies on ideal levels of personal life satisfaction-the most common operationalization of happiness in psychology-involving respondents from 61 countries. We conclude that societies whose members seek to maximize happiness tend to be characterized as WEIRD, and generalizing this across societies can prove problematic if adopted at the ideological and policy level.
Corrigendum to "Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet)?"
Individual-Specific Animated Profiles of Mental Health
How important is the timing of the pretreatment evaluation? If we consider mental health to be a relatively fixed condition, the specific timing (e.g., day, hour) of the evaluation is immaterial and often determined on the basis of technical considerations. Indeed, the fundamental assumption underlying the vast majority of psychotherapy research and practice is that mental health is a state that can be captured in a one-dimensional snapshot. If this fundamental assumption, underlying 80 years of empirical research and practice, is incorrect, it may help explain why for decades psychotherapy failed to rise above the 50% efficacy rate in the treatment of mental-health disorders, especially depression, a heterogeneous disorder and the leading cause of disability worldwide. Based on recent studies suggesting within-individual dynamics, this article proposes that mental health and its underlying therapeutic mechanisms have underlying intrinsic dynamics that manifest across dimensions. Computational psychotherapy is needed to develop individual-specific pretreatment animated profiles of mental health. Such individual-specific animated profiles are expected to improve the ability to select the optimal treatment for each patient, devise adequate treatment plans, and adjust them on the basis of ongoing evaluations of mental-health dynamics, creating a new understanding of therapeutic change as a transition toward a more adaptive animated profile.
A Novel, Network-Based Approach to Assessing Romantic-Relationship Quality
How should romantic-relationship quality be approached psychometrically? This is a complicated theoretical and methodological challenge that we begin to address through three studies. In Study 1a, we identified 25 distinct romantic-relationship categories among 754 items from 26 romantic-relationship-quality instruments with a weak Jaccard index (0.38), indicating that the scales' item content was extremely heterogeneous. Study 1b then demonstrated limited structure validity evidence in 43 scale-development-validation articles of 23 of these 26 instruments. Finally, Study 2 surveyed 587 French-speaking participants in a romantic relationship on romantic-relationship quality. Applying a network-based model, we identified four dimensions, three of which are central to relationship quality. The inferences were mostly limited to French-speaking, monogamous, heterosexual women. To resolve challenges detected in the literature, we recommend a multicountry qualitative approach, more diverse sampling, better definitions of romantic-relationship quality, and a dynamic-systems approach to measuring romantic-relationship quality.