Conducting research among hard-to-reach populations is a difficult endeavor because some of their characteristics are known to be associated with survey nonresponse and panel attrition. In the case of the Parchemins study, which followed undocumented migrants over their process of regularization and during the first years of regularized life in Geneva, we underscore the difficulties in recruiting and keeping respondents who come from such a hard-to-reach population. Factors hindering their participation include the fear of being denounced as undocumented, missing time due to high workload, health issues, or language problems. Using unique data from the recruitment and the follow-up processes, we demonstrate that investing high resources and time is particularly beneficial to reach such a population and to reduce attrition over successive data collection waves. In addition, we present the strategies adopted to draw a convenient sample from our targeted population, which mainly relies on generating trust.

The American Family Health Study (AFHS) collected family health and fertility data from a national probability sample of persons aged 18-49 between September 2021 and May 2022, using web and mail exclusively. In July 2022, we surveyed AFHS respondents and gauged their willingness to become part of a national web panel that would create novel longitudinal data on these topics. We focus on predictors of willingness to participate, identifying the potential selection bias that this type of approach may introduce. We found that efforts of this type to create a national web panel may introduce potential selection bias in estimates based on the panel respondents, with individuals having higher socio-economic status being more cooperative. Thus, alternative recruitment strategies and re-weighting of the subsample may be needed to further reduce selection bias. We present methodological implications of our results, limitations of our approach, and suggestions for further research on this topic.

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is believed to fail in recognizing real data. To reduce that risk, some authors have proposed benchmark tables that put a limit on the number of exogenous factors given a certain number of cases. Many applied researchers looking for methodological guidance have since adhered to these tables. We argue that fears of inferential breakdown in QCA due to an "unfavorable" case-to-factor ratio are without foundation. What is more, we demonstrate that these benchmarks induce more fallacious inferences than they prevent. For valid causal inference, researchers are better off relying on the current state of knowledge in their respective fields.

Researchers have become increasingly interested in response times to survey items as a measure of cognitive effort. We used machine learning to develop a prediction model of response times based on 41 attributes of survey items (e.g., question length, response format, linguistic features) collected in a large, general population sample. The developed algorithm can be used to derive reference values for expected response times for most commonly used survey items.

Audit correspondence studies are field experiments that test for discriminatory behavior in active markets. Researchers measure discrimination by comparing how responsive individuals ("audited units") are to correspondences from different types of people. This paper elaborates on the tradeoffs researchers face between sending audited units only one correspondence and sending them multiple correspondences, especially when including less common identity signals in the correspondences. We argue that when researchers use audit correspondence studies to measure discrimination against individuals that infrequently interact with audited units, they raise the risk that these audited units become aware they are being studied or otherwise act differently. We also argue that sending multiple pieces of correspondence can increase detection risk. We present the result of an audit correspondence study that demonstrates how detection can occur for these reasons, leading to significantly attenuated (biased towards zero) estimates of discrimination.

Conducting field research with a vulnerable population is difficult under the most auspicious conditions, and these difficulties only increase during a pandemic. Here, we describe the practical challenges and ethical considerations surrounding a recent data collection effort with a high-risk population during the COVID-19 pandemic. We detail our strategies related to research design, site selection, and ethical review.

This study presents results from a randomized experiment in the 2015-2017 National Survey of Family Growth, where a large national sample of U.S. individuals aged 15-49 was randomly assigned to one of two different versions of a survey question about sexual identity (one with three response options, including heterosexual, gay/lesbian, and bisexual, and one adding the option "something else"). Analyses of changes in the associations of sexual identity with alcohol, tobacco, and other drug use across these treatments revealed evidence of significant differences in the associations that remained robust after adjusting for socio-demographics. The results suggest that when individuals choose their sexual identity from a more limited number of response options, the heterogeneity of the sexual identity subgroups increases, weakening estimated associations of sexual identity with these behaviors. Open-ended questions may therefore be necessary to measure sexual identity and estimate its associations with substance use behaviors accurately in surveys.

Natural experiments are often used for answering research questions in which randomization is implausible. Effective recruitment strategies are well documented for observational cohort studies and clinical trials, unlike recruitment methods for time-sensitive natural experiments. In this time-sensitive study of the impact of a minimum wage policy, we aimed to recruit 900 low-wage workers in Minneapolis, Minnesota and Raleigh, North Carolina. We present our recruitment strategies, challenges, and successes for participant screening and enrollment of a difficult-to-reach population.

Body mapping methods are used in sexual and reproductive health studies to encourage candid discussion of sex and sexuality, pleasure and pain, sickness and health, and to understand individuals' perceptions of their bodies. VOICE-D, a qualitative follow-up study to the VOICE trial, developed and used a body map tool in the context of individual in-depth interviews with women in South Africa, Uganda, and Zimbabwe. The tool showed the outline of a nude female figure from the front and back perspective. We asked women to identify, label, and discuss genitalia and other body parts associated with sexual behaviors, pain, and pleasure. Respondents could indicate body parts without having to verbalize potentially embarrassing anatomical terms, enabling interviewers to clarify ambiguous terminology that may have otherwise been open to misinterpretation. Body maps provided women with a non-intimidating way of discussing and disclosing their sexual practices, and minimized miscommunication of anatomical and behavioral terminology.

Researchers based in low- and middle-income countries (LMIC) often cannot access conventional but high-priced ethnographic tools. I developed a low-cost methodology as an exercise in meeting the needs of both LMIC-based researchers and the broader qualitative community. As demonstrated in this proof of concept, ethnographic researchers should strive for a suite of open access software tools and common and affordable hardware to reduce inequities in knowledge generation and dissemination.

To gain a complex understanding of willingness to participate in genomics research among African Americans, we developed a technique specifically suited to studying decision making in a relaxed social setting. The "Qualitative Story Deck," (QSD) is a gamified, structured elicitation technique that allows for the spontaneous creation of scenarios with variable attributes. We used the QSD to create research scenarios that varied on four details (race/ethnicity of the researcher; research goal; biospecimen requested; and institutional affiliation). Participants created scenarios by randomly choosing cards from these categories and provided: (1) a judgement about their willingness to participate in the research project represented; and (2) their thought process in reaching a decision. The QSD has applicability to topics involving decision making or in cases where it would be beneficial to provide vignettes with alternate attributes. Additional benefits include: rapid establishment of rapport and engagement and the facilitation of discussion of little known or sensitive topics.

This study describes a method for collecting data from non-literate, non-English speaking populations. Our Audio computer-assisted self-interview instrument with color-labeled response categories was designed for use with a helper assistance. The study included 30 dyads of non-literate older Hmong respondents and family helpers answering questions about health. Analysis of video recordings identified respondents' problems and helpers' strategies to address these problems. Seven dyads displayed the paradigmatic question-answer sequence for all items, while 23 departed from the paradigmatic sequence at least once. Reports and pauses were the most common signs of problems displayed by respondents. Paraphrasing questions or response categories and providing examples were the most common helper strategies. Future research could assess the impact of helpers' strategies on data quality.

Acquiescence is often defined as the systematic selection of agreeable ("strongly agree") or affirmative ("yes") responses to survey items, regardless of item content or directionality. This definition implies that acquiescence is immune to item characteristics; however, the influence of item characteristics on acquiescence remains largely unexplored. We examined the influence of eight item characteristics on acquiescence in a telephone survey of 400 Latinos and non-Latino whites: qualified wording, mental comparisons, negated wording, unfamiliar terms, ambiguous wording, knowledge accessibility, item length, and polysyllabic wording. Negated and ambiguous wording was associated with reduced acquiescence for the full sample, as well as subsamples stratified by ethnicity and sociodemographic characteristics. This effect was strongest among younger, more educated, and non-Latino white respondents. No other item characteristics had a significant influence on respondent acquiescence. Findings from this study suggest that acquiescence may be affected by interactions between respondent and item characteristics.

Previous research has shown that increasing the size of incentives can increase response rates for probability-based, cross-sectional surveys. However, the effects of incentives on web panels have not been extensively studied. We sought to answer the question: What is the effect of larger, postpaid incentives on (1) response, (2) data quality, and (3) nonresponse bias for individuals in a web panel? We analyzed data from the 2015 and 2016 National Internet Flu Survey, a survey that uses the GfK KnowledgePanel as its sampling frame. We compare panel members who received a postpaid, standard 1,000-point (the equivalent of US$1) incentive in 2015 to panelists who received a larger, 5,000-point (the equivalent of US$5) incentive in 2016. We found that larger incentives were associated with increased interview completion rates with minimal impact on data quality or bias.

Web, or online, probing has the potential to supplement existing questionnaire design processes by providing structured cognitive data on a wider sample than typical qualitative-only question evaluation methods can achieve. One of the practical impediments to the further integration of web probing is the concern of survey managers about how the probes themselves may affect response to other items and to a questionnaire as a whole. This study explores the effects web probes had on response to a self-administered web survey by comparing two rounds of this survey-one without web probes and one with web probes-that were administered to a probability-based panel of approximately 100,000 American adults. While the item response to the probes themselves appears to be related to the way they are formatted, the findings indicate that web probes do not have an overall negative effect on a questionnaire in which they are embedded.

One of the implicit assumptions in survey research is lower response rates by sexual minorities than non-minorities. With rapidly changing public attitudes towards same-sex marriage, we reconsider this assumption. We used data from the 2013 and 2014 National Health Interview Survey (NHIS) that include contact history data for all sample families (n=117,589) as well as sexual orientation information about adults sampled from responding families (n=71,110). We created proxy nonresponse indicators based on contact efforts and reluctance from contact history data and linked them to sexual orientation of the sample adult and simulated nonresponse. The data did not support the assumption: straight adults were more difficult to get cooperation from than non-straights. With female sexual minorities showing higher nonresponse than the male counterpart, special considerations are required. Replication analyses may provide insights into what factors influence study participation decisions, which will inform how nonresponse may impact the accuracy of research findings.

Social network analysis depends on how social ties to others are elicited during interviews, a process easily affected by respondent and interviewer behaviors. We investigate how the number of self-reported important social contacts varied within a single data collection round. Our data come from HAALSI, a comprehensive population-based survey of individuals aged 40 years and older conducted over thirteen months at the Agincourt health and demographic surveillance site in rural South Africa. As part of HAALSI, interviewers elicited detailed egocentric network data. The average number of contacts reported by the 5059 respondents both varied significantly across interviewers and fell over time as the data collection progressed, even after adjusting for respondent, interviewer and respondent-interviewer dyad characteristics. Contact numbers rose substantially after a targeted interviewer intervention. We conclude that checking (and adjusting) for interviewer effects, even within one data collection round, is critical to valid and reliable social network analysis.

Surveys of egocentric networks are especially vulnerable to methods effects. This study combines a true experiment-random assignment of respondents to receive essentially identical questions from either an in-person interviewer or an online survey--with audio recordings of the in-person interviews. We asked over 850 respondents from a general population several different name-eliciting questions. Face-to-face interviews yielded more cooperation and higher quality data but names than did the web surveys. Exploring several explanations, we determine that interviewer differences account for the mode difference: Interviewers who consistently prompted respondents elicited as many alters as did the web survey and substantially more than did less active interviewers. Although both methods effects substantially influenced the volume of alters listed, they did not substantially modify associations of other variables with volume.

Event history calendars (EHCs) are frequently used in social measurement to capture important information about the time ordering of events in people's lives, and enable inference about the relationships of the events with other outcomes of interest. To date, EHCs have primarily been designed for face-to-face or telephone survey interviewing, and few calendar tools have been developed for more private, self-administered modes of data collection. Web surveys offer benefits in terms of both self-administration, which can reduce social desirability bias, and timeliness. We developed and tested a web application enabling the calendar-based measurement of contraceptive method use histories. These measures provide valuable information for researchers studying family planning and fertility behaviors. This study describes the development of the web application, and presents a comparison of data collected from online panels using the application with data from a benchmark face-to-face survey collecting similar measures (the National Survey of Family Growth).