A Half Century of Health Data for the U.S. Population: The Integrated Health Interview Series
The U.S. National Health Interview Survey (NHIS) is the world's longest survey time series of health data and a rich source of information on health conditions, behaviors, and care from the 1960s to the present. NHIS public-use files are difficult to use for long-term analysis, due to complex file structure, changes in questionnaire content, and evolving variable names and coding schemes. Researchers at the Minnesota Population Center have created the Integrated Health Interview Series (IHIS) to overcome these problems. IHIS provides access to thousands of consistently coded and well-documented NHIS variables on the Internet and makes it easy to analyze health trends and differentials. IHIS multiplies the value of NHIS data by allowing researchers to make consistent comparisons over half a century and thus to study U.S. health status as a dynamic process. This article describes the main features of IHIS and suggests fruitful avenues for historical research using these invaluable health data.
Big Data: Large-Scale Historical Infrastructure from the Minnesota Population Center
The Minnesota Population Center (MPC) provides aggregate data and microdata that have been integrated and harmonized to maximize crosstemporal and cross-spatial comparability. All MPC data products are distributed free of charge through an interactive Web interface that enables users to limit the data and metadata being analyzed to samples and variables of interest to their research. In this article, the authors describe the integrated databases available from the MPC, report on recent additions and enhancements to these data sets, and summarize new online tools and resources that help users to analyze the data over time. They conclude with a description of the MPC's newest and largest infrastructure project to date: a global population and environment data network.
New Methods of Census Record Linking
The Minnesota Population Center (MPC) has released linked datasets through its NAPP and IPUMS projects, making them readily accessible to researchers. Prior to the availability of complete count census microdata from the MPC, researchers applied various forms of record-linking software. This essay describes the techniques used in the MPC's linking program and briefly compares this technique with those used by other researchers. The key feature of the MPC linking method is the construction of cumulative name similarity scores, based on approximately 2.5 billion record comparisons; we also use support vector mechanics to classify potential links. This article explains modifications made for the final linked datasets and includes a discussion of the role of weighting variables when using linked data.
Mapping America in 1880: The Urban Transition Historical GIS Project
The Urban Transition Historical GIS Project is a new data resource for United States counties and cities that takes advantage of NAPP's 100% digital transcription of records from the 1880 Census. It has developed several additional resources to make possible analysis of social patterns at the level of individuals and households while also taking into account information about their communities. One key contribution is the creation of historically accurate GIS maps showing the boundaries of enumeration districts in 39 major cities. These materials are now publicly available through a web-based mapping system. Addresses of all households in these cities are also being geocoded, a step that will enable spatial analyses of residential patterns at any geographic scale. Preliminary analyses demonstrate the utility of multiple scales and the ability to combine information about individuals with data about their neighborhoods.
The North Atlantic Population Project: Progress and Prospects
The North Atlantic Population Project (NAPP) is a massive database of historical census microdata from European and North American countries. The backbone of the project is the unique collection of completely digitized censuses providing information on the entire enumerated populations of each country. In addition, for some countries, the NAPP includes sample data from surrounding census years. In this article, the authors provide a brief history of the project, describe their progress to data and plans for the future, and discuss some potential implications of this unique data resource for social and economic research.
Harmonizing Disparate Data across Time and Place: The Integrated Spatio-Temporal Aggregate Data Series
In this article, the authors describe a new data infrastructure project being developed at the Minnesota Population Center. The Integrated Spatio-Temporal Aggregate Data Series (ISTADS) will make it easier for researchers to use publicly available aggregate data for the United States over a time span that covers virtually the entire life of the nation: 1790-2012. In addition to facilitating access and ease of use, ISTADS will facilitate the use of these various data sets in mapping and spatial analysis.
Decennial Life Tables for the White Population of the United States, 1790-1900
This article constructs new life tables for the white population of the United States in each decade between 1790 and 1900. Drawing from several recent studies, it suggests best estimates of life expectancy at age 20 for each decade. These estimates are fitted to new standards derived from the 1900-02 rural and 1900-02 overall DRA life tables using a two-parameter logit model with fixed slope. The resulting decennial life tables more accurately represent sex-and age-specific mortality rates while capturing known mortality trends.
War-related excess mortality in The Netherlands, 1944-45: New estimates of famine- and non-famine-related deaths from national death records
Despite there being several estimates for famine-related deaths in the west of The Netherlands during the last stage of World War II, no such information exists for war-related excess mortality among the civilian population from other areas of the country. Previously unavailable data files from Statistics Netherlands allow researchers to estimate the number of war-related excess deaths during the last stage of the war in the whole country. This study uses a seasonal-adjusted mortality model combined with a difference-in-difference approach to estimate the number of excess deaths in the period between January 1944 and July 1945 at a total of close to 91,000 (75%) excess deaths. Almost half of all war-related excess mortality during the last year of the war occurred outside the west.
Simple Strategies for Improving Inference with Linked Data: A Case Study of the 1850-1930 IPUMS Linked Representative Historical Samples
New large-scale linked data are revolutionizing quantitative history and demography. This paper proposes two complementary strategies for improving inference with linked historical data: the use of validation variables to identify higher quality links and a simple, regression-based weighting procedure to increase the representativeness of custom research samples. We demonstrate the potential value of these strategies using the 1850-1930 Integrated Public Use Microdata Series Linked Representative Samples (IPUMS-LRS)-a high quality, publicly available linked historical dataset. We show that, while incorrect linking rates appear low in the IPUMS-LRS, researchers can reduce error rates further using validation variables. We also show how researchers can reweight linked samples to balance observed characteristics in the linked sample with those in a reference population using a simple regression-based procedure.
A New Strategy for Linking U.S. Historical Censuses: A Case Study for the IPUMS Multigenerational Longitudinal Panel
This paper presents a probabilistic method of record linkage, developed using the U.S. full count censuses of 1900 and 1910 but applicable to many sources of digitized historical records. The method links records using a two-step approach, first establishing high confidence matches among men by exploiting a comprehensive set of individual and contextual characteristics. The method then proceeds to link both men and women by leveraging links between households established in the first step. While only the first stage links can be directly comparable to other popular methods in research on the U.S., our method yields both considerably higher linkage rates and greater accuracy while only performing negligibly worse than other algorithms in resembling the target population.
Reconstruction of Birth Histories for the Study of Fertility in the United States, 1830-1910
This paper describes a method to reconstruct complete birth histories for women in the 1900 and 1910 U. S. census IPUMS samples. The method is an extension of an earlier method developed by Luther and Cho (1988). The basic method relies on the number of children ever born, number of children surviving, number of children coresident in the household and age-specific fertility rates for the population to probabilistically assign an "age" to deceased and unmatched children. Modifications include the addition of an iterative Poisson regression model to fine-tune age-specific fertility inputs. The potential of complete birth histories for the study of the U.S. fertility transition is illustrated with a few examples.
Introduction to Special Issues on Historical Record Linking
Linking the 1940 U.S. Census with Modern Data
The U.S. Census Bureau has created a set of linkable census, survey, and administrative records that provides longitudinal data on the American population across the past eight decades. While these files include modern decennial censuses, Census Bureau surveys, and administrative records files from other federal agencies, the long time span is only possible with the addition of the complete count 1940 Census microdata. In this paper, we discuss the development of this linked data infrastructure and provide an overview of the record linkage techniques used. We primarily focus on the techniques used to produce a beta version of a linkable 1940 Census microdata file and discuss the potential to further document and extend the infrastructure.
The Creation of LIFE-M: The Longitudinal, Intergenerational Family Electronic Micro-Database Project
This paper describes the creation of the Longitudinal, Intergenerational Family Electronic Micro-Database (LIFE-M), a new data resource linking vital records and decennial censuses for millions of individuals and families living in the late 19th and 20th centuries in the United States. This combination of records provides a life-course and intergenerational perspective on the evolution of health and economic outcomes. Vital records also enable the linkage of women, because they contain a crosswalk between women's birth (i.e., "maiden") and married names. We describe (1) the data sources, coverage, and linking sequence; (2) the process and supervised machine-learning methods to linking records longitudinally and across generations; and (3) the resulting linked samples, including linking rates, representativeness, and weights.
Frozen Film and FOSDIC Forms: Restoring the 1960 U.S. Census of Population and Housing
In this article, the authors describe a collaboration of the Minnesota Population Center (MPC), the U.S. Census Bureau, and the National Archives and Records Administration to restore the lost data from the 1960 Census. The data survived on refrigerated microfilm in a cave in Lenexa, Kansas. The MPC is now converting the data to usable form. Once the restored data are processed, the authors intend to develop three new data sources based on the 1960 census. These data will replace the most inadequate sample in the series of public-use census microdata spanning the years from 1850 to 2000, extend the chronological scope of the public census summary files, and provide a powerful new resource for the Census Bureau and its Research Data Centers.
Union Army Veterans, All Grown Up
This paper overviews the research opportunities made possible by a NIA-funded program project, Early Indicators, Intergenerational Processes, and Aging. Data collection began almost three decades ago on 40,000 soldiers from the Union Army in the US Civil War. The sample contains extensive demographic, economic, and medical data from childhood to death. In recent years, a large sample of African-American soldiers and an oversampling of soldiers from major US cities have been added. Hundreds of historical maps containing public health data have been geocoded to place soldiers and their family members in a geospatial context. With newly granted funding, thousands of veterans will be linked to the demographic information available from the census and vital records of their children.
Segregation and Neighborhood Change in Northern Cities: New Historical GIS Data from 1900-1930
Most quantitative research on segregation and neighborhood change in American cities prior to 1940 has utilized data published by the Census Bureau at the ward level. The transcription of census manuscripts has made it possible to aggregate individual records to a finer level, the enumeration district (ED). Advances in Geographic Information Systems (GIS) have facilitated mapping these data, opening new possibilities for historical GIS research. We report here the creation of a mapped public use data set for EDs in ten northern cities for each decade from 1900 to 1930. We illustrate a range of research topics that can now be pursued: recruitment into ethnic neighborhoods, the effects of comprehensive zoning on neighborhood change, and white flight from black neighbors.
Indirectly Estimating International Net Migration Flows by Age and Gender: The Community Demographic Model International Migration (CDM-IM) Dataset
Although data for the total number of international migrant flows is now available, no global dataset concerning demographic characteristics, such as the age and gender composition of migrant flows exists. This paper reports on the methods used to generate the CDM-IM dataset of age and gender specific profiles of bilateral net (not gross) migrant flows. We employ raw data from the United Nations Global Migration Database and estimate net migrant flows by age and gender between two time points around the year 2000, accounting for various demographic processes (fertility, mortality). The dataset contains information on 3,713 net migrant flows. Validation analyses against existing data sets and the historical, geopolitical context demonstrate that the CDM-IM dataset is of reasonably high quality.
Children and Grandchildren of Union Army Veterans: New Data Collections to Study the Persistence of Longevity and Socioeconomic Status Across Generations
This paper introduces four new intergenerational and multigenerational datasets which follow both sons and daughters and which can be used to study the persistence of longevity, socioeconomic status, family structure, and geographic mobility across generations. The data follow the children of Black and White Union Army (US Civil War, 1861-5) veterans from birth to death, linking them to the available censuses. The White samples include an over-sample of children of ex-POWs. A separate collection links grandchildren of White Union Army veterans to their death records. The data were created with high quality manual linkage procedures utilizing a wide variety of records to establish links.