An open API service indexing awesome lists of open source software.

https://github.com/ipums/ipumsr

Request, download, and read IPUMS data in R
https://github.com/ipums/ipumsr

Last synced: 5 months ago
JSON representation

Request, download, and read IPUMS data in R

Awesome Lists containing this project

README

          

---
output: github_document
---

```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-"
)
```

# ipumsr

[![Project
Status:Active](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/ipumsr)](https://CRAN.R-project.org/package=ipumsr)
[![R build
status](https://github.com/ipums/ipumsr/workflows/R-CMD-check/badge.svg)](https://github.com/ipums/ipumsr/actions/workflows/R-CMD-check.yaml)
[![Codecov test
coverage](https://codecov.io/gh/ipums/ipumsr/branch/main/graph/badge.svg)](https://app.codecov.io/gh/ipums/ipumsr?branch=main)

ipumsr provides an R interface for handling
[IPUMS](https://www.ipums.org) data, allowing users to:

- Easily read files downloaded from the IPUMS extract system

- Request data, download files, and get metadata from certain IPUMS
collections

- Interpret and process data using the contextual information that is
included with many IPUMS files

## Installation

To install the package from CRAN, use

```{r, eval=FALSE}
install.packages("ipumsr")
```

To install the development version of the package, use

```{r, eval=FALSE}
remotes::install_github("ipums/ipumsr")
```

## What is IPUMS?

[IPUMS](https://www.ipums.org) is the world's largest
publicly available population database, providing census and survey data
from around the world integrated across time and space. IPUMS
integration and documentation make it easy to study change, conduct
comparative research, merge information across data types, and analyze
individuals within family and community context. Data and services are
available free of charge.

IPUMS consists of multiple projects, or *collections*, that provide
different data products.

- **Microdata** projects distribute data for individual survey units,
like people or households.
- **Aggregate data** projects distribute summary tables of aggregate
statistics for particular geographic units along with corresponding
GIS mapping files.

ipumsr supports different levels of functionality for each IPUMS project, as
summarized in the table below.

```{r}
#| echo: false

checkmark <- function() {
"Check mark"
}

# Difficult to use `proj_config()` to auto-produce this table, as we
# group projects together (e.g. Global Health projects), which is not
# reflected in `proj_config()`. Currently we will need to keep this up-to-date
# manually.
tbl_config <- list(
list(
img = "IPUMS USA logo",
proj = "IPUMS USA",
type = "Microdata",
desc = "U.S. Census and American Community Survey microdata (1850-present)",
read = checkmark(),
request = checkmark(),
metadata = ""
),
list(
img = "IPUMS CPS logo",
proj = "IPUMS CPS",
type = "Microdata",
desc = "Current Population Survey microdata including basic monthly surveys and supplements (1962-present)",
read = checkmark(),
request = checkmark(),
metadata = ""
),
list(
img = "IPUMS International logo",
proj = "IPUMS International",
type = "Microdata",
desc = "Census microdata covering over 100 countries, contemporary and historical",
read = checkmark(),
request = checkmark(),
metadata = ""
),
list(
img = "IPUMS NHGIS logo",
proj = "IPUMS NHGIS",
type = "Aggregate Data",
desc = "Tabular U.S. Census data and GIS mapping files (1790-present)",
read = checkmark(),
request = checkmark(),
metadata = checkmark()
),
list(
img = "IPUMS IHGIS logo",
proj = "IPUMS IHGIS",
type = "Aggregate Data",
desc = "Tabular and GIS data from population, housing, and agricultural censuses around the world",
read = checkmark(),
request = checkmark(),
metadata = checkmark()
),
list(
img = "IPUMS Time Use logo",
proj = "IPUMS Time Use",
type = "Microdata",
desc = "Time use microdata from the U.S. (1930-present) and thirteen other countries (1965-present)",
read = checkmark(),
request = checkmark(),
metadata = ""
),
list(
img = "IPUMS Health Surveys logo",
proj = "IPUMS Health Surveys",
type = "Microdata",
desc = paste0(
"Microdata from the U.S. ",
"National Health Interview Survey (NHIS) (1963-present) and ",
"Medical Expenditure Panel Survey (MEPS) (1996-present)"
),
read = checkmark(),
request = checkmark(),
metadata = ""
),
list(
img = "IPUMS Global Health logo",
proj = "IPUMS Global Health",
type = "Microdata",
desc = paste0(
"Health survey microdata for low- and middle-income countries, including ",
"harmonized data collections for Demographic and Health Surveys (DHS) ",
"and Performance Monitoring for Action (PMA) surveys"
),
read = checkmark(),
request = "DHS only",
metadata = ""
),
list(
img = "IPUMS Higher Ed logo",
proj = "IPUMS Higher Ed",
type = "Microdata",
desc = "Survey microdata on the science and engineering workforce in the U.S. from 1993 to 2013",
read = checkmark(),
request = "",
metadata = ""
)
)

knitr::kable(
purrr::map_dfr(tbl_config, tibble::as_tibble),
table.attr = "class=\'table-hover table-proj-summary\'",
format = "html",
escape = FALSE,
col.names = c(
"",
"",
"Data Type",
"Description",
"Read Data Extracts",
"Request & Download Data",
"Browse Metadata"
),
align = c("clllccc")
)
```

ipumsr uses the [IPUMS API](https://developer.ipums.org/) to submit data
requests, download data extracts, and get metadata, so the scope of
functionality generally corresponds to that [available via the API](https://developer.ipums.org/docs/v2/apiprogram/apis/). As
the IPUMS team extends the API to support more functionality for more
projects, we aim to extend ipumsr capabilities accordingly.

## Getting started

If you're new to IPUMS data, learn more about what's available through
the [IPUMS Projects Overview](https://www.ipums.org/overview). Then, see
`vignette("ipums")` for an overview of how to obtain IPUMS data.

The package vignettes are the best place to explore what ipumsr has to offer:

- To read IPUMS data extracts into R, see `vignette("ipums-read")`.

- To interact with the IPUMS extract and metadata system via the IPUMS API,
see `vignette("ipums-api")`.

- For additional details about microdata and aggregate data extract requests,
see `vignette("ipums-api-micro")` and `vignette("ipums-api-agg")`.

- To work with labelled values in IPUMS data, see `vignette("value-labels")`.

- For techniques for working with large data extracts, see
`vignette("ipums-bigdata")`.

The [IPUMS support website](https://www.ipums.org/support) also houses
many project-specific R-based [training
exercises](https://www.ipums.org/support/exercises). However, note that some
of these exercises may not be be up to date with ipumsr's current
functionality.

## Related work

- The [survey](http://r-survey.r-forge.r-project.org/survey/) and
[srvyr](https://github.com/gergness/srvyr/) packages can help you
incorporate IPUMS survey weights into your analysis for various
survey designs.

- See [haven](https://haven.tidyverse.org/index.html) for more
information about value labels and labelled vectors

- [hipread](https://github.com/ipums/hipread) underlies the
hierarchical file reading functions in ipumsr

## Getting help + contributing

We greatly appreciate feedback and development contributions. Please
submit any bug reports, pull requests, or other suggestions on
[GitHub](https://github.com/ipums/ipumsr/issues). Before contributing,
please be sure to read the [Contributing
Guidelines](https://tech.popdata.org/ipumsr/CONTRIBUTING.html)
and the
[Code of Conduct](https://tech.popdata.org/ipumsr/CODE_OF_CONDUCT.html).

If you have general questions or concerns about IPUMS data, check out
our [user forum](https://forum.ipums.org) or send an email to
[ipums\@umn.edu](mailto:ipums@umn.edu){.email}.