https://github.com/globaldothealth/list
Repository for Global.health: a data science initiative to enable rapid sharing of trusted and open public health data to advance the response to infectious diseases.
https://github.com/globaldothealth/list
covid19 curation epidemiology
Last synced: 4 months ago
JSON representation
Repository for Global.health: a data science initiative to enable rapid sharing of trusted and open public health data to advance the response to infectious diseases.
- Host: GitHub
- URL: https://github.com/globaldothealth/list
- Owner: globaldothealth
- License: mit
- Created: 2020-04-15T16:42:05.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2023-04-17T15:02:36.000Z (over 2 years ago)
- Last Synced: 2024-02-19T16:12:06.412Z (almost 2 years ago)
- Topics: covid19, curation, epidemiology
- Language: TypeScript
- Homepage:
- Size: 729 MB
- Stars: 39
- Watchers: 4
- Forks: 8
- Open Issues: 195
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# Global.health List
[Global.health](https://global.health)'s mission is to enable rapid sharing of trusted and open public health data to
advance the response to infectious diseases.
This repository contains the servers and scripts that support its data curation efforts.
Should you have any questions please feel free to get in touch via: info@global.health
## COVID-19 Data
The data exposed on Global.health was curated using two methods. ~60,000 cases were manually curated by humans analyzing sources and inputting data into spreadsheets. This data was ported from the spreadsheets into the Curator Portal as described [here](https://github.com/globaldothealth/list/blob/main/data-serving/scripts/convert-data#converting-line-list-data). The rest of the data was automatically ingested from sources through a process described [here](https://github.com/globaldothealth/list/tree/main/ingestion/functions#ingestion-functions). Each case is marked as `VERIFIED` if a human has confirmed this data is valid or `UNVERIFIED` if it has not yet been reviewed.
You can tell if a case was imported from the manually created spreadsheets data in a couple of ways. The case will be marked as created by covid19_spreadsheets@googlegroups.com. It will also have a source URL that links to this documentation. The source URL that was used to find data about these cases can be found in the additional sources section of the detailed case view (found by clicking on the table row).
## Frontends
### COVID-19
- [dev](https://dev-data.covid-19.global.health)
- [prod](https://data.covid-19.global.health)
## Daily exports of case data
A daily export of case data can be downloaded from the data portal. The data
is generated using this [script](data-serving/scripts/export-data/README.md), with
this [data dictionary](data-serving/scripts/export-data/data_dictionary.txt).
## CI/CD status
* [**All actions**](https://github.com/globaldothealth/list/actions)
* **Docker images**
* **Tests**
* **Monitoring**
## Components
- [The data service](data-serving/data-service) in `data-serving/data-service` facilitates CRUD operations with the
MongoDB database storing case data.
- [The curator service](verification/curator-service/api) in `verification/curator-service/api` serves as the backend
for the curator portal, which enables curators to view, enter, update, and verify cases; manage data sources and their
ingestion; and manage portal access.
- [The geocoding service](geocoding/location-service) geocodes locations and is
used by the data service, but can be used standalone as well.
- [The curator UI](verification/curator-service/ui) in `verification/curator-service/ui` is the frontend for the curator
portal.
## Developer documentation
### READMEs
- Getting set up
- [Local development](dev/README.md)
- [Production infrastructure set-up and management](aws/README.md)
- Component documentation
- [Authentication & authorization](verification/curator-service/auth.md)
- [Data ingestion functions](ingestion/functions/README.md)
- [Curator service](verification/curator-service/api/README.md)
- [Curator UI](verification/curator-service/ui/README.md)
- [Data service](data-serving/data-service/README.md)
- [Geocoding](data-serving/data-service/src/geocoding/README.md)
- [API](verification/curator-service/api/openapi/openapi.yaml)
- [Load testing](loadtest/README.md)
- Scripts
- Data service
- [Converting legacy CSV data to schema-conformant JSON](data-serving/scripts/convert-data/README.md)
- [Converting & importing legacy data into MongoDb](data-serving/scripts/data-pipeline/README.md)
- [Exporting MongoDb data to CSV/JSON](data-serving/scripts/export-data/README.md)
- [Setting up your MongoDB instance](data-serving/scripts/setup-db/README.md)
- [Curator portal](verification/scripts/README.md)
- How do I...
- [Update the case schema](data-serving/README.md)
- [Rotate secrets](aws/README.md#secrets)
### API docs
- [Curator service](https://data.covid-19.global.health/api-docs)
- [Local curator service](http://localhost:3001/api-docs)
- [Local data service](http://localhost:3000/api-docs)
## Sources licenses and terms of use
This repository and daily data exports are published under the MIT license.
Each automatically ingested data source used has a required license and terms of use attachment, forcing curators to look-up the sources they are setting-up for ingestion.
If you are the owner of a data source included here and would like us to remove data, add or alter an attribution, or add or alter license information, please open an issue on this repository and we will happily consider your request.
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fglobaldothealth%2Flist?ref=badge_large)
## Stats
 
[](https://app.fossa.com/projects/git%2Bgithub.com%2Fglobaldothealth%2Flist?ref=badge_shield)
