https://github.com/sodascience/synthetic_youth_pilot
Synthetic data pilot for YOUth study questionnaires, using metasyn
https://github.com/sodascience/synthetic_youth_pilot
questionnaire-survey synthetic-data youth-data
Last synced: 8 months ago
JSON representation
Synthetic data pilot for YOUth study questionnaires, using metasyn
- Host: GitHub
- URL: https://github.com/sodascience/synthetic_youth_pilot
- Owner: sodascience
- License: mit
- Created: 2025-05-07T09:56:56.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-07-08T09:34:40.000Z (11 months ago)
- Last Synced: 2025-08-28T05:15:50.543Z (10 months ago)
- Topics: questionnaire-survey, synthetic-data, youth-data
- Language: Python
- Homepage:
- Size: 84 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# YOUth pilot privacy-friendly synthetic data
This repository implements a pilot for creating privacy-friendly questionnaire datasets from the YOUth cohort. It is built on [metasyn](https://github.com/sodascience/metasyn) with the [disclosure control plugin](https://github.com/sodascience/metasyn-disclosure-control).
## Installation
To install the dependencies of this project, follow the following steps:
1. We use [uv](https://docs.astral.sh/uv) to manage dependencies and environments. Install it first.
2. Clone this repository.
3. Instantiate the environment by running `uv sync` from this folder.
## Synthesizing data
1. Obtain the following datasets from the YOUth study and put them in the `raw_data` folder: `CECPAQ_2.csv`, `M_DEMOGRAFY_1.csv`, `P_DEMOGRAFY_1.csv`, `P_LIFSTYLE_1_MED_STOREY.csv`, `P_LIFSTYLE_1_MEDICATIONY.csv`, `P_LIFSTYLE_1.csv`, `Q_1.csv`
2. Obtain the following metadata files and put them in the `raw_data\metadata` folder: `YOUth_baby_en_kind-metadata.csv`, `YOUth_baby_en_kind-valuelabels.csv`.
3. Create the synthetic data by running `uv run synthesize.py`
Now, the folders `output/csv` and `output/gmf` should be populated with synthetic data and metadata, respectively:
```
📁 synthetic_youth_pilot/
├── 📖 README.md
├── 📄 test_analysis.py
├── 📄 synthesize.py
├── pyproject.toml
├── uv.lock
├── 📁 raw_data/
│ ├── 📜 CECPAQ_2.csv
│ ├── 📜 M_DEMOGRAFY_1.csv
│ ├── 📜 P_DEMOGRAFY_1.csv
│ ├── 📜 P_LIFSTYLE_1.csv
│ ├── 📜 P_LIFSTYLE_1_MEDICATIONY.csv
│ ├── 📜 P_LIFSTYLE_1_MED_STOREY.csv
│ ├── 📜 Q_1.csv
│ └── 📁 metadata/
│ ├── 📜 YOUth_baby_en_kind-metadata.csv
│ └── 📜 YOUth_baby_en_kind-valuelabels.csv
└── 📁 output/
├── 📁 csv/
│ ├── 📜 CECPAQ_2.csv
│ ├── 📜 M_DEMOGRAFY_1.csv
│ ├── 📜 P_DEMOGRAFY_1.csv
│ ├── 📜 P_LIFSTYLE_1.csv
│ ├── 📜 P_LIFSTYLE_1_MEDICATIONY.csv
│ ├── 📜 P_LIFSTYLE_1_MED_STOREY.csv
│ └── 📜 Q_1.csv
└── 📁 gmf/
├── 📜 CECPAQ_2.json
├── 📜 M_DEMOGRAFY_1.json
├── 📜 P_DEMOGRAFY_1.json
├── 📜 P_LIFSTYLE_1.json
├── 📜 P_LIFSTYLE_1_MEDICATIONY.json
├── 📜 P_LIFSTYLE_1_MED_STOREY.json
└── 📜 Q_1.json
5 directories, 28 files
📖README 📜Data 📄Code 📁Folder
```
_(Made with [`scitree`](https://github.com/J535D165/scitree))_
## Test analysis
This repo includes a test analysis on both the real and synthetic data to display medication use by age bracket. You can find the analysis in the file [test_analysis.py](./test_analysis.py). To run this analysis, run `uv run test_analysis.py`. It will show something like the following:
```
Paracetamol use in real data:
Age: 10 - 19 | ____ ███████████████████
Age: 20 - 29 | ____ ████████████████
Age: 30 - 39 | ____ ███████████████
Age: 40 - 49 | ____ ██████████████
Age: 50 - 59 | ____ █████████
Paracetamol use in synthetic data:
Age: 10 - 19 | 0.81 ████████████████████
Age: 20 - 29 | 0.81 ████████████████████
Age: 30 - 39 | 0.78 ███████████████████
Age: 40 - 49 | 0.74 ██████████████████
Age: 50 - 59 | 0.81 ████████████████████
```
_(numbers redacted & bars fuzzed in real data analysis for privacy)_
Three things are noteworthy here:
1. The analysis code is __exactly__ the same between the synthetic and real analyses
2. The ranges of the individual variables (age and paracetamol use) are similar
3. The relation between age and paracetamol use is removed from the synthetic data
## Contact
This is a project by the [ODISSEI Social Data Science team](https://odissei-soda.nl/). Do you have questions, suggestions, or remarks on the technical implementation? Create an issue in the issue tracker or feel free to contact [Erik-Jan van Kesteren](https://github.com/vankesteren).
