https://github.com/jackdbd/learning-data-engineering
https://github.com/jackdbd/learning-data-engineering
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/jackdbd/learning-data-engineering
- Owner: jackdbd
- Created: 2024-07-25T09:04:39.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-08-27T15:09:50.000Z (almost 2 years ago)
- Last Synced: 2025-01-17T21:29:35.738Z (over 1 year ago)
- Language: Python
- Size: 353 KB
- Stars: 0
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Learning data engineering
Exercises and projects to learn data engineering.
## Setup & Installation
This projects uses [Nix](https://nixos.org/) and [devenv](https://devenv.sh/) to define a reproducible developer environment.
You can install Nix and devenv by following the instructions in the [devenv documentation](https://devenv.sh/getting-started/).
If you have Nix and devenv installed, the developer environment will be automatically created and activated when you enter the repository root directory (thanks to direnv).
## Repository structure
### `dagster_university`
The `dagster_university` directory contains the [Dagster project](https://docs.dagster.io/guides/dagster/recommended-project-structure) for the course [Dagster Essentials](https://courses.dagster.io/courses/dagster-essentials). The project was scaffolded using the following command (do **not** run this command):
```sh
dagster project from-example \
--example project_dagster_university_start \
--name dagster_university
```
If you want to check the Dagster project developed in that course, enter the `dagster_university` directory and follow [the instructions](./dagster_university/README.md).
## Reference
- [Dagster Essentials](https://courses.dagster.io/courses/dagster-essentials)
- [Dagster & dbt](https://courses.dagster.io/courses/dagster-dbt)
- [dbt Learn](https://www.getdbt.com/dbt-learn)