https://github.com/correlaid/schule-ein-interface-geben
https://github.com/correlaid/schule-ein-interface-geben
Last synced: 9 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/correlaid/schule-ein-interface-geben
- Owner: CorrelAid
- Created: 2025-07-16T15:55:12.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2025-08-26T22:38:57.000Z (11 months ago)
- Last Synced: 2025-08-27T05:51:51.680Z (11 months ago)
- Language: Jupyter Notebook
- Size: 3.97 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# SeGg Data Pipeline
- Bucket URL: https://cdl-segg.fra1.cdn.digitaloceanspaces.com
- See `exploration.ipynb` for examples on how to load the data and simple descriptive statistics.
- Main pipeline script: `pipeline.py`. Stages are described in the log messages.
- Also see `/tests`, e.g. `tests/test_post_main_scraping.py` to get a feel for the data and how it is structured.
## Misc. info about data Sources
All sources have a schema in `lib/src/lib/models.py`
### Knowledge Modules / Posts
- SeGg uses the Elementor website builder, which allows designing posts via a GUI (the output is stored in the API as HTML).
- There are knowledge modules as posts with:
- A main section
- An optional download section
- An optional “More Info” section
- If a download section exists, it refers to a dedicated chapter in the SV archive: [https://meinsvwissen.de/sv-archiv/](https://meinsvwissen.de/sv-archiv/).
- The “More Info” section contains links to other posts and other chapters in the SV archive.
- The main section can include various media (see `lib/src/lib/models.py` and `exploration.ipynb` for an overview)