https://github.com/philippdrebes/mscids_cip02
This project aims to extract hiking routes data from the websites SAC, Schweizmobil, and Komoot.
https://github.com/philippdrebes/mscids_cip02
beautifulsoup4 hiking mariadb python selenium switzerland webscraping
Last synced: about 2 months ago
JSON representation
This project aims to extract hiking routes data from the websites SAC, Schweizmobil, and Komoot.
- Host: GitHub
- URL: https://github.com/philippdrebes/mscids_cip02
- Owner: philippdrebes
- License: mit
- Created: 2023-04-14T12:34:38.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-05-19T18:40:37.000Z (about 3 years ago)
- Last Synced: 2025-11-21T03:03:14.499Z (7 months ago)
- Topics: beautifulsoup4, hiking, mariadb, python, selenium, switzerland, webscraping
- Language: Python
- Homepage:
- Size: 7.14 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Hiking Routes in Switzerland
This project aims to extract hiking routes data from the websites SAC, Schweizmobil, and Komoot. The data includes route descriptions, difficulty, distance, elevation change, and other relevant information.
## Prerequisites
To run the project, you need to have Python 3 installed on your machine, as well as the following packages:
- selenium
- beautifulsoup4
- pandas
- mariadb
- plus a few others... (see `requirements.txt`)
We recommend using virtualenv to manage dependencies.
## Installation
1. Clone the repository to your local machine.
2. Create a virtual environment using virtualenv.
3. Activate the virtual environment.
4. Install the dependencies using `pip install -r requirements.txt`.
5. Run the code.
## Repository Structure
The repository is organized as follows:
- `docs`: documentation files
- `src`: source code files
- `database`: database-related code
- `extractors`: website-specific extraction code
- `KomootExtractor.py`: extractor for Komoot website
- `sac`: extractor for SAC website
- `schweizmobil`: extractor for Schweizmobil website
- `SeleniumUtil.py`: utility functions for Selenium web driver
- `loaders`: code to load data into database
- `models`: data models
- `output`: output files
- `transformers`: code to transform data
- `main.py`: main script to run the data extraction process
- `Makefile`: makefile with common tasks
- `requirements.txt`: list of required packages
- `README.md`: this file
## Contributing
If you find any issues or have suggestions for improvement, please feel free to submit a pull request or open an issue.
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.