https://github.com/pawelgoj/exemplary_scraping_project
https://github.com/pawelgoj/exemplary_scraping_project
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/pawelgoj/exemplary_scraping_project
- Owner: pawelgoj
- Created: 2022-03-19T22:24:00.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-05-16T21:44:14.000Z (almost 3 years ago)
- Last Synced: 2025-01-27T16:47:46.782Z (3 months ago)
- Language: Python
- Size: 31.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Exemplary scraping project
## Description
Program scraping the https://www.oryxspioenkop.com/. Collect data about destroyed military equipment of Russia and Ukraine.
## Tools
1. Python 3.9
2. Scrapy
3. sqlite3
4. click
5. pandas
## Use
Run following commands:
```
#create database
python cli_client.py create_database --name name_your_database
#run scraping. Data are collected to database
python cli_client.py scraping
#process data and get simplified data for a given day
python cli_client.py data_processing --date yyyy-mm-dd --file your_csv_file_with_processed_data
#If you want drop database
python cli_client.py drop_database
```