https://github.com/altaris/ielove-scraper
ielove.co.jp scraper
https://github.com/altaris/ielove-scraper
Last synced: 3 months ago
JSON representation
ielove.co.jp scraper
- Host: GitHub
- URL: https://github.com/altaris/ielove-scraper
- Owner: altaris
- License: mit
- Created: 2023-07-07T01:48:05.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-16T04:28:48.000Z (3 months ago)
- Last Synced: 2025-02-16T05:21:27.778Z (3 months ago)
- Language: Python
- Size: 137 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# ielove
[](https://choosealicense.com/licenses/mit/)
[](https://pypi.org/project/black)
ielove.co.jp scraper
# Howto
## Start a celery worker
```sh
. ./secret.env
export MONGO_USER=$MONGO_INITDB_ROOT_USERNAME
export MONGO_PASSWORD=$MONGO_INITDB_ROOT_PASSWORD
celery -A ielove.tasks worker --loglevel=INFO
```
## Scrape a property page
```sh
python3 -m ielove get-property https://www.ielove.co.jp/chintai/c1-397758400
```
If commiting to database:
```sh
. ./secret.env
export MONGO_USER=$MONGO_INITDB_ROOT_USERNAME
export MONGO_PASSWORD=$MONGO_INITDB_ROOT_PASSWORD
python3 -m ielove get-property --commit https://www.ielove.co.jp/chintai/c1-397758400
```
## Start the webui
```sh
python3 -m ielove.webui
```
# Contributing
## Dependencies
- `python3.10` or newer;
- `requirements.txt` for runtime dependencies;
- `requirements.dev.txt` for development dependencies.
- `make` (optional);
Simply run
```sh
virtualenv venv -p python3.10
. ./venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements.dev.txt
```
## Documentation
Simply run
```sh
make docs
```
This will generate the HTML doc of the project, and the index file should be at
`docs/index.html`. To have it directly in your browser, run
```sh
make docs-browser
```
## Code quality
Don't forget to run
```sh
make
```
to format the code following [black](https://pypi.org/project/black/),
typecheck it using [mypy](http://mypy-lang.org/), and check it against coding
standards using [pylint](https://pylint.org/).