Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bahattincinic/crawler-challenge
https://github.com/bahattincinic/crawler-challenge
django docker github-actions mypy python3 react unit-testing
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/bahattincinic/crawler-challenge
- Owner: bahattincinic
- License: mit
- Created: 2020-04-17T19:36:46.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-01-05T19:33:06.000Z (almost 2 years ago)
- Last Synced: 2023-04-27T16:49:00.055Z (over 1 year ago)
- Topics: django, docker, github-actions, mypy, python3, react, unit-testing
- Language: Python
- Size: 4.8 MB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 23
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Crawler Challenge
## Requirements
Backend:
- Use Python
- Use Django
- Use Django Rest Framework in the API layer.
- Write unit tests
Frontend:
- Use React and Hooks
- Use Semantic UI React
- Write unit tests.
## Building the Project (with Docker)
Clone the repository
```
$ git clone [email protected]:bahattincinic/crawler-challenge.git
$ cd crawler-challenge
$ cp backend/crawler/settings/dev.py-dist backend/crawler/settings/dev.py
```
build docker images with the following link
```
$ docker-compose up -d --build
```
Apply Database migrations
```
$ docker exec -it djangoapp bash
$ python manage.py migrate
```
Import data with crawler
```
$ docker exec -it djangoapp bash
$ python manage.py crawl_products --help
usage: manage.py crawl_products [-h] [--category-url CATEGORY_URL] [--version]
[-v {0,1,2,3}] [--settings SETTINGS]
[--pythonpath PYTHONPATH] [--traceback]
[--no-color] [--force-color] [--skip-checks]
$ python manage.py crawl_products --category-url=https://example.com/category/list
```
Open http://localhost:3000
## Running Unit Tests
Frontend:
```
$ docker exec -it frontend bash
$ yarn run test
```
Backend:
```
$ docker exec -it djangoapp bash
$ python manage.py test
```
## API
Fetch products
```
URl: GET http://localhost:8000/api/products/
```
Compare HTML Files
```
URl: POST http://localhost:8000/api/compare/
{
"from_product": 1,
"to_product": 2
}
```
## Linter & Github Actions
I also added a linter and static analyzer to the projects.
Backend Linter (Github Actions)
https://github.com/bahattincinic/crawler-challenge/blob/master/.github/workflows/backend_linter.yml
Frontend Linter (Github Actions)
https://github.com/bahattincinic/crawler-challenge/blob/master/.github/workflows/frontend_linter.yml
