https://github.com/gerapy/gerapyitempipeline
Item Pipeline Components for Gerapy & Scrapy
https://github.com/gerapy/gerapyitempipeline
Last synced: 11 months ago
JSON representation
Item Pipeline Components for Gerapy & Scrapy
- Host: GitHub
- URL: https://github.com/gerapy/gerapyitempipeline
- Owner: Gerapy
- Created: 2020-07-26T04:00:56.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-09-20T17:10:29.000Z (over 2 years ago)
- Last Synced: 2024-04-15T04:13:29.146Z (almost 2 years ago)
- Language: Python
- Size: 12.7 KB
- Stars: 1
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# Gerapy Item Pipeline
This is a package for supporting Item Pipelines in Scrapy, also this
package is a module in [Gerapy](https://github.com/Gerapy/Gerapy).
## Installation
```shell script
pip3 install gerapy-item-pipeline
```
## Usage
These are all kinds of the storage implemented.
### MongoDB
```python
MONGODB_CONNECTION_STRING = 'localhost'
MONGODB_DATABASE_NAME = 'default'
MONGODB_UPSERT = True
MONGODB_COLLECTION_NAME_FIELD = 'mongodb_collection_name'
MONGODB_COLLECTION_NAME_DEFAULT = 'default'
MONGODB_ITEM_PRIMARY_KEY_FIELD = 'primary_key'
MONGODB_ITEM_PRIMARY_KEY_DEFAULT = 'id'
```