Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/artemrys/scrapy-rabbitmq-publisher

Scrapy Item Pipeline to send items to RabbitMQ. From https://medium.com/python4you/rabbitmq-scrapy-item-publisher-in-python-4c66a985e3cb
https://github.com/artemrys/scrapy-rabbitmq-publisher

medium python rabbitmq scrapy

Last synced: 7 days ago
JSON representation

Scrapy Item Pipeline to send items to RabbitMQ. From https://medium.com/python4you/rabbitmq-scrapy-item-publisher-in-python-4c66a985e3cb

Host: GitHub
URL: https://github.com/artemrys/scrapy-rabbitmq-publisher
Owner: artemrys
License: mit
Created: 2019-02-18T17:51:19.000Z (over 5 years ago)
Default Branch: master
Last Pushed: 2022-01-07T11:33:18.000Z (almost 3 years ago)
Last Synced: 2024-10-03T19:05:00.822Z (about 1 month ago)
Topics: medium, python, rabbitmq, scrapy
Language: Python
Homepage: https://pypi.org/project/scrapy-rabbitmq-publisher/
Size: 7.81 KB
Stars: 13
Watchers: 1
Forks: 3
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

## A RabbitMQ Item Publisher for Scrapy Framework.

Scrapy-RabbitMQ-Publisher allows you to save publish items to RabbitMQ.

## Installation

Using pip, type in your command-line prompt

```
pip install scrapy-rabbitmq-publisher
```

Or clone the repo and inside the `scrapy-rabbitmq-publisher` directory, type

```
python setup.py install
```

## Usage

Add following code to your scrapy settings.

```python
RABBITMQ_HOST = "localhost"
RABBITMQ_PORT = 5672
RABBITMQ_USER = "guest"
RABBITMQ_PASSWORD = "guest"
RABBITMQ_VIRTUAL_HOST = "/"
RABBITMQ_EXCHANGE = "scrapy"
RABBITMQ_ROUTING_KEY = "item"
RABBITMQ_QUEUE = "item"

ITEM_PIPELINES = {
"scrapy_rabbitmq_publisher.pipelines.RabbitMQItemPublisherPipeline": 1,
}
```