https://github.com/blainerothrock/python-twitter-kafka-producer
https://github.com/blainerothrock/python-twitter-kafka-producer
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/blainerothrock/python-twitter-kafka-producer
- Owner: blainerothrock
- Created: 2019-03-30T15:35:37.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-03-30T17:20:51.000Z (about 6 years ago)
- Last Synced: 2025-02-08T18:30:43.307Z (4 months ago)
- Language: Python
- Size: 4.88 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# python-twitter-kafka-producer
A very simple kafka producer written in python that steams tracked keywords
## Run
* install requirements from `requirements.txt`
* start zookeeper & kafka locally (assumes broker @ `127.0.0.1:9092` or `KAFKA_BROKER` environment variable)
* add twitter environment variables to `.bash_profile`
* `TWITTER_CONSUMER_KEY`
* `TWITTER_CONSUMER_SECRET`
* `TWITTER_ACCESS_TOKEN`
* `TWITTER_ACCESS_SECRET`
* run `python producer.py TOPIC KEYWORD1 KEYWORD2 ... KEYWORDN`
* example: `python producer.py twitter-topic01 kafka python`
* start a kafka consumer
* example: `kafkakat -b 127.0.0.1:9092 -t twitter-topic01`
## Details
* Messages are posted as JSON:
```JSON
{
"user": "",
"message": ""
}
```