https://github.com/denisecase/kafka-case
Custom Kafka Producer and Consumer
https://github.com/denisecase/kafka-case
Last synced: over 1 year ago
JSON representation
Custom Kafka Producer and Consumer
- Host: GitHub
- URL: https://github.com/denisecase/kafka-case
- Owner: denisecase
- Created: 2019-10-24T03:27:36.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2024-03-14T13:12:26.000Z (over 2 years ago)
- Last Synced: 2025-03-25T10:17:17.328Z (over 1 year ago)
- Language: Java
- Size: 30.3 KB
- Stars: 2
- Watchers: 3
- Forks: 22
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# kafka-case
Custom Kafka Producer and Consumer apps.
- Article:
Goal:
* Create your own CustomConsumer.
* Create your own CustomProducer to auto-generate and send at least 10 messages to a topic using the Twitter API.
* Use a custom namespace to avoid naming conflicts with other "CustomProducer" classes.
Alternate:
* Use Facebook API, or create a random word generator, choose a random noun-verb-noun, or cycle through a list sending one message every second.
* Experiment - the goal is to be comfortable adding external messaging to any Java app.
## Recommended Environment
* [Notepad++](https://notepad-plus-plus.org/)
* [VS Code](https://code.visualstudio.com/)
* [VS Code Extension - Java Extension Pack](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-pack)
## Prerequisities
* Twitter Developer Account
* OpenJDK or JDK (8 or up)
* Apache Maven
* Apache Zookeeper
* Apache Kafka
Also:
* Configure system environment variables including Path.
* Configure Zookeeper
* Configure Kafka
## Complete your Project
* Determine your namespace.
* Follow the Maven standard project directory structure.
* Use src\main\java for your code (with namespace folders)
* Use src\main\resources for your proporties files.
* Write your custom Kafka Producer in your namespace.
* Write your custome Kafka Consumer in your namespace.
* Add pom.xml
* Add README.md
* Add .gitignore
## Compile and Build a Fat Jar File
Open PowerShell as Administrator in the root project folder, compile the code using Maven and create an executable jar file. Generated artificacts can be found in the new 'target' folder.
Recommendation: Run this command each time you make code changes.
```PowerShell
mvn clean compile assembly:single
```
## 1 - Run Zookeeper Service
Start and keep running the Zookeeper service.
## 2 - Run Kafka Service
Start and keep running the Kafka service.
## 3 - Start Consumer
Open PowerShell as Administrator in the root project folder, start the original consumer app:
```PowerShell
java -cp target/kafka-case-1.0-SNAPSHOT-jar-with-dependencies.jar edu.nwmissouri.isl.professorcase.kafka.CustomConsumer
```
## 4 - Start Producer
Open a new PowerShell as Administrator in the root project folder, start the Producer app using topic test:
```PowerShell
java -cp target/kafka-case-1.0-SNAPSHOT-jar-with-dependencies.jar edu.nwmissouri.isl.professorcase.kafka.CustomProducer
```
## Test Communications
Verify messages are output by the Consumer.
## Accessing Twitter with Code
See http://www.baeldung.com/twitter4j
For help registering your Twitter API see this Tweepy article:
-
Register app:
- https://apps.twitter.com
Developer Agreement:
- https://developer.twitter.com/en/developer-terms/agreement-and-policy
Twitter App Attributes Example:
- Name: KafkaCaseCustomProducer
- Description: Sample app to illustrate working with the Twitter API and using Kafka from a Java application
- Website: https://bitbucket.org/professorcase/h08
- Callback URL: (blank)
## See also
- Simple examples at
-
## Repository
- Source: