https://github.com/dinstone/kafka-assistant
kafka assistant toolkit
https://github.com/dinstone/kafka-assistant
Last synced: about 1 year ago
JSON representation
kafka assistant toolkit
- Host: GitHub
- URL: https://github.com/dinstone/kafka-assistant
- Owner: dinstone
- Created: 2016-06-30T07:57:43.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2022-09-06T01:51:34.000Z (almost 4 years ago)
- Last Synced: 2025-04-07T03:34:36.917Z (about 1 year ago)
- Language: Java
- Size: 73.2 KB
- Stars: 7
- Watchers: 1
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: license/ApacheV2.txt
Awesome Lists containing this project
README
# kafka assistant
# 1.Install kafka
## with a docker compose file
```
> cd kafka-docker
> docker-compose up -d
```
## produce test
```
> docker exec -it kafka_kafkaNode_1 sh
> kafka-console-producer.sh --broker-list localhost:9092 --topic kafeidou
>> hello world
```
## consume test
```
> docker exec -it kafka_kafkaNode_1 sh
> kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic kafeidou --from-beginning
```
# 2.API access kafka
## produce message to kafka topic
```
public class TopicProducerTest {
public static void main(String[] args) {
ProducerKafkaConfig config = new ProducerKafkaConfig("config-producer-test.xml");
TopicProducer producer = new TopicProducer(config);
long st = System.currentTimeMillis();
int count = 100;
for (int i = 0; i < count; i++) {
producer.send("" + i, "a" + i);
}
producer.flush();
long et = System.currentTimeMillis();
System.out.println("ok," + count + " take's " + (et - st) + ", " + count * 1000 / (et - st) + "tps");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
producer.destroy();
}
}
```
## parallel consume message from kafka topic
```
public class TopicConsumerTest {
private static final Logger LOG = LoggerFactory.getLogger(TopicConsumerTest.class);
public static void main(String[] args) {
MessageHandler handleService = new MessageHandler() {
@Override
public void handle(ConsumerRecord consumerRecord) throws Exception {
Thread.sleep(new Random().nextInt(10) * 1000);
LOG.error("{}-{} record: {}", consumerRecord.topic(), consumerRecord.partition(), consumerRecord.key());
}
};
ConsumerKafkaConfig consumeConfig = new ConsumerKafkaConfig("config-consumer-test.xml");
// consumeConfig.setParallelConsumerSize(1);
// consumeConfig.setMessageQueueSize(3);
TopicConsumer process = new TopicConsumer(consumeConfig, handleService);
process.start();
try {
System.in.read();
} catch (IOException e) {
}
process.stop();
}
}
```