https://github.com/teivah/franz
A collection of Kafka utility tools (load testing, replication)
https://github.com/teivah/franz
kafka load-testing performance-testing replication rust
Last synced: 7 months ago
JSON representation
A collection of Kafka utility tools (load testing, replication)
- Host: GitHub
- URL: https://github.com/teivah/franz
- Owner: teivah
- Created: 2020-06-09T16:32:34.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-06-23T20:34:13.000Z (over 5 years ago)
- Last Synced: 2025-03-19T09:14:27.626Z (7 months ago)
- Topics: kafka, load-testing, performance-testing, replication, rust
- Language: Rust
- Homepage:
- Size: 125 KB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# franz
A collection of Kafka utility tools:
* [franz-load](#franz-load): Kafka load tester.
* [franz-replicator](#franz-replicator): replicate Kafka topics.
A tiny Kafka producer load tester, written in Rust. It exposes REST endpoints to create load producer jobs and query results.
## franz-load
### Run
#### Parameters
| Short | Long | Description |
|---|---|---|
| -p | --http-port | HTTP port used by franz. |
| -k | --kafka-hosts | Kafka hosts, comma separated. |
#### Docker
```shell script
$ docker run -p 8080:8080 teivah/franz-load -- -p 8080 -k kafka:9092
```
#### Local
```shell script
$ cargo build --release
$ ./target/release/franz -p 8080 -k kafka:9092
```
### API
#### Produce
Description: Triggers a job
Path: POST `/produce`
JSON request:
* `topic`: Kafka topic
* `payload`: Kafka payload
* `expected_total`: Total number of messages
* `requested_required_acks`: Kafka required ack (-1: All, 0: None, 1: One)
* `producers`: Number of concurrent producers
JSON example:
```json
{
"topic": "foo",
"payload": "{true}",
"expected_total": 100,
"requested_required_acks": -1,
"producers": 10
}
```
The response returns an 201 containing an HATEAOS response with the job ID created.
```json
{
"links": {
"status": "/status/7bb17429-50bf-4f79-b8de-db1916683294"
}
}
```
#### Status
Description: Get job status.
Path: GET `/status/{id}` (`id` being the one returned by `/produce`)
If the job has been completed, it will return a 200:
```json
{
"messages_sent": 1000000,
"average_latency_ms": 6.9
}
```
Otherwise, it returns a 404 if the job does not exist, or a 204 if the job has not completed yet.
## franz-replicator
### Run
#### Parameters
| Short | Long | Description |
|---|---|---|
| -k | --kafka-hosts | Kafka hosts, comma separated. |
| -r | --replication | Replication configuration, e.g. _source1=target1;source2=target2_. |
| -o | --offset | Offset: 0 (earliest) or 1 (latest). |
| -a | --required-acks | Number of required acks for the producer: 0 (none), 1 (one), -1 (all). |
| -g | --group | Consumer group. |
| -t | --producer-timeout | Producer timeout in ms. |
#### Docker
```shell script
$ docker run -p 8080:8080 teivah/franz-load -- -k kafka:9092 -r source=target -o 1 -a 0, -g consumer_group -t 3000
```
#### Local
```shell script
$ cargo build --release
$ ./target/release/franz-load -k kafka:9092 -r source=target -o 1 -a 0, -g consumer_group -t 3000
```