Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/bluebrown/iot-stream-analytics

emqx iot kafka mqtt stream stream-analysis stream-analytics stream-processing

Last synced: 8 days ago
JSON representation

mqtt & kafka based stream processing playground

Host: GitHub
URL: https://github.com/bluebrown/iot-stream-analytics
Owner: bluebrown
Created: 2023-12-15T16:04:23.000Z (about 1 year ago)
Default Branch: main
Last Pushed: 2023-12-17T22:58:26.000Z (about 1 year ago)
Last Synced: 2024-12-14T11:32:34.788Z (2 months ago)
Topics: emqx, iot, kafka, mqtt, stream, stream-analysis, stream-analytics, stream-processing
Language: Jupyter Notebook
Homepage:
Size: 474 KB
Stars: 0
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

# IOT Playground

## Quick Start

First start the stack:

```bash
bin/install.sh # download connect plugins
docker compose up -d # boot the system
docker compose logs migration -f # observe the migration
```

After a few minutes, there should be data in the [kafka
bucket](http://localhost:9001/browser/kafka). You can authenticate with
`minio:minio123`.

Once data is available, you can use the [notebook](./notebook.ipynb), to
visualize it.

While you are waiting, you can inspect the system. For example try some of the
below commands.

## Useful Commands

### List Kafka Topics

```bash
docker compose exec kafka kafka-topics.sh \
--bootstrap-server kafka:9092 --list
```

### Consume Kafka Messages

```bash
docker compose exec kafka kafka-console-consumer.sh \
--bootstrap-server localhost:9092 --topic mqtt
```

### Inspect Message Structure

```bash
bin/ksql.sh -e "print 'device-parameter';"
```

### list Consumer Groups

```bash
docker compose exec kafka kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 --all-groups --describe
```