Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ariffinzulkifli/iot-middleware

This repository host a middleware solution designed for Internet of Things (IoT) applications using Docker.
https://github.com/ariffinzulkifli/iot-middleware

adminer docker esp32 grafana hibiscus-sense influxdb iot iot-application mosquitto mqtt mysql node-red

Last synced: about 2 months ago
JSON representation

This repository host a middleware solution designed for Internet of Things (IoT) applications using Docker.

Awesome Lists containing this project

README 

          
```

   __  ___        __     _         

  /  |/  /_ _____/ /_ __(_)__  ___ 

 / /|_/ / // / _  / // / / _ \/ _ \

/_/  /_/\_, /\_,_/\_,_/_/_//_/\___/

       /___/                       

ai.iot.education.solutions.marketplace ==================================================



Ts. Mohamad Ariffin Zulkifli

ariffin@myduino.com

```



# Docker IoT Middleware



This repository contains a skeleton to setup a remote server to become a powerful IoT middleware using [Docker Compose](https://docs.docker.com/compose/). It is built around the **MING** stack, a popular combination of open-source applications for IoT inspired by the [LAMP](https://en.wikipedia.org/wiki/LAMP_(software_bundle)) stack. The MING stack was first introduced in 2019 by [Alex Lennon](https://blog.balena.io/ming-stack-mqtt-influxdb-nodered-grafana-balena/), a balena Ambassador, as a reusable template for IoT projects. It is now widely adopted in [Industrial IoT (IIoT)](https://www.iiot-world.com/industrial-iot/connected-industry/the-ming-stack-what-it-is-and-how-it-works/) applications as well, including integration with [Siemens S7 PLCs](https://flowfuse.com/blog/2025/01/integrating-siemens-s7-plcs-with-node-red-guide/):



- **M** (MQTT) [Mosquitto](https://mosquitto.org/) for MQTT protocols

- **I** (InfluxDB) [InfluxDB](https://www.influxdata.com/) for time series database

- **N** (Node-RED) [Node-RED](https://nodered.org/) for Javascript low-code flow-based programming

- **G** (Grafana) [Grafana](https://grafana.com/) for interactive dashboard



Additionally, the stack includes:

- [MySQL](https://www.mysql.com/) for SQL database

- [Adminer](https://www.adminer.org/) for SQL database management system



In Docker, each application above is containerized into a single image and is readily available in [Docker Hub](https://hub.docker.com/). For example, everything you need to host Node-RED in a server including NodeJS and other libraries or dependencies is containerized as [nodered/node-red](https://hub.docker.com/r/nodered/node-red) image.



Docker Compose allows you to define your entire application stack in a single, easy-to-read configuration file called `docker-compose.yml`. In this file, you specify which containers should run, how they should be configured, and how they should connect to each other.



This repository contains the `docker-compose.yml` with definition of the stack of applications stated above so you can easily setup an IoT middleware.



**Disclaimer:** Please use this `docker-compose.yml` file as a starting point for training and testing

but keep in mind that for production usage it might need modifications, especially on security.



## Directory layout



* `docker-compose.yml`: the docker-compose file containing the services

* `configuration/mosquitto`: directory containing the Mosquitto (MQTT broker) configuration

* `configuration/nodered`: directory containing the Node-RED configuration



## Installation



**Prerequisites:** [Git](https://git-scm.com/) must be installed on your system.



Choose the installation guide that matches your platform:



- [Option 1: PC or Mac (Docker Desktop)](#option-1-pc-or-mac-docker-desktop)

- [Option 2: Raspberry Pi (Terminal)](#option-2-raspberry-pi-terminal)

- [Option 3: VPS Cloud Server (GBCloud)](#option-3-vps-cloud-server-gbcloud)



### Option 1: PC or Mac (Docker Desktop)



1. Download and install [Docker Desktop](https://www.docker.com/products/docker-desktop/) for your operating system (Windows or Mac).



2. Make sure Docker Desktop is running.



3. Open Terminal (Mac) or PowerShell (Windows) and clone this repository.

```bash

git clone https://github.com/ariffinzulkifli/iot-middleware.git

```



4. Change your working directory to the cloned repository directory.

```bash

cd iot-middleware

```



5. Launch the Docker Compose services in detached mode.

```bash

docker compose up -d

```

Wait until all containers is successfully `Created` like below.



```

[+] up 84/84

 ✔ Image eclipse-mosquitto:latest       Pulled                                                13.3s

 ✔ Image adminer:latest                 Pulled                                                25.5s

 ✔ Image nodered/node-red:latest        Pulled                                                45.0s

 ✔ Image influxdb:latest                Pulled                                                34.4s

 ✔ Image grafana/grafana:latest         Pulled                                                47.9s

 ✔ Image mysql:latest                   Pulled                                                50.3s

 ✔ Network iot-middleware_iotstack      Created                                                0.0s

 ✔ Container iot-middleware-grafana-1   Created                                                0.5s

 ✔ Container iot-middleware-adminer-1   Created                                                0.5s

 ✔ Container iot-middleware-influxdb-1  Created                                                0.5s

 ✔ Container iot-middleware-mosquitto-1 Created                                                0.5s

 ✔ Container iot-middleware-mysql-1     Created                                                0.5s

 ✔ Container iot-middleware-nodered-1   Created                                                0.5s

```



6. Verify the containers are running in Docker Desktop.



![Docker Desktop](images/docker-desktop.png)



7. Access the services by clicking on the port link in Docker Desktop or open your browser and navigate to the service URL.



7.1 Node-RED http://localhost:1880 (username: `admin`, password: `password`)



![Docker Desktop Node-RED](images/docker-desktop-nodered.png)



7.2 InfluxDB http://localhost:8086 (username: `admin`, password: `password`)



![Docker Desktop InfluxDB](images/docker-desktop-influxdb.png)



7.3 Grafana http://localhost:3000 (username: `admin`, password: `password`)



![Docker Desktop Grafana](images/docker-desktop-grafana.png)



7.4 Adminer http://localhost:8060 (username: `root`, password: `password`)



![Docker Desktop Adminer](images/docker-desktop-adminer.png)



You can also verify the containers using command like below:

```bash

docker compose ps

```



```

NAME                         IMAGE                      COMMAND                  SERVICE     CREATED          STATUS                    PORTS

iot-middleware-adminer-1     adminer:latest             "entrypoint.sh docke…"   adminer     16 seconds ago   Up 15 seconds             0.0.0.0:8060->8080/tcp

iot-middleware-grafana-1     grafana/grafana:latest     "/run.sh"                grafana     16 seconds ago   Up 15 seconds             0.0.0.0:3000->3000/tcp

iot-middleware-influxdb-1    influxdb:latest            "/entrypoint.sh infl…"   influxdb    16 seconds ago   Up 15 seconds (healthy)   0.0.0.0:8086->8086/tcp

iot-middleware-mosquitto-1   eclipse-mosquitto:latest   "/docker-entrypoint.…"   mosquitto   16 seconds ago   Up 15 seconds             0.0.0.0:1883->1883/tcp, 0.0.0.0:9001->9001/tcp

iot-middleware-mysql-1       mysql:latest               "docker-entrypoint.s…"   mysql       16 seconds ago   Up 15 seconds             0.0.0.0:3306->3306/tcp

iot-middleware-nodered-1     nodered/node-red:latest    "./entrypoint.sh"        nodered     16 seconds ago   Up 15 seconds (healthy)   0.0.0.0:1880->1880/tcp

```



```bash

docker compose logs

```



### Option 2: Raspberry Pi (Terminal)



Here is the guide to install Docker on a 64-bit version of Debian Bullseye, Bookworm or Trixie running on your Raspberry Pi. If you are unsure, you can check the architecture using the `dpkg --print-architecture` command.



1. Update the package list and install the required dependencies:

```bash

sudo apt update

sudo apt install ca-certificates curl

```



2. Add Docker's official GPG key:

```bash

sudo install -m 0755 -d /etc/apt/keyrings

sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc

sudo chmod a+r /etc/apt/keyrings/docker.asc

```



3. Add the Docker repository to your package sources:

```bash

sudo tee /etc/apt/sources.list.d/docker.sources <8080/tcp

iot-middleware-grafana-1     grafana/grafana:latest     "/run.sh"                grafana     9 minutes ago   Up 9 minutes             0.0.0.0:3000->3000/tcp

iot-middleware-influxdb-1    influxdb:latest            "/entrypoint.sh infl…"   influxdb    9 minutes ago   Up 9 minutes (healthy)   0.0.0.0:8086->8086/tcp

iot-middleware-mosquitto-1   eclipse-mosquitto:latest   "/docker-entrypoint.…"   mosquitto   9 minutes ago   Up 9 minutes             0.0.0.0:1883->1883/tcp, 0.0.0.0:9001->9001/tcp

iot-middleware-mysql-1       mysql:latest               "docker-entrypoint.s…"   mysql       9 minutes ago   Up 9 minutes             0.0.0.0:3306->3306/tcp

iot-middleware-nodered-1     nodered/node-red:latest    "./entrypoint.sh"        nodered     9 minutes ago   Up 9 minutes (healthy)   0.0.0.0:1880->1880/tcp

```



9. Access the services from your Raspberry Pi using `localhost`, hostname or IP address from other devices on the same network.



9.1 Node-RED http://`ip-address`:1880 (username: `admin`, password: `password`)



![Docker Pi Node-RED](images/docker-pi-nodered.png)



9.2 InfluxDB http://`ip-address`:8086 (username: `admin`, password: `password`)



![Docker Pi InfluxDB](images/docker-pi-influxdb.png)



9.3 Grafana http://`ip-address`:3000 (username: `admin`, password: `password`)



![Docker Pi Grafana](images/docker-pi-grafana.png)



9.4 Adminer http://`ip-address`:8060 (username: `root`, password: `password`)



![Docker Pi Adminer](images/docker-pi-adminer.png)



### Option 3: VPS Cloud Server (GBCloud)



#### Create Account on GBCloud



1. Click the `Create account` link on [GBCloud](https://billing.gbcloud.net/aff.php?aff=87) login page.

2. Fill in your `Personal Information`, `Billing Address`, `Additional Information` and `Account Security`.

3. Check the `I have read and agree to the Terms of Service`. and click the `Register` button.



#### Remote SSH Ubuntu Server



1. Open Terminal on your PC.



**Note:** You can use any suitable software such as [PowerShell](https://learn.microsoft.com/en-us/powershell/scripting/install/installing-powershell-on-windows), [PuTTY](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html), [Cmder](https://github.com/cmderdev/cmder), etc.



2. Use the SSH command to connect to the remote server.



**Note:** change the `ip-address` with your server ip-address.

```bash

ssh root@ip-address

```

**Note:** When you attempt to connect for the first time, SSH will display the authenticity message you provided. It shows the server's fingerprint, which is a unique identifier for the server's SSH key. You need to verify this fingerprint to ensure you're connecting to the correct server. Type in `yes` to continue.



```bash

The authenticity of host 'ip-address (ip-address)' can't be established.

ED25519 key fingerprint is SHA256:oUuTnMQM2qCp7Oqip8gBpclMRBJFbL/hbQR5kbQnNOk.

This key is not known by any other names.

Are you sure you want to continue connecting (yes/no/[fingerprint])?

```



3. After confirming the authenticity of the server, SSH will prompt you to enter your password to log in.

```bash

root@ip-address's password:

```



If you provided the correct credentials, you should now be logged into the remote server, and you can start using it. You'll see the Ubuntu server terminal as below.



```

Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-52-generic x86_64)



 * Documentation:  https://help.ubuntu.com

 * Management:     https://landscape.canonical.com

 * Support:        https://ubuntu.com/advantage



  System information as of Tue Oct 17 19:21:02 +08 2023



  System load:           0.1962890625

  Usage of /:            3.7% of 48.27GB

  Memory usage:          10%

  Swap usage:            0%

  Processes:             95

  Users logged in:       0

  IPv4 address for eth0: ipv4-ipaddress

  IPv6 address for eth0: ipv6-ipaddress



0 updates can be applied immediately.



The list of available updates is more than a week old.

To check for new updates run: sudo apt update



Last login: Tue Nov  1 15:31:57 2022 from 104.208.107.150

root@iot-middleware:~#

```



#### Docker Installation



1. Update the package list to ensure you have the latest information about available packages.

```bash

sudo apt update

```



2. Install the necessary packages and dependencies for Docker.

```bash

sudo apt install apt-transport-https ca-certificates curl software-properties-common

```



3. Add Docker's GPG key to your system.

```bash

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

```



4. Add the Docker repository to your system's sources list.

```bash

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

```



5. Update the package list once more to include Docker repository information.

```bash

sudo apt update

```



6. Check the available Docker packages and their versions.

```bash

apt-cache policy docker-ce

```



You'll see the output like below, although the version number for Docker may be different:

```

docker-ce:

  Installed: (none)

  Candidate: 5:20.10.14~3-0~ubuntu-jammy

  Version table:

     5:20.10.14~3-0~ubuntu-jammy 500

        500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages

     5:20.10.13~3-0~ubuntu-jammy 500

        500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages

```



7. Install Docker using the following command.

```bash

sudo apt install docker-ce

```



8. Check the status of the Docker service to verify that it's running.

```bash

sudo systemctl status docker

```



You'll see the output like below, showing that the service is active and running:

```

Output

● docker.service - Docker Application Container Engine

     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)

     Active: active (running) since Fri 2022-04-01 21:30:25 UTC; 22s ago

TriggeredBy: ● docker.socket

       Docs: https://docs.docker.com

   Main PID: 7854 (dockerd)

      Tasks: 7

     Memory: 38.3M

        CPU: 340ms

     CGroup: /system.slice/docker.service

             └─7854 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

```

Hold `Ctrl + C` on keyboard to exit the status back to terminal prompt.



#### Docker Compose Installation



1. Make sure the directory where Docker Compose should be installed exists. This command creates the necessary directory if it doesn't exist.

```bash

mkdir -p ~/.docker/cli-plugins/

```



2. Download the Docker Compose binary to the specified directory.

```bash

curl -SL https://github.com/docker/compose/releases/download/v2.38.2/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose

```



3. Make the downloaded Docker Compose binary executable.

```bash

chmod +x ~/.docker/cli-plugins/docker-compose

```



4. Check the Docker Compose version to verify that it's installed correctly.

```bash

docker compose version

```



You'll see output like below, showing that Docker Compose is successfully installed with stated version:

```

Output

Docker Compose version v2.38.2

```



#### Clone and Run



1. Clone this repository into your server.

```bash

git clone https://github.com/ariffinzulkifli/iot-middleware.git

```



2. Change your working directory to the cloned repository directory.

```bash

cd ~/iot-middleware

```



3. Launch the Docker Compose services in detached mode.

```bash

docker compose up -d

```

Wait until all containers is successfully `Started` like below.



```

[+] Running 6/6

 ✔ Container iot-middleware-nodered-1    Running            0.0s

 ✔ Container iot-middleware-mosquitto-1  Running            0.0s

 ✔ Container iot-middleware-influxdb-1   Running            0.0s

 ✔ Container iot-middleware-grafana-1    Running            0.0s

 ✔ Container iot-middleware-mysql-1      Running            0.0s

 ✔ Container iot-middleware-adminer-1    Running            0.0s

```



4. Verify that the Docker Compose containers are running.

```bash

docker compose ps

```



```

NAME                         IMAGE                      COMMAND                  SERVICE     CREATED          STATUS                    PORTS

iot-middleware-adminer-1     adminer:latest             "entrypoint.sh docke…"   adminer     12 minutes ago   Up 12 minutes             0.0.0.0:8060->8080/tcp

iot-middleware-grafana-1     grafana/grafana:latest     "/run.sh"                grafana     12 minutes ago   Up 12 minutes             0.0.0.0:3000->3000/tcp

iot-middleware-influxdb-1    influxdb:latest            "/entrypoint.sh infl…"   influxdb    12 minutes ago   Up 12 minutes (healthy)   0.0.0.0:8086->8086/tcp

iot-middleware-mosquitto-1   eclipse-mosquitto:latest   "/docker-entrypoint.…"   mosquitto   12 minutes ago   Up 12 minutes             0.0.0.0:1883->1883/tcp, 0.0.0.0:9001->9001/tcp

iot-middleware-mysql-1       mysql:latest               "docker-entrypoint.s…"   mysql       12 minutes ago   Up 11 minutes             0.0.0.0:3306->3306/tcp

iot-middleware-nodered-1     nodered/node-red:latest    "./entrypoint.sh"        nodered     12 minutes ago   Up 12 minutes (healthy)   0.0.0.0:1880->1880/tcp

```



5. View the logs of the Docker Compose containers to monitor their output and any potential issues.

```bash

docker compose logs

```



### Running Specific Containers



**Note:** If you would like to run a specific service only, such as `nodered`:

```bash

docker compose up -d nodered

```



or several services such as `nodered`, `mysql` and `adminer`:

```bash

docker compose up -d nodered mysql adminer

```



or several services such as `nodered`, `mosquitto` and `influxdb`:

```bash

docker compose up -d nodered mosquitto influxdb

```



## Quick Reference



**Note:** Replace `ip-address` with `localhost`, hostname or IP address.



| Service | URL / Port | Username | Password |

|---------|------------|----------|----------|

| Mosquitto (MQTT) | `ip-address`:1883 (TCP) / `ip-address`:9001 (Websockets) | admin | password |

| InfluxDB | http://`ip-address`:8086 | admin | password |

| Node-RED | http://`ip-address`:1880 | admin | password |

| Grafana | http://`ip-address`:3000 | admin | password |

| MySQL | `ip-address`:3306 | root | password |

| Adminer | http://`ip-address`:8060 | root | password |



**InfluxDB additional configuration:**

- Bucket name: `iot-sensors`

- Organisation name: `my-organisation`

- Admin token: `A8C2B071-35F6-43F8-9F51-C5F584B2366B`