https://github.com/firefly-cpp/succulent
Collect POST requests
https://github.com/firefly-cpp/succulent
data-collection data-preprocessing-pipelines data-science esp32 machine-learning raspberry-pi
Last synced: about 2 months ago
JSON representation
Collect POST requests
- Host: GitHub
- URL: https://github.com/firefly-cpp/succulent
- Owner: firefly-cpp
- License: mit
- Created: 2023-04-18T08:02:34.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-06T09:41:38.000Z (3 months ago)
- Last Synced: 2025-03-26T23:11:34.505Z (2 months ago)
- Topics: data-collection, data-preprocessing-pipelines, data-science, esp32, machine-learning, raspberry-pi
- Language: Python
- Homepage:
- Size: 357 KB
- Stars: 3
- Watchers: 2
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
Awesome Lists containing this project
README
succulent
Collect POST requests easily
🔍 Detailed Insights •
📦 Installation •
🐳 Container •
🚀 Usage •
🔧 Configuration •
📄 Cite us •
🔑 License •
🫂 Contributors
Do you ever find it challenging and tricky to send sensor measurements 📏, data 📊, or GPS positions from embedded devices 📱, microcontrollers, and [smartwatches](https://github.com/firefly-cpp/AST-Monitor) to a central server? 📡 Setting up the primary data collection scripts can be a time-consuming ⏳ process, involving selecting a protocol, framework, API, and testing them out. Moreover, these scripts are often tailored for specific tasks, making them difficult to adapt to different scenarios.
But fear not! Introducing succulent 🌵, a pure Python framework that simplifies the configuration, management, collection, and preprocessing of data collected via POST requests. This framework draws inspiration from real-world data collection challenges in [smart agriculture](https://github.com/firefly-cpp/smart-agriculture-datasets/tree/main/plant-monitoring-esp32) 🧠🌿, specifically plant monitoring using ESP32 devices. The main goal behind succulent is to streamline the process of configuring various data parameters and provide a range of useful functions for data transformations. By leveraging succulent, you can set up your entire data collection endpoint within minutes, freeing you from the hassle of dealing with server-side scripts. 🚀🔧
* **Free software:** MIT license
* **Documentation:** [https://succulent.readthedocs.io/en/latest](https://succulent.readthedocs.io/en/latest/)
* **Python versions:** 3.8.x, 3.9.x, 3.10.x, 3.11.x, 3.12.x
* **Tested OS:** Windows, Ubuntu, Fedora, Alpine, Arch, macOS. **However, that does not mean it does not work on others**
## 🔍 Detailed Insights
The current version of succulent comes packed with exciting features, including, but not limited to:
- **Hassle-free generation of request URLs** for seamless data collection 🌐
- **Effortless data retrieval** from POST requests 📥
- **Versatile data storage options**, such as CSV, JSON, SQLite, XML, and even images 🗂️📊🖼️
- **Customisable boundaries for collected data**, allowing you to set minimum and maximum thresholds ⚙️
With succulent, the process of collecting, managing, and preprocessing data becomes a breeze, empowering you to focus on what truly matters—gaining valuable insights from your embedded devices, microcontrollers, and smartwatches. ⌚ So why waste precious time? ⏳ Dive into the world of succulent and unlock the true potential of your data! 💪📈
## 📦 Installation
### pip
To install `succulent` with pip, use:
```sh
pip install succulent
```
### Alpine Linux
To install `succulent` on Alpine Linux, use:
```sh
$ apk add py3-succulent
```
### Arch Linux
To install `succulent` on Arch Linux, use an [AUR helper](https://wiki.archlinux.org/title/AUR_helpers):
```sh
$ yay -Syyu python-succulent
```
### Fedora Linux
To install `succulent` on Fedora, use:
```sh
$ dnf install python3-succulent
```
## 🐳 Container
Create a `docker-compose.yml` file with the following content in the root directory:
```yml
version: '3.8'
services:
app:
image: codeberg.org/firefly-cpp/succulent:v6
ports:
- "8080:8080"
volumes:
- ./run.py:/succulent-app/run.py
- ./configuration.yml:/succulent-app/configuration.yml
environment:
- GUNICORN_WORKERS=2
```
Next create a `configuration.yml` file in the root directory. Here's an example of a configuration file:
```yml
data:
- name: 'temperature'
- name: 'humidity'
- name: 'light'
results:
- enable: true
- export: true
timestamp: true
```
More information regarding the configuration file and its settings can be found in the [configuration](#-configuration) section.
Then create a Python file named `run.py` with the following content in the root directory:
```python
from succulent.api import SucculentAPI
api = SucculentAPI(config='configuration.yml', format='csv')
# Flask app instance, called by gunicorn
app = api.app
```
Once you have set up the configuration file and the Python file, build the Docker image with the following command:
```bash
docker compose build
```
Finally, run the Docker container with the following command:
```bash
docker compose up
```
## 🚀 Usage
### Example
```python
from succulent.api import SucculentAPI
api = SucculentAPI(host='0.0.0.0', port=8080, config='configuration.yml', format='csv')
api.start()
```
## 🔧 Configuration
### Data collection
In the root directory, create a `configuration.yml` file and define the following:
```yml
data:
- name: # Measure name
min: # Minimum value (optional)
max: # Maximum value (optional)
```
To collect images, create a `configuration.yml` file in the root directory and define the following:
```yml
data:
- key: # Key in POST request
```
To store data collection timestamps, define the following setting in the `configuration.yml` file in the root directory:
```yml
timestamp: true # false by default
```
To access the URL for data collection, send a GET request (or navigate) to [http://localhost:8080/measure](http://localhost:8080/measure).
To restrict access to the collected data, define the following setting in the `configuration.yml` file in the root directory:
```yml
password: 'password' # Password for data access
```
To store data using a password, append the password parameter to the request URL: `?password=password`.
### Data access
To access data via the Succulent API, define the following setting in the `configuration.yml` file in the root directory:
```yml
results:
- enable: true # false by default
```
To access the collected data, send a GET request (or navigate) to [http://localhost:8080/data](http://localhost:8080/data). To access password-protected data, append the password parameter to the request URL: `?password=password`.
### Data export
To export the data, enable the export option in the configuration file:
```yml
results:
- export: true # false by default
```
To export the data, send a GET request (or navigate) to [http://localhost:8080/export](http://localhost:8080/export). To export password-protected data, append the password parameter to the request URL: `?password=password`. The data will be downloaded in the format specified in the configuration file.
## 📄 Cite us
Fister Jr., Iztok, and Tadej Lahovnik. Succulent. 0.4.0, doi:[10.5281/zenodo.10402365](https://doi.org/10.5281/zenodo.10402365).
## 🔑 License
This package is distributed under the MIT License. This license can be found online at .
## Disclaimer
This framework is provided as-is, and there are no guarantees that it fits your purposes or that it is bug-free. Use it at your own risk!
## 🫂 Contributors
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
Tadej Lahovnik
💻 🐛 🤔 📖 ✅
Ayan Das
💻 ⚠️
Iztok Fister Jr.
💻 🤔 🧑🏫
Oromion
🐛 📦
rhododendrom
🎨
Zala Lahovnik
📖
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!