Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sintef/sensapp
SensApp, time-series with ease.
https://github.com/sintef/sensapp
amqp data-engineering gateway influxdb iot-platform mqtt opcua opcua-gateway pipelines prometheus-remote-write rust sensors-data-collection time-series
Last synced: about 2 months ago
JSON representation
SensApp, time-series with ease.
- Host: GitHub
- URL: https://github.com/sintef/sensapp
- Owner: SINTEF
- License: apache-2.0
- Created: 2011-09-23T18:24:12.000Z (over 13 years ago)
- Default Branch: main
- Last Pushed: 2024-08-27T14:02:36.000Z (4 months ago)
- Last Synced: 2024-08-27T15:34:42.522Z (4 months ago)
- Topics: amqp, data-engineering, gateway, influxdb, iot-platform, mqtt, opcua, opcua-gateway, pipelines, prometheus-remote-write, rust, sensors-data-collection, time-series
- Language: Rust
- Homepage:
- Size: 8.96 MB
- Stars: 18
- Watchers: 6
- Forks: 5
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# 
SensApp is an open-source sensor data platform developed by SINTEF for research purposes. It manages time-series data from a multitude of sensors.
It enables the handling of small time series data of the edge efficiently to large-scale big data digital twins.
## Features
- **Flexible Time Series DataBase Storage**: Supports various time-series databases such as SQLite, PostgreSQL (with optional TimeScaleDB plugin), DuckDB, and ClickHouse, with the potential to extend support to other databases in the future.
- **Data Lake Storage**: Supports Parquet files over S3 compatible object stores for long-term time-series data storage.
- **Multiple Data Ingestion Protocols**: Easy data ingestion via HTTP REST API, MQTT, AMQP, KAFKA, OPCUA, and NATS.
- **Compatibility with Existing Pipelines**: Offers Prometheus Remote Write and InfluxDB line format support for seamless integration into existing sensor data pipelines.
- **Data formats**: Supports various data formats like JSON, CSV, Parquet, or SenML.
## Architecture
SensApp should be stateless and scale from the edge to big data. The message queue software and the database software solve the complex problems. SensApp is a simple adapter between.
* SensApp supports simple deployments without requiring a message queue and only an embedded SQLite database.
* SensApp supports medium deployments with a single message broker and a PostgreSQL database.
* For larger deployments, SensApp advises a distributed message queue, an automatic load balancer for the SensApp instances, and a ClickHouse cluster.
Check the [ARCHITECTURE.md](docs/ARCHITECTURE.md) file for more details.
## Built With Rust™️
SensApp is developed using Rust, a language known for its performance, memory safety, and annoying borrow checker. SensApp used to be written in Scala, but the new author prefers Rust.
Not only the language, it's also the extensive high quality open-source ecosystem that makes Rust a great choice for SensApp:
* [Tokio](https://tokio.rs/) asynchronous runtime
* [Serde](https://serde.rs/) serialization framework
* [Axum](https://github.com/tokio-rs/axum) web framework
* [SQLx](https://github.com/launchbadge/sqlx) database driver
* [Polars](https://pola.rs) data frame library
* [nom](https://github.com/rust-bakery/nom) parser combinator library
* *and many more…*
## Contributing
We appreciate your interest in contributing to SensApp! Contributing is as simple as submitting an issue or a merge/pull request. Please read the [CONTRIBUTING.md](CONTRIBUTING.md) file for more details.
## License
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
The SensApp software is provided "as is," with no warranties, and the creators of SensApp are not liable for any damages that may arise from its use.
## You may not want to use it in production (yet)
SensApp is currently under development. It is not yet ready for production.
## Acknowledgments
We thank [the historical authors of SensApp](https://github.com/SINTEF/sensapp/graphs/contributors) who created the first version a decade ago.
SensApp is developed by
[SINTEF](https://www.sintef.no) ([Digital division](https://www.sintef.no/en/digital/), [Sustainable Communication Technologies department](https://www.sintef.no/en/digital/departments-new/department-of-sustainable-communication-technologies/), [Smart Data research group](https://www.sintef.no/en/expertise/digital/sustainable-communication-technologies/smart-data/)).
It is made possible thanks to the research and development of many research projects, founded notably by the [European Commission](https://ec.europa.eu/programmes/horizon2020/en) and the [Norwegian Research Council](https://www.forskningsradet.no/en/).
We also thank the open-source community for all the tools they create and maintain that allow SensApp to exist.