Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/andreped/dss
:vibration_mode: From training and deployment of ViTs to development of real-time cross-platform mobile apps!
https://github.com/andreped/dss
android cnn data-visualization deep-learning dss flutter internet-of-things ios iot mobile real-time recording rnn sensor tensorboard tensorflow tf2 tflite vision-transformer vit
Last synced: about 2 months ago
JSON representation
:vibration_mode: From training and deployment of ViTs to development of real-time cross-platform mobile apps!
- Host: GitHub
- URL: https://github.com/andreped/dss
- Owner: andreped
- License: mit
- Created: 2022-09-07T15:16:25.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-04-19T19:51:03.000Z (over 1 year ago)
- Last Synced: 2024-06-21T06:35:44.644Z (6 months ago)
- Topics: android, cnn, data-visualization, deep-learning, dss, flutter, internet-of-things, ios, iot, mobile, real-time, recording, rnn, sensor, tensorboard, tensorflow, tf2, tflite, vision-transformer, vit
- Language: Dart
- Homepage:
- Size: 45.7 MB
- Stars: 7
- Watchers: 4
- Forks: 4
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
DSS: Deep Sensor Systems
:vibration_mode: From training of transformers to real-time development in cross-platform mobile apps!
[](https://opensource.org/licenses/MIT)
[](https://github.com/andreped/DSS/releases)
[](https://doi.org/10.5281/zenodo.7568040)
[](https://codecov.io/gh/andreped/DSS)
**DSS** was developed by SINTEF Medical Image Analysis with aim to integrate AIs into sensor systems.
This project serves as a demonstration on how to do it, and does not claim to be a generic framework.
Below there are described some of the key features of this project, but to see what else is possible, please, see [the wiki](https://github.com/andreped/DSS/wiki).
## [Continuous integration](https://github.com/andreped/DSS#continuous-integration)
| Build Type | Status |
| - | - |
| **Test Training** |  |
| **Test Flutter** | |
| **Build APK** |  |
## [How to your train own model?](https://github.com/andreped/DSS#how-to-train-your-own-model)
### [Setup](https://github.com/andreped/DSS#setup)
When using this framework, it is a good idea to setup a virtual environment:
```
virtualenv -ppython3 venv --clear
source venv/bin/activate
pip install -r requirements.txt
```
The following dependencies will be installed:
* `pandas<=1.5.3`
* `tensorflow<=2.12.0`
* `tensorflow-addons<=0.19.0`
* `tensorflow-datasets<=4.8.3`
Tested with Python 3.7.9 on Win10, macOS, and Ubuntu Linux operating systems. Also tested with Python 3.10.4 on GitHub Codespaces.
Note that to activate the virtual environment on Windows instead run `./venv/Scripts/activate`.
### [Usage](https://github.com/andreped/DSS#usage)
To train a model, simply run:
```
python main.py
```
The script supports multiple arguments. To see supported arguments, run `python main.py -h`.
### [Training history](https://github.com/andreped/DSS#training-history)
To visualize training history, use TensorBoard (with example):
```
tensorboard --logdir ./output/logs/gesture_classifier_arch_rnn/
```
Example of training history for a Recurrent Neural Network (RNN) can be seen underneath:
The figure shows macro-averaged F1-score for each step during training, with black curve for training and blue curve for validation sets.
Best model reached a macro-averaged F1 score of 99.66 % on the validation set, across all 20 classes.
**Disclaimer:** This model was only trained for testing purposes. The input features were stratified on sample-level and not patient-level, and thus validation performance will likely not represent true performance on new data. However, having a trained model enables us to test it in a Mobile app.
### [Available datasets](https://github.com/andreped/DSS#available-datasets)
#### [SmartWatch Gestures](https://github.com/andreped/DSS#smartwatch-gestures)
The current data used to train the AI model is the SmartWatch Gestures dataset,
which is available in [tensorflow-datasets](https://www.tensorflow.org/datasets/catalog/smartwatch_gestures). The dataset has the
following structure:
```
FeaturesDict({
'attempt': tf.uint8,
'features': Sequence({
'accel_x': tf.float64,
'accel_y': tf.float64,
'accel_z': tf.float64,
'time_event': tf.uint64,
'time_millis': tf.uint64,
'time_nanos': tf.uint64,
}),
'gesture': ClassLabel(shape=(), dtype=tf.int64, num_classes=20),
'participant': tf.uint8,
})
```
## [How to test the model in a mobile app?](https://github.com/andreped/DSS#how-to-test-the-model-in-a-mobile-app)
### [Converting model to TF-Lite](https://github.com/andreped/DSS#converting-model-to-tf-lite)
In order to be able to use the trained model in a mobile app, it is necessary to convert the model to a compatible format. TensorFlow Lite is an inference engine tailored for mobile devices. To convert the model to TF-Lite, simply run this command:
```
python dss/keras2tflite.py -m /path/to/pretrained/saved_model/ -o /path/to/save/converted/model.tflite
```
### [Model integration and testing in app](https://github.com/andreped/DSS#model-integration-and-testing-in-app)
A simple Mobile app was developed in Flutter, which demonstrates the AI in action using the accelerometer data from the mobile phone in real time. The data can also be stored and deleted locally.
To use the app, you need an Android phone and have developer mode enabled (see [here](https://developer.android.com/studio/debug/dev-options) for how to enable it). Then simply download the APK from [here](https://github.com/andreped/DSS/releases), double-click to install, and use the app as you normally would.
Info on how the mobile app was developed (and how to make your own app), can be found [in the wiki](https://github.com/andreped/DSS/wiki/Getting-started-with-mobile-development).
## [Acknowledgements](https://github.com/andreped/DSS#acknowledgements)
The training framework was mainly developed using [Keras](https://github.com/keras-team/keras) with [TensorFlow](https://github.com/tensorflow/tensorflow) backend.
The mobile app was developed using [Flutter](https://github.com/flutter/flutter), which is a framework developed by Google.
For the app, the following _open_ packages were used (either MIT, BSD-2, or BSD-3 licensed):
* [flutter_sensors](https://pub.dev/packages/flutter_sensors)
* [tflite_flutter](https://pub.dev/packages/tflite_flutter)
* [wakelock](https://pub.dev/packages/wakelock)
* [sqflite](https://pub.dev/packages/sqflite)
* [intl](https://pub.dev/packages/intl)
* [csv](https://pub.dev/packages/csv)
* [path_provider](https://pub.dev/packages/path_provider)
## [How to cite?](https://github.com/andreped/DSS#how-to-cite)
If you found this project useful, please, consider citing it in your research article:
```
@software{andre_pedersen_2023_7701510,
author = {André Pedersen and Ute Spiske and Javier Pérez de Frutos},
title = {andreped/DSS: v0.2.0},
month = mar,
year = 2023,
publisher = {Zenodo},
version = {v0.2.0},
doi = {10.5281/zenodo.7701510},
url = {https://doi.org/10.5281/zenodo.7701510}
}
```