Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cloudmercato/ai-benchmark
https://github.com/cloudmercato/ai-benchmark
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/cloudmercato/ai-benchmark
- Owner: cloudmercato
- License: other
- Created: 2020-12-05T03:21:55.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2023-11-27T19:57:42.000Z (about 1 year ago)
- Last Synced: 2023-11-27T21:29:12.658Z (about 1 year ago)
- Language: Python
- Size: 19.9 MB
- Stars: 6
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[AI Benchmark Alpha](http://ai-benchmark.com/alpha) is an open source python library for evaluating AI performance of various hardware platforms, including CPUs, GPUs and TPUs. The benchmark is relying on [TensorFlow](https://www.tensorflow.org) machine learning library, and is providing a lightweight and accurate solution for assessing inference and training speed for key Deep Learning models.
In total, AI Benchmark consists of 42 tests and 19 sections provided below:
1. MobileNet-V2 `[classification]`
2. Inception-V3 `[classification]`
3. Inception-V4 `[classification]`
4. Inception-ResNet-V2 `[classification]`
5. ResNet-V2-50 `[classification]`
6. ResNet-V2-152 `[classification]`
7. VGG-16 `[classification]`
8. SRCNN 9-5-5 `[image-to-image mapping]`
9. VGG-19 `[image-to-image mapping]`
10. ResNet-SRGAN `[image-to-image mapping]`
11. ResNet-DPED `[image-to-image mapping]`
12. U-Net `[image-to-image mapping]`
13. Nvidia-SPADE `[image-to-image mapping]`
14. ICNet `[image segmentation]`
15. PSPNet `[image segmentation]`
16. DeepLab `[image segmentation]`
17. Pixel-RNN `[inpainting]`
18. LSTM `[sentence sentiment analysis]`
19. GNMT `[text translation]`
For more information and results, please visit the project website: [http://ai-benchmark.com/alpha](http://ai-benchmark.com/alpha)
#### Installation Instructions
The benchmark requires TensorFlow machine learning library to be present in your system.
On systems that do not have Nvidia GPUs, run the following commands to install AI Benchmark:
```bash
pip install tensorflow
pip install ai-benchmark
```
If you want to check the performance of Nvidia graphic cards, run the following commands:
```bash
pip install tensorflow-gpu
pip install ai-benchmark
```
`Note 1:` If Tensorflow is already installed in your system, you can skip the first command.
`Note 2:` For running the benchmark on Nvidia GPUs, `NVIDIA CUDA` and `cuDNN` libraries should be installed first. Please find detailed instructions [here](https://www.tensorflow.org/install/gpu).
#### Getting Started
To run AI Benchmark, use the following code:
```bash
from ai_benchmark import AIBenchmark
benchmark = AIBenchmark()
results = benchmark.run()
```
Alternatively, on Linux systems you can type `ai-benchmark` in the command line to start the tests.
To run inference or training only, use `benchmark.run_inference()` or `benchmark.run_training()`.
#### Advanced settings
```bash
AIBenchmark(use_CPU=None, verbose_level=1):
```
> use_CPU=`{True, False, None}`: whether to run the tests on CPUs (if tensorflow-gpu is installed)
> verbose_level=`{0, 1, 2, 3}`: run tests silently | with short summary | with information about each run | with TF logs
```bash
benchmark.run(precision="normal"):
```
> precision=`{"normal", "high"}`: if `high` is selected, the benchmark will execute 10 times more runs for each test.
### Additional Notes and Requirements
GPU with at least 2GB of RAM is required for running inference tests / 4GB of RAM for training tests.
The benchmark is compatible with both `TensorFlow 1.x` and `2.x` versions.
### Contacts
Please contact `[email protected]` for any feedback or information.