https://github.com/inevitable-team/image-classification-benchmarker
This is an open source benchmarking tool for Machine Learning practitioners, the purpose of which is to accurately compare multiple Machine Learning algorithms and techniques against one another, under a standardised condition.
https://github.com/inevitable-team/image-classification-benchmarker
ai artificial-intelligence benchmarking image-classification machine-learning ml tensorflow tfjs
Last synced: 5 months ago
JSON representation
This is an open source benchmarking tool for Machine Learning practitioners, the purpose of which is to accurately compare multiple Machine Learning algorithms and techniques against one another, under a standardised condition.
- Host: GitHub
- URL: https://github.com/inevitable-team/image-classification-benchmarker
- Owner: inevitable-team
- Created: 2019-10-10T15:19:16.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2025-06-13T20:02:55.000Z (7 months ago)
- Last Synced: 2025-06-13T21:19:15.142Z (7 months ago)
- Topics: ai, artificial-intelligence, benchmarking, image-classification, machine-learning, ml, tensorflow, tfjs
- Language: JavaScript
- Homepage:
- Size: 2.21 MB
- Stars: 0
- Watchers: 2
- Forks: 2
- Open Issues: 41
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Image Classification Benchmarker
This is an open source benchmarking tool for Machine Learning practitioners, the purpose of which is to accurately compare multiple Machine Learning algorithms and techniques against one another, under a standardised condition.
## Setup
Some implemented algorithms use TensorFlow.js, which require you to install the [`windows-build-tools`](https://www.npmjs.com/package/windows-build-tools) for the TFJS binaries to build from Python.
Once downloaded, run `npm i` to install the dependencies, execute `npm start` to start the server and then visit `http://localhost:5000/`.
### Using Different Algorithms
You will find the controller for loading the various algorithms in `~/server/models/index.js`, this will then load all of the JavaScript files in the `~/server/models/wrappers` directory.
These wrappers are exported classes that require a name, description with a create function. The `create` function will be used to return a new instance of a model, which will be ran per each API call, which will return the results.
Also, along with a `run` function in the instance, a `predictOne` function is also needed, which takes a file buffer as in input to return a prediction of the class.
An example of which would be:
```js
const fs = require("fs");
class exampleWrapper {
constructor() {
this.name = "Algorithm #1";
this.desc = "Lorem ipsum, dolor sit amet consectetur adipisicing elit. Quam vel labore quos sit minus corrupti voluptatibus.";
}
create(imagePercentage, imageAbsPath, params) {
params.classCount = fs.readdirSync(imageAbsPath).length;
return new e(params);
}
}
class e {
constructor(params) {
this.params = params;
}
async predictOne(fileBuffer) {
return "N/A";
}
async run() {
return {
setUpTime: 0,
trainTime: 0,
evaluateTime: 0,
confusionMatrix: Array.from({ length: this.params.classCount }).map(() => Array.from({ length: this.params.classCount }).fill(0)),
accuracy: 0
};
}
}
module.exports = exampleWrapper;
```
### Using Image Datasets
The directory the image datasets should be placed in is `~/server/imageDatasets/datasets/`, which then should have the following folder structure:
```
datasets
+-- cifar10
| +-- airplane
| | +-- image_x.jpeg
| | +-- image_z.png
| | +-- image_l.png
| +-- automobile
| | +-- image_q.png
| | +-- image_a.jpg
| | +-- image_d.jpeg
+-- MNIST
| +-- 0
| | +-- image_er.png
| | +-- image_do.jpg
| | +-- image_b.jpg
| +-- 1
| | +-- ima_0.jpeg
| | +-- imatg.jpg
```
You can find the following images datasets by clicking the following links:
cifar10 - [https://pjreddie.com/projects/cifar-10-dataset-mirror/](https://pjreddie.com/projects/cifar-10-dataset-mirror/)
MNIST - [https://www.kaggle.com/scolianni/mnistasjpg/download](https://www.kaggle.com/scolianni/mnistasjpg/download)
### Using `@tensorflow/tfjs-node-gpu`
In the various model wrappers there are lines of code to use the gpu version of tfjs, these might require further debugging to install on a local machine, but will increase performance once `npm i @tensorflow/tfjs-node-gpu` is ran successfully on a machine.
If you get the following error:
```
internal/modules/cjs/loader.js:805
return process.dlopen(module, path.toNamespacedPath(filename));
```
Try rebuilding the package from source (`npm rebuild @tensorflow/tfjs-node-gpu --build-from-source`).
## Roadmap
Below is our current plan for this project:
