https://github.com/paperswithcode/model-index
Create a source of truth for ML model results and browse it on Papers with Code
https://github.com/paperswithcode/model-index
Last synced: 2 months ago
JSON representation
Create a source of truth for ML model results and browse it on Papers with Code
- Host: GitHub
- URL: https://github.com/paperswithcode/model-index
- Owner: paperswithcode
- License: mit
- Created: 2021-02-10T17:38:12.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-06-09T13:27:50.000Z (about 4 years ago)
- Last Synced: 2025-03-23T03:32:18.067Z (3 months ago)
- Language: Python
- Homepage:
- Size: 209 KB
- Stars: 26
- Watchers: 6
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# model-index: maintain a source of truth for ML models
`model-index` has two goals:
- Make it easy to maintain a source-of-truth index of Machine Learning model metadata
- Enable the community browse this model metadata on [Papers with Code](https://paperswithcode.com/)
The main design principle of `model-index` is **flexibility**. You can store your model metadata however is the
most convenient for you - as JSONs, YAMLs or as annotations inside markdown. `model-index` provides a convenient
way to collect all this metadata into a single file that's browsable, searchable and comparable.
You can use this library locally or choose to upload the metadata to [Papers with Code](https://paperswithcode.com)
to have your library featured on the website.
## How it works
There is a root file for the model index: `model-index.yml` that links to (or contains) metadata.
```yaml
Models:
- Name: Inception v3
Metadata:
FLOPs: 5731284192
Parameters: 23834568
Training Data: ImageNet
Training Resources: 8x V100 GPUs
Results:
- Task: Image Classification
Dataset: ImageNet
Metrics:
Top 1 Accuracy: 74.67%
Top 5 Accuracy: 92.1%
Paper: https://arxiv.org/abs/1512.00567v3
Code: https://github.com/rwightman/pytorch-image-models/blob/timm/models/inception_v3.py#L442
Weights: https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth
README: docs/inception-v3-readme.md
```
All fields except for `Name` are **optional**. You can add any fields you like, but the ones above have a
standard meaning across different models and libraries.
We recommend putting the `model-index.yml` file in the root of your repository (so that relative links such as
`docs/inception-v3-readme.md` are easier to write), but you can also put it anywhere else in the repository (e.g.
in your `docs/` or `models/` folder).
### Storing metadata in markdown files
Metadata can also be directly stored in a model's README file. For example in this `docs/rexnet.md` file:
```markdown
# Summary
Rank Expansion Networks (ReXNets) follow a set of new design
principles for designing bottlenecks in image classification models.
## Usage
import timm
m = timm.create_model('rexnet_100', pretrained=True)
m.eval()
```
In this case, you just need to include this markdown file into the global `model-index.yml` file:
```yaml
Models:
- docs/rexnet.md
```
## Get started
Check out our [official documentation](https://model-index.readthedocs.io/en/latest/) on how to get started.
## Uploading to Papers with Code
To feature your library on Papers with Code, get in touch with `[email protected]` and the model index
of your library will be automatically included into Papers with Code.