https://github.com/neuralmagic/model-validation-configs
https://github.com/neuralmagic/model-validation-configs
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/neuralmagic/model-validation-configs
- Owner: neuralmagic
- Created: 2025-04-01T02:35:30.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-01-22T18:20:14.000Z (5 months ago)
- Last Synced: 2026-01-22T23:58:43.761Z (5 months ago)
- Size: 228 KB
- Stars: 2
- Watchers: 12
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# model-validation-configs
This repository contains configurations for model validation.
The configs are organized by model. Each top level folder should be named after the model stub and contain `accuracy` and `performance` sub-folders. Both folders contain configs specific to the activity. Shared configs, e.g. `storage.yml` should be in the common area.
The `accuracy` folder contains YAML configs needed for the model to be validated through the [llm-eval-test](https://github.com/openshift-psap/llm-eval-test). There are 3 config files for each model:
* server.yml: contains settings to start a vllm server with the model
* client.yml: contains settings for the llm-eval-test harness for the model
* accuracy.yml: contains evaluation tasks and accuracy expectations for the model
The `performance` folder contains YAML configs needed for the model to be validated through the [guidellm](https://github.com/neuralmagic/guidellm). There are 2 config files for each model:
* server.yml: contains settings to start a vllm server with the model
* client.yml: contains settings for the guidellm for the model