https://github.com/nvidia/isv-ncp-validation-suite
Validation and management tools for NVIDIA ISV Lab environments.
https://github.com/nvidia/isv-ncp-validation-suite
gpu kubernetes nvidia slurm testing validation
Last synced: about 1 month ago
JSON representation
Validation and management tools for NVIDIA ISV Lab environments.
- Host: GitHub
- URL: https://github.com/nvidia/isv-ncp-validation-suite
- Owner: NVIDIA
- License: apache-2.0
- Created: 2026-01-30T03:49:08.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-05-08T18:58:07.000Z (about 2 months ago)
- Last Synced: 2026-05-08T20:45:22.331Z (about 2 months ago)
- Topics: gpu, kubernetes, nvidia, slurm, testing, validation
- Language: Python
- Homepage: https://www.nvidia.com/en-us/data-center/isv-validation-program/
- Size: 2.17 MB
- Stars: 11
- Watchers: 1
- Forks: 9
- Open Issues: 113
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
- Agents: AGENTS.md
Awesome Lists containing this project
README
# NVIDIA ISV NCP Validation Suite
[](LICENSE)
Validation and management tools for NVIDIA ISV Lab environments.
> [!WARNING]
> **Experimental Preview Release**
> This is an experimental/preview release of ISV-NCP-Validation-Suite. Use at your own risk in production environments. The software is provided "as is" without warranties of any kind. Features, APIs, and configurations may change without notice in future releases. For production deployments, thoroughly test in non-critical environments first.
## What Is This?
The ISV NCP Validation Suite is a test framework for validating that developers and compute providers get the most from their NVIDIA hardware across a range of common compute offerings.
It consists of a very flexible set of tests, which ensure that a system is able to support AI training, inferencing, and running AI-enabled applications, along with more traditional cloud services.
This validation suite is meant to be run against an existing cloud system, specifically one that is running NVIDIA hardware. This suite is not itself a cloud software platform, nor does it target a single specific cloud platform. Instead, it maps high-level requirements to a set of *stub* functions, which allow you to run high-level operations (like "Create a Virtual Machine") which you can then use for direct validation, or as steps in validating more complex specifications.
## Quick Start
The fastest way to try running parts of the validation suite is against an existing cloud service, such as AWS. This can be run by setting up your environment with your AWS keys and running a simple test, as follows:
### Prerequisites
- [uv](https://docs.astral.sh/uv/getting-started/installation/) (Python package manager)
Install:
```bash
git clone https://github.com/NVIDIA/ISV-NCP-Validation-Suite.git
cd ISV-NCP-Validation-Suite
uv sync
```
Configure credentials:
```bash
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=...
export AWS_SESSION_TOKEN=... # only required for temporary/SSO credentials
```
Execution:
```bash
$ uv run isvctl test run -f isvctl/configs/providers/aws/config/control-plane.yaml
Loaded configuration (1 import).
Validating configuration...
Running phases: ['setup', 'test', 'teardown']
... [~80 lines abridged]
------------------------------------------------------------
[PASS] All phases completed successfully
```
## Adding your own platform
See the **[my-isv scaffold](isvctl/configs/providers/my-isv/scripts/README.md)** --
copy-and-fill-in stubs with a demo-mode fallback. Preview the whole pipeline
before writing any code:
```bash
make demo-test
```
## Documentation
See [docs/](docs/) for full documentation:
- [Getting Started](docs/getting-started.md) - Installation and first steps
### Guides
- [Configuration](docs/guides/configuration.md) - Config file format and options
- [External Validation](docs/guides/external-validation-guide.md) - Create custom validations without modifying the repo
- [Remote Deployment](docs/guides/remote-deployment.md) - Deploy and run tests remotely
- [Local Development](docs/guides/local-development.md) - MicroK8s setup for local testing
### References
- [my-isv Scaffold](isvctl/configs/providers/my-isv/scripts/README.md) - Copy-and-fill-in stubs for adding your own platform
- [Validation Test Suites](isvctl/configs/suites/README.md) - Provider-agnostic validation contract
- [AWS Reference Implementation](docs/references/aws.md) - Working AWS examples for all validation domains
### Package Reference
- [isvctl](docs/packages/isvctl.md) - Controller documentation
- [isvtest](docs/packages/isvtest.md) - Validation framework
- [isvreporter](docs/packages/isvreporter.md) - Reporter documentation
## Development
```bash
make help # Show available targets
make test # Run tests for all packages
make lint # Run linting
make build # Build all packages
make plan # Render docs/test-plan.yaml to AsciiDoc and interactive HTML
```
See [Contributing](CONTRIBUTING.md) for development setup and guidelines.
## Environment Variables
| Variable | Description |
| -------- | ----------- |
| `ISV_SERVICE_ENDPOINT` | Required for ISV Lab Service uploads |
| `ISV_SSA_ISSUER` | Required for ISV Lab Service uploads |
| `ISV_CLIENT_ID` | Required for ISV Lab Service uploads |
| `ISV_CLIENT_SECRET` | Required for ISV Lab Service uploads |
| `NGC_API_KEY` | Required for NIM model benchmarks |
## Security
Report vulnerabilities via the [NVIDIA Security Vulnerability Submission Form](https://www.nvidia.com/object/submit-security-vulnerability.html) or email psirt@nvidia.com. **Do not open a public GitHub issue for security vulnerabilities.** See [SECURITY.md](SECURITY.md) for details.
## License
This project is licensed under the [Apache License 2.0](LICENSE).
This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.