https://github.com/southernmethodistuniversity/hpc_mamba_envs
https://github.com/southernmethodistuniversity/hpc_mamba_envs
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/southernmethodistuniversity/hpc_mamba_envs
- Owner: SouthernMethodistUniversity
- License: mit
- Created: 2024-09-16T20:03:06.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-17T14:32:58.000Z (almost 2 years ago)
- Last Synced: 2025-11-30T21:38:51.073Z (7 months ago)
- Size: 5.86 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# HPC Mamba Environments
The goal of this repository is to create a versioned and scripted method for creating stackable Mamba environments that can be loaded in an HPC environment.
## Desired outcomes
1. Reproducible environments based on `json` config files
2. Scoped environments so users can load packages most useful to them without additional bloat
3. Reduced reliance on *per user* created environments which has caused needless and excessive duplication of packages on high performance storage
## Basic design principles
1. Base environments `json` files are created without versions, when possible
1. All builds are scripted
2. When building environmnents, versions are specified as necessary and otherwise try to use the latest compatible version
1. Dated and versioned `yml` files are saved when an environment is built with the exact packages used
2. Dated and versioned `lua` module files are generated to load the environment
3. All `json`, `yml` and `lua` files and any associated scripts are tracked in Github
1. Environments that trigger security alerts may be depricated and archived
3. Environments are nested, e.g. access to environments built with `Python 3.11` first requires loading the module associated with `python/3.11`
4. Environmnets should build basic documentation
1. At minimum creates a markdown file with a brief summary of the expected use and included packages / versions.
5. Environments should have unit tests that can be run on the installed system.
1. At minumun verify `import package_name` works for all specified packages in environment
2. Verify dependencies are correctly loaded, e.g. Cuda enabled packages can see a GPU (if present)
3. Job scripts to submit unit tests