https://github.com/killiansheriff/sbatchpy
A python package that allows easy sbatch job script creation and submissions on hpc clusters, directly from python.
https://github.com/killiansheriff/sbatchpy
bash hpc python sbatch slurm
Last synced: 9 months ago
JSON representation
A python package that allows easy sbatch job script creation and submissions on hpc clusters, directly from python.
- Host: GitHub
- URL: https://github.com/killiansheriff/sbatchpy
- Owner: killiansheriff
- License: mit
- Created: 2022-08-13T01:33:47.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2023-08-11T21:31:38.000Z (almost 3 years ago)
- Last Synced: 2025-07-17T12:32:20.605Z (11 months ago)
- Topics: bash, hpc, python, sbatch, slurm
- Language: Python
- Homepage:
- Size: 33.2 KB
- Stars: 10
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# SbatchPy
 
A python package that allows easy ``sbatch`` job script creation and submissions on ``hpc clusters``, directly from ``python``.
# Installation
```bash
# to install latest PyPI release
pip install sbatchpy
# to install latest GitHub commit
pip install --upgrade git+https://github.com/killiansheriff/sbatchpy
```
# Usage
An example on how to run a python script called ``my_script.py`` taking 2 arguments ``var1`` and ``var2`` as inputs, and using the ``base`` environement is provided below. A complete example with outputs can be found [here](examples/).
```python
from sbatchpy import run
config = {
"mem": "1gb",
"time": "00:01:00",
"account": "myaccount",
"cpus-per-task": "5",
"partition": "shared",
"ntasks-per-node": "1",
"nodes": "1",
}
for var1, var2 in zip([1, 2, 3], ["A", "B", "C"]):
config["job-name"] = f"myjob_{var1}var1_{var2}var2.sh"
config["output"] = f"out/myjob_{var1}var1_{var2}var2.out"
run(
config,
code=f"source activate base \n python my_script.py {var1} {var2}",
)
```
By default, sbatchpy will check that ``config["output"]``'s directory folder exists. If it doesn't, you will be notified and the job will not run.
Additionally, each sbatch submission script is saved inside the ``f"{os.getcwd()}/.job"`` folder. Another saving folder can be chosen by passing ``job_directory=my_saving_directory_path`` to the ``run`` function which is responsible of creating the submission script and running it using ``sbatch job_directory/config["job-name"]``.