{"id":15787262,"url":"https://github.com/ysig/spython","last_synced_at":"2025-03-14T11:31:33.246Z","repository":{"id":157456368,"uuid":"633449910","full_name":"ysig/spython","owner":"ysig","description":"Making sbatch more user-friendly (for python users of Jean-Zay).","archived":false,"fork":false,"pushed_at":"2023-06-07T09:35:54.000Z","size":41,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-05T21:06:40.401Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ysig.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-27T14:19:39.000Z","updated_at":"2024-09-24T09:04:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"d9cc25e8-8f32-4bce-8202-f71d8c660038","html_url":"https://github.com/ysig/spython","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ysig%2Fspython","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ysig%2Fspython/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ysig%2Fspython/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ysig%2Fspython/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ysig","download_url":"https://codeload.github.com/ysig/spython/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243569654,"owners_count":20312475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T21:06:57.590Z","updated_at":"2025-03-14T11:31:33.232Z","avatar_url":"https://github.com/ysig.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `spython` making sbatch \u0026 python more user-friendly :revolving_hearts:\nDo you still want to write `sbatch` files yourself?\n`spython` is meant to improve your user experience of python when working on Jean-Zay's slurm system.\nIt is in an alpha-development stage (i.e. developed for a single user and now open to be tested and improved by many).\n\n## Motivation\nJean Zay is one of the coolest thing that has happened to France in terms of computing (optimized, well managed, abundant, open). Its support team is the best example of a French institution: extremely polite, trying to help you in everything and quick to answer (unfortunately not how French beaurocracy looks on average). \n\nHowever, there are two main problems with using Jean-Zay: it's like coding in C instead of python and it's counter-productive. Most of this comes from the fact of both complex rules that need to be read and understood (similar to law - but more pragmatic) but also for most operations it's simply a matter of copy pasting sbatch scripts around that are hard to read, understanding and digesting usage regulations and learning how to allocate modules, priorities and compute nodes.\n\nHere is where this script comes in. `spython` is a simple script that does all that for you and lifts interaction with `sbatch` instead to a more higher level of commands, implementing the Jean-Zay [documentation](http://www.idris.fr/media/eng/ia/guide_nouvel_utilisateur_ia-eng.pdf). For example, you can directy write the number of GPUs you want ot use the memory of each in gb's or the total cpu ram with any of these arguments `--ngpu 2` `--gb 16` `--ram l` (if arguments are missing it can self configure itself to optimal values) or e.g. select time `--t 19` or place it easily to `--debug` priority quickly. It has even a function called `--live` where the user experience of running a program imitates that of directly running python!* Output logs and scripts are automatically saved under a log directory in `$STORE` and tagged according to a tag `--tag` hyperparameter. \n\n\\*without ctrl-c for practical purposes.\n\n### Contribute\nWe hope that the Jean-Zay team \u0026 community get's to converge in high-level scripts or libraries that help improve quick and wasteless integration. Using such scripts high level practices can directly get implemented on a program level (instead of [documentation](http://www.idris.fr/media/eng/ia/guide_nouvel_utilisateur_ia-eng.pdf)) so that Jean-Zay is used much more optimally.\n\nAs this package sprawls from the user-experience of a certain user it can be limited to their explored use-cases. However you can easily incorporate your use-case to this package, by making pull-requests. If this package becomes useful, documentation and a python installation will be the next steps.\n\n## Usage\nTo use `spython` you firstly need to install it in your system, which takes not more than 10 seconds. It carries no-python dependencies and can work on native jean-zay python.\n\n### Installation\nCurrently the project requires manual installation, which can be done in 2 simple steps:\n\n```python\nwget -O ~/jean-zay.py https://raw.githubusercontent.com/ysig/spython/main/jean-zay/jean-zay.py # installs the main-script\nwget -O - https://raw.githubusercontent.com/ysig/spython/main/jean-zay/.bashrc \u003e\u003e ~/.bashrc # installs macros\n```\n\nAfter reopening a terminal you will be exposed to the cli-commands: \n- `spython`\n- `saccelerate`\n- `sl`\n- (hopefully more to be contributed in the future).\n\n### Examples\nFor all our examples we will suppose that what we really want is to run a `train.py`.\n\n#### Basic example\nimply append to your python script command `:` adding a set of Jean Zay allocation specific arguments after it:\n\n```bash\nspython train.py \u003cARGS\u003e : --ngpu 2 --ncpu 10 --module-load pytorch-gpu/py3/1.11.0 --tag EXP --gb 32 --env diffusers --post-script jean-zay/server.sh --email \u003cyour-email\u003e --name test\n```\n\nFrom this script we can firstly notice that we we allocate 2 gpus and 10 cpus per task.\nWe then `module load` a certain package (but multiple can be chained) and `conda activate geography` after the module is loaded.\n\nWe also copy paste server a script `jean-zay/server.sh` to run after environment is loaded and send emails to a specific address about the status of the experiment using `test` as a name.\n\nRunning directly:\n`spython --ngpu 2 --ncpu 10 --module-load pytorch-gpu/py3/1.11.0 --tag EXP --gb 32 --post-script jean-zay/server.sh`\n\nwill just print a command to run to generate the same environment:\n\n`srun --pty --job-name=test --constraint=v100-32g --nodes=1 --gres=gpu:2 --cpus-per-task=10 --ntasks=2 --ntasks-per-node=2 --time=72:00:00 --qos=qos_gpu-t4 -A hkt@v100 bash`\n\nbut omitting any in environment commands like running the `server.sh` or module-load.\n\n#### Logging\n\nAll our logs can be located by default in the `$SUB` folder.\nTo quickly access them we provide an `sl` macro that takes 3 arguments:\n\n```bash\nsl \u003ctag\u003e \u003ck=-1\u003e [--script]\n```\n\nwhere:\n\n- running `sl` alone prints all available tags\n- adding k (that defaults to -1) indexes the available logs as an index in a chronological list\n- using `--script` prints the sbatch script instead of the log.\n\n\n#### Help\nFor more covered functionality below follows the detailed `--help` output of the script:\n\n```bash\nusage: jean-zay.py [-h] [--gb {16,32,40,80}] [--ram {l,m,h}] [--debug]\n                   [--ngpu NGPU] [--ncpu NCPU] [--hours HOURS]\n                   [--minutes MINUTES] [--name NAME]\n                   [--module-load MODULE_LOAD [MODULE_LOAD ...]]\n                   [--ntasks NTASKS] [--ntasks-per-node NTASKS_PER_NODE]\n                   [--submission-dir SUBMISSION_DIR] [--error-file ERROR_FILE]\n                   [--output-file OUTPUT_FILE] [--script-file SCRIPT_FILE]\n                   [--conda_path CONDA_PATH] [--command COMMAND]\n                   [--email EMAIL] [--env ENV] [--preload] [--prepost]\n                   [--account ACCOUNT] [--tag TAG] [--path PATH] [--live]\n                   [--post-script POST_SCRIPT]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --gb {16,32,40,80}    Number of GBs GPU to reserve (default: None)\n  --ram {l,m,h}         Memory Mode (default: None)\n  --debug               Debug (default: False)\n  --ngpu NGPU, -g NGPU  Num GPUs (default: 1)\n  --ncpu NCPU, -c NCPU  Num CPUs (default: 10)\n  --hours HOURS, -t HOURS\n                        Max Time (hrs) (default: 72)\n  --minutes MINUTES, -m MINUTES\n                        Max Time (mins) (default: 0)\n  --name NAME, -n NAME  Job Name (default: None)\n  --module-load MODULE_LOAD [MODULE_LOAD ...], -ml MODULE_LOAD [MODULE_LOAD ...]\n  --ntasks NTASKS       Number of MP tasks (default: None)\n  --ntasks-per-node NTASKS_PER_NODE\n                        Number of tasks per node (default: None)\n  --submission-dir SUBMISSION_DIR\n                        Directory where submission files and outputs are\n                        stored (default: None)\n  --error-file ERROR_FILE, -e ERROR_FILE\n                        Error file (default: log.txt)\n  --output-file OUTPUT_FILE, -o OUTPUT_FILE\n                        Output file (default: log.txt)\n  --script-file SCRIPT_FILE, -s SCRIPT_FILE\n                        Script file (default: script.txt)\n  --conda_path CONDA_PATH\n                        Path of conda (default: None)\n  --command COMMAND     Main command to execute (default: python)\n  --email EMAIL         Email of user (default: None)\n  --env ENV             Environment (default: None)\n  --preload             Preload - if not set modules wil be purged (default:\n                        True)\n  --prepost             Set on a prepost node (default: False)\n  --account ACCOUNT     Manually set account name (default: None)\n  --tag TAG             Set an experiment tag (default: None)\n  --path PATH           Set explicit path from which to start the experiment\n                        (default: /gpfs7kw/linkhome/rech/genlgm01/uoi78rt)\n  --live                Debug (default: False)\n  --post-script POST_SCRIPT\n                        A script to be executed before the main command\n                        (default: None)\n```\n\n#### Behind the scenes.\nAs a BTS example we will show what happens with our current support `saccelerate`.\nThe user will simply run:\n\n`saccelerate \u003cARGS\u003e : --ngpu 2 --ncpu 10 --module-load pytorch-gpu/py3/1.11.0 --tag \u003cTAG\u003e --gb 32 --post-script jean-zay/server.sh --debug --live`\n\nbut the following script gets generated:\n\nFor example \nOn `/gpfsstore/rech/hkt/$USER/submissions/EXP/20230427-153034/` you will find two files: \n- `log.txt` on the same address will save all std output.\n- `script.txt` contains the `sbatch script`\n\n```bash\n  #!/bin/bash\n  #SBATCH --hint=nomultithread\n  #SBATCH --distribution=block:block\n  #SBATCH --job-name=$USER\n  #SBATCH --constraint=v100-32g\n  #SBATCH --nodes=1\n  #SBATCH --gres=gpu:2\n  #SBATCH --cpus-per-task=10\n  #SBATCH --ntasks=1\n  #SBATCH --ntasks-per-node=1\n  #SBATCH --time=1:00:00\n  #SBATCH --qos=qos_gpu-dev\n  #SBATCH --output=/gpfsstore/rech/hkt/$USER/submissions/\u003cTAG\u003e/20230425- 222102/log.txt\n  #SBATCH --error=/gpfsstore/rech/hkt/$USER/submissions/\u003cTAG\u003e/20230425-222102/log.txt\n  #SBATCH -A hkt@v100\n  \n  source /linkhome/rech/genlgm01/$USER/.bashrc\n  module purge\n  module load pytorch-gpu/py3/1.11.0\n  set -x\n\n  cd /gpfsdswork/projects/rech/hkt/\u003cuser-id\u003e/\u003cPROJECT-PATH\u003e\n\n  module load singularity\n\n  singularity exec --bind $SCRATCH:/data $SINGULARITY_ALLOWED_DIR/tileserver.sif node /usr/src/app/ /data/planet.mbtiles \u0026\u003e /dev/null \u0026\n\n  export HOSTNAMES=`scontrol show hostnames \"$SLURM_JOB_NODELIST\"`\n  export MASTER_ADDR=$(scontrol show hostnames \"$SLURM_JOB_NODELIST\" | head -n 1)\n  export MASTER_PORT=6000\n\n  srun accelerate launch --num_processes 2 --multi_gpu --main_process_ip $MASTER_ADDR --main_process_port $MASTER_PORT train.py \n  ```\n\nDo you still want to write `sbatch` files yourself?\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fysig%2Fspython","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fysig%2Fspython","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fysig%2Fspython/lists"}