{"id":23449955,"url":"https://github.com/natefoo/slurm-drmaa","last_synced_at":"2025-04-05T12:07:05.709Z","repository":{"id":47307516,"uuid":"96942376","full_name":"natefoo/slurm-drmaa","owner":"natefoo","description":"DRMAA for Slurm: Implementation of the DRMAA C bindings for Slurm","archived":false,"fork":false,"pushed_at":"2025-03-19T21:41:31.000Z","size":258,"stargazers_count":49,"open_issues_count":25,"forks_count":24,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-29T11:09:14.059Z","etag":null,"topics":["cluster","clusters","distributed-computing","drmaa","hpc","resource-management","slurm","slurm-job-scheduler"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/natefoo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-11T22:27:06.000Z","updated_at":"2025-03-19T21:41:32.000Z","dependencies_parsed_at":"2024-12-23T23:31:29.137Z","dependency_job_id":"07fbe1a2-7e09-407e-bf88-11026af3efe1","html_url":"https://github.com/natefoo/slurm-drmaa","commit_stats":{"total_commits":142,"total_committers":15,"mean_commits":9.466666666666667,"dds":0.4014084507042254,"last_synced_commit":"4f1b7c44ce541a961aef10204be9431da4c08e1b"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fslurm-drmaa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fslurm-drmaa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fslurm-drmaa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fslurm-drmaa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/natefoo","download_url":"https://codeload.github.com/natefoo/slurm-drmaa/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332609,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","clusters","distributed-computing","drmaa","hpc","resource-management","slurm","slurm-job-scheduler"],"created_at":"2024-12-23T23:31:13.275Z","updated_at":"2025-04-05T12:07:05.676Z","avatar_url":"https://github.com/natefoo.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"DRMAA for Slurm\n===============\n\n**Please note:** DRMAA for Slurm is a continuation of [PSNC DRMAA for SLURM][psnc-slurm-drmaa-trac] by [an unrelated developer][natefoo].\n\n[psnc-slurm-drmaa-trac]: http://apps.man.poznan.pl/trac/slurm-drmaa\n[natefoo]: https://github.com/natefoo/\n\nIntroduction\n------------\n\nDRMAA for [Slurm Workload Manager][slurm] (Slurm) is an implementation of [Open Grid Forum][ogf] [Distributed Resource Management Application API][drmaa] (DRMAA) [version 1][drmaa-v133] for submission and control of jobs to Slurm. Using DRMAA, grid applications builders, portal developers and ISVs can use the same high-level API to link their software with different cluster/resource management systems.\n\n[slurm]: https://slurm.schedmd.com/\n[ogf]: http://www.gridforum.org/\n[drmaa]: http://www.drmaa.org/\n[drmaa-v133]: http://www.ogf.org/documents/GFD.133.pdf\n\n### History\n\nDRMAA for Slurm was originally developed at the [Poznań Supercomputing and Networking Center][psnc] as [PSNC DRMAA for SLURM][psnc-slurm-drmaa-trac]. Following the unexpected death in 2013 of its primary maintainer, Mariusz Mamoński, there has been little additional development from PSNC, and no new releases.\n\n[This fork][slurm-drmaa] is maintained by [Nate Coraor][natefoo] and was originally created in 2014 to add support for Slurm's `--clusters` (`-M`) option. Since that time, others have found it useful and additional features and bug fixes have been added. However, the majority of the credit for this work belongs to the original authors (found below). In 2017, current maintainer Piotr Kopta created [psnc-apps/slurm-drmaa][psnc-slurm-drmaa-github] in Github, a snapshot of the unreleased 1.2.0 version (upon which this fork is also based) that has seen occasional work.\n\nThe title of the software in this fork has been changed from *PSNC DRMAA for SLURM* to simply *DRMAA for Slurm*. This change was made in order to alleviate confusion and differentiate from the canonical version. Additionally, this fork is not affiliated with PSNC; as such, releasing this software under PSNC's name would not be appropriate. However, this fork's maintainer is incredibly grateful for the work of the original authors of this software and the name change is in no way intended to minimize the efforts of those people.\n\n[psnc]: http://www.man.poznan.pl/\n[slurm-drmaa]: https://github.com/natefoo/slurm-drmaa/\n[psnc-slurm-drmaa-github]: https://github.com/psnc-apps/slurm-drmaa\n\nDownload\n--------\n\nDRMAA for Slurm is distributed as a source package which can be downloaded via the [Releases](https://github.com/natefoo/slurm-drmaa/releases) section.\n\nInstallation\n------------\n\nTo compile and install the library, go to main source directory and type:\n\n```console\n$ ./configure [options] \u0026\u0026 make\n$ sudo make install\n```\n\nThe library uses the standard GNU Autotools system, so standard `configure` arguments are available; see `./configure --help` for a full list. The library was tested with Slurm versions 16.05 and 18.08. If you encountered any problems using the library on different systems, please see the [contact](#contact) section.\n\nNotable `./configure` script options:\n\n\u003e `--with-slurm-inc` SLURM\\_INCLUDE\\_PATH\n\u003e\n\u003e \u003e Path to Slurm header files (i.e. directory containing `slurm/slurm.h` ). By default the library tries to guess the `SLURM_INCLUDE_PATH` and `SLURM_LIBRARY_PATH` based on location of the `srun` executable.\n\n\u003e `--with-slurm-lib` SLURM\\_LIBRARY\\_PATH\n\u003e\n\u003e \u003e Path to Slurm libraries (i.e. directory containing `libslurm.a` ).\n\n\u003e `--prefix` INSTALLATION\\_DIRECTORY\n\u003e\n\u003e \u003e Root directory where PSNC DRMAA for Slurm shall be installed. When not given library is installed in `/usr/local`.\n\n\u003e `--enable-debug`\n\u003e\n\u003e \u003e Compiles library with debugging enabled (with debugging symbols not stripped, without optimizations, and with many log messages enabled). Useful when you are to debug DRMAA enabled application or investigate problems with DRMAA library itself.\n\nThere are no unusual requirements for basic usage of library: a C99 compiler and standard make program should suffice. If you have taken sources directly from this repository or wish to run test-suite you would need additional [developer tools](#developer-tools).\n\nConfiguration\n-------------\n\nDuring DRMAA session initialization (`drmaa_init`), the library tries to read its configuration parameters from locations: `/etc/slurm_drmaa.conf`, `~/.slurm_drmaa.conf` and from the file given in the `$SLURM_DRMAA_CONF` environment variable (if set to a non-empty string). If multiple configuration sources are present then all configurations are merged with values from user-defined files taking precedence (in the following order: `$SLURM_DRMAA_CONF`, `~/.slurm_drmaa.conf`, `/etc/slurm_drmaa.conf`).\n\nCurrently recognized configuration parameters are:\n\n\u003e `cache_job_state`\n\u003e\n\u003e \u003e According to the DRMAA specification, every `drmaa_job_ps()` call should query the DRM system for job state. With this option one may optimize communication with the DRM. If set to a positive integer, `drmaa_job_ps()` returns the remembered job state without communicating with the DRM for `cache_job_state` seconds since the last update. By default the library conforms to the specification (no caching will be performed).\n\u003e \u003e\n\u003e \u003e Type: integer, default: 0\n\n\u003e `job_categories`\n\u003e\n\u003e \u003e Dictionary of job categories. Its keys are job categories names mapped to [native specification](#native-specification) strings. Attributes set by job category can be overridden by corresponding DRMAA attributes or native specification. The special category name `default` is used when the `drmaa_job_category` job attribute is not set.\n\u003e \u003e\n\u003e \u003e Type: dictionary with string values, default: empty dictionary\n\n### Configuration file syntax\n\nThe configuration file is in a form of a dictionary. A dictionary is set of zero or more key-value pairs. A key is a string, while a value can be a string, an integer or another dictionary.\n\n```\n  configuration: dictionary | dictionary_body\n  dictionary: '{' dictionary_body '}'\n  dictionary_body: (string ':' value ',')*\n  value: integer | string | dictionary\n  string: unquoted-string | single-quoted-string | double-quoted-string\n  unquoted-string: [^ \\t\\n\\r:,0-9][^ \\t\\n\\r:,]*\n  single-quoted-string: '[^']*'\n  double-quoted-string: \"[^\"]*\"\n  integer: [0-9]+\n```\n\nNative specification\n--------------------\n\nThe DRMAA interface allows passing DRM-dependent job submission options. Those options may be specified directly by setting the `drmaa_native_specification` job template attribute or indirectly by the `drmaa_job_category` job template attribute. The legal format of the native options looks like:\n\n```\n  -A My_job_name -s -N 1-10\n```\n\nList of parameters that can be passed in the `drmaa_native_specification` attribute:\n\n| Native specification              | Description                                                                                                                      |\n|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------|\n| -A, --account=*name*              | Charge job to specified accounts                                                                                                 |\n| --acctg-freq=*list*               | Define the job accounting sampling interval                                                                                      |\n| --comment=*string*                | An arbitrary comment                                                                                                             |\n| -C, --constraint=*list*           | Specify a list of constraints                                                                                                    |\n| -c, --cpus-per-task=*n*           | Number of processors per task                                                                                                    |\n| --contiguous                      | If set, then the allocated nodes must form a contiguous set                                                                      |\n| -d, --dependency=*list*           | Defer the start of this job until the specified dependencies have been satisfied completed                                       |\n| --exclusive                       | Allocate nodenumber of tasks to invoke on each nodes in exclusive mode when cpu consumable resource is enabled                   |\n| --gres=*list*                     | Specifies a comma delimited list of generic consumable resources                                                                 |\n| -k, --no-kill                     | Do not automatically terminate a job of one of the nodes it has been allocated fails                                             |\n| -L, --licenses=*license*          | Specification of licenses                                                                                                        |\n| -M, --clusters=*list*             | Comma delimited list of clusters to issue commands to                                                                            |\n| --mail-type=*type*                | Notify user by email when certain event types occur. Valid type values are BEGIN, END, FAIL, REQUEUE, and ALL (any state change) |\n| --mem=*MB*                        | Minimum amount of real memory                                                                                                    |\n| --mem-per-cpu=*MB*                | Maximum amount of real memory per allocated cpu required by a job                                                                |\n| --mincpus=*n*                     | Minimum number of logical processors (threads) per node                                                                          |\n| -N, --nodes=*minnodes[-maxnodes]* | Number of nodes on which to run                                                                                                  |\n| -n, --ntasks=*n*                  | Number of tasks                                                                                                                  |\n| --no-requeue                      | Specifies that the batch job should not be requeued after node failure                                                           |\n| --ntasks-per-node=*n*             | Number of tasks to invoke on each node                                                                                           |\n| -p, --partition=*partition*       | Partition requested                                                                                                              |\n| --qos=*qos*                       | Quality of Serice                                                                                                                |\n| --requeue                         | If set, permit the job to be requeued                                                                                            |\n| --reservation=*name*              | Allocate resources from named reservation                                                                                        |\n| -s, --share                       | Job allocation can share nodes with other running jobs                                                                           |\n| --tmp=*size[units]*               | Specify a minimum amount of temporary disk space                                                                                 |\n| -w, --nodelist=*hosts*            | Request a specific list of hosts                                                                                                 |\n| -x, --exclude=*nodelist*          | Explicitly exclude certain nodes from the resources granted to the job                                                           |\n\nAdditionally, the following parameters to `drmaa_native_specification` are supported, but their use is discouraged in\nfavor of the corresponding DRMAA job attributes:\n\n| Native specification       | DRMAA job attribute | Description                                                                                   |\n|----------------------------|---------------------|-----------------------------------------------------------------------------------------------|\n| -e, --error=*pattern*      | drmaa_output_path   | Connect the batch script's standard error directly to the file name specified in the pattern  |\n| -J, --job-name=*name*      | drmaa_job_name      | Specify a name for the job allocation                                                         |\n| -o, --output=*pattern*     | drmaa_error_path    | Connect the batch script's standard output directly to the file name specified in the pattern |\n| -t, --time=*hours:minutes* | drmaa_wct_hlimit    | Set a maximum job wallclock time                                                              |\n\nDescriptions of each parameter can be found in `man sbatch`.\n\nChangelog\n---------\n\nSee [CHANGELOG.md](CHANGELOG.md)\n\n### Known bugs and limitations\n\nThe library covers all of the [DRMAA 1.0 specification][drmaa-v133] with exceptions listed below. It was successfully tested with [Slurm 16.05 and 18.08][slurm]. Known limitations:\n\n-   `drmaa_control` options `DRMAA_CONTROL_HOLD`, `DRMAA_CONTROL_RELEASE` are only available for users being Slurm administrators (in version prior 2.2)\n-   `drmaa_control` options `DRMAA_CONTROL_SUSPEND`, `DRMAA_CONTROL_RESUME` are only available for users being Slurm administrators\n-   `drmaa_wct_slimit` not implemented\n-   optional attributes `drmaa_deadline_time`, `drmaa_duration_hlimit`, `drmaa_duration_slimit`, `drmaa_transfer_files` not implemented\n-   The SPANK client side (i.e. not remote) plugins chain is not invoked in DRMAA run job call. For this reason we advice you to use [TASK BLOCKS](http://code.google.com/p/slurm-spank-plugins/wiki/UseEnvSyntax#TASK_BLOCKS) in the `UseEnv` SPANK plugin.\n\nDevelopment and Pre-releases\n----------------------------\n\n**note:** This repository depends on [FedStage DRMAA Utils][drmaa-utils], which is configured as a submodule. When cloning this repository, you should clone recursively, e.g.:\n\n```console\n$ git clone --recursive https://github.com/natefoo/slurm-drmaa.git\n```\n\nThe source repository does not contain Autotools-generated artifacts such as `configure` and `Makefile`. Please note the `./autogen.sh` and `./autoclean.sh` scripts which call the Autotools command chain in the appropriate order to generate these artifacts.\n\n**note:** You need some developer tools to compile the source from git.\n\n[drmaa-utils]: https://github.com/natefoo/drmaa-utils/\n\n### Developer tools\n\nAlthough not needed to use the library or to compile from source distribution tarballs, user the following tools may be required if you intend to develop DRMAA for Slurm from git:\n\n-   GNU autotools\n    -   autoconf (tested with version 2.67)\n    -   automake (tested with version 1.11)\n    -   libtool (tested with version 2.2.8)\n    -   m4 (tested with version 1.4.14)\n-   [Bison](http://www.gnu.org/software/bison/) parser generator,\n-   [RAGEL](http://www.complang.org/ragel/) State Machine Compiler,\n-   [gperf](http://www.gnu.org/software/gperf/) gperf - a perfect hash function generator.\n\nAuthors\n-------\n\nThe library was developed by:\n\n-   Michal Matloka - first implementation\n-   Mariusz Mamonski - maintainer since version 1.0.3\n-   Piotr Kopta - maintainer since version 1.0.7\n\nThis library relies heavily on the *Fedstage DRMAA utils* code developed by:\n\n-   Lukasz Ciesnik\n\nThe maintainer of [this fork][slurm-drmaa] is:\n\n-   Nate Coraor\n\nwith [additional contributors](https://github.com/natefoo/slurm-drmaa/graphs/contributors).\n\n### Contact\n\nYou can submit [issues](https://github.com/natefoo/slurm-drmaa/issues) and [pull requests](https://github.com/natefoo/slurm-drmaa/pulls) for DRMAA for Slurm in GitHub.\n\nLinks\n-----\n\n- [PSNC DRMAA for SLURM][psnc-slurm-drmaa-trac]\n- [PSNC DRMAA for SLURM][psnc-slurm-drmaa-github] in Github\n- [DRMAA][drmaa]\n- [Open Grid Forum][ogf]\n- [DRMAA 1.0 specification][drmaa-v133]\n- [Official DRMAA test-suite](http://drmaa.org/testsuite.php)\n- [Slurm Workload Manager][slurm]\n\n### Software using DRMAA for Slurm\n\n- [QCG-Computing](http://www.qoscosgrid.org/trac/qcg-computing/): remote multi-user job submission and control over Web Services\n- [Galaxy](https://galaxyproject.org/): an open, web-based platform for accessible, reproducible, and transparent computational biomedical research\n\nLicense\n-------\n\nCopyright (C) 2011-2015 Poznan Supercomputing and Networking Center\nCopyright (C) 2014-2019 The Pennsylvania State University\n\nThis program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with this program. If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/).\n\nSome portions of this program are copied or derived from Slurm, which is licensed under the GNU General Public License Version 2. For details, including the list of Slurm copyright holders,  see \u003c[https://slurm.schedmd.com/](https://slurm.schedmd.com/)\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatefoo%2Fslurm-drmaa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnatefoo%2Fslurm-drmaa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatefoo%2Fslurm-drmaa/lists"}