{"id":13412540,"url":"https://github.com/dgruber/drmaa","last_synced_at":"2025-04-23T23:26:27.299Z","repository":{"id":7485739,"uuid":"8834393","full_name":"dgruber/drmaa","owner":"dgruber","description":"Compute cluster (HPC) job submission library for Go (#golang) based on the open DRMAA standard.","archived":false,"fork":false,"pushed_at":"2024-07-09T06:25:48.000Z","size":1350,"stargazers_count":48,"open_issues_count":0,"forks_count":20,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-07-31T20:51:03.265Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dgruber.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-03-17T12:58:02.000Z","updated_at":"2024-07-09T06:25:48.000Z","dependencies_parsed_at":"2024-06-17T14:44:38.071Z","dependency_job_id":"8bd94274-891a-45c9-bf3e-701aec619be9","html_url":"https://github.com/dgruber/drmaa","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgruber%2Fdrmaa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgruber%2Fdrmaa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgruber%2Fdrmaa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgruber%2Fdrmaa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dgruber","download_url":"https://codeload.github.com/dgruber/drmaa/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250530685,"owners_count":21445837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:25.875Z","updated_at":"2025-04-23T23:26:27.273Z","avatar_url":"https://github.com/dgruber.png","language":"Go","funding_links":[],"categories":["分布式系统","Distributed Systems","Relational Databases","\u003cspan id=\"分布式系统-distributed-systems\"\u003e分布式系统 Distributed Systems\u003c/span\u003e","分佈式系統"],"sub_categories":["SQL 查询语句构建库","检索及分析资料库","Advanced Console UIs","Search and Analytic Databases","高级控制台界面","\u003cspan id=\"高级控制台用户界面-advanced-console-uis\"\u003e高级控制台用户界面 Advanced Console UIs\u003c/span\u003e","高級控制台界面"],"readme":"go-drmaa\n========\n[![GoDoc](http://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/dgruber/drmaa)\n[![Apache V2 License](http://img.shields.io/badge/license-BSD-blue.svg)](https://github.com/dgruber/drmaa/blob/master/COPYING)\n[![Go Report Card](http://goreportcard.com/badge/github.com/dgruber/drmaa)](https://goreportcard.com/report/github.com/dgruber/drmaa)\n\nThis is a job submission library for Go (#golang) which is compatible to \nthe [DRMAA](http://drmaa.org) standard. The Go library is a wrapper around\nthe [DRMAA C library implementation](https://www.ogf.org/documents/GFD.22.pdf) provided\nby many distributed resource managers (cluster schedulers).\n\nThis library fully supports [Open Cluster Scheduler](https://github.com/hpc-gridware/clusterscheduler) and [Gridware Cluster Scheduler](https://www.hpc-gridware.com/).\n\nThe library was originally developed using [Univa Grid Engine's](http://www.univa.com) libdrmaa.so. It\nwas tested with different versions of Grid Engine, Torque, and [SLURM](https://github.com/natefoo/slurm-drmaa), but it should work also other resource managers / cluster schedulers which provide _libdrmaa.so_.\n\nThe \"gestatus\" subpackage only works with Grid Engine (some values are only available\non Univa Grid Engine).\n\nThe DRMAA (Distributed Resource Management Application API) standard is meanwhile\navailable in version 2. DRMAA2 provides more functionality around cluster monitoring\nand job session management. DRMAA and DRMAA2 are not compatible hence it is expected\nthat both libraries are co-existing for a while. The Go DRMAA2 can be found [here](https://github.com/dgruber/drmaa2os).\n\n## Compilation ##\n\nFirst download the package:\n\n~~~\n   export GOPATH=${GOPATH:-~/src/go}\n   mkdir -p $GOPATH\n   go get -d github.com/dgruber/drmaa\n   cd $GOPATH/github.com/dgruber/drmaa\n~~~\n\nNext, we need to compile the code.\n\nFor Univa Grid Engine and original SGE:\n\n~~~\n   source /path/to/grid/engine/installation/default/settings.sh\n   ./build.sh\n   cd examples/simplesubmit\n   go build\n   export LD_LIBRARY_PATH=$SGE_ROOT/lib/lx-amd64\n   ./simplesubmit\n~~~\n\nFor Son of Grid Engine (\"loveshack\"):\n\n~~~\n   source /path/to/grid/engine/installation/default/settings.sh\n   ./build.sh --sog\n   cd examples/simplesubmit\n   go build\n   export LD_LIBRARY_PATH=$SGE_ROOT/lib/lx-amd64\n   ./simplesubmit\n~~~\n\n\nFor [Torque](https://github.com/adaptivecomputing/torque/tree/master/src/drmaa):\n\nIf your Torque drmaa.h header file is not located under /usr/include/torque,\nyou will have to modify the build.sh script before running it.\n\n~~~\n   ./build.sh --torque\n   cd examples/simplesubmit\n   go build\n   ./simplesubmit\n~~~\n\nFor [SLURM](https://apps.man.poznan.pl/trac/slurm-drmaa) and [the updated \nSLURM C drmaa binding](https://github.com/natefoo/slurm-drmaa)\n\n~~~\n   ./build.sh --slurm /usr/local\n~~~\n\n\nThe example program submits a sleep job into the system and prints out detailed\njob information as soon as the job is started.\n\n## Short Introduction in Go DRMAA ##\n\nGo DRMAA applications need to open a DRMAA session before the DRMAA calls\ncan be executed. Opening a DRMAA session usually establishes a connection\nto the cluster scheduler (distributed resource manager). Hence if no more\nDRMAA calls are made the Exit() method of the session must be executed.\nThis tears down the connection. When an application does not call the Exit()\nmethod this can leave a communication handle open on the cluster scheduler\nside (which can take a while to be removed automatically). It should\nbe always avoided not to call Exit(). In Go the **defer** statement can be\nused but remember that the function is not executed when an *os.Exit()* call \nis made.\n\nCreating a DRMAA session:\n\n    s, err := drmaa.MakeSession()\n\nUsually jobs and job workflows are submitted within DRMAA applications.\nIn order to submit a job first a job template needs to be allocated:\n\n    jt, errJT := s.AllocateJobTemplate()\n    if errJT != nil {\n       fmt.Printf(\"Error during allocating a new job template: %s\\n\", errJT)\n       return\n    }\n\nUnderneath a C job template is allocated which is out-of-scope of the\nGo system. Hence it must be ensured that the job template is deleted\nwhen it is not used anymore. Also here the Go **defer** statement is useful.\n\n    // prevent memory leaks by freeing the allocated C job template at the end\n    defer s.DeleteJobTemplate(\u0026jt)\n\nThe job template contains the specification of the job, like the command\nto be executed and its parameters. Those can be set by the setter methods\nof the job.\n\n    // set the application to submit\n    jt.SetRemoteCommand(\"sleep\")\n    // set the parameter (use SetArgs() when having more parameters)\n    jt.SetArg(\"1\")\n\nA job can be executed with the session **RunJob()** method. If the same\ncommand should be executed many times, running it as a job array \nwould make sense. In Grid Engine each instance gets a task ID assigned\nwhich the job can see in the SGE_TASK_ID environment variable (which \nis set to **unknown** for normal jobs). This task ID can be used for \nfinding the right data set the job (array job task) needs to process.\nSubmitting an array job is done with the **RunBulkJobs()** method.\n\n\n    jobID, errSubmit := s.RunJob(\u0026jt)\n\n    // submitting 1000 instances of the same job\n    jobIDs, errBulkSubmit := s.RunBulkJobs(\u0026jt, 1, 1000, 1)\n\nA job state can also be changed (suspended / resumed / put in hold / deleted):\n\n    errTerm := s.TerminateJob(jobID)\n\nThe JobInfo data structure contains the runtime information of the \njob, like exit status or the amount of used resources (memory / IO / etc.).\nThe JobInfo data structure can be get with the **Wait()** method.\n\n    jinfo, errWait := s.Wait(jobID, drmaa.TimeoutWaitForever)\n\nFor more details please consult the documentation and the DRMAA \nstandard [specifications](https://www.ogf.org/ogf/doku.php/standards/standards).\n\nMore examples can be found on my blog at http://www.gridengine.eu.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgruber%2Fdrmaa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdgruber%2Fdrmaa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgruber%2Fdrmaa/lists"}