{"id":27918768,"url":"https://github.com/juliaparallel/clustermanagers.jl","last_synced_at":"2025-05-06T18:25:25.849Z","repository":{"id":9389066,"uuid":"11250727","full_name":"JuliaParallel/ClusterManagers.jl","owner":"JuliaParallel","description":"Run jobs on different job queue systems (schedulers) commonly used on HPC compute clusters","archived":false,"fork":false,"pushed_at":"2025-04-29T05:10:10.000Z","size":233,"stargazers_count":255,"open_issues_count":12,"forks_count":76,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-29T05:30:23.475Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaParallel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"custom":"https://numfocus.salsalabs.org/donate-to-julia/index.html"}},"created_at":"2013-07-08T09:32:59.000Z","updated_at":"2025-04-29T04:48:45.000Z","dependencies_parsed_at":"2023-01-13T15:19:56.738Z","dependency_job_id":"c8eba60a-b67e-400f-b0de-4db863d47656","html_url":"https://github.com/JuliaParallel/ClusterManagers.jl","commit_stats":{"total_commits":150,"total_committers":54,"mean_commits":"2.7777777777777777","dds":0.8733333333333333,"last_synced_commit":"2dac92c3d0ed137655654902b474a8db767b2600"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FClusterManagers.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FClusterManagers.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FClusterManagers.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FClusterManagers.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaParallel","download_url":"https://codeload.github.com/JuliaParallel/ClusterManagers.jl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252742668,"owners_count":21797290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-06T18:25:24.993Z","updated_at":"2025-05-06T18:25:25.838Z","avatar_url":"https://github.com/JuliaParallel.png","language":"Julia","funding_links":["https://numfocus.salsalabs.org/donate-to-julia/index.html"],"categories":[],"sub_categories":[],"readme":"# ClusterManagers.jl\n\nThe `ClusterManagers.jl` package implements code for different job queue systems commonly used on compute clusters.\n\n\u003e [!WARNING]\n\u003e This package is not currently being actively maintained or tested.\n\u003e\n\u003e We are in the process of splitting this package up into multiple smaller packages, with a separate package for each job queue systems.\n\u003e\n\u003e We are seeking maintainers for these new packages. If you are an active user of any of the job queue systems listed below and are interested in being a maintainer, please open a GitHub issue - say that you are interested in being a maintainer, and specify which job queue system you use.\n\n## Available job queue systems\n\n### In this package\n\nThe following managers are implemented in this package (the `ClusterManagers.jl` package):\n\n| Job queue system | Command to add processors |\n| ---------------- | ------------------------- |\n| Local manager with CPU affinity setting | `addprocs(LocalAffinityManager(;np=CPU_CORES, mode::AffinityMode=BALANCED, affinities=[]); kwargs...)` |\n\n### Implemented in external packages\n\n| Job queue system | External package | Command to add processors |\n| ---------------- | ---------------- | ------------------------- |\n| Slurm | [SlurmClusterManager.jl](https://github.com/JuliaParallel/SlurmClusterManager.jl) | `addprocs(SlurmManager(); kwargs...)` |\n| Load Sharing Facility (LSF) | [LSFClusterManager.jl](https://github.com/JuliaParallel/LSFClusterManager.jl) | `addprocs_lsf(np::Integer; bsub_flags=``, ssh_cmd=``)` or `addprocs(LSFManager(np, bsub_flags, ssh_cmd, retry_delays, throttle))` |\n| ElasticManager | [ElasticClusterManager.jl](https://github.com/JuliaParallel/ElasticClusterManager.jl) | `addprocs(ElasticManager(...); kwargs...)` |\n| Kubernetes (K8s) | [K8sClusterManagers.jl](https://github.com/beacon-biosignals/K8sClusterManagers.jl) | `addprocs(K8sClusterManager(np; kwargs...))` |\n| Azure scale-sets | [AzManagers.jl](https://github.com/ChevronETC/AzManagers.jl) | `addprocs(vmtemplate, n; kwargs...)` |\n\n### Not currently being actively maintained\n\n\u003e [!WARNING]\n\u003e The following managers are not currently being actively maintained or tested.\n\u003e\n\u003e We are seeking maintainers for the following managers. If you are an active user of any of the following job queue systems listed and are interested in being a maintainer, please open a GitHub issue - say that you are interested in being a maintainer, and specify which job queue system you use.\n\u003e\n\n| Job queue system | Command to add processors |\n| ---------------- | ------------------------- |\n| Sun Grid Engine (SGE) via `qsub` | `addprocs_sge(np::Integer; qsub_flags=``)` or `addprocs(SGEManager(np, qsub_flags))` |\n| Sun Grid Engine (SGE) via `qrsh` | `addprocs_qrsh(np::Integer; qsub_flags=``)` or `addprocs(QRSHManager(np, qsub_flags))` |\n| PBS (Portable Batch System) | `addprocs_pbs(np::Integer; qsub_flags=``)` or `addprocs(PBSManager(np, qsub_flags))` |\n| Scyld | `addprocs_scyld(np::Integer)` or `addprocs(ScyldManager(np))` |\n| HTCondor | `addprocs_htc(np::Integer)` or `addprocs(HTCManager(np))` |\n\n### Custom managers\n\nYou can also write your own custom cluster manager; see the instructions in the [Julia manual](https://docs.julialang.org/en/v1/manual/distributed-computing/#ClusterManagers).\n\n## Notes on specific managers\n\n### Slurm: please see [SlurmClusterManager.jl](https://github.com/JuliaParallel/SlurmClusterManager.jl)\n\nFor Slurm, please see the [SlurmClusterManager.jl](https://github.com/JuliaParallel/SlurmClusterManager.jl) package.\n\n### Using `LocalAffinityManager` (for pinning local workers to specific cores)\n\nSee [`docs/local_affinity.md`](docs/local_affinity.md)\n\n### Using `ElasticManager` (dynamically adding workers to a cluster)\n\nFor `ElasticManager`, please see the [ElasticClusterManager.jl](https://github.com/JuliaParallel/ElasticClusterManager.jl) package.\n\n### Sun Grid Engine (SGE)\n\nSee [`docs/sge.md`](docs/sge.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaparallel%2Fclustermanagers.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuliaparallel%2Fclustermanagers.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaparallel%2Fclustermanagers.jl/lists"}