{"id":18964042,"url":"https://github.com/JaneliaSciComp/ray-janelia","last_synced_at":"2025-04-16T05:31:40.817Z","repository":{"id":38279665,"uuid":"498373294","full_name":"JaneliaSciComp/ray-janelia","owner":"JaneliaSciComp","description":"Run Python Ray on the Janelia cluster","archived":false,"fork":false,"pushed_at":"2022-09-08T17:07:37.000Z","size":19,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-29T08:03:00.488Z","etag":null,"topics":["cluster","compute-clusters","distributed-computing","janelia","ray"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JaneliaSciComp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-31T14:35:08.000Z","updated_at":"2022-05-31T21:35:23.000Z","dependencies_parsed_at":"2022-08-18T11:31:24.625Z","dependency_job_id":null,"html_url":"https://github.com/JaneliaSciComp/ray-janelia","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fray-janelia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fray-janelia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fray-janelia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fray-janelia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JaneliaSciComp","download_url":"https://codeload.github.com/JaneliaSciComp/ray-janelia/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249201117,"owners_count":21229004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","compute-clusters","distributed-computing","janelia","ray"],"created_at":"2024-11-08T14:22:38.364Z","updated_at":"2025-04-16T05:31:40.601Z","avatar_url":"https://github.com/JaneliaSciComp.png","language":"Shell","readme":"# ray-janelia\n\nThese scripts let you run [Ray](https://github.com/ray-project/ray) on the Janelia cluster (and maybe other LSF clusters).\n\nYou must have a Conda environment with [Ray installed](https://docs.ray.io/en/latest/ray-overview/installation.html).\n\n\n## Create a cluster\n\nThis command will start a 20 slot cluster, using a conda environment called `ray-python`:\n\n```bash\nray-janelia/ray-launch.sh -n 20 -e ray-python\n```\n\nBy default, the cluster will be divided into nodes of 4 slots each. To use a different tiling, specify the number of nodes you want with `-d \u003cnodes\u003e`.\n\nThis command will start a cluster with 20 CPU and 2 GPU slots on a GPU enabled queue `gpu_queue`:\n\n```bash\nray-janelia/ray-launch.sh -n 20 -e ray-python -b \"-q gpu_queue -gpu num=2\"\n```\n\n## Run a job on a cluster\n\nThe output of launching the cluster above will print a remote address like `ray://head_node:10001`. You can simply pass this address into your job when creating your Ray client, like this:\n\n```python\nray.init(address=\"ray://head_node:10001\")\n```\n\nThe output will also print the address of the [Ray dashboard](https://docs.ray.io/en/latest/ray-core/ray-dashboard.html) for the launched Ray cluster.\n\n## Create a cluster, run a job, then shut it down\n\nAnother option is to create a cluster and run a python job with a single command:\n\n```bash\n./ray-launch.sh -n 20 -e ray-python -p \"/path/to/job.py --options\"\n```\n\nIn this case, to connect to the Ray cluster created with the `ray-launch.sh` script, the python script `job.py` should contain:\n\n```python\nray.init(address=\"auto\")\n```\n\nWhen the python script completes, the Ray cluster will be automatically shut down and the Janelia cluster job will be terminated.\n","funding_links":[],"categories":["Janelia Site-specific Software"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJaneliaSciComp%2Fray-janelia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJaneliaSciComp%2Fray-janelia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJaneliaSciComp%2Fray-janelia/lists"}