{"id":17604277,"url":"https://github.com/danielgafni/dagster-ray","last_synced_at":"2026-01-25T23:17:50.744Z","repository":{"id":191955724,"uuid":"686084317","full_name":"danielgafni/dagster-ray","owner":"danielgafni","description":"Ray integration for Dagster","archived":false,"fork":false,"pushed_at":"2026-01-23T16:47:36.000Z","size":4298,"stargazers_count":61,"open_issues_count":40,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-01-24T07:24:55.393Z","etag":null,"topics":["dagster","dagster-pipes","kubernetes","orchestration","ray"],"latest_commit_sha":null,"homepage":"https://danielgafni.github.io/dagster-ray","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danielgafni.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-09-01T17:48:01.000Z","updated_at":"2026-01-23T15:47:51.000Z","dependencies_parsed_at":"2024-04-18T16:32:16.034Z","dependency_job_id":"7ae56b6f-ac32-47ec-8293-31335b8c279e","html_url":"https://github.com/danielgafni/dagster-ray","commit_stats":null,"previous_names":["danielgafni/dagster-ray"],"tags_count":42,"template":false,"template_full_name":null,"purl":"pkg:github/danielgafni/dagster-ray","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgafni%2Fdagster-ray","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgafni%2Fdagster-ray/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgafni%2Fdagster-ray/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgafni%2Fdagster-ray/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danielgafni","download_url":"https://codeload.github.com/danielgafni/dagster-ray/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielgafni%2Fdagster-ray/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28761813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T23:06:19.311Z","status":"ssl_error","status_checked_at":"2026-01-25T23:03:50.555Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dagster","dagster-pipes","kubernetes","orchestration","ray"],"created_at":"2024-10-22T14:07:52.082Z","updated_at":"2026-01-25T23:17:50.738Z","avatar_url":"https://github.com/danielgafni.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `dagster-ray`\n\n[![PyPI version](https://img.shields.io/pypi/v/dagster-ray.svg)](https://pypi.python.org/pypi/dagster-ray)\n[![License](https://img.shields.io/pypi/l/dagster-ray.svg)](https://pypi.python.org/pypi/dagster-ray)\n[![Python versions](https://img.shields.io/pypi/pyversions/dagster-ray.svg)](https://pypi.python.org/pypi/dagster-ray)\n[![CI](https://github.com/danielgafni/dagster-ray/actions/workflows/CI.yml/badge.svg)](https://github.com/danielgafni/dagster-ray/actions/workflows/CI.yml)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit)\n[![basedpyright - checked](https://img.shields.io/badge/basedpyright-checked-42b983)](https://docs.basedpyright.com)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\n---\n\n**Ray integration for Dagster.**\n\n`dagster-ray` enables working with distributed Ray compute from Dagster pipelines, combining Dagster's excellent orchestration capabilities and Ray's distributed computing power together.\n\n\u003e [!NOTE]\n\u003e This project is ready for production use, but some APIs may change between minor releases.\n\nLearn more in the [docs](https://danielgafni.github.io/dagster-ray)\n\n## 🚀 Key Features\n\n- **🎯 Run Launchers \u0026 Executors**: Submit Dagster runs or individual steps by submitting Ray jobs\n- **🔧 Ray Resources**: Automatically create and destroy ephemeral Ray clusters and connect to them in client mode\n- **📡 Dagster Pipes Integration**: Submit external scripts as Ray jobs, stream back logs and rich Dagster metadata\n- **☸️ KubeRay Support**: Utilize `RayJob` and `RayCluster` custom resources in client or job submission mode ([tutorial](https://danielgafni.github.io/dagster-ray/tutorial/kuberay))\n- **🏭 Production Ready**: Tested against a matrix of core dependencies, integrated with Dagster+\n- **⚡ Instant Startup**: Leverage `RayCluster` with [Cluster Sharing](https://danielgafni.github.io/dagster-ray/tutorial/kuberay/#cluster-sharing) for lightning-fast development cycles with zero cold start times (intended for development environments)\n\n## Installation\n\n```shell\npip install dagster-ray\n```\n\n## 📚 Docs\n\n**📖 [Full Documentation](https://danielgafni.github.io/dagster-ray)**\n\n- **[Tutorial](https://danielgafni.github.io/dagster-ray/tutorial/)**: Step-by-step guide with examples\n- **[API Reference](https://danielgafni.github.io/dagster-ray/api/)**: Complete API reference\n\n## 🤝 Contributing\n\nContributions are very welcome! To get started:\n\n```bash\nuv sync --all-extras --all-groups\nuv run pre-commit install\n```\n\n### 🧪 Testing\n\n```bash\nuv run pytest\n```\n\nRunning KubeRay tests requires the following tools to be present:\n- `docker`\n- `kubectl`\n- `helm`\n- `minikube`\n\n❄️ Nix users will find them provided in the dev shell:\n\n```\nnix develop\n```\n\n### Documentation\n\nTo build and serve the documentation locally:\n\n```bash\n# Serve documentation locally\nuv run --group docs mkdocs serve\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielgafni%2Fdagster-ray","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanielgafni%2Fdagster-ray","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielgafni%2Fdagster-ray/lists"}