{"id":49504294,"url":"https://github.com/nebari-dev/nebari-rayserve-pack","last_synced_at":"2026-05-01T14:01:22.562Z","repository":{"id":347404047,"uuid":"1184305855","full_name":"nebari-dev/nebari-rayserve-pack","owner":"nebari-dev","description":null,"archived":false,"fork":false,"pushed_at":"2026-04-14T20:50:22.000Z","size":28,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-14T21:10:12.807Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nebari-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-17T13:11:06.000Z","updated_at":"2026-03-27T21:11:32.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nebari-dev/nebari-rayserve-pack","commit_stats":null,"previous_names":["nebari-dev/nebari-rayserve-pack"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nebari-dev/nebari-rayserve-pack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nebari-dev%2Fnebari-rayserve-pack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nebari-dev%2Fnebari-rayserve-pack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nebari-dev%2Fnebari-rayserve-pack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nebari-dev%2Fnebari-rayserve-pack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nebari-dev","download_url":"https://codeload.github.com/nebari-dev/nebari-rayserve-pack/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nebari-dev%2Fnebari-rayserve-pack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32499691,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-01T14:01:21.920Z","updated_at":"2026-05-01T14:01:22.552Z","avatar_url":"https://github.com/nebari-dev.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nebari Ray Serve Software Pack\n\nA [Nebari Software Pack](https://github.com/nebari-dev/nebari-software-pack-template) that deploys [Ray Serve](https://docs.ray.io/en/latest/serve/index.html) on Kubernetes using the [RayService CRD](https://docs.ray.io/en/latest/serve/production-guide/kubernetes.html), with optional routing, TLS, and OIDC authentication via the [nebari-operator](https://github.com/nebari-dev/nebari-operator).\n\n## Overview\n\nThis pack deploys a production-ready Ray Serve instance using the RayService CRD (the [recommended approach](https://docs.ray.io/en/latest/serve/production-guide/kubernetes.html) for running Ray Serve on Kubernetes).\n\n**What gets deployed:**\n\n- KubeRay operator (manages Ray cluster and Serve lifecycle)\n- RayService (Ray cluster + Serve proxy, pre-initialized with `host: 0.0.0.0`)\n- Stable Kubernetes Services for the dashboard and serve endpoint\n- NebariApp resources for external access via Envoy Gateway (optional)\n\n**Two access patterns:**\n\n| Access from | Path | Auth required? |\n|-------------|------|----------------|\n| Jupyter notebook (in-cluster) | Direct to K8s service | No |\n| Browser / external client | Envoy Gateway via NebariApp | Yes (if enabled) |\n\n## Prerequisites\n\n- [kubectl](https://kubernetes.io/docs/tasks/tools/)\n- [Helm 3](https://helm.sh/docs/intro/install/)\n- A Kubernetes cluster (or [kind](https://kind.sigs.k8s.io/) for local dev)\n\n## Quick Start\n\n### Standalone (no Nebari)\n\n```bash\ncd chart\nhelm dependency update .\nhelm install rayserve . --create-namespace -n rayserve --wait --timeout 5m\n```\n\nAccess via port-forward:\n\n```bash\n# Ray Dashboard\nkubectl port-forward svc/rayserve-nebari-rayserve-head-svc 8265:8265 -n rayserve\n\n# Ray Serve endpoint\nkubectl port-forward svc/rayserve-nebari-rayserve-serve-svc 8000:8000 -n rayserve\n```\n\n### On a Nebari cluster (via ArgoCD)\n\nAdd this to your GitOps repo as `apps/rayserve-pack.yaml`:\n\n```yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: rayserve-pack\n  namespace: argocd\n  annotations:\n    argocd.argoproj.io/sync-wave: \"7\"\n  finalizers:\n    - resources-finalizer.argocd.argoproj.io\nspec:\n  project: default\n  source:\n    repoURL: https://github.com/nebari-dev/nebari-rayserve-pack.git\n    targetRevision: main\n    path: chart\n    helm:\n      releaseName: rayserve\n      values: |\n        nebariapp:\n          enabled: true\n          serve:\n            enabled: false  # Keep serve endpoint internal-only\n          dashboard:\n            enabled: true\n            hostname: ray-dashboard.example.com\n          auth:\n            enabled: true\n            provider: keycloak\n            provisionClient: true\n            redirectURI: /oauth2/callback\n  destination:\n    server: https://kubernetes.default.svc\n    namespace: rayserve\n  syncPolicy:\n    automated:\n      prune: true\n      selfHeal: true\n    managedNamespaceMetadata:\n      labels:\n        nebari.dev/managed: \"true\"\n    syncOptions:\n      - CreateNamespace=true\n      - ServerSideApply=true\n      - SkipDryRunOnMissingResource=true\n      - RespectIgnoreDifferences=true\n    retry:\n      limit: 5\n      backoff:\n        duration: 5s\n        factor: 2\n        maxDuration: 3m\n  # The KubeRay controller modifies RayService and Service resources at\n  # runtime (adding selectors, status fields, etc.), causing a permanent\n  # OutOfSync state without these ignore rules.\n  ignoreDifferences:\n    - group: \"\"\n      kind: Service\n      jsonPointers:\n        - /spec/selector\n        - /spec/clusterIP\n        - /spec/clusterIPs\n    - group: ray.io\n      kind: RayService\n      jsonPointers:\n        - /spec/rayClusterConfig\n        - /status\n```\n\n**Important:**\n- `managedNamespaceMetadata` with `nebari.dev/managed: \"true\"` is required for the nebari-operator to manage NebariApp resources\n- `redirectURI` must be `/oauth2/callback` (Envoy Gateway rejects `/`)\n- Set `serve.enabled: false` to keep the serve endpoint internal-only (recommended — notebooks access it via cluster DNS)\n\n## Connecting from Jupyter\n\nFrom a notebook running in the same cluster (e.g., via the [nebari-data-science-pack](https://github.com/nebari-dev/nebari-data-science-pack)):\n\n```python\nimport ray\nfrom ray import serve\nimport requests\n\n# Connect to the Ray cluster\nray.init(\"ray://rayserve-nebari-rayserve-head-svc.rayserve.svc.cluster.local:10001\")\n\n# Deploy a model\n@serve.deployment\nclass Hello:\n    async def __call__(self, request):\n        return \"Hello from Ray Serve!\"\n\nserve.run(Hello.bind(), name=\"hello\", route_prefix=\"/hello\")\n\n# Run inference\nresp = requests.get(\"http://rayserve-nebari-rayserve-serve-svc.rayserve.svc.cluster.local:8000/hello\")\nprint(resp.text)\n# Hello from Ray Serve!\n```\n\nNo manual Serve initialization is needed — the RayService CRD starts the Serve proxy with `host: 0.0.0.0` automatically.\n\n**Note:** The Ray and Python versions in your Jupyter environment must match the Ray cluster. This chart deploys Ray 2.43.0 with Python 3.9 by default. If using [Nebi](https://github.com/nebari-dev/nebari-nebi-pack) for environment management, create a workspace with:\n\n```toml\n[workspace]\nname = \"ray-serve\"\nchannels = [\"conda-forge\"]\nplatforms = [\"linux-64\"]\n\n[dependencies]\npython = \"3.9.*\"\nray-serve = \"2.43.*\"\nipykernel = \"\u003e=6.0\"\n```\n\n## Deploying Models (Production)\n\nFor production, bake your model code into a custom Docker image and declare applications in `values.yaml`:\n\n```yaml\nimage:\n  repository: your-registry/your-ray-image\n  tag: \"2.43.0-custom\"\n\nserveApplications:\n  - name: my-model\n    route_prefix: /predict\n    import_path: myapp.model:app\n    deployments:\n      - name: MyModel\n        num_replicas: 2\n```\n\nThe RayService controller handles deployment, health monitoring, and zero-downtime upgrades automatically.\n\n## Chart Configuration\n\nKey values in `chart/values.yaml`:\n\n### Nebari Integration\n\n| Value | Default | Description |\n|-------|---------|-------------|\n| `nebariapp.enabled` | `false` | Create NebariApp resources for routing/TLS/auth |\n| `nebariapp.serve.enabled` | `true` | Expose the serve endpoint externally (set `false` to keep internal-only) |\n| `nebariapp.hostname` | - | Hostname for the Ray Serve endpoint (required when serve.enabled) |\n| `nebariapp.dashboard.enabled` | `true` | Create a separate NebariApp for the Ray Dashboard |\n| `nebariapp.dashboard.hostname` | - | Hostname for the Ray Dashboard (required when dashboard enabled) |\n| `nebariapp.auth.enabled` | `false` | Enable OIDC authentication via Keycloak |\n| `nebariapp.auth.redirectURI` | `/oauth2/callback` | OAuth callback path (Envoy Gateway rejects `/`) |\n| `nebariapp.gateway` | `public` | Gateway to use (`public` or `internal`) |\n\n### Ray Cluster\n\n| Value | Default | Description |\n|-------|---------|-------------|\n| `image.repository` | `rayproject/ray` | Ray container image |\n| `image.tag` | `2.43.0` | Ray version |\n| `head.resources.requests.cpu` | `1` | Head node CPU request |\n| `head.resources.requests.memory` | `2Gi` | Head node memory request |\n| `head.runtimeClassName` | - | Runtime class for head pod (e.g., `nvidia` for GPU) |\n| `worker.replicas` | `1` | Number of worker nodes |\n| `worker.minReplicas` | `1` | Min workers (for autoscaling) |\n| `worker.maxReplicas` | `1` | Max workers (for autoscaling) |\n| `worker.resources.requests.cpu` | `1` | Worker CPU request |\n| `worker.resources.requests.memory` | `2Gi` | Worker memory request |\n| `worker.runtimeClassName` | - | Runtime class for worker pods (e.g., `nvidia` for GPU) |\n\n### Serve Applications\n\n| Value | Default | Description |\n|-------|---------|-------------|\n| `serveApplications` | `[]` | Declarative Serve applications (see [Ray Serve config](https://docs.ray.io/en/latest/serve/production-guide/config.html)) |\n\n## Architecture\n\n```mermaid\nflowchart TD\n    subgraph KO[\"KubeRay Operator\"]\n        op[\"Manages RayService lifecycle\"]\n    end\n\n    subgraph RS[\"RayService CRD\"]\n        subgraph RC[\"RayCluster\"]\n            head[\"Head Pod\\n:8265 dashboard\\n:8000 serve\\n:10001 client\"]\n            workers[\"Worker Pod(s)\\nRay Workers\"]\n        end\n    end\n\n    subgraph SVC[\"Kubernetes Services\"]\n        headsvc[\"-head-svc\\n:8265 :10001 :6379\"]\n        servesvc[\"-serve-svc\\n:8000\"]\n    end\n\n    subgraph NB[\"NebariApp (optional)\"]\n        route[\"HTTPRoute + OIDC auth\\nvia Envoy Gateway\"]\n    end\n\n    jupyter[\"Jupyter Notebook\\n(in-cluster)\"]\n    browser[\"Browser\\n(external)\"]\n\n    KO --\u003e RS\n    head --- workers\n    head --\u003e headsvc\n    head --\u003e servesvc\n    servesvc --\u003e route\n\n    jupyter --\u003e|\"ray:// :10001\"| headsvc\n    jupyter --\u003e|\"HTTP :8000\"| servesvc\n    browser --\u003e|\"HTTPS\"| route\n\n    style KO fill:#fef0db,stroke:#e8952c,color:#7c4a03\n    style RS fill:#eeeef3,stroke:#4a4a6a,color:#1a1a2e\n    style RC fill:#e8faf8,stroke:#20aaa1,color:#0d5d57\n    style SVC fill:#d4f5f2,stroke:#20aaa1,color:#0d5d57\n    style NB fill:#f3e8fc,stroke:#c840e9,color:#6b21a8\n```\n\n## Repository Structure\n\n```\nnebari-rayserve-pack/\n  chart/\n    Chart.yaml                 # Depends on kuberay-operator\n    values.yaml                # RayService + NebariApp configuration\n    templates/\n      _helpers.tpl             # Name, label, and service name helpers\n      rayservice.yaml          # RayService CRD\n      services.yaml            # Stable head and serve K8s Services\n      nebariapp.yaml           # NebariApp CRDs (conditional)\n      NOTES.txt                # Post-install usage instructions\n  dev/\n    Makefile                   # Local dev with full Nebari stack on kind\n```\n\n## Troubleshooting\n\n### Ray Dashboard returns 500 via NebariApp\n\nThe NebariApp may be pointing at a service that doesn't exist. Check the actual service name:\n\n```bash\nkubectl get svc -n rayserve\n```\n\nThe stable services are `\u003crelease\u003e-nebari-rayserve-head-svc` and `\u003crelease\u003e-nebari-rayserve-serve-svc`.\n\n### Version mismatch connecting from Jupyter\n\nThe Ray and Python versions in your notebook environment must match the cluster:\n\n```bash\nkubectl exec -n rayserve $(kubectl get pod -n rayserve -l ray.io/node-type=head -o name) -- ray --version\nkubectl exec -n rayserve $(kubectl get pod -n rayserve -l ray.io/node-type=head -o name) -- python --version\n```\n\n### NebariApp not reaching Ready\n\nCheck that the namespace has the managed label:\n\n```bash\nkubectl get namespace rayserve --show-labels | grep nebari.dev/managed\n```\n\nIf missing, add it (or use `managedNamespaceMetadata` in the ArgoCD app):\n\n```bash\nkubectl label namespace rayserve nebari.dev/managed=true\n```\n\n### JupyterHub notebooks can't reach Ray\n\nThe default JupyterHub singleuser network policy blocks egress to private IPs. Add this to your data science pack values:\n\n```yaml\njupyterhub:\n  singleuser:\n    networkPolicy:\n      egressAllowRules:\n        privateIPs: true\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnebari-dev%2Fnebari-rayserve-pack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnebari-dev%2Fnebari-rayserve-pack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnebari-dev%2Fnebari-rayserve-pack/lists"}