{"id":20812536,"url":"https://github.com/superstreamlabs/superstream-engine","last_synced_at":"2026-02-28T20:02:46.689Z","repository":{"id":225338117,"uuid":"764695179","full_name":"superstreamlabs/superstream-engine","owner":"superstreamlabs","description":"Improve And Optimize Your Kafka In Literally Minutes.","archived":false,"fork":false,"pushed_at":"2025-12-19T16:28:08.000Z","size":676,"stargazers_count":5,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-12-22T06:50:50.431Z","etag":null,"topics":["kafka"],"latest_commit_sha":null,"homepage":"https://superstream.ai","language":"Smarty","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superstreamlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-28T14:48:58.000Z","updated_at":"2025-12-19T16:28:12.000Z","dependencies_parsed_at":"2024-04-14T19:23:14.123Z","dependency_job_id":"dad9055b-59bc-467e-b95a-85e034793148","html_url":"https://github.com/superstreamlabs/superstream-engine","commit_stats":null,"previous_names":["memphisdev/superstream-dp-deployment","memphisdev/superstream-engine","superstreamlabs/superstream-engine"],"tags_count":186,"template":false,"template_full_name":null,"purl":"pkg:github/superstreamlabs/superstream-engine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fsuperstream-engine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fsuperstream-engine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fsuperstream-engine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fsuperstream-engine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superstreamlabs","download_url":"https://codeload.github.com/superstreamlabs/superstream-engine/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fsuperstream-engine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29951075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T18:42:55.706Z","status":"ssl_error","status_checked_at":"2026-02-28T18:42:48.811Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka"],"created_at":"2024-11-17T20:55:20.060Z","updated_at":"2026-02-28T20:02:46.664Z","avatar_url":"https://github.com/superstreamlabs.png","language":"Smarty","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"https://github.com/user-attachments/assets/35899c78-24eb-4507-97ed-e87e84c49fea#gh-dark-mode-only\" width=\"500\"\u003e\n\u003cimg src=\"https://github.com/user-attachments/assets/8a7bca49-c362-4a8c-945e-a331fb26d8eb#gh-light-mode-only\" width=\"500\"\u003e\n\n\u003cp\u003e\n  Superstream is an autonomous platform designed\u003cbr\u003eto optimize data infrastructure for cost reduction and reduced operations.\u003cbr\u003e\n  \u003ca href=\"https://superstream.ai\"\u003eWebsite\u003c/a\u003e | \u003ca href=\"https://docs.superstream.ai\"\u003eDocs\u003c/a\u003e\n\u003c/p\u003e\n\n\u003c/div\u003e\n\n#### This chart is for Superstream customers who prefer to host a local engine\n\n## Configure Environment Tokens\n\nFor easiness, create `custom_values.yaml` file and edit the following values:\n```yaml\n############################################################\n# GLOBAL configuration for Superstream Engine\n############################################################\nglobal:\n  engineName: \"\"                   # Define the superstream engine name within 32 characters, excluding '.', and using only lowercase letters, numbers, '-', and '_'.\n  superstreamAccountId: \"\"         # Provide the account ID associated with the deployment, which could be used for identifying resources or configurations tied to a specific account.\n  superstreamActivationToken: \"\"   # Enter the activation token required for services or resources that need an initial token for activation or authentication.\n  skipLocalAuthentication: true\n\n############################################################\n# NATS config\n############################################################\n# NATS HA Deployment. Default \"true\"\nnats:\n  config:\n    cluster:\n      enabled: true\n# NATS storageClass configuration. The default is blank \"\".\n    jetstream:\n      fileStore:\n        pvc:\n          storageClassName: \"\"\n############################################################\n# Kafka Autoscaler config\n############################################################\n# Optional service to automatically scale the Kafka cluster up/down based on CPU and memory metrics  \nautoScaler:\n  enabled: true \n```\n\nTo deploy it, run the following:\n```bash\nhelm repo add superstream https://k8s.superstream.ai/ --force-update \u0026\u0026 helm upgrade --install superstream superstream/superstream -f custom_values.yaml --create-namespace --namespace superstream --wait\n```\n\n## Parameters\nThe following table lists the configurable parameters of the SuperStream chart and their default values:\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `global.engineName`                                       | Define the superstream engine name within 32 characters, excluding '.', and using only lowercase letters, numbers, '-', and '_'. | `\"\"`|\n| `global.superstreamAccountId`                             | Provide the account ID associated with the deployment, which could be used for identifying resources or configurations tied to a specific account. | `\"\"`|\n| `global.superstreamActivationToken`                       | Enter the activation token required for services or resources that need an initial token for activation or authentication. | `\"\"` |\n| `global.skipLocalAuthentication`                          | Specifies whether to skip local authentication.                                      | `true`                             |\n| `global.image.pullPolicy`                                 | Global image pull policy to use for all container images in the chart. Can be overridden by individual image pullPolicy. | `\"\"` |\n| `global.image.pullSecretNames`                            | Global list of secret names to use as image pull secrets for all pod specs in the chart. Secrets must exist in the same namespace. | `[]` |\n| `global.image.registry`                                   | Global registry to use for all container images in the chart. Can be overridden by individual image registry. | `\"\"` |\n| `global.labels`                                   | Global labels to use for all container images in the chart. | `\"\"` |\n| `nats.config.cluster.enabled`                             | Indicates whether the NATS cluster is enabled.                                      | `true`                             |\n| `nats.config.jetstream.fileStore.pvc.storageClassName`    | Specifies the storage class name for the Jetstream file store PVC.                  | `\"\"`                               |\n| `nats.config.nats.tls.enabled`                             | Enables or disables TLS (Transport Layer Security) for the NATS server. Set to `true` to enable TLS.                                      | `false`                             |\n| `nats.config.nats.tls.secretName`                             | If provided, mounts an existing secret to the directory for TLS credentials. Useful for referencing pre-existing certificates and keys.                                      | `\"\"`                             |\n| `nats.config.nats.tls.localCa.enabled`                             | Enables or disables the use of a local Certificate Authority (CA) for generating TLS certificates.                                      | `false`                             |\n| `nats.config.nats.tls.localCa.secretName`                             | The name of the secret containing the local CA’s certificates. Required if `nats.config.nats.tls.localCa.enabled` is set to `true`.                                      | `false`                             |\n| `superstreamEngine.releaseDate`                           | Release date for the backend component.                                             | `\"\"`               |\n| `superstreamEngine.replicaCount`                          | Number of replicas for the backend deployment.                                      | `2`                                |\n| `superstreamEngine.image.repository`                      | Docker image repository for the backend service.                                    | `superstreamlabs/superstream-data-plane-be` |\n| `superstreamEngine.image.pullPolicy`                      | Policy for pulling the image.                                                       | `Always`                           |\n| `superstreamEngine.image.tag`                             | Overrides the image tag.                                                            | `\"latest\"`                         |\n| `superstreamEngine.imagePullSecrets`                      | Image pull secrets.                                                                 | `[]`                               |\n| `superstreamEngine.nameOverride`                          | Overrides for Helm's default naming conventions.                                    | `\"\"`                               |\n| `superstreamEngine.fullnameOverride`                      | Full name override for Helm's default naming conventions.                           | `\"\"`                               |\n| `superstreamEngine.initContainers.image`                  | Image used for readiness checks or setup tasks before the main containers start.    | `curlimages/curl:8.6.0`            |\n| `superstreamEngine.configMap.enable`                      | Enable ConfigMap settings.                                                          | `\"\"`                               |\n| `superstreamEngine.configMap.useExisting`                 | Determines whether to use an existing ConfigMap instead of creating a new one.      | `false`                            |\n| `superstreamEngine.configMap.existingConfigMapName`       | Name of the existing ConfigMap to use if `useExisting` is `true`.                   | `my-existing-configmap`            |\n| `superstreamEngine.configMap.name`                        | Name used when creating a new ConfigMap.                                            | `\"\"`                               |\n| `superstreamEngine.configMap.fileData.mountPath`          | Path where the ConfigMap will be mounted in the pod.                                | `/etc/superstream-conf/`           |\n| `superstreamEngine.configMap.fileData.fileName`           | Name of the file to be used from the ConfigMap.                                     | `\"\"`                               |\n| `superstreamEngine.configMap.fileData.fileContent`        | Content of the file to be used from the ConfigMap.                                  | `\"\"`                               |\n| `superstreamEngine.secret.name`                           | Secret configuration for sensitive information.                                     | `superstream-creds`                |\n| `superstreamEngine.secret.encryptionSecretKey`            | Encryption secret key used for sensitive information.                               | `\"\"`                               |\n| `superstreamEngine.secret.activationToken`                | Activation token for services or resources.                                         | `\"\"`                               |\n| `superstreamEngine.secret.useExisting`                    | Specifies whether to use an existing secret.                                        | `false`                            |\n| `superstreamEngine.podAnnotations.prometheus.io/path`     | Path for Prometheus to scrape metrics from the pod.                                 | `\"/monitoring/metrics\"`            |\n| `superstreamEngine.podAnnotations.prometheus.io/scrape`   | Specifies whether Prometheus should scrape metrics from the pod.                    | `'true'`                           |\n| `superstreamEngine.podAnnotations.prometheus.io/port`     | Port for Prometheus to scrape metrics from the pod.                                 | `\"7777\"`                           |\n| `superstreamEngine.podSecurityContext`                    | Security context settings for the pod.                                              | `{}`                               |\n| `superstreamEngine.securityContext`                       | Security context for containers within the pod.                                     | `{}`                               |\n| `superstreamEngine.serviceAccount.create`                 | Specifies whether a service account should be created.                              | `true`                             |\n| `superstreamEngine.serviceAccount.annotations`            | Annotations to add to the service account.                                          | `{}`                               |\n| `superstreamEngine.serviceAccount.name`                   | The name of the service account to use.                                             | `\"\"`                               |\n| `superstreamEngine.extraEnv`                    |\tA map of additional environment variables for the application.\t| `{}` |\n| `superstreamEngine.service.enabled`                       | Enable service for the backend.                                                     | `true`                             |\n| `superstreamEngine.service.type`                          | Type of service for the backend.                                                    | `ClusterIP`                        |\n| `superstreamEngine.service.port`                          | Port for the backend service.                                                       | `7777`                             |\n| `superstreamEngine.resources.limits.cpu`                  | CPU limit for the backend pod.                                                      | `8`                                |\n| `superstreamEngine.resources.limits.memory`               | Memory limit for the backend pod.                                                   | `8Gi`                              |\n| `superstreamEngine.resources.requests.cpu`                | CPU request for the backend pod.                                                    | `500m`                             |\n| `superstreamEngine.resources.requests.memory`             | Memory request for the backend pod.                                                 | `1Gi`                              |\n| `superstreamEngine.autoscaling.enabled`                   | Enable autoscaling for the backend.                                                 | `true`                             |\n| `superstreamEngine.autoscaling.minReplicas`               | Minimum number of replicas for autoscaling.                                         | `2`                                |\n| `superstreamEngine.autoscaling.maxReplicas`               | Maximum number of replicas for autoscaling.                                         | `5`                                |\n| `superstreamEngine.autoscaling.targetCPUUtilizationPercentage` | CPU utilization percentage for autoscaling.                                         | `75`                               |\n| `superstreamEngine.autoscaling.targetMemoryUtilizationPercentage` | Memory utilization percentage for autoscaling.                                      | `75`                               |\n| `superstreamEngine.nodeSelector`                          | Node selectors to control the placement of pods.                                    | `{}`                               |\n| `superstreamEngine.tolerations`                           | Tolerations for pods to tolerate certain node conditions or taints.                 | `[]`                               |\n| `superstreamEngine.affinity`                              | Affinity rules for pod scheduling.                                                  | `{}`                               |\n| `superstreamEngine.internalNatsConnection.host`           | Host for the internal NATS connection.                                              | `\"\"`                               |\n| `superstreamEngine.internalNatsConnection.port`           | Port for the internal NATS connection.                                              | `4222`                             |\n| `superstreamEngine.controlPlane.host`                     | Host for the control plane connection.                                              | `\"broker.superstream.dev\"`         |\n| `superstreamEngine.controlPlane.port`                     | Port for the control plane connection.                                              | `4222`                             |\n| `superstreamEngine.syslog.enabled`                        | Determines whether the syslog is enabled for the superstream engine.                | `true`                             |\n| `superstreamEngine.syslog.remoteSyslog`                   | Remote syslog server to send logs to.                                               | `\"superstream-syslog\"`             |\n| `superstreamEngine.releaseDate`                           | Release date for the backend component.                                             | `\"2024-03-20-11-12\"` \n| `autoScaler.enabled`                           | Enables the Kafka auto-scaler.                                             | `\"true\"`                |\n| `autoScaler.releaseDate`                           | Release date for the autoscaler.                                             | `\"\"`               |\n| `autoScaler.replicaCount`                           | Enables the Kafka auto-scaler.                                             | `\"false\"`                |\n| `autoScaler.image.repository`                           | Docker image repository for the Kafka auto-scaler.                                             | `\"superstreamlabs/superstream-kafka-auto-scaler\"`                |\n| `autoScaler.image.pullPolicy`                           | Policy for pulling the Docker image.                                             | `Always`                |\n| `autoScaler.image.tag`                           | Docker image tag (Overrides the image).                                             | `\"latest\"`                |\n| `autoScaler.resources.limits.cpu`                           | CPU limit for the auto-scaler.                                             | `\"2\"`                |\n| `autoScaler.resources.limits.memory`                           | Memory limit for the auto-scaler.                                             | `\"2Gi\"`                |\n| `autoScaler.resources.requests.cpu`                           | CPU request for the auto-scaler.                                             | `\"500m\"`                |\n| `autoScaler.resources.requests.memory`                           | Memory request for the auto-scaler.                                             | `\"500Mi\"`                |\n| `autoScaler.autoscaling.enabled`                   | Enable autoscaling for the backend.                                                 | `true`                             |\n| `autoScaler.autoscaling.minReplicas`               | Minimum number of replicas for autoscaling.                                         | `2`                                |\n| `autoScaler.autoscaling.maxReplicas`               | Maximum number of replicas for autoscaling.                                         | `5`                                |\n| `autoScaler.autoscaling.targetCPUUtilizationPercentage` | CPU utilization percentage for autoscaling.                                         | `75`                               |\n| `autoScaler.autoscaling.targetMemoryUtilizationPercentage` | Memory utilization percentage for autoscaling.                                      | `75`                               |\n| `autoScaler.extraEnv`                    |\tA map of additional environment variables for the application.\t| `{}` |\n| `autoScaler.nodeSelector`                          | Node selectors to control the placement of pods.                                    | `{}`                               |\n| `autoScaler.tolerations`                           | Tolerations for pods to tolerate certain node conditions or taints.                 | `[]`                               |\n| `autoScaler.affinity`                              | Affinity rules for pod scheduling.                                                  | `{}`                               |\n| `syslog.replicaCount`                                     | Number of replicas for the syslog deployment.                                       | `1`                                |\n| `syslog.image.repository`                                 | Docker image repository for syslog.                                                 | `linuxserver/syslog-ng`            |\n| `syslog.image.pullPolicy`                                 | Pull policy for the syslog image.                                                   | `IfNotPresent`                     |\n| `syslog.image.tag`                                        | Tag for the syslog image.                                                           | `\"4.5.0\"`                          |\n| `syslog.imagePullSecrets`                                 | Image pull secrets.                                                                 | `[]`                               |\n| `syslog.service.type`                                     | Type of service for syslog.                                                         | `ClusterIP`                        |\n| `syslog.service.port`                                     | Port for the syslog service.                                                        | `5514`                             |\n| `syslog.service.protocol`                                 | Protocol for the syslog service.                                                    | `UDP`                              |\n| `syslog.extraEnv`                    |\tA map of additional environment variables for the application.\t| `{}` |\n| `syslog.resources.limits.cpu`                             | CPU limit for the syslog pod.                                                       | `\"100m\"`                           |\n| `syslog.resources.limits.memory`                          | Memory limit for the syslog pod.                                                    | `\"256Mi\"`                          |\n| `syslog.resources.requests.cpu`                           | CPU request for the syslog pod.                                                     | `\"50m\"`                            |\n| `syslog.resources.requests.memory`                        | Memory request for the syslog pod.                                                  | `\"128Mi\"`                          |\n| `syslog.podAnnotations.prometheus.io/scrape`              | Specifies whether Prometheus should scrape metrics from the syslog pod.             | `'false'`                          |\n| `syslog.remoteSyslog.destinationHost`                     | Destination host for the remote syslog.                                             | `telegraf`                         |\n| `syslog.remoteSyslog.port`                                | Port for the remote syslog.                                                         | `6514`                             |\n| `syslog.remoteSyslog.protocol`                            | Protocol (e.g., UDP) for the remote syslog.                                         | `udp`                              |\n| `syslog.configMap.enabled`                                | Enable ConfigMap for syslog.                                                        | `true`                             |\n| `syslog.configMap.name`                                   | Name of the ConfigMap for syslog.                                                   | `syslog-config`                    |\n| `syslog.configMap.mountPath`                              | Mount path for the syslog ConfigMap.                                                | `/config/syslog-ng.conf`           |\n| `syslog.configMap.subPath`                                | Specific file to mount from the ConfigMap.                                          | `syslog-ng.conf`                   |\n| `syslog.persistence.enabled`                              | Enable persistence for syslog.                                                      | `false`                            |\n| `syslog.persistence.size`                                 | Size of the persistent volume for syslog.                                           | `\"1Gi\"`                            |\n| `syslog.persistence.accessModes`                          | Access modes (e.g., ReadWriteOnce) for the syslog persistent volume.                | `[\"ReadWriteOnce\"]`                |\n| `syslog.persistence.storageClassName`                     | Storage class name for the syslog persistent volume.                                | `\"standard\"`                       |\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fsuperstream-engine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperstreamlabs%2Fsuperstream-engine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fsuperstream-engine/lists"}