{"id":19528175,"url":"https://github.com/sneaksanddata/arcane-operator","last_synced_at":"2026-02-09T13:18:57.054Z","repository":{"id":236321764,"uuid":"778349061","full_name":"SneaksAndData/arcane-operator","owner":"SneaksAndData","description":"Kubernetes-native data streaming service based on Akka.NET","archived":false,"fork":false,"pushed_at":"2025-06-30T09:20:37.000Z","size":475,"stargazers_count":1,"open_issues_count":17,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-30T10:27:38.248Z","etag":null,"topics":["actor","akka","arcane","c-sharp","data-ingestion","reactive","streaming"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SneaksAndData.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-27T14:57:46.000Z","updated_at":"2025-05-08T13:08:28.000Z","dependencies_parsed_at":"2024-05-28T15:42:02.601Z","dependency_job_id":null,"html_url":"https://github.com/SneaksAndData/arcane-operator","commit_stats":null,"previous_names":["sneaksanddata/arcane-operator"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/SneaksAndData/arcane-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SneaksAndData%2Farcane-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SneaksAndData%2Farcane-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SneaksAndData%2Farcane-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SneaksAndData%2Farcane-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SneaksAndData","download_url":"https://codeload.github.com/SneaksAndData/arcane-operator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SneaksAndData%2Farcane-operator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266296812,"owners_count":23907014,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-21T11:47:31.412Z","response_time":64,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actor","akka","arcane","c-sharp","data-ingestion","reactive","streaming"],"created_at":"2024-11-11T01:17:51.575Z","updated_at":"2026-02-09T13:18:56.973Z","avatar_url":"https://github.com/SneaksAndData.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# About the project\n\n\u003cimg src=\"docs/images/arcane-logo.png\" width=\"100\" height=\"100\" alt=\"logo\"\u003e \n\n[![Run tests with coverage](https://github.com/SneaksAndData/arcane-operator/actions/workflows/build.yaml/badge.svg)](https://github.com/SneaksAndData/arcane-operator/actions/workflows/build.yaml)\n[![C#](https://img.shields.io/badge/C%23-8-blue.svg)](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-versioning)\n![GitHub Release Date](https://img.shields.io/github/release-date/sneaksanddata/arcane-operator)\n\n\n\n**Arcane** is a Kubernetes-native data streaming platform powered by extendable plugin architecture.\n\nArcane is designed to be:\n- **Kubernetes-native**: It runs data streaming plugins as a Kubernetes jobs and uses Kubernetes API to manage the\n   lifecycle of data streams.\n\n- **Extendable**: It allows you to extend the platform with your own data streaming plugins, which can be written in\n   any programming language.\n\n- **Scalable**: It can scale horizontally by adding more Kubernetes nodes to the cluster or vertically by adding more\n   resources to the streaming jobs. It does not require any stateful components like Zookeeper or Kafka brokers which \n   makes it easy to scale and manage.\n\n- **Lightweight**: It has a small footprint and can run on any Kubernetes cluster, including local clusters like\n   [Minikube](https://minikube.sigs.k8s.io/docs/) or [Kind](https://kind.sigs.k8s.io/).\n \n- **Cloud-agnostic**: It can run on any cloud provider that supports Kubernetes, including AWS, Azure, GCP, and\n   on-premises clusters.\n\nThis repository contains the **Arcane Operator**, which is responsible for managing the lifecycle of Arcane data streams.\n\n# Table of Contents\n - [Getting started](#getting-started)\n   - [Verify the installation](#verify-the-installation)\n - [Streaming plugins](#streaming-plugins)\n    - [Available ZIO-based streaming plugins](#available-ziobased-streaming-plugins)\n    - [Available Akka-based streaming plugins](#available-akkabased-streaming-plugins)\n - [Monitoring and observability](#monitoring-and-observability)\n - [Contributing](#contributing)\n   - [Extending the platform with your own plugins](#extending-the-platform-with-your-own-plugins)\n\n# Getting started\nRun the following command to install the Arcane Operator in your Kubernetes cluster:\n\n```bash\n# Create a namespace for the operator installation\n$ kubectl create namespace arcane\n\n# Install the operator in the created namespace\n$ helm install arcane oci://ghcr.io/sneaksanddata/helm/arcane-operator \\\n  --version v0.0.14 \\\n  --namespace arcane\n```\n\nThis command creates a namespace `arcane` and installs the stream operator in it. By default, the Helm chart installs two CRDs:\n- StreamClass\n- StreamingJobTemplate\n\nThe resources of both kinds are being installed by the streaming plugins.\n\n## Verify the installation\n\nTo verify the operator installation run the following command:\n\n```bash\n$ kubectl get pods -l app.kubernetes.io/name=arcane-operator --namespace arcane\n```\n\nIt should produce the similar output:\n\n```bash\nNAME                               READY   STATUS    RESTARTS   AGE\narcane-operator-55988bbfcb-ql7qr   1/1     Running   0          25m\n```\n\nOnce operator is installed, you can install the streaming plugins.\n\n# Platform roadmap\n\nPlease refer the roadmap for the Arcane Streaming Platform on the [Arcane](https://github.com/orgs/SneaksAndData/projects/21) project page.\n\nThe most significant milestones are listed below: \n\n- [x] Support for ZIO-based streaming plugins\n- [ ] Add contribution guidelines\n- [ ] Rewrite the operator in Go\n- [ ] Complete transition from Akka.NET to ZIO for the remaining streaming plugins\n\n# Streaming plugins\n\n### Available ZIO-based streaming plugins\n* Sql Server streaming plugin: [arcane-stream-sqlserver-change-tracking](https://github.com/SneaksAndData/arcane-stream-sqlserver-change-tracking)\n* Microsoft Synapse streaming plugin: [arcane-stream-microsoft-synapse-link](https://github.com/SneaksAndData/arcane-stream-microsoft-synapse-link)\n\n### Available Akka-based streaming plugins\n* REST-api streaming plugin: [arcane-stream-rest-api](https://github.com/SneaksAndData/arcane-stream-rest-api)\n\n\n# Monitoring and observability\n-- TBD --\n\n# Contributing\n-- TBD --\n\n## Extending the platform with plugins\n-- TBD --\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsneaksanddata%2Farcane-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsneaksanddata%2Farcane-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsneaksanddata%2Farcane-operator/lists"}