{"id":13742393,"url":"https://github.com/lyft/flinkk8soperator","last_synced_at":"2025-05-16T03:04:34.215Z","repository":{"id":36069554,"uuid":"189491745","full_name":"lyft/flinkk8soperator","owner":"lyft","description":"Kubernetes operator that provides control plane for managing Apache Flink applications","archived":false,"fork":false,"pushed_at":"2024-08-29T07:06:11.000Z","size":10259,"stargazers_count":570,"open_issues_count":88,"forks_count":155,"subscribers_count":377,"default_branch":"master","last_synced_at":"2025-04-12T04:45:18.735Z","etag":null,"topics":["lyft"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lyft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-05-30T22:41:57.000Z","updated_at":"2025-02-23T05:46:16.000Z","dependencies_parsed_at":"2024-04-29T21:29:36.614Z","dependency_job_id":"6ece81e9-2ddc-41ef-95c1-c4953237944e","html_url":"https://github.com/lyft/flinkk8soperator","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyft%2Fflinkk8soperator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyft%2Fflinkk8soperator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyft%2Fflinkk8soperator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyft%2Fflinkk8soperator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lyft","download_url":"https://codeload.github.com/lyft/flinkk8soperator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254459088,"owners_count":22074605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lyft"],"created_at":"2024-08-03T05:00:31.247Z","updated_at":"2025-05-16T03:04:29.188Z","avatar_url":"https://github.com/lyft.png","language":"Go","funding_links":[],"categories":["Repository is obsolete"],"sub_categories":["Awesome Operators in the Wild"],"readme":"[![Current Release](https://img.shields.io/github/release/lyft/flinkk8soperator.svg)](https://github.com/lyft/flinkk8soperator/releases/latest)\n[![Build Status](https://travis-ci.org/lyft/flinkk8soperator.svg?branch=master)](https://travis-ci.org/lyft/flinkk8soperator)\n[![GoDoc](https://godoc.org/github.com/lyft/flinkk8soperator?status.svg)](https://godoc.org/github.com/lyft/flinkk8soperator)\n[![License](https://img.shields.io/badge/LICENSE-Apache2.0-ff69b4.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)\n[![CodeCoverage](https://img.shields.io/codecov/c/github/lyft/flinkk8soperator.svg)](https://codecov.io/gh/lyft/flinkk8soperator)\n[![Go Report Card](https://goreportcard.com/badge/github.com/lyft/flinkk8soperator)](https://goreportcard.com/report/github.com/lyft/flinkk8soperator)\n![Commit activity](https://img.shields.io/github/commit-activity/w/lyft/flinkk8soperator.svg?style=plastic)\n![Commit since last release](https://img.shields.io/github/commits-since/lyft/flinkk8soperator/latest.svg?style=plastic)\n[![Slack](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack\u0026style=social)](http://go.lyft.com/flinkoperator_slack)\n\n\n# Flinkk8soperator\nFlinkK8sOperator is a [Kubernetes operator](https://coreos.com/operators/) that manages [Flink](https://flink.apache.org/) applications on Kubernetes. The operator acts as control plane to manage the complete deployment lifecycle of the application.\n\n\n## Project Status\n\n*Beta*\n\nThe operator is in use for some less-critical jobs at Lyft. At this point the focus is on testing and stability. While in \nBeta, we will attempt to limit the number of backwards-incompatible changes, but they may still occur as necessary. \n\n## Prerequisites\n* Version \u003e= 1.10 of Kubernetes (versions \u003c 1.13 require `--feature-gates=CustomResourceSubresources=true`)\n* Version \u003e= 1.7 of Apache Flink.\n\n## Overview\n\n![Flink operator overview](docs/flink-operator-overview.svg)\n\nThe goal of running Flink on Kubernetes is to enable more flexible, lighter-weight deployment of streaming applications, without needing to manage infrastructure. The Flink operator aims to abstract out the complexity of hosting, configuring, managing and operating Flink clusters from application developers. It achieves this by extending any kubernetes cluster using [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources).\n\nThe Operator creates flink clusters dynamically using the specified custom resource. Flink clusters in kubernetes consist of the following:\n* JobManager [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/)\n* TaskManager [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/)\n* JobManager [Service](https://kubernetes.io/docs/concepts/services-networking/service/)\n* JobManager [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) for the UI (optional)\n\nDeploying and managing Flink applications in Kubernetes involves two steps:\n\n* **Building Flink application packaged as a docker image:** A docker image is built containing the application source code with the necessary dependencies built in. This is required to bootstrap the Jobmanager and Taskmanager pods. At Lyft we use Source-To-Image [S2I](https://docs.openshift.com/container-platform/3.11/architecture/core_concepts/builds_and_image_streams.html#source-build) as the image build tool that provides a common builder image with Apache Flink pre-installed. The docker image could be built using any pre-existing workflows at an organization.\n\n* **Creating the Flink application custom resource:** The custom resource for Flink application provides the spec for configuring and managing flink clusters in Kubernetes. The FlinkK8sOperator, deployed on Kubernetes, continuously monitors the resource and the corresponding flink cluster, and performs actions based on the diff.\n\n## Documentation\n\n* [Quick start guide](/docs/quick-start-guide.md)\n* [User guide](/docs/user_guide.md)\n* [Flink application custom resource](/docs/crd.md)\n* [Operator state machine](/docs/state_machine.md)\n\n## Community\n\n* Join our [Slack](http://go.lyft.com/flinkoperator_slack) channel.\n* Check out [who is using FlinkK8sOperator](docs/who-is-using.md).\n\nWe welcome you to contribute and make the operator better! For questions, and changes please create an [issue](https://github.com/lyft/flinkk8soperator/issues/new) or submit a [pull request](https://github.com/lyft/flinkk8soperator/compare).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyft%2Fflinkk8soperator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flyft%2Fflinkk8soperator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyft%2Fflinkk8soperator/lists"}