{"id":26267834,"url":"https://github.com/oslabs-beta/kafka-kare","last_synced_at":"2025-04-30T19:21:32.080Z","repository":{"id":225946427,"uuid":"765984102","full_name":"oslabs-beta/Kafka-Kare","owner":"oslabs-beta","description":"🦋 Kafka Kare is a Kafka visualization and error handling web app with plug and play functionality. 🚀","archived":false,"fork":false,"pushed_at":"2024-03-30T16:29:50.000Z","size":121132,"stargazers_count":42,"open_issues_count":0,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-08T10:53:18.204Z","etag":null,"topics":["express","grafana","grafana-dashboard","kafka","kafka-cluster","kafka-consumer","kafka-streams","nextjs","oauth","prometheus","react","visualizer"],"latest_commit_sha":null,"homepage":"https://kafkakare.com","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oslabs-beta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"open-source-labs","custom":["https://donorbox.org/donations-to-oslabs-inc"]}},"created_at":"2024-03-02T02:55:09.000Z","updated_at":"2024-11-30T19:14:32.000Z","dependencies_parsed_at":"2024-03-05T05:30:18.235Z","dependency_job_id":"3cdf7532-0164-4338-ab6c-735f5b450b08","html_url":"https://github.com/oslabs-beta/Kafka-Kare","commit_stats":null,"previous_names":["oslabs-beta/kafka-kare"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FKafka-Kare","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FKafka-Kare/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FKafka-Kare/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FKafka-Kare/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oslabs-beta","download_url":"https://codeload.github.com/oslabs-beta/Kafka-Kare/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251767210,"owners_count":21640470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["express","grafana","grafana-dashboard","kafka","kafka-cluster","kafka-consumer","kafka-streams","nextjs","oauth","prometheus","react","visualizer"],"created_at":"2025-03-14T04:18:27.041Z","updated_at":"2025-04-30T19:21:32.057Z","avatar_url":"https://github.com/oslabs-beta.png","language":"JavaScript","funding_links":["https://github.com/sponsors/open-source-labs","https://donorbox.org/donations-to-oslabs-inc"],"categories":[],"sub_categories":[],"readme":"# Kafka-Kare\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./public/kafka-kare-logo-v2.png\" width=500px\u003e\n\u003c/p\u003e\n\n\n## Table of Contents\n1. [About](#about-kafka-kare)\n2. [How it Works](#how-it-works)\n3. [Getting Started](#getting-started-with-kafka-kare)\n4. [Interested in Contributing?](#interested-in-contributing)\n5. [Meet the Team](#meet-the-team)\n6. [License](#license)\n\n\n## About Kafka Kare\nIn the complex landscape of Apache Kafka ecosystems, monitoring data streams and detecting anomalies can be challenging. Kafka Kare addresses this challenge by offering a user-friendly interface to visualize Kafka topics, partitions, consumer groups, and more. \n\nWith built-in alerting mechanisms, it empowers users to proactively identify issues such as lagging consumers, high throughput, or unusual patterns, ensuring the reliability and performance of Kafka deployments.\n\nKafka Kare is an open-source web application designed to provide powerful visualization capabilities and real-time alerting functionalities for Apache Kafka environments. It offers seamless integration with Kafka clusters, allowing users to monitor data streams, visualize key metrics, and set up Slack alerts for critical events.\n\n## How it Works\nKafka Kare is built with:\n\n[![JavaScript](https://img.shields.io/badge/JavaScript-F7DF1E?style=for-the-badge\u0026logo=javascript\u0026logoColor=black)](https://www.javascript.com/)\n[![Apache Kafka](https://img.shields.io/badge/Apache%20Kafka-000?style=for-the-badge\u0026logo=apachekafka)](https://kafka.apache.org/)\n[![HTML5](https://img.shields.io/badge/html5-%23E34F26.svg?style=for-the-badge\u0026logo=html5\u0026logoColor=white)](https://html.com/html5/)\n[![CSS](https://img.shields.io/badge/CSS3-1572B6?style=for-the-badge\u0026logo=css3\u0026logoColor=white)](https://developer.mozilla.org/en-US/docs/Web/CSS)\n[![Node.js](https://img.shields.io/badge/Node.js-43853D?style=for-the-badge\u0026logo=node.js\u0026logoColor=white)](https://nodejs.org/)\n[![React](https://img.shields.io/badge/React-20232A?style=for-the-badge\u0026logo=react\u0026logoColor=61DAFB)](https://react.dev/)\n[![Chakra UI](https://img.shields.io/badge/Chakra--UI-319795?style=for-the-badge\u0026logo=chakra-ui\u0026logoColor=white)](https://chakra-ui.com/)\n[![Next.js](https://img.shields.io/badge/next%20js-000000?style=for-the-badge\u0026logo=nextdotjs\u0026logoColor=white)](https://nextjs.org/)\n[![Prometheus](https://img.shields.io/badge/Prometheus-E6522C?style=for-the-badge\u0026logo=Prometheus\u0026logoColor=white)](https://prometheus.io/)\n[![Grafana](https://img.shields.io/badge/grafana-%23F46800.svg?style=for-the-badge\u0026logo=grafana\u0026logoColor=white)](https://grafana.com/)\n[![Express](https://img.shields.io/badge/Express.js-404D59?style=for-the-badge\u0026logo=express)](https://expressjs.com/)\n[![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)](https://www.docker.com/)\n[![Chart.js](https://img.shields.io/badge/Chart%20js-FF6384?style=for-the-badge\u0026logo=chartdotjs\u0026logoColor=white)](https://www.chartjs.org/)\n[![MongoDB](https://img.shields.io/badge/MongoDB-4EA94B?style=for-the-badge\u0026logo=mongodb\u0026logoColor=white)](https://www.mongodb.com/)\n[![JWT](https://img.shields.io/badge/JWT-000000?style=for-the-badge\u0026logo=JSON%20web%20tokens\u0026logoColor=white)](https://jwt.io/)\n\n\nKafka Kare's key features include:\n  - With its plug-and-play functionality, Kafka Kare allows users to add cluster connections with their port numbers. \n  - Once connected, real-time visualization of 22 key Kafka cluster metrics such as consumer lag, overall health of the cluster, and latency are available via Grafana-generated dashboards that \n    are supported by Prometheus.\n  - Additionally, the alerting system allows users to define custom alert rules based on various Kafka metrics and thresholds. When predefined conditions are met, the system triggers alerts via \n    Slack, enabling timely response to potential issues and ensuring the stability of Kafka infrastructures.\n\nRead more about our build process on Medium, [here](https://medium.com/@konomp/kafka-kare-simplifying-kafka-error-handling-for-small-teams-524a555a7eff)!   \n    \n## Getting Started With Kafka Kare\n\n1. Fork and clone this repository then open it on your code editor of choice.\n   - If you don't have it installed already, please set up Docker. The Kafka Kare team uses Docker Desktop: https://www.docker.com/products/docker-desktop/\n   - To use Kafka Kare, you must have a containerized Kafka cluster set up to expose JMX data.\n   - You can use [Confluent's Kafka images](https://registry.hub.docker.com/r/confluentinc/cp-kafka) with the `KAFKA_OPTS` environmental variable to run the Prometheus [JMX \n      Exporter](https://github.com/prometheus/jmx_exporter) as a Java agent. Your cluster should also be set up using a Docker bridge network to allow the Prometheus container to connect. See \n     [here](https://github.com/oslabs-beta/Kafka-Kare/tree/main/kafka-cluster) for an example.\n2. Setup the .env file\n   - Locate the **.env.example** file in the **root directory**. You will need to add your own JSON Web Token (JWT) secret key for security purposes: https://jwt.io/\n   - You will also need to set up your own Next Auth URL and secret key for security purposes: https://next-auth.js.org/configuration/options\n   - Optional: If you would prefer to use Github or Google OAuth for your login, please fill in the corresponding ID and Secret Key in the .env file as well.\n     \n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./public/kafkaKare-creatingEnvfile.gif\" alt=\"create .env file\" width=\"\"\u003e\n\u003c/p\u003e\n     \n3. Spin up the application container by running the below command:\n```\ndocker compose up -d\n```\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./public/kafkakare-dockercomposeup-d.gif\" alt=\"create .env file\" width=\"\"\u003e\n\u003c/p\u003e\n\n\n### New to Kafka? Use steps 1 - 5 below to play around with our demo Kafka cluster!\n\n  _1. Change directory to /kafka-cluster_\n  ```\n  cd kafka-cluster\n  ```\n\n  _2. (First time running the application) Build kafka cluster image_\n  ```\n  docker build -t dockerpromkafka:latest .\n  ```\n\n  _3. Spin up demo kafka-cluster container (demo Kafka-cluster container must be spun up after application container)_\n  ```\n  docker compose up -d\n  ```\n\n  _4. Run the consumer followed by producer script_\n  ```\n  node consumer.js\n  node producer.js\n  ```\n  _5. Congratulations! You just set up your first Kafka cluster!_\n     \n     \n4. Visit localhost:3000, create an account or log in. Click the 'Add Cluster' button, enter in the port number of your cluster, then click the dashboard button to start viewing your metrics!\n   - Optional: Navigate to localhost:8080 to view your database. The username and password is located in the logs of the mongo-express container.\n6. Click the 3 lines to enter in your Slack URL and head to the Alerts page to configure your custom alerting thresholds.\n7. Enjoy!\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./public/kafkakareAddingcluster.gif\" alt=\"create .env file\" width=\"\"\u003e\n\u003c/p\u003e\n\n## Instructions to stop the application\n1. Spin down application container\n```\ndocker compose down\n```\n\n2. Change directory to /kafka-cluster\n```\ncd kafka-cluster\n```\n\n3. Spin down demo kafka-cluster container\n```\ndocker compose down\n```\n## Interested in Contributing?\nNow seeking contributors to join the Kafka Kare team!\n\n### How to Contribute \n\nTo start contributing, please fork and clone Kafka Kare, create a feature branch with the pattern \"(issue or feature)/what-you-are-working-on\". \n\nWhen you are ready to submit a pull request to the **dev branch**, please follow the checklist closely and request at least two people to review and approve your code. You can refer to the Kafka Kare team members [here](#meet-the-team).\n\nPrioritize any linked issues first before tackling the roadmap features and feel free to add any roadmap features as well that will continue to bring value to developers! We appreciate your support!\n\n### Roadmap\n\n| Feature                                                                               | Status    |\n|---------------------------------------------------------------------------------------|-----------|\n| Ability to plug in Kafka cluster to app                                               | ✅        |\n| Metrics visualization with Grafana                                                    | ✅        |\n| Save custom dashboard for users                                                       | ⏳        |\n| Save historical data                                                                  | ⏳        |\n| Alert history                                                                         | ⏳        |\n| Custom notification configuration                                                     | 🙏🏻        |\n| Migrate to KRaft                                                                      | 🙏🏻        |\n| Shared state between any component shortcode                                          | 🙏🏻        |\n\n- ✅ = Ready to use\n- ⏳ = In progress\n- 🙏🏻 = Looking for contributors\n\n# Meet the Team!\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cimg src=\"https://github.com/AllisonScott97.png\" width=\"140px;\" alt=\"profile picture of allison\"/\u003e\n      \u003cbr /\u003e\n      \u003csub\u003e\u003cb\u003eAllison Scott\u003c/b\u003e\u003c/sub\u003e\n      \u003cbr /\u003e\n      \u003ca href=\"https://www.linkedin.com/in/allisonscott01/\"\u003e🖇️\u003c/a\u003e\n      \u003ca href=\"https://github.com/AllisonScott97\"\u003e🐙\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cimg src=\"https://github.com/MattKonop.png\" width=\"140px;\" alt=\"profile picture of matt\"/\u003e\n      \u003cbr /\u003e\n      \u003csub\u003e\u003cb\u003eMatt Konop\u003c/b\u003e\u003c/sub\u003e\n      \u003cbr /\u003e\n      \u003ca href=\"https://www.linkedin.com/in/matt-konop-58a82b189/\"\u003e🖇️\u003c/a\u003e\n      \u003ca href=\"https://github.com/MattKonop\"\u003e🐙\u003c/a\u003e\n    \u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\n      \u003cimg src=\"https://github.com/JustinTzHLin.png\" width=\"140px;\" alt=\"profile picture of justin\"/\u003e\n      \u003cbr /\u003e\n      \u003csub\u003e\u003cb\u003eJustin Lin\u003c/b\u003e\u003c/sub\u003e\n      \u003cbr /\u003e\n      \u003ca href=\"https://www.linkedin.com/in/justin-tzuhung-lin\"\u003e🖇️\u003c/a\u003e\n      \u003ca href=\"https://github.com/JustinTzHLin\"\u003e🐙\u003c/a\u003e\n    \u003c/td\u003e\n     \u003ctd align=\"center\"\u003e\n      \u003cimg src=\"https://github.com/julesdmai.png\" width=\"140px;\" alt=\"profile picture of jules\"/\u003e\n      \u003cbr /\u003e\n      \u003csub\u003e\u003cb\u003eJules Mai\u003c/b\u003e\u003c/sub\u003e\n      \u003cbr /\u003e\n      \u003ca href=\"http://www.linkedin.com/in/julesmai\"\u003e🖇️\u003c/a\u003e\n      \u003ca href=\"https://github.com/julesdmai\"\u003e🐙\u003c/a\u003e\n    \u003c/td\u003e\n     \u003ctd align=\"center\"\u003e\n      \u003cimg src=\"https://github.com/josuerole.png\" width=\"140px;\" alt=\"profile picture of ronal\"/\u003e\n      \u003cbr /\u003e\n      \u003csub\u003e\u003cb\u003eRonal Garcia\u003c/b\u003e\u003c/sub\u003e\n      \u003cbr /\u003e\n      \u003ca href=\"https://www.linkedin.com/in/ronalgarcia/\"\u003e🖇️\u003c/a\u003e\n      \u003ca href=\"https://github.com/josuerole\"\u003e🐙\u003c/a\u003e\n    \u003c/td\u003e\n\u003c/table\u003e\n\n\n- 🖇️ = LinkedIn\n- 🐙 = Github\n\n## License\nThis product is licensed under the MIT License without restriction.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslabs-beta%2Fkafka-kare","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foslabs-beta%2Fkafka-kare","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslabs-beta%2Fkafka-kare/lists"}