{"id":26268064,"url":"https://github.com/oslabs-beta/atomickafka","last_synced_at":"2025-04-30T19:26:27.046Z","repository":{"id":50119556,"uuid":"366071850","full_name":"oslabs-beta/AtomicKafka","owner":"oslabs-beta","description":null,"archived":false,"fork":false,"pushed_at":"2023-01-03T21:07:58.000Z","size":972,"stargazers_count":107,"open_issues_count":0,"forks_count":16,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-12-31T13:04:27.715Z","etag":null,"topics":["apache","apache-kafka","atom","atomic","consumer","javascript","kafka","kafka-clusters","producer","react","typescript","websocket"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oslabs-beta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-10T14:30:55.000Z","updated_at":"2024-04-07T16:26:56.000Z","dependencies_parsed_at":"2023-02-01T10:16:32.954Z","dependency_job_id":null,"html_url":"https://github.com/oslabs-beta/AtomicKafka","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FAtomicKafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FAtomicKafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FAtomicKafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslabs-beta%2FAtomicKafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oslabs-beta","download_url":"https://codeload.github.com/oslabs-beta/AtomicKafka/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243521211,"owners_count":20304188,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","apache-kafka","atom","atomic","consumer","javascript","kafka","kafka-clusters","producer","react","typescript","websocket"],"created_at":"2025-03-14T04:19:20.450Z","updated_at":"2025-03-14T04:19:21.039Z","avatar_url":"https://github.com/oslabs-beta.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![AtomicKafka_MastHead](./assets/atomic-kafka-logo.png)\n\n___\n\n\n\u003cp align=\"center\"\u003eAtomicKafka is a lightweight \u003ca href=\"https://www.npmjs.com/package/atomic-kafka\"\u003e NPM Package\u003c/a\u003e developed to simplify the process of establishing bidirectional, real-time data streaming with Apache Kafka in your web-app.\n\u003cbr\u003e\n\u003ca href=\"http://www.atomickafka.com/\"\u003eWebsite\u003c/a\u003e\u003cspan\u003e\u0026nbsp; | \u0026nbsp;\u003c/span\u003e\u003ca href=\"https://github.com/oslabs-beta/AtomicKafka\"\u003eLibrary\u003c/a\u003e\u003cspan\u003e\u0026nbsp; | \u0026nbsp;\u003c/span\u003e\u003ca href=\"https://github.com/AtomicKafka\"\u003eDemo Apps\u003c/a\u003e\u003cspan\u003e\u0026nbsp; | \u0026nbsp;\u003c/span\u003e\u003ca href=\"https://medium.com/@dbehmoaras/2eb79b20eaae?source=friends_link\u0026sk=843b83b81eb79f37f0d2b8a96ce26212\"\u003eFeatured on Medium\u003c/a\u003e\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n\n  \u003ca href=\"https://www.npmjs.com/package/atomic-kafka\"\u003e\n    \u003cimg alt=\"npm\" src=\"https://img.shields.io/npm/v/atomic-kafka?color=%2366FCF1\u0026style=for-the-badge\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/oslabs-beta/atomickafka/graphs/contributors\"\u003e\n    \u003cimg alt=\"GitHub contributors\" src=\"https://img.shields.io/github/contributors/oslabs-beta/atomickafka?color=%2366FCF1\u0026style=for-the-badge\"\u003e\n  \u003c/a\u003e\n  \u003ca id=\"star\" href=\"https://github.com/oslabs-beta/AtomicKafka/stargazers\"\u003e\n    \u003cimg alt=\"GitHub Repo stars\" src=\"https://img.shields.io/github/stars/oslabs-beta/AtomicKafka?color=%2366FCF1\u0026style=for-the-badge\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/oslabs-beta/atomickafka/blob/main/LICENSE\"\u003e\n    \u003cimg alt=\"Last Commit\" src=\"https://img.shields.io/github/last-commit/oslabs-beta/AtomicKafka?color=%2366FCF1\u0026style=for-the-badge\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/oslabs-beta/atomickafka/blob/main/LICENSE\"\u003e\n    \u003cimg alt=\"NPM\" src=\"https://img.shields.io/npm/l/atomic-kafka?color=%2366FCF1\u0026style=for-the-badge\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n___\n\n**\u003ch2 id=\"\"\u003eTable of Contents\u003c/h2\u003e**\n1. [Features](#features)\n2. [Benefits of AtomicKafka](#ben)\n3. [Getting Started](#gs)\n4. [Contribution](#contribution)\n5. [Maintainers](#maintainers)\n6. [Built With](#dependencies)\n7. [License](#license)\n___\n**\u003ch2 id=\"features\"\u003eFeatures\u003c/h2\u003e**\n\n- Websocket connections between the client and the server that accept user-defined event strings and callbacks\n- Broker initialization and connection to Apache Kafka\n- Consumer and Producer classes are predefined to be as modular as possible\n- Consumer functions accept user-defined callback functions to support lightweight stream processing\n- React Hook that throttles the websocket event listener with a time interval to maintain client performance\n- Supports multiple Kafka streams\n___\n**\u003ch2 id=\"ben\"\u003eBenefits of AtomicKafka\u003c/h2\u003e**\n![AtomicKafka Supports Multiple Streams](./assets/howAKworks2.png)\n\n___\n\n**\u003ch2 id=\"gs\"\u003eGetting Started\u003c/h2\u003e**\n\n### **1. Initialize Kafka cluster**\n\nAtomicKafka currently supports running Apache Kafka clusters either using a Docker image or by connecting to Confluent Cloud.\n\n\n**Docker:**\n- Download this [.yml](https://github.com/AtomicKafka/atomicKafkaConsumer/blob/main/docker-compose.yml) and run the following command in your terminal:\n  ```\n  docker-compose up -d\n  ```\n\n**Confluent Cloud:**\n- Follow the steps on [Confluent Cloud](https://www.confluent.io/confluent-cloud/) to create a free account with Confluent cloud. Obtain the ***API_ACCESS_KEY***, **_API_ACCESS_SECRET_**, and ***BOOTSTRAP_SERVER***\n\n___\n### **2. Configure *.env* file**\nInclude the following lines in your .env depending on your Kafka environment. Set the PORT variable to the port where **AtomicKafkaServer** will be initialized in the next step.\n\n- Docker .env config: (***API_KEY*** and ***API_SECRET*** are intentionally left blank)\n  ```js\n  PORT=\u003cUSER_DEFINED\u003e\n  API_KEY=\n  API_SECRET=\n  KAFKA_BOOTSTRAP_SERVER=localhost:9092\n  ```\n- Confluent Cloud .env config: (***PORT*** intentionally left blank)\n  ```js\n  PORT=\n  API_KEY=\u003cAPI_ACCESS_KEY\u003e\n  API_SECRET=\u003cAPI_ACCESS_SECRET\u003e\n  KAFKA_BOOTSTRAP_SERVER=\u003cBOOTSTRAP_SERVER\u003e\n  ```\n___\n### **3. Install AtomicKafka**\n\n```sh\n$ npm install atomic-kafka\n```\n___\n### **4. Create Server Instance**\nInitialize a server instance of your choice (HTTP, Node.js, etc). The example below contemplates a Node.js Express server.\n\n***ATTENTION: a Server instance must be created for every remote AtomicKafkaClient.***\n1. Initialize and configure ***expressApp*** according to desired specifications.\n2. Require in ***AtomicKafkaServer***.\n3. Define a ***server*** that listens on the user-defined PORT environment variable.\n4. Initialize an AtomicKafkaServer instance ***aks*** by passing in the ***server***.\n```js\n/* initialize and configure Node.js expressApp according to user specifications\nthen add the following: */\n\nconst AtomicKafkaServer = require('atomic-kafka/server');\n\nconst server = expressApp.listen(process.env.PORT, () =\u003e {\n  console.log(`Listening on port ${process.env.PORT}`);\n})\n\nconst aks = new AtomicKafkaServer(server);\n```\n___\n### **5A. Create the Consumer and enable the built-in websocket on the server**\n1. Initialize a **_newConsumer_** on the **_aks_** instance and pass in the **_group_ID_string_**.\n2. Enable the built-in websocket by invoking **_socketConsume_** and passing in the **_group_ID_string_**, an **_event_string_**, and the **_topic_string_**.\n\n```js\n/* AKS_Producer_Init */\naks.newProducer('topic');\naks.globalProduce('produceMessageEvent', 'topic');\n```\n\n___\n### **5B. Create the Producer and enable the built-in websocket on the server**\n1. Initialize a ***newProducer*** on the ***aks*** instance and pass in the ***topic_string***.\n2. Enable the built-in websocket by invoking ***globalProducer*** and passing in an ***event_string*** and the ***topic_string***.\n```js\n/* AKS_Consumer_Init */\naks.newConsumer('group_ID');\naks.socketConsume('group_ID', 'consumeMessageEvent', 'topic');\n```\n___\n### **6A. JavaScript - Import Client Interface (React \u0026 Hooks)**\n```js\n/* in your React.jsx Component */\nimport AtomicKafkaClient from 'atomic-kafka/client';\n```\n### **6B. TypeScript - Import Client Interface (React \u0026 Hooks)**\n```js\n/* in your TypeScript React Component */\ndeclare function require(name:string);\nconst AtomicKafkaClient = require('atomic-kafka/client').default;\n\n```\n___\n### **7A. Create and implement Consumer client component (JS \u0026 TS)**\n  1. Initialize ***akc*** as an ***AtomicKafkaClient***. Pass in ***AtomicKafkaServer*** instance host's ***URI_STRING***\n  2. Define a callback to process message ***payload*** through the React state management tool of your choice.\n  3. Implement ***useInterval*** to consume from the kafka cluster on interval.\n  4. Return the invocation of the ***consumer*** function on the ***akc*** instance. Pass in a user-defined websocket ***event_string***, the previously defined ***callback***, and the ***interval_delay*** in milliseconds.\n```js\nfunction ConsumerComponent() {\n  const akc = new AtomicKafkaClient('ATOMIC_KAFKA_SERVER_URI_STRING');\n\n  const callback = (payload) =\u003e {\n    /* user-provided data stream processing function definition\n    that effects state change */\n  }\n\n  /* Throttles message consumption. Interval in milliseconds,\n  can be any number */\n  akc.useInterval(() =\u003e akc.consumer('consumeMessageEvent', callback), 4000);\n}\n```\n\n### **7B. Create and implement Producer client component (JS \u0026 TS)**\n\n1. Initialize **_akc_** as an **_AtomicKafkaClient_**. Pass in **_AtomicKafkaServer_** instance host's **_URI_STRING_**\n2. Generate a ***payload*** formatted as an arbitrarily-nested JSON object. The example below defines a payload, but it can be generated at any point in the client according to the user's specification.\n3. Invoke the consumer function. Pass in the websocket ***event_string*** and the ***payload***.\n\n```js\nfunction ProducerComponent() {\n  const akc = new AtomicKafkaClient('ATOMIC_KAFKA_SERVER_URI_STRING');\n\n  const payload = {\n    /* Data to be sent to the cluster. Arbitrarily-nested JSON format.\n    Can be defined anywhere in the app. */\n  }\n\n  akc.producer('produceMessageEvent', payload);\n}\n```\n___\n**\u003ch2 id=\"contribution\"\u003eContribute\u003c/h2\u003e**\n\nWe want this open-sourced project to continue to improve. If you would like to make a contribution to AtomicKafka, please fork [this repo](https://github.com/oslabs-beta/AtomicKafka), add your awesome changes to a well-named feature branch of this repository, and make a pull request. We look forward to your input! And if you want to support AtomicKafka, please click on the [⭐](https://github.com/oslabs-beta/AtomicKafka/stargazers)  button for us! \n\n___\n**\u003ch2 id=\"maintainers\"\u003eMaintainers\u003c/h2\u003e**\n[Nikhil Massand](https://github.com/nikhilmassand)\n\n[Vicki Yang](https://github.com/vickiwyang)\n\n[David Behmoaras](https://github.com/dbehmoaras)\n\n[Joseph Lee](https://github.com/josephjslee)\n\n___\n**\u003ch2 id=\"dependencies\"\u003eBuilt With\u003c/h2\u003e**\n\n- [KafkaJS](https://kafka.js.org/)\n- [React](https://reactjs.org/)\n- [TypeScript](https://www.typescriptlang.org/)\n- [Socket-IO](https://socket.io/)\n- [dotenv](https://github.com/motdotla/dotenv#readme)\n- The support of [OSLabs](https://opensourcelabs.io/)\n\n___\n## \u003ch2 id=\"license\"\u003eLicense\u003c/h2\u003e\nThis product is released under the MIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslabs-beta%2Fatomickafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foslabs-beta%2Fatomickafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslabs-beta%2Fatomickafka/lists"}