{"id":23112396,"url":"https://github.com/lixx21/kafka-ibm-stocking","last_synced_at":"2025-04-03T23:45:20.235Z","repository":{"id":207952338,"uuid":"720488837","full_name":"lixx21/kafka-IBM-Stocking","owner":"lixx21","description":"Move data to JSON file real time using Kafka for IBM Stocking Data","archived":false,"fork":false,"pushed_at":"2023-11-18T16:39:09.000Z","size":4,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-09T11:41:44.106Z","etag":null,"topics":["data-engineer","kafka","kafka-streams"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lixx21.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-18T16:36:16.000Z","updated_at":"2023-11-18T16:37:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"8fca9fcc-7623-44ca-a2d4-7450ddaa2f3f","html_url":"https://github.com/lixx21/kafka-IBM-Stocking","commit_stats":null,"previous_names":["lixx21/kafka-ibm-stocking"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fkafka-IBM-Stocking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fkafka-IBM-Stocking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fkafka-IBM-Stocking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lixx21%2Fkafka-IBM-Stocking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lixx21","download_url":"https://codeload.github.com/lixx21/kafka-IBM-Stocking/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247097975,"owners_count":20883127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineer","kafka","kafka-streams"],"created_at":"2024-12-17T02:17:51.553Z","updated_at":"2025-04-03T23:45:20.223Z","avatar_url":"https://github.com/lixx21.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka IBM Real Time Stocking\n\n## Dataset\n\nIn this project, we will use this dataset:\n[https://www.kaggle.com/datasets/bhanuprasanna527/stock-market-prediction](https://www.kaggle.com/datasets/bhanuprasanna527/stock-market-prediction)\n\n## Install Environment\n\n- install kafka ```wget https://downloads.apache.org/kafka/3.3.1/kafka_2.12-3.3.1.tgz```\n- extract kafka ```tar -xvf kafka_2.12-3.3.1.tgz```\n- install java jdk 1.8.0 (set environment for JAVA_HOME)\n\n\n## Start Kafka Server\n\n- move directory to kafka: ```cd kafka_2.12-3.3.1```\n- Start Zookeeper: ```bin/zookeeper-server-start.sh config/zookeeper.properties```\n- Start Broker: ```bin/kafka-server-start.sh config/server.properties```\n- Create Topic:```bin/kafka-topics.sh --create --topic demo_kafka --bootstrap-server localhost:9092```\n- To see Topic's lists:  ```bin/kafka-topics.sh --bootstrap-server localhost:9092 --list```\n- Start Producer: ```bin/kafka-console-producer.sh --topic demo_kafka --bootstrap-server localhost:9092```\n- Start Consumer: ```bin/kafka-console-consumer.sh --topic demo_kafka --bootstrap-server localhost:9092```\n\n## How to use\n\n- clone this github ```git clone https://github.com/lixx21/kafka-IBM-Stocking.git```\n- install all libraries ```pip install -r requirements.txt```\n- run all shell in **kafka_consumer.ipynb** and **kafka_producer.ipynb**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flixx21%2Fkafka-ibm-stocking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flixx21%2Fkafka-ibm-stocking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flixx21%2Fkafka-ibm-stocking/lists"}