https://github.com/shikhar97/kafka-neo4j

Implementing graph processing algorithms in a distributed environment using Docker, Neo4j, Kafka, and Kubernetes.
https://github.com/shikhar97/kafka-neo4j

bfs docker gds helm-charts kafka kafka-connect kubernetes neo4j pagerank python statefulsets

Last synced: about 1 month ago
JSON representation

Implementing graph processing algorithms in a distributed environment using Docker, Neo4j, Kafka, and Kubernetes.

• Host: GitHub
• URL: https://github.com/shikhar97/kafka-neo4j
• Owner: Shikhar97
• License: mit
• Created: 2023-05-10T21:54:31.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-05-20T22:12:58.000Z (over 1 year ago)
• Last Synced: 2024-12-25T19:49:31.863Z (about 1 month ago)
• Topics: bfs, docker, gds, helm-charts, kafka, kafka-connect, kubernetes, neo4j, pagerank, python, statefulsets
• Language: Python
• Homepage:
• Size: 8.79 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# sgupt330-project-2-phase2

1. To install:

   1. I used `brew install minikube` to install.

   2. Also, I used `minikube config set driver docker` to set docker as my default driver

2. To deploy:

   1. I configured the service and deployment sections in the yaml file

   2. Used `minikube start -p CSE511-project2-phase2` to start the cluster

   3. Used `kubectl apply -f ./kafka-setup.yaml` and `kubectl apply -f ./zookeeper-setup.yaml` to deploy the pods.

3. I verified my deployment in two ways:

Sending data from local host to Kafka using the following command:

    kubectl port-forward  29092

    echo "hello Kafka" | kcat -P -b localhost:29092 -t test

    kcat -C -b localhost:29092 -t test

Checking connectivity between zookeeper and kafka and listing the kafka broker id and topics.

    kubectl exec -it  -- /bin/bash

    zookeeper-shell.sh zookeeper-service:2181

    ls /brokers/ids

    ls /brokers/topics