{"id":22922435,"url":"https://github.com/dhruvikdonga/grpc-apps","last_synced_at":"2026-02-18T04:37:17.043Z","repository":{"id":267402724,"uuid":"900322611","full_name":"DhruvikDonga/grpc-apps","owner":"DhruvikDonga","description":"GRPC Go apps in memory k8s distributed communication ","archived":false,"fork":false,"pushed_at":"2024-12-11T12:27:45.000Z","size":23,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-12T22:54:01.764Z","etag":null,"topics":["golang","grpc","headless-service","k8s","kube"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DhruvikDonga.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-08T13:39:50.000Z","updated_at":"2024-12-13T18:04:50.000Z","dependencies_parsed_at":"2024-12-10T06:28:06.078Z","dependency_job_id":"a1f8ef75-577d-49b0-a5bb-63071c7f8495","html_url":"https://github.com/DhruvikDonga/grpc-apps","commit_stats":null,"previous_names":["dhruvikdonga/grpc-apps"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DhruvikDonga%2Fgrpc-apps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DhruvikDonga%2Fgrpc-apps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DhruvikDonga%2Fgrpc-apps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DhruvikDonga%2Fgrpc-apps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DhruvikDonga","download_url":"https://codeload.github.com/DhruvikDonga/grpc-apps/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253837389,"owners_count":21971981,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","grpc","headless-service","k8s","kube"],"created_at":"2024-12-14T08:10:16.209Z","updated_at":"2025-10-08T16:33:32.403Z","avatar_url":"https://github.com/DhruvikDonga.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GRPC Message Distributed Service\n\nThis guide explains how to set up a Go-based service with two APIs:\n\n1. One to fetch all messages across pods.\n2. Another to save messages in memory.\n\nWe will cover two scenarios:\n\n- How two local Go apps communicate and display all messages over gRPC.\n- How to achieve this communication in Kubernetes (K8s).\n\nThe service uses one-directional gRPC streaming (response streaming) to stream all messages stored in memory in the Go app.\n\n## 1. Setup gRPC with Protobuf\n\nTo define the gRPC service, we use Protocol Buffers (Protobuf). Generate Go code from the `.proto` file using the following command:\n\n```bash\nprotoc --go_out=. --go-grpc_out=. proto/messages.proto\n```\n\n## 2. Local Setup\n\n- **APP 1**: Exposes ports `8081` (HTTP) and `9091` (gRPC).\n- **APP 2**: Exposes ports `8082` (HTTP) and `9092` (gRPC).\n\n### 3. Load Image into Kubernetes Cluster\n\nTo load the Docker image into the cluster, use the following command:\n\n```bash\nkind load docker-image grpc-app:latest --name \u003ccluster-name\u003e\n```\n\n## 4. Service Configuration\n\nIn Kubernetes, we create a **headless service** to allow direct communication between individual pods. A headless service doesn't allocate a cluster IP for load balancing but allows clients to communicate directly with the pods.\n\nWe have two apps: one for HTTP and the other for gRPC. The configuration steps are as follows:\n\n### Key Features of a Headless Service:\n\n- **No ClusterIP**: Setting `clusterIP: None` means no load-balancing IP is assigned, and traffic is routed directly to individual pods.\n- **DNS Resolution**: Kubernetes creates DNS entries for each pod in the headless service, enabling direct connections using DNS queries like `pod-name.grpc-service.default.svc.cluster.local`.\n- **StatefulSets**: Headless services are often used with StatefulSets where each pod needs to be addressed individually by its DNS name.\n- **Pod IPs**: Instead of routing traffic to a single service IP, clients query DNS and get back a list of pod IPs.\n\n### Example Headless Service Configuration:\n\n#### `grpc-service.yaml`\n\n```yaml\napiVersion: v1\nkind: Service\nmetadata:\n  name: grpc-service\n  labels:\n    app: grpc-service\nspec:\n  clusterIP: None # Headless service (no load balancing, direct pod communication)\n  selector:\n    app: grpc-service\n  ports:\n    - name: grpc-port # Name for the gRPC port\n      protocol: TCP\n      port: 9091 # gRPC port\n      targetPort: 9091 # Port on the pod to forward to\n```\n\n## 5. Deployment Configuration\n\nWithin Kubernetes, services are resolved using the DNS convention:\n\n```\n\u003cservice-name\u003e.\u003cnamespace\u003e.svc.cluster.local\n```\n\nThis format is standard for referring to services in a Kubernetes cluster.\n\n### Fetching Pod IPs:\n\nTo enable gRPC streaming, our function `GetPODIPs()` fetches the IP addresses of all pods associated with the service. These IPs are then used to stream messages.\n\n```go\nfunc GetPODIPs() []string {\n    // Logic to fetch pod IPs\n}\n```\n\n## 6. Example Output in Kubernetes\n\nWhen fetching the messages across pods, the result would look like this:\n\n```json\n[\n  {\n    \"room\": \"cool_room\",\n    \"client_name\": \"cool_client_pod_3_k8s\",\n    \"message\": \"COOOL MESSAGE\"\n  },\n  {\n    \"room\": \"cool_room\",\n    \"client_name\": \"cool_client_pod_2_k8s\",\n    \"message\": \"COOOL MESSAGE\"\n  },\n  {\n    \"room\": \"cool_room\",\n    \"client_name\": \"cool_client_pod_1_k8s\",\n    \"message\": \"COOOL MESSAGE\"\n  }\n]\n```\n\nThis result demonstrates how messages from multiple pods are fetched and displayed over the gRPC stream.\n\n---\n\nBy following this setup, we ensure that Go apps can communicate across pods within a Kubernetes environment using gRPC streaming, while also managing the flow of messages directly from each pod in memory.\n\n## Using loadbalancer or ingress\n\n### Ingress\n\n`ingress.yaml` sets up a nginx ingress controller over `grpc-http-service.yaml` to loadbalance in local you can update hosts file and add url grpc-app.lc\nIngress controller:- `kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml`\n\n\u003e Note: \n\u003e if you are using headless clusterip instead of grpc-http-service.yaml nginx ingress controller will resolve dns like we are doing in our app main.go for grpc it will do for http 8081 but its http and we don't need to but good for websocket and stateful apps\n\n### Loadbalancer\n\nUse metallb loadbalancer for kind `metallbconfig.yaml` has appropriate commands for it and `loadbalancer.yaml` implements it . Loadbalancer will use the selector and get pods .\n\n# Why not using a database instead of storing in memory .\n\nThere are scenarios where in memory becomes a necessity primary one is websockets my side project [simplysocket](https://github.com/DhruvikDonga/simplysocket) which is websocket . A websocket upgrades clients https session to receive and send message its like a bridge which actual sense connects to the app . If the app or a pod is loosed client disconnects from websocket itself .\nConsider `client_name` as ws connection it self .\n\nIt means 2 things here :-\n\n- Pods can be scaled but clients logic won't be same as normal http request does client will connect to it .\n- Pods down scaling is a challenge as down scale will empty all data like in grpc-apps we can see message will be lost if deployment replicas are reduced .\n\nEach pod will have clients in it which are connected . There is a option to use a pub/sub module but why to rely on external service if something can be solved internally if tried 🌟 .\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhruvikdonga%2Fgrpc-apps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdhruvikdonga%2Fgrpc-apps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhruvikdonga%2Fgrpc-apps/lists"}