{"id":13564649,"url":"https://github.com/shijuvar/go-distributed-sys","last_synced_at":"2025-04-04T10:08:36.680Z","repository":{"id":39607864,"uuid":"140957018","full_name":"shijuvar/go-distributed-sys","owner":"shijuvar","description":"Guidance for building event-driven distributed systems and microservices in Go with NATS JetStream, gRPC and CockroachDB ","archived":false,"fork":false,"pushed_at":"2023-12-13T13:56:56.000Z","size":191,"stargazers_count":648,"open_issues_count":2,"forks_count":118,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-03-28T09:07:38.520Z","etag":null,"topics":["cockroachdb","cqrs","event-sourcing","go","golang","grpc","grpc-go","microservices","nats","nats-jetstream","nats-streaming"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shijuvar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-07-14T15:30:30.000Z","updated_at":"2025-03-26T13:53:02.000Z","dependencies_parsed_at":"2023-12-13T14:56:41.919Z","dependency_job_id":null,"html_url":"https://github.com/shijuvar/go-distributed-sys","commit_stats":{"total_commits":48,"total_committers":2,"mean_commits":24.0,"dds":0.04166666666666663,"last_synced_commit":"45a577ccd09ea0c850fd90cc0f9c7952ac4d56a8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shijuvar%2Fgo-distributed-sys","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shijuvar%2Fgo-distributed-sys/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shijuvar%2Fgo-distributed-sys/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shijuvar%2Fgo-distributed-sys/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shijuvar","download_url":"https://codeload.github.com/shijuvar/go-distributed-sys/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247157283,"owners_count":20893220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cockroachdb","cqrs","event-sourcing","go","golang","grpc","grpc-go","microservices","nats","nats-jetstream","nats-streaming"],"created_at":"2024-08-01T13:01:34.036Z","updated_at":"2025-04-04T10:08:36.660Z","avatar_url":"https://github.com/shijuvar.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\n# Guidance for building distributed systems and Microservices in Go\n\n## Articles\n* [Building Event-Driven Distributed Systems in Go with gRPC, NATS JetStream and CockroachDB](https://shijuvar.medium.com/building-event-driven-distributed-systems-in-go-with-grpc-nats-jetstream-and-cockroachdb-c4b899c8636d)\n\n## Technologies Used: \n* Go\n* NATS, NATS JetStream, NATS Micro\n* gRPC\n* CockroachDB\n* Go kit\n* Zipkin (for distributed tracing)\n\n\n## Compile Proto files\nRun the command below from the eventstream directory:\n\n```\nprotoc eventstore/*.proto \\\n\t\t--go_out=. \\\n\t\t--go-grpc_out=. \\\n\t\t--go_opt=paths=source_relative \\\n\t\t--go-grpc_opt=paths=source_relative \\\n\t\t--proto_path=.\n```\t\t\n\n\n## Set up CockroachDB \n\n#### Set up CockroachDB  Cluster with three nodes\n\n```\ncockroach start \\\n--insecure \\\n--store=orders-1 \\\n--listen-addr=localhost:26257 \\\n--http-addr=localhost:8080 \\\n--join=localhost:26257,localhost:26258,localhost:26259 \\\n--background\n```\n\n```\ncockroach start \\\n--insecure \\\n--store=orders-2 \\\n--listen-addr=localhost:26258 \\\n--http-addr=localhost:8081 \\\n--join=localhost:26257,localhost:26258,localhost:26259 \\\n--background\n```\n```\ncockroach start \\\n--insecure \\\n--store=orders-3 \\\n--listen-addr=localhost:26259 \\\n--http-addr=localhost:8082 \\\n--join=localhost:26257,localhost:26258,localhost:26259 \\\n--background\n```\n\n#### cockroach init command to perform a one-time initialization of the cluster\n```\ncockroach init --insecure --host=localhost:26257\n```\n\n#### Start a SQL Shell for CockroachDB:\n```\ncockroach sql --insecure --host=localhost:26257\n```\n\n#### Create user\n```\ncockroach user set shijuvar --insecure\n```\n\n#### Create Databases\n```\ncockroach sql --insecure -e 'CREATE DATABASE eventstoredb'\n```\n\n```\ncockroach sql --insecure -e 'CREATE DATABASE ordersdb'\n```\n\n#### Grant privileges to the shijuvar user\n```\ncockroach sql --insecure -e 'GRANT ALL ON DATABASE ordersdb TO shijuvar'\n```\n```\ncockroach sql --insecure -e 'GRANT ALL ON DATABASE eventstoredb TO shijuvar'\n```\n\n## Run NATS JetStream Server \n\n```\nnats-server -js\n```\n\n## Run JetStream with config file\n\n```\nnats-server -c js.conf\n```\n\n// js.conf file\n```\njetstream {\n    // jetstream data will be in /data/jetstream-server/jetstream\n    store_dir: \"/data/jetstream-server\"\n\n    // 1GB\n    max_memory_store: 1073741824\n\n    // 10GB\n    max_file_store: 10737418240\n}\n\nhttp_port: 8222\n```\n\n## Prerequisites for running the eventstream demo\n\n* Run the NATS JetStream Server\n* Run CockroachDB\n* Run bootstrapper app (eventstream/bootstrapper) as a one time activity for creating stream in JetStream and create two databases in CockroachDB\n* In order to use distributed tracing with gokit-ordersvc (order service with Go kit), run Zipkin distributed tracing system  \n\n## Basic Workflow in the example (eventstream directory):\n\n1. A client app post an Order to an HTTP API (ordersvc)\n2. An HTTP API (ordersvc) receives the order, then executes a command onto Event Store, which is an immutable log of events of domain events, to create an event via its gRPC API (eventstoresvc).\n3. The Event Store API executes the command and then publishes an event \"ORDERS.created\" to NATS JetStream server to let other services know that a domain event is created.\n4. The Payment worker (paymentworker) subscribes the event \"ORDERS.created\", then make the payment, and then create another event \"ORDERS.paymentdebited\" via Event Store API.\n5. The Event Store API executes a command onto Event Store to create an event \"ORDERS.paymentdebited\" and publishes an event to NATS JetStream server to let other services know that the payment has been debited.\n6. The Query synchronising worker (querymodelworker) subscribes the event \"ORDERS.created\" that synchronise the query data model to provide state of the aggregates for query views.\n7. The review worker (reviewworker) subscribes the event \"ORDERS.paymentdebited\" and finally approves the order, and then create another event \"ORDERS.approved\" via Event Store API.\n8. A Saga coordinator manages the distributed transactions and makes void transactions on failures (to be implemented)\n\n\n## Training and Consulting in India\nAs a Consulting Solutions Architect, I do provide [training and consulting on Go programming language and distributed systems architectures](https://github.com/shijuvar/shijuvar/blob/master/masterclass.md), in India.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshijuvar%2Fgo-distributed-sys","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshijuvar%2Fgo-distributed-sys","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshijuvar%2Fgo-distributed-sys/lists"}