{"id":17101744,"url":"https://github.com/nicholasjackson/grpc-consul-resolver","last_synced_at":"2025-04-13T00:25:48.239Z","repository":{"id":57485902,"uuid":"143825966","full_name":"nicholasjackson/grpc-consul-resolver","owner":"nicholasjackson","description":"HashiCorp Consul Resolver for use with gRPC load balancer","archived":false,"fork":false,"pushed_at":"2020-03-16T15:34:24.000Z","size":3777,"stargazers_count":19,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-26T18:21:25.316Z","etag":null,"topics":["consul","golang","grpc","grpc-resolver"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nicholasjackson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-07T05:51:35.000Z","updated_at":"2024-08-24T14:04:51.000Z","dependencies_parsed_at":"2022-09-18T18:16:06.330Z","dependency_job_id":null,"html_url":"https://github.com/nicholasjackson/grpc-consul-resolver","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholasjackson%2Fgrpc-consul-resolver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholasjackson%2Fgrpc-consul-resolver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholasjackson%2Fgrpc-consul-resolver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicholasjackson%2Fgrpc-consul-resolver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nicholasjackson","download_url":"https://codeload.github.com/nicholasjackson/grpc-consul-resolver/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248649618,"owners_count":21139524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consul","golang","grpc","grpc-resolver"],"created_at":"2024-10-14T15:26:38.728Z","updated_at":"2025-04-13T00:25:48.208Z","avatar_url":"https://github.com/nicholasjackson.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gRPC Resolver for Consul\n\n[![CircleCI](https://circleci.com/gh/nicholasjackson/grpc-consul-resolver.svg?style=svg)](https://circleci.com/gh/nicholasjackson/grpc-consul-resolver)\n[![GoDoc](https://godoc.org/github.com/nicholasjackson/grpc-consul-resolver?status.svg)](https://godoc.org/github.com/nicholasjackson/grpc-consul-resolver)\n\nThis repository implements a naming.Resolver for Consul which can be used with gRPC load balancers.\n\nFor information on load balancing concepts with gRPC please see the documentation:   \n[https://github.com/grpc/grpc/blob/master/doc/load-balancing.md](https://github.com/grpc/grpc/blob/master/doc/load-balancing.md)\n\nWhen creating a gRPC load balancer a resolver must be passed as a dependency:\n\n```\nfunc RoundRobin(r naming.Resolver) Balancer\n```\n\nIt is the resolver's job is to determine the endpoints for the given service name.  When `grpc.Dial` has been setup with a load balancer and you make a call to a service, internal the gRPC framework\nrequests an endpoint from the load balancer.  The load balancer gets this information from the resolver at creation time, this is supplied by the resolver function `Next()`.   \n\nOnce this first batch of endpoints has been retrieved it is the resolvers job to watch the service catalog and to return any updated information, letting the load balancer know of any added or deleted records.  The `Next()` function blocks until there is updated service information, internally inside the load balancer the resolvers `Next()` function is continually called, a return from this function informs it that it needs to update the internal endpoint list.  \n\n![](https://github.com/grpc/grpc/raw/master/doc/images/load-balancing.png)\n\nConnections in gRPC are persistent, it is common to have a single client which is shared across all go routines, it is the clients job to marshal access to the connections, spawning additional connections as required.  When a load balancer is used then the client will maintain at least one connection for each endpoint in the load balanced list.  With each call to a service these will be rotated according to the policy implemented by the load balancer.  For example if you have two endpoints `127.0.0.1:8080` and `127.0.0.1:8081` using the built in `RoundRobin` load balancer would ensure that every call to a service endpoint would rotate through the endpoints returned from the resolver in turn.\n\nInternally this implementation of a gRPC Resolver leverages Consuls Service Catalog, endpoints are retrieved from the catalog based\non their registered name.  The resolver continually polls Consul (60 seconds by default) to ensure the endpoint list is kept up to date.\n\n## Basic usage:\n```\nr := resolver.NewServiceQueryResolver(\"http://consulAddr:8500\")\n\n// use the default poll interval of 60 seconds\n// the poll interval can be changed by setting the resolvers PollInterval field\n// r.PollInterval = 10 * time.Second\n\n// Create the gRPC load balancer\nlb := grpc.RoundRobin(r)\n\n// create a new gRPC client connection\nc, err := grpc.Dial(\n\t\"test_grpc\",\n\tgrpc.WithInsecure(),\n\tgrpc.WithBalancer(lb),\n\tgrpc.WithTimeout(5*time.Second),\n)\n\n// create the instance of our test client\ncc := echo.NewEchoServiceClient(c)\n\n// call the service method\n// The first call would be routed to the first endpoint which was returned from \n// the Resolver and thus the Consul Service Catalog\ncc.Echo(context.Background(), \u0026echo.Message{Data: \"hello world\"})\n\n// call the service method again\n// The second call would create a new connection, this time the endpoint used \n// would be the second listed in the Consul Service Catalog\ncc.Echo(context.Background(), \u0026echo.Message{Data: \"hello world\"})\n```\n\n## Consul Connect usage:\n```\nr, dialer, _ := resolver.NewConnectServiceQueryResolver(\"http://consulAddr:8500\",\"my_service\")\nlb := grpc.RoundRobin(r)\n\n// create a new gRPC client connection\nc, err := grpc.Dial(\n\t\"test_grpc\",\n\tgrpc.WithInsecure(),\n\tgrpc.WithBalancer(lb),\n\tgrpc.WithTimeout(5*time.Second),\n  dialer,\n)\n\n```\n\n## Testing\nThis package has both `unit` and `integration` tests, the unit tests are pure Go tests with mocks replacing the dependency for Consul.  To execute unit tests:\n\n```bash\n$ make test_unit\n\n# or\n\n$ go test -v -race .\n\n=== RUN   TestCreatesNewResolver\n--- PASS: TestCreatesNewResolver (0.00s)\n=== RUN   TestResolveRerturnsWatcher\n--- PASS: TestResolveRerturnsWatcher (0.00s)\n=== RUN   TestNewConsulWatcherReturnsWatcher\n--- PASS: TestNewConsulWatcherReturnsWatcher (0.00s)\n=== RUN   TestNextReturnsErrorWhenConsulError\n--- PASS: TestNextReturnsErrorWhenConsulError (0.00s)\n=== RUN   TestNextReturnsInitialUpdatesFromConsul\n--- PASS: TestNextReturnsInitialUpdatesFromConsul (0.00s)\n=== RUN   TestNextReturnsInitialUpdatesFromConsulSetsNodeWhenNoAddr\n--- PASS: TestNextReturnsInitialUpdatesFromConsulSetsNodeWhenNoAddr (0.00s)\n=== RUN   TestNextReturnsUpdatesContainingAddedItemsFromConsul\n--- PASS: TestNextReturnsUpdatesContainingAddedItemsFromConsul (0.00s)\n=== RUN   TestNextReturnsUpdatesContainingDeletedItemsFromConsul\n--- PASS: TestNextReturnsUpdatesContainingDeletedItemsFromConsul (0.00s)\n=== RUN   TestNextBlocksWhenNoChangesFromConsul\n--- PASS: TestNextBlocksWhenNoChangesFromConsul (0.05s)\nPASS\nok      github.com/nicholasjackson/grpc-consul-resolver 1.073s\n```\n\nIn addition to the unit tests there is also an integration test suite, the test suite requires a `Consul` server to be running on localhost with the default ports as a dependency. The integration tests start two dummy gRPC servers and register them with the Consul server's Service Catalog.  A gRPC client is then created to ensure the function of the Resolver.  Integration tests can be found in the sub folder `./functional_tests`, the [GoDog](https://github.com/DATA-DOG/godog) Cucumber BDD framework is used to execute these tests.  To execute theintegration tests:\n\n```bash\n$ make test_functional\n\n# or\n\n$ cd functional_tests\n$ go test -v --godog.format=pretty --godog.random\n\nFeature: As a developer, I want to ensure that the\n  loabalancer functions correctly\n\n  Scenario: Calls two different upstreams                   # features/consul_service.feature:10\n    Given that Consul is running                            # main_test.go:47 -\u003e thatConsulIsRunning\n    And the services are running and registered             # main_test.go:60 -\u003e theServicesAreRunningAndRegistered\nServer id localhost:7711 Echo request hello world\nServer id localhost:7712 Echo request hello world\n    When I call use the client 2 times                      # main_test.go:88 -\u003e iCallUseTheClientTimes\n    Then I expect 2 different endpoints to have been called # main_test.go:123 -\u003e iExpectDifferentEndpointsToHaveBeenCalled\n\n  Scenario: Calls one upstream                              # features/consul_service.feature:4\n    Given that Consul is running                            # main_test.go:47 -\u003e thatConsulIsRunning\n    And the services are running and registered             # main_test.go:60 -\u003e theServicesAreRunningAndRegistered\nServer id localhost:7712 Echo request hello world\n    When I call use the client 1 times                      # main_test.go:88 -\u003e iCallUseTheClientTimes\n    Then I expect 1 different endpoints to have been called # main_test.go:123 -\u003e iExpectDifferentEndpointsToHaveBeenCalled\n\n2 scenarios (2 passed)\n8 steps (8 passed)\n3.664543687s\n\nRandomized with seed: 54915\ntesting: warning: no tests to run\nPASS\nok      github.com/nicholasjackson/grpc-consul-resolver/functional_tests        3.699s\n``` \n\n\n\n## TODO\n[x] Implement Consul Connect Services lookup   \n[x] Implement prepared queries  \n[ ] Implement prepared queries with Connect Services  \n[ ] Finish implementing query options  \n[ ] Investigate why functional tests hang on CircleCI but run fine locally  \n[ ] Tidy readme and documentation  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicholasjackson%2Fgrpc-consul-resolver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnicholasjackson%2Fgrpc-consul-resolver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicholasjackson%2Fgrpc-consul-resolver/lists"}