{"id":21354841,"url":"https://github.com/salrashid123/envoy_discovery","last_synced_at":"2025-04-07T15:07:58.727Z","repository":{"id":91309826,"uuid":"131563158","full_name":"salrashid123/envoy_discovery","owner":"salrashid123","description":"Envoy Discovery service 'hello world'","archived":false,"fork":false,"pushed_at":"2024-11-20T14:17:14.000Z","size":12237,"stargazers_count":70,"open_issues_count":1,"forks_count":21,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-31T12:09:35.436Z","etag":null,"topics":["envoy","envoyproxy","go","golang","grpc","service-discovery"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salrashid123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-30T06:30:12.000Z","updated_at":"2024-12-22T18:55:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"38fb7bc8-6d85-4e9c-9907-6fa38a002076","html_url":"https://github.com/salrashid123/envoy_discovery","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_discovery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_discovery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_discovery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fenvoy_discovery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salrashid123","download_url":"https://codeload.github.com/salrashid123/envoy_discovery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247675597,"owners_count":20977376,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["envoy","envoyproxy","go","golang","grpc","service-discovery"],"created_at":"2024-11-22T04:14:44.191Z","updated_at":"2025-04-07T15:07:58.685Z","avatar_url":"https://github.com/salrashid123.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Envoy EDS \"hello world\"\n\nA simple app demonstrating a small part of [Envoy's Endpoint Discovery Service](https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/eds.proto#envoy-api-file-envoy-api-v2-eds-proto). \n\nSome of the configurations are hardcoded in the `envoy_config.yaml` file just as a demonstration.  Specifically, the service, cluster and bootstrap endpoint\nto get discovery information.\n\n\n## References\n - [Endpoint Discovery Service](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/endpoint/endpoint)\n - [Endpoing Overview](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/service_discovery#arch-overview-service-discovery-types-sds)\n - [SDS at Lyft](https://github.com/lyft/discovery)\n - [Envoy Dynamic Configuration](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/dynamic_configuration.html)\n - [Envoy API for developers](https://github.com/envoyproxy/data-plane-api/blob/master/API_OVERVIEW.md)\n\n### Prerequsites\n\n- [envoy binary ](https://envoyproxy.io)\n- python (and virtualenv)\n- golang\n\nOverall there are three components:\n\n1. Envoy server (listen  port `:10000`)\n   This is the core proxy that will startup without any upstream endpoints.  It will ask the EDS server for valid endpoints for a given cluster\n2. EDS gRPC server (listen port: `:8080`).  This gRPC server returns a list of endpoints that it was told about back to envoy\n3. Upstream Endpoints (listen ports `:8081`, `:8082`).  THese are the actual upstream webservers that envoy will ultimately connect to.  On startup, these python webservers will 'tell' the GRPC server of their existence and later on the grpc server will inform envoy.\n\n![arch.png](images/arch.png)\n\n---\n\n### Start EDS Server\n\n```bash\ncd eds_server\ngo run grpc_server.go\n```\n\n## Start Envoy with SDS\n\n\nGet Envoy binary\n\n```bash\ndocker cp `docker create envoyproxy/envoy-dev:latest`:/usr/local/bin/envoy .\n```\n\nSo start envoy with debug enabled:\n\n```bash\n./envoy -c envoy_config.yaml -l debug\n```\n\nAt this point, envoy attempts to connect to the upstream EDS gRPC cluster at `127.0.0.1:8080` but since your EDS isn't running yet, nothing additional config takes place.\n\n\n```bash\n curl -v  http://localhost:10000/\n\n\u003e GET / HTTP/1.1\n\u003e Host: localhost:10000\n\u003e User-Agent: curl/7.72.0\n\u003e Accept: */*\n\u003e \n\n\u003c HTTP/1.1 503 Service Unavailable\n\u003c content-length: 19\n\u003c content-type: text/plain\n\u003c date: Fri, 25 Dec 2020 13:26:05 GMT\n\u003c server: envoy\n\u003c \n\nno healthy upstream\n```\n\n\nAs mentioned, this is the gRPC server that envoy will connect to.  Since the EDS server doesn't 'know' about any other webservers, its list of endpoints is blank\n\nWhen envoy contacts the EDS server, it will return an empty list\n\nThe following shows the EDS Server returning a cache snapshot back to envoy\n\n```bash\n$ go run grpc_server.go \nINFO[0000] Starting control plane                       \nINFO[0000] management server listening                   port=8080\nINFO[0022] OnStreamOpen 1 open for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment \nINFO[0022] OnStreamRequest type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment \nINFO[0022] OnStreamRequest ResourceNames [myservice]    \nINFO[0022] []                                           \nINFO[0022] \u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e creating snapshot Version 1 \nINFO[0022] OnStreamResponse...                          \nINFO[0022] cb.Report()  callbacks                        fetches=0 requests=1\nINFO[0022] OnStreamRequest type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment \nINFO[0022] OnStreamRequest ResourceNames [myservice]    \n\n```\n\nhowever, the cache doesn't contain any endpoints so envoy can't proxy to any webserver (clearly since no upstream server is even running!)\n\n\n## Start Upstream services\n\nNow in a new window, start the upstream service on a given the default port for the script (```:8081```)\n\n```bash\ncd upstream/\n\nvirtualenv env --python=/usr/bin/python3.7\nsource env/bin/activate\npip install -r requirements.txt\n\n$ python3 server.py -p 8081\n```\n\nOn startup, the webserver will make a rest API call back to the EDS server over HTTP just to let it know its alive and the host/port it listens on.\n\n\u003e\u003e NOTE:\n\nThis is important bit:  I just happened to use the endpoint webservers startup stage to let EDS know about its existence...you can use **ANY** technique..that bit is not specified..\n\neg, the EDS Server runs both grpc on port `:8080` AND and HTTP server on port `:5000`.  The HTTP server is simply a way for upstream servers to 'register' itself...\n\n```python\ndef main(argv):\n   port = 18080\n   print(\"Registering endpoint: 127.0.0.1:\", port)\n   url = 'http://localhost:5000/edsservice/register?endpoint=127.0.0.1:' + port\n   f = urllib.request.urlopen(url)\n   print(f.read().decode('utf-8'))\n```\n\nThe REST endpoint on the EDS server `/edsservice/register?endpoint=` is something i just made up\n\n\nYou'll see that the EDS server's next snapshot contains the  host/port back to envoy:\n\n```log\nINFO[0382] OnStreamRequest type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment \nINFO[0382] OnStreamRequest ResourceNames [myservice]    \nINFO[0442] \u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e creating cluster, remoteHost, nodeID myservice,  127.0.0.1:8081, test-id \nINFO[0442] [lb_endpoints:{endpoint:{address:{socket_address:{address:\"127.0.0.1\" port_value:8081}}}}] \nINFO[0442] \u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e\u003e creating snapshot Version 8 \nINFO[0442] OnStreamResponse...                          \nINFO[0442] cb.Report()  callbacks \n```\n\n### Check client connectivity via envoy\n\nSince we already started the upstream service above, you can connect to it via envoy:\n\n```bash\n$ curl -v  http://localhost:10000/\n \n\u003c HTTP/1.1 200 OK\n\u003c content-type: text/html; charset=utf-8\n\u003c content-length: 36\n\u003c server: envoy\n\u003c date: Mon, 30 Apr 2018 06:21:43 GMT\n\u003c x-envoy-upstream-service-time: 3\n\u003c \n* Connection #0 to host localhost left intact\n40b9bc6f-77b8-49b7-b939-1871507b0fcc\n```\n\n(note the `server: envoy` part in the header)\n\nNote, if you can also directly remove an upstream host:port from EDS by invoking the deregister :\n\n `curl http://localhost:5000/edsservice/deregister?endpoint=127.0.0.1:8081`\n\n\n## Rinse and repeat\n\nOk, you can continue to play with the endpoints by adding and adding or stopping new upstream services on different ports:\n\neg:\n\n```bash\n$ python server.py -p 8082\n$ python server.py -p 8083\n```\n\neach successive calls to envoy should show the various endpoints\n## Conclusion\n\nI wrote this up just in an effort to play around with `envoy` i'm pretty much new to this so i likely have numerous \nmisunderstanding on what i just did here...if you see something amiss, please do let me know.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fenvoy_discovery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalrashid123%2Fenvoy_discovery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fenvoy_discovery/lists"}