{"id":21354964,"url":"https://github.com/salrashid123/cs_mesh","last_synced_at":"2025-03-16T05:23:34.500Z","repository":{"id":195772621,"uuid":"693630197","full_name":"salrashid123/cs_mesh","owner":"salrashid123","description":"Service-\u003eService mTLS using Envoy and Consul Service Discovery on GCP Confidential Space","archived":false,"fork":false,"pushed_at":"2023-09-19T12:11:01.000Z","size":227,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-22T17:47:12.384Z","etag":null,"topics":["confidential-computing","google-cloud-platform","hashicorp-consul","tls","trusted-execution-environment"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salrashid123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-09-19T12:06:40.000Z","updated_at":"2023-11-11T11:42:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"d6ad035f-454d-4df8-a2c7-fc5e72f15dcc","html_url":"https://github.com/salrashid123/cs_mesh","commit_stats":null,"previous_names":["salrashid123/cs_mesh"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fcs_mesh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fcs_mesh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fcs_mesh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salrashid123%2Fcs_mesh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salrashid123","download_url":"https://codeload.github.com/salrashid123/cs_mesh/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243828627,"owners_count":20354533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["confidential-computing","google-cloud-platform","hashicorp-consul","tls","trusted-execution-environment"],"created_at":"2024-11-22T04:15:25.664Z","updated_at":"2025-03-16T05:23:34.462Z","avatar_url":"https://github.com/salrashid123.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n## Service-\u003eService mTLS using Envoy and Consul Service Discovery on GCP Confidential Space\n\nTutorial demonstrating running [Consul](https://www.consul.io/) and `service-\u003eservice mTLS` on GCP [Confidential Space](https://cloud.google.com/blog/products/identity-security/announcing-confidential-space).\n\nEssentially, this demo will run a Consul Server and two Confidential Space containers acting as microservices.   Each microservice will register itself with the consul server and use its DNS resolution for service-\u003eservice traffic. \n\nThe service-\u003eservice traffic is encrypted using mTLS certificates read in from GCP `Secret Manager`.\n\nThis repo is an implementation of the following:\n\n- [Constructing Trusted Execution Environment (TEE) with GCP Confidential Space](https://github.com/salrashid123/confidential_space/blob/main/README.md#service-discovery-and-tee-tee-traffic)\n- [mTLS proxy containers for GCP Confidential Compute](https://github.com/salrashid123/tee_server_proxy)\n\n\n\u003e\u003e this repo is not supported by google\n\n\nNote, this is a very, very  DIY \"service mesh\"  (and i use that term very loosely).  The challenge with running systems within confidential computing is each secret and constraint but be carefully released and even the TLS traffic's secrets should be secure (i.,e we 'just can't run istio/k8s' on confidential space).\n\nThis repo just happens to use consul and envoy and there are clearly alternatives to this mesh (envoy ofcourse has its own way to distribute secrets and coordinate service discovery...(i.,e istio))\n\n---\n\nIn this demo, the consul server runs locally as a VM though in real life several replicas could run as a cluster in confidential space, remotely or hosted elsewhere.\n\nEach mTLS certs/keys and consul agent/app configuration is are _pregenerated_ and saved hosted on `GCP Secret Manager` statically.  In real life, the certificates and keys could be generated preferably by private keys generated on each service instance and \"signed\" by  trusted authority (eg, Hashicorp Vault, GCP Private CA, etc).  That way the private keys only exist inside the service.   \n\nFor more information, see [mTLS using acquired Keys](https://github.com/salrashid123/confidential_space/blob/main/README.md#mtls-using-acquired-keys).  However, for simplicity, we are just going to pre-generate the certs and save them to secret manager.\n\n\nAt a high level, this setup includes several steps where each microservice container itself comprises of processes\n\n1. `bootstrap`\n   \n   This is a small go application  which starts up first.  Its entire reason for being is to read certificates saved (in this case) from Secret Manager and launch envoy, consul and the actual service.  In real life, in stead of reading from Secret Manager using the ambient credentials (metadata credential), it should use the [confidential space JWT](https://github.com/salrashid123/confidential_space/blob/main/README.md#attestation-token-and-jwt-bearer-token) to access the backend system return mTLS keys and the consul configuration.\n\n   Once the certificates and consul secrets are acquired, this binary will write them to specific folders where envoy and consul and find them.\n\n   From there, the bootstrap container will launch consul, envoy and the actual service\n\n2. `ingress/egress proxy (envoy)`\n\n   All inbound traffic is to the container intercepted by an envoy proxy which, if configured will enforce mTLS.  The frontend service does not enforce mTLS but the second microservices does.\n   \n   If the frontend microservice needs to talk to another microservice, there are two ways to do this:   directly invoke the second microservice using mTLS certificates or use envoy as an egress https proxy.\n   \n   In the second case, envoy acts will use the client certificates and act as an https proxy for any local traffic intended for the backend.  Envoy in egress mode uses consul for DNS resolution.\n\n3. `consul agent`\n\n   Each node has a consul agent which receives its encryption and configuration keys from secret manager.\n\n   This demo uses pregenerated consul node and application keys.  In real life, these would by dynamically generated as the node pool grows.\n\n   Consul agent registers itself with the consul server and receives DNS updates for valid backend service addresses to connect to\n\n4. `microservice`\n\n   This is the backend application thats the actual microservice.\n\n   The frontend microservice communicates with the backend in two ways:  \n   \n   - do a DNS lookup to consul, read mtls client certificates and invoke the endpoint\n   - connect to envoy egress proxy and let envoy read the mtls certificates.  Envoy will use consul as the DNS server\n\n\n![images/cs_envoy_consul.png](images/cs_envoy_consul.png)\n\n\nAdditional References:\n\n- [Envoy Secret Distribution Service](https://github.com/salrashid123/envoy_control#3-sds-dynamic)\n- [Envoy control plane \"hello world\"](https://github.com/salrashid123/envoy_control)\n- [Envoy EDS \"hello world\"](https://github.com/salrashid123/envoy_discovery)\n- [Envoy Dynamic Forward Proxy configuration with Downstream SNI for Google APIs and HTTPBin](https://github.com/salrashid123/envoy_dynamic_forward_proxy_with_sni)\n- [Envoy mTLS](https://github.com/salrashid123/envoy_mtls)\n- [gRPC xDS Loadbalancing](https://github.com/salrashid123/grpc_xds)\n- [Secure service communication with Consul service mesh and Envoy](https://developer.hashicorp.com/consul/tutorials/developer-mesh/service-mesh-with-envoy-proxy?utm_source=docs)\n\n---\n\n### Setup\n\nThis repo comes with a consul server that is pre-configured with keys and config files for the server and agent.  IT also uses preset mTLS certificates for envoy. \n\nIt will run a standalone consul server in an ordinary VM, then two confidential space instances where each instance is a microservice.\n\nFor the impatient, you can just run this on any project where you have a zone with a subnet of `10.128.0.0/20`\n\n```bash\nexport OPERATOR_PROJECT_ID=`gcloud config get-value core/project`\nexport OPERATOR_PROJECT_NUMBER=`gcloud projects describe $OPERATOR_PROJECT_ID --format='value(projectNumber)'`\n\ngcloud services enable  compute.googleapis.com confidentialcomputing.googleapis.com secretmanager.googleapis.com\n\ngcloud projects add-iam-policy-binding $OPERATOR_PROJECT_ID   --member=serviceAccount:$OPERATOR_PROJECT_NUMBER-compute@developer.gserviceaccount.com    --role=roles/logging.logWriter\n\ngcloud projects add-iam-policy-binding $OPERATOR_PROJECT_ID   --member=serviceAccount:$OPERATOR_PROJECT_NUMBER-compute@developer.gserviceaccount.com    --role=roles/confidentialcomputing.workloadUser\n\ngcloud secrets create ca --replication-policy=automatic   --data-file=envoy/ca.pem\ngcloud secrets create server-cert --replication-policy=automatic   --data-file=envoy/server.crt\ngcloud secrets create server-key --replication-policy=automatic   --data-file=envoy/server.key\n\ngcloud secrets create client-cert --replication-policy=automatic   --data-file=envoy/client.crt\ngcloud secrets create client-key --replication-policy=automatic   --data-file=envoy/client.key\n\ngcloud secrets create be-cert --replication-policy=automatic   --data-file=envoy/be.crt\ngcloud secrets create be-key --replication-policy=automatic   --data-file=envoy/be.key\n\ngcloud secrets create consul-agent-ca --replication-policy=automatic   --data-file=consul/consul-agent-ca.pem\ngcloud secrets create fe_config --replication-policy=automatic   --data-file=consul/client_config_fe/fe_config.json\ngcloud secrets create fe_agent-hcl --replication-policy=automatic   --data-file=consul/client_config_fe/fe_agent.hcl\n\ngcloud secrets create be_agent-hcl --replication-policy=automatic   --data-file=consul/client_config_be/be_agent.hcl\ngcloud secrets create be_config --replication-policy=automatic   --data-file=consul/client_config_be/be_config.json\n\n## allow access to all secrets;  this is just a demo so we'e not even using workload federation no restricting which secrets which container (frontend/backend) should have access to\ngcloud  projects add-iam-policy-binding $OPERATOR_PROJECT_ID --member=serviceAccount:$OPERATOR_PROJECT_NUMBER-compute@developer.gserviceaccount.com  --role=roles/secretmanager.secretAccessor \n\n\ngcloud compute instances create instance-1 --private-network-ip=10.128.0.2\n\ngcloud compute ssh instance-1\n#### install consul https://developer.hashicorp.com/consul/downloads\n#### sudo apt-get install zip -y \u0026\u0026 wget https://releases.hashicorp.com/consul/1.16.1/consul_1.16.1_linux_amd64.zip \u0026\u0026 unzip consul_1.16.1_linux_amd64.zip \u0026\u0026 rm consul_1.16.1_linux_amd64.zip\n\ngit clone https://github.com/salrashid123/cs_systemd\ncd cs_systemd/consul\nconsul agent  -config-file server_config/server.hcl -client 0.0.0.0  -bind 10.128.0.2 -advertise 10.128.0.2  -bootstrap-expect 1\n\n\ngcloud compute ssh instance-1 -- -NL 8500:localhost:8500\n\n\n# goto http://localhost:8500/ui/dc1/overview/server-status  and use TOKEN=7c4518b5-c944-0bc8-6d9e-508ec76e830a\n```\n\nAt this point, we're ready to run the microservices in Confidential Space\n\n\n\n```bash\n## if you want to build these images by hand or 'just run' the images i setup here:\n# docker build -f Dockerfile.fe -t docker.io/salrashid123/cs_fe .\n# docker build -f Dockerfile.be -t docker.io/salrashid123/cs_be .\n\ndocker push docker.io/salrashid123/cs_fe\ndocker push docker.io/salrashid123/cs_be\n\ngcloud compute instances create cs-2  --confidential-compute    \\\n     --shielded-secure-boot --tags=tee-vm  --private-network-ip=10.128.0.41  \\\n               --maintenance-policy=TERMINATE --service-account=\"$OPERATOR_PROJECT_NUMBER-compute@developer.gserviceaccount.com\"   \\\n                         --scopes=cloud-platform  --zone=us-central1-a        \\\n                               --image-project=confidential-space-images --image-family=confidential-space-debug     \\\n                                           --metadata ^~^tee-image-reference=docker.io/salrashid123/cs_fe@sha256:7581f9ad07469c8076b2fc1db5619c9e6d3b548aa5ce436c17d1cd5a9457aa5a~tee-restart-policy=Never~tee-container-log-redirect=true\n\n\ngcloud compute instances create cs-3  --confidential-compute    \\\n     --shielded-secure-boot --tags=tee-vm  --private-network-ip=10.128.0.42  \\\n               --maintenance-policy=TERMINATE --service-account=\"$OPERATOR_PROJECT_NUMBER-compute@developer.gserviceaccount.com\"   \\\n                         --scopes=cloud-platform  --zone=us-central1-a        \\\n                               --image-project=confidential-space-images --image-family=confidential-space-debug     \\\n                                           --metadata ^~^tee-image-reference=docker.io/salrashid123/cs_be@sha256:3ee113dac1e06fc39fc7ad6af66026a37f48f3016cd488fbb9b0b5205a70da37~tee-restart-policy=Never~tee-container-log-redirect=true\n\n# now ssh to instance-1 (the consul server)\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/getfe\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/getbe\n```\n\nYou'll see mtls traffic to the first microservice\n\n```bash\n$ curl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/getfe\n* Added server.domain.com:8081:10.128.0.41 to DNS cache\n* Hostname server.domain.com was found in DNS cache\n*   Trying 10.128.0.41:8081...\n* Connected to server.domain.com (10.128.0.41) port 8081 (#0)\n* ALPN, offering h2\n* ALPN, offering http/1.1\n* successfully set certificate verify locations:\n*  CAfile: envoy/ca.pem\n*  CApath: /etc/ssl/certs\n* TLSv1.3 (OUT), TLS handshake, Client hello (1):\n* TLSv1.3 (IN), TLS handshake, Server hello (2):\n* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):\n* TLSv1.3 (IN), TLS handshake, Certificate (11):\n* TLSv1.3 (IN), TLS handshake, CERT verify (15):\n* TLSv1.3 (IN), TLS handshake, Finished (20):\n* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):\n* TLSv1.3 (OUT), TLS handshake, Finished (20):\n* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384\n* ALPN, server did not agree to a protocol\n* Server certificate:\n*  subject: C=US; O=Google; OU=Enterprise; CN=server.domain.com\n*  start date: Aug 17 17:59:13 2022 GMT\n*  expire date: Nov 24 17:59:13 2024 GMT\n*  subjectAltName: host \"server.domain.com\" matched cert's \"server.domain.com\"\n*  issuer: C=US; O=Google; OU=Enterprise; CN=Enterprise Subordinate CA\n*  SSL certificate verify ok.\n\u003e GET /getfe HTTP/1.1\n\u003e Host: server.domain.com:8081\n\u003e User-Agent: curl/7.74.0\n\u003e Accept: */*\n\n* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):\n* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):\n* old SSL session ID is stale, removing\n* Mark bundle as not supporting multiuse\n\u003c HTTP/1.1 200 OK\n\u003c date: Tue, 19 Sep 2023 01:35:35 GMT\n\u003c content-length: 2\n\u003c content-type: text/plain; charset=utf-8\n\u003c x-envoy-upstream-service-time: 0\n\u003c server: envoy\n\u003c \n\nfe\n```\n\nand traffic to the backend from the first\n\n```bash\n$ curl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/getbe\n* Added server.domain.com:8081:10.128.0.41 to DNS cache\n* Hostname server.domain.com was found in DNS cache\n*   Trying 10.128.0.41:8081...\n* Connected to server.domain.com (10.128.0.41) port 8081 (#0)\n* ALPN, offering h2\n* ALPN, offering http/1.1\n* successfully set certificate verify locations:\n*  CAfile: envoy/ca.pem\n*  CApath: /etc/ssl/certs\n* TLSv1.3 (OUT), TLS handshake, Client hello (1):\n* TLSv1.3 (IN), TLS handshake, Server hello (2):\n* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):\n* TLSv1.3 (IN), TLS handshake, Certificate (11):\n* TLSv1.3 (IN), TLS handshake, CERT verify (15):\n* TLSv1.3 (IN), TLS handshake, Finished (20):\n* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):\n* TLSv1.3 (OUT), TLS handshake, Finished (20):\n* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384\n* ALPN, server did not agree to a protocol\n* Server certificate:\n*  subject: C=US; O=Google; OU=Enterprise; CN=server.domain.com\n*  start date: Aug 17 17:59:13 2022 GMT\n*  expire date: Nov 24 17:59:13 2024 GMT\n*  subjectAltName: host \"server.domain.com\" matched cert's \"server.domain.com\"\n*  issuer: C=US; O=Google; OU=Enterprise; CN=Enterprise Subordinate CA\n*  SSL certificate verify ok.\n\u003e GET /getbe HTTP/1.1\n\u003e Host: server.domain.com:8081\n\u003e User-Agent: curl/7.74.0\n\u003e Accept: */*\n\n* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):\n* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):\n* old SSL session ID is stale, removing\n* Mark bundle as not supporting multiuse\n\u003c HTTP/1.1 200 OK\n\u003c date: Tue, 19 Sep 2023 01:35:38 GMT\n\u003c content-length: 7\n\u003c content-type: text/plain; charset=utf-8\n\u003c x-envoy-upstream-service-time: 12\n\u003c server: envoy\n\u003c \n\nbe cs-3\n```\n\nIf you had the consul admin console running, you'll see:\n\nNodes:\n![images/nodes.png](images/nodes.png)\n\nServices:\n![images/services.png](images/services.png)\n\n\n---\n\n#### Run on VMs\n\nIf you want to run directly on VMs and not deal with confidential space (i.,e just to demo),\n\nfirst create using the following `INTERNAL_IP` addresses:\n\n```bash\n$ gcloud compute instances list\nNAME        ZONE           MACHINE_TYPE  PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\ninstance-1  us-central1-a  e2-medium                  10.128.0.2   34.172.77.225  RUNNING\ncs-2        us-central1-a  e2-medium                  10.128.0.41  34.28.202.134  RUNNING\ncs-2        us-central1-a  e2-small                   10.128.0.42  34.41.27.185   RUNNING\n\n### start Consul Server\n\n## ssh to instance-1\nconsul agent  -config-file server_config/server.hcl -client 0.0.0.0  -bind 10.128.0.2 -advertise 10.128.0.2  -bootstrap-expect 1\n\n# to view the consul ui, in a new window\n# export CONSUL_HTTP_TOKEN=7c4518b5-c944-0bc8-6d9e-508ec76e830a\n#  gcloud compute ssh instance-1 -- -NL 8500:localhost:8500\n# then goto http://localhost:8500/ui/dc1/overview/server-status\n## use CONSUL_HTTP_TOKEN\n\n\n### start Consul Client (node-1)\n# on cs-2, install docker\n\ndocker build -f Dockerfile.fe -t fe .\ndocker run  -p 8081:8081 -p 8500:8500 -p 8500:8500/udp -p 8301:8301 -p 8301:8301/udp fe\n\n\n### Consul Client (node-2)\n# on cs-3, install docker\ndocker build -f Dockerfile.be -t be .\ndocker run  -p 8082:8082 -p 8500:8500 -p 8500:8500/udp -p 8301:8301 -p 8301:8301/udp be\n\n### test service-\u003eservice traffic on node-1\n#### client-\u003efe\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.40  https://server.domain.com:8081/getfe\n####  client-\u003efe-\u003ebe:\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.40  https://server.domain.com:8081/getbe\n\n## If you want to verify that node-1 can infact access the cosul apis\n### get the running CONTAINER_ID\n$ docker ps\nCONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS         \n84d1ef7b46a0   fe        \"/usr/bin/supervisord\"   6 minutes ago   Up 6 minutes   0.0.0.0:8081-\u003e8081/tcp,    fervent_shannon\n\n$ docker exec -ti 84d1ef7b46a0 /bin/bash\n\n## in container:\napt-get install curl wget gettext-base\n\n### get jwt login token for consul\ncd /tmp/\nwget https://raw.githubusercontent.com/salrashid123/consul_jwt_auth/main/acl/jwt.tmpl\nexport URL=https://idp-on-cloud-run-3kdezruzua-uc.a.run.app\nexport IAT=`date -u +%s`\nexport EXP=`date -u +%s -d \"+3600 seconds\"`\nexport EMAIL=\"operator-svc-account@vegas-codelab-5.iam.gserviceaccount.com\"\nexport SUB=\"https://www.googleapis.com/compute/v1/projects/vegas-codelab-5/zones/us-central1-a/instances/vm1\"\nexport ISS=\"https://confidentialcomputing.googleapis.com/\"\nexport NAME=\"alice\"\nexport AUD=\"https://sts.googleapis.com\"\nenvsubst \u003c \"jwt.tmpl\" \u003e \"jwt.json\"\nexport JWT_TOKEN=`curl -s -X POST -d @jwt.json  $URL/token`\necho $JWT_TOKEN\necho -n $JWT_TOKEN \u003e /tmp/jwt_token.txt\n\n## use consul cli to login\n/usr/local/bin/consul login -method=cs-jwt-auth -type=jwt -token-sink-file=/tmp/consul.token -bearer-token-file /tmp/jwt_token.txt -ca-file=/consul/consul-agent-ca.pem\n/usr/local/bin/consul catalog nodes -service=fe -detailed -ca-file=/consul/consul-agent-ca.pem -token=`cat /tmp/consul.token`\n\n## use ordinary dns on consul's port:\n\n$ dig @127.0.0.1 -p 8600 fe.service.consul SRV\n\n    ;; ANSWER SECTION:\n    fe.service.consul.\t0\tIN\tSRV\t1 1 18081 node-1.node.dc1.consul.\n\n    ;; ADDITIONAL SECTION:\n    node-1.node.dc1.consul.\t0\tIN\tA\t10.128.0.41\n    node-1.node.dc1.consul.\t0\tIN\tTXT\t\"consul-version=1.16.1\"\n    node-1.node.dc1.consul.\t0\tIN\tTXT\t\"consul-network-segment=\"\n\n\n$ dig @127.0.0.1 -p 8600 be.service.consul SRV\n\n    ;; ANSWER SECTION:\n    be.service.consul.\t0\tIN\tSRV\t1 1 18082 node-2.node.dc1.consul.\n\n    ;; ADDITIONAL SECTION:\n    node-2.node.dc1.consul.\t0\tIN\tA\t10.128.0.42\n    node-2.node.dc1.consul.\t0\tIN\tTXT\t\"consul-network-segment=\"\n    node-2.node.dc1.consul.\t0\tIN\tTXT\t\"consul-version=1.16.1\"\n```\n\n---\n\n#### Setup full end-to-end\n\nIf you really want to set this  up and run this from scratch without using pregenerated consul config, there are a lot more steps.\n\nYou'll need to create the three vms' with the IP addresses specified.\n\nOn each vm install consul, envoy and golang\n\n```bash\n\n## create a new consul key\ncd consul\n$ consul keygen\n  EYWAIfW4DqcblgN1PLQhZWF82mHekspKkgqIBU0ef40=\n\nexport CONSUL_KEY=EYWAIfW4DqcblgN1PLQhZWF82mHekspKkgqIBU0ef40=\n\n# edit `server_config/server.hcl` and set your value of `$CONSUL_KEY`\n\nrm -rf data/ data_agent/ consul-agent* dc1-*\nconsul tls ca create\nconsul tls cert create -server -dc dc1\n\nconsul agent  -config-file server_config/server.hcl -client 0.0.0.0  -bind 10.128.0.2 -advertise 10.128.0.2  -bootstrap-expect 1\n\n# read SecretID as CONSUL_HTTP_TOKEN\n$ consul acl \n\n      AccessorID:       2d0c2cec-25ae-c558-3d30-41339145d8c7\n      SecretID:         7c4518b5-c944-0bc8-6d9e-508ec76e830a\n      Description:      Bootstrap Token (Global Management)\n      Local:            false\n      Create Time:      2023-09-16 20:48:56.069149359 -0400 EDT\n      Policies:\n        00000000-0000-0000-0000-000000000001 - global-management\n\nexport CONSUL_HTTP_TOKEN=7c4518b5-c944-0bc8-6d9e-508ec76e830a\n\n# use the key here\n# to view the consul ui, in a new window\n#  gcloud compute ssh instance-1 -- -NL 8500:localhost:8500\n# then goto http://localhost:8500/ui/dc1/overview/server-status\n## use CONSUL_HTTP_TOKEN\n\n\n# read SecretID as export NODE_1_TOKEN=\n$ consul acl token create -description \"node-1 agent token\"  -node-identity \"node-1:dc1\"\n      AccessorID:       11a5156a-42e2-1ed3-2a97-82175816d148\n      SecretID:         991a4761-c0ad-3314-c822-89900d53f84c\n      Description:      node-1 agent token\n      Local:            false\n      Create Time:      2023-09-16 20:49:55.172203103 -0400 EDT\n      Node Identities:\n        node-1 (Datacenter: dc1)\n\n# read SecretID as export NODE_2_TOKEN=\n$ consul acl token create -description \"node-1 agent token\"  -node-identity \"node-2:dc1\"\n      AccessorID:       28fbe6f7-0ef1-0ee6-0e59-1aa90f891504\n      SecretID:         4de1d92f-9469-4348-09cc-4b0e4904e400\n      Description:      node-1 agent token\n      Local:            false\n      Create Time:      2023-09-16 20:50:23.182768349 -0400 EDT\n      Node Identities:\n        node-2 (Datacenter: dc1)\n\nexport NODE_1_TOKEN=991a4761-c0ad-3314-c822-89900d53f84c\nexport NODE_2_TOKEN=4de1d92f-9469-4348-09cc-4b0e4904e400\n\nconsul acl policy create -name node-1-policy -rules @acl/node-1-policy.hcl\nconsul acl policy create -name node-2-policy -rules @acl/node-2-policy.hcl\n\nconsul acl policy create -name fe-policy -rules @acl/fe-policy.hcl\nconsul acl policy create -name be-policy -rules @acl/be-policy.hcl\n\n\nconsul acl token create -description \"Token for fe\" -service-identity \"fe\"\n      AccessorID:       a8a46bd5-9807-2eb3-2046-b60792cbb448\n      SecretID:         9befa2c2-a8ba-e145-f628-aa678a625cdc\n      Description:      Token for fe\n      Local:            false\n      Create Time:      2023-09-16 20:52:07.512674558 -0400 EDT\n      Service Identities:\n        fe (Datacenters: all)\n\nexport APP_FE_TOKEN=9befa2c2-a8ba-e145-f628-aa678a625cdc\n\nconsul acl token create -description \"Token for be\" -service-identity \"be\"\n      AccessorID:       c7bfb775-087c-4094-f667-cdaed4639f89\n      SecretID:         85a7b172-d570-2a79-ff45-cfa58cea1f05\n      Description:      Token for be\n      Local:            false\n      Create Time:      2023-09-16 20:52:38.934574596 -0400 EDT\n      Service Identities:\n        be (Datacenters: all)\n\nexport APP_BE_TOKEN=85a7b172-d570-2a79-ff45-cfa58cea1f05\n\nconsul acl policy create -name node-reader -rules @acl/node-reader.hcl\n\nconsul acl policy create -name anonymous-dns-read -rules @acl/anonymous-dns-read.hcl\nconsul acl token update -id anonymous -policy-name=anonymous-dns-read\n\nconsul acl role create -name \"node-reader-role\" -description \"node-reader role\" -policy-name \"node-reader\"\nconsul acl role create -name \"node-1-role\" -description \"node role\" -policy-name \"node-1-policy\"\nconsul acl role create -name \"node-2-role\" -description \"node role\" -policy-name \"node-2-policy\"\n\nconsul acl auth-method create -name=cs-jwt-auth -type jwt  -description 'conf space jwt' -config=@acl/cs-jwt-auth.json\nconsul acl binding-rule create -method=cs-jwt-auth -bind-type=role -bind-name=node-reader-role  -selector=\"value.swname==CONFIDENTIAL_SPACE and value.image_digest==\\\"sha256:c693f5cf4f447b31e8c0ae7f784fc754f783f2e64f8836913c22264004204f6b\\\" and value.swversion==1\"\n\n\n# on instance-2\n##  edit client_config_fe/fe_agent.hcl and enter CONSUL_KEY as encrypt_key $NODE_1_TOKEN as acl.token.agent\n##  edit client_config_fe/fe_config.json and enter APP_FE_TOKEN as service.token\n# consul  agent   -config-dir=client_config_fe\ndocker build -f Dockerfile.fe -t fe .\ndocker run  -p 8081:8081 -p 8500:8500 -p 8500:8500/udp -p 8301:8301 -p 8301:8301/udp fe\n\n# on instance-3\n##  edit client_config_be/be_agent.hcl and enter CONSUL_KEY as encrypt_key $NODE_2_TOKEN as acl.token.agent\n##  edit client_config_be/be_config.json and enter APP_BE_TOKEN as service.token\n# consul  agent   -config-dir=client_config_be\ndocker build -f Dockerfile.be -t be .\ndocker run  -p 8082:8082 -p 8500:8500 -p 8500:8500/udp -p 8301:8301 -p 8301:8301/udp be\n\n### test on instance-2\n\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/\n\ncurl -v --cacert envoy/ca.pem  --resolve  server.domain.com:8081:10.128.0.41  https://server.domain.com:8081/getbe\n```\n\nfor DNS lookups:\n\n```bash\n$ dig @127.0.0.1 -p 8600 fe.service.consul SRV\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n;; QUESTION SECTION:\n;fe.service.consul.\t\tIN\tSRV\n\n;; ANSWER SECTION:\nfe.service.consul.\t0\tIN\tSRV\t1 1 18081 node-1.node.dc1.consul.\n\n;; ADDITIONAL SECTION:\nnode-1.node.dc1.consul.\t0\tIN\tA\t10.128.0.41\nnode-1.node.dc1.consul.\t0\tIN\tTXT\t\"consul-version=1.16.1\"\nnode-1.node.dc1.consul.\t0\tIN\tTXT\t\"consul-network-segment=\"\n```\n\n```bash\n$ dig @127.0.0.1 -p 8600 be.service.consul SRV\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n;; QUESTION SECTION:\n;be.service.consul.\t\tIN\tSRV\n\n;; ANSWER SECTION:\nbe.service.consul.\t0\tIN\tSRV\t1 1 18082 node-2.node.dc1.consul.\n\n;; ADDITIONAL SECTION:\nnode-2.node.dc1.consul.\t0\tIN\tA\t10.128.0.42\nnode-2.node.dc1.consul.\t0\tIN\tTXT\t\"consul-network-segment=\"\nnode-2.node.dc1.consul.\t0\tIN\tTXT\t\"consul-version=1.16.1\"\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fcs_mesh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalrashid123%2Fcs_mesh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalrashid123%2Fcs_mesh/lists"}