{"id":15137336,"url":"https://github.com/didier-durand/microservices-on-cloud-kubernetes","last_synced_at":"2025-04-09T21:20:36.111Z","repository":{"id":114263178,"uuid":"294435580","full_name":"didier-durand/microservices-on-cloud-kubernetes","owner":"didier-durand","description":"Microservices demo application on cloud-hosted Kubernetes cluster","archived":false,"fork":false,"pushed_at":"2021-04-22T05:03:16.000Z","size":2283,"stargazers_count":256,"open_issues_count":0,"forks_count":68,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-02T19:07:30.429Z","etag":null,"topics":["cloud","envoy","gcp","gke","grafana","istio","jaeger","kiali","kubernetes","mesh","microservices","opentelemetry","polaris","prometheus","sidecar"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/didier-durand.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-10T14:35:45.000Z","updated_at":"2025-03-14T12:28:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"dd115189-81e0-49c5-81b9-15207c5b7d44","html_url":"https://github.com/didier-durand/microservices-on-cloud-kubernetes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fmicroservices-on-cloud-kubernetes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fmicroservices-on-cloud-kubernetes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fmicroservices-on-cloud-kubernetes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/didier-durand%2Fmicroservices-on-cloud-kubernetes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/didier-durand","download_url":"https://codeload.github.com/didier-durand/microservices-on-cloud-kubernetes/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248112351,"owners_count":21049644,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","envoy","gcp","gke","grafana","istio","jaeger","kiali","kubernetes","mesh","microservices","opentelemetry","polaris","prometheus","sidecar"],"created_at":"2024-09-26T07:00:38.271Z","updated_at":"2025-04-09T21:20:36.038Z","avatar_url":"https://github.com/didier-durand.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"img/header-logos.jpg\" height=\"265\"\u003e\n\n# Microservices on cloud-based Kubernetes\n\n![workflow badge](https://github.com/didier-durand/microservices-on-cloud-kubernetes/workflows/Microservices%20on%20GKE/badge.svg)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\nThe purpose of this repository is to provide the fully automated setup of a nice-looking (see [screenshots](https://github.com/GoogleCloudPlatform/microservices-demo#screenshots)) \nshowcase / testbed for a cloud-native ([precisely defined](https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/definition) application\nby Microsoft) on a cloud-hosted Kubernetes cluster (here [GKE by Google Cloud](https://cloud.google.com/kubernetes-engine)) based on an interesting [service mesh](https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh). \nSo, additionally, the setup will install tooling (coming from the [CNCF Cloud Trail Map](https://github.com/cncf/trailmap) for many of them) to make the application and its service mesh observable and manageable.\n\nThis application, licensed under Apache terms (same terms for all components used in this worklfow - So, allowing free reuse) is the [\"Online Boutique\"](https://github.com/GoogleCloudPlatform/microservices-demo)\n (formerly known as Hipster Shop - developed by a Google team but not an official product of them). It is composed of 10 polyglot microservices behind a nice-looking web frontend calling them to serve client requests. \n A load-generator - part of the package - will generate traffic while the application is running to make use of tools (Prometheus, OpenTelemetry, \n etc.) more attractive.\n\nAnother goal of this repository is to help people exploring the cloud-native architecture: when you fork it, you rapidly get a working cluster with a somewhat \n\"real-life\" application and decent tooling to experiment with, without the need for a long trial-and-error process starting with infrastructure to set it \nup from scratch. It makes it much faster to grasp the philosophy of the distributed architecture proposed by Kubernetes.\n\nSo, happy forking for your own use! (see [Setup section](#setup-for-forks) for all \ntechnical details) And come back regularly or get notified by following this repository: we will add additional tools in subsequent updates.\n\nWe implement here a Github workflow ([microservices-on-gke.yml](.github/workflows/microservices-on-gke.yml) \u0026 [shells in sh directory](sh/) - see our [other repository](https://github.com/didier-durand/gcp-workflows-on-github) for other workflows)\nwhich allows to automatically deploy a fresh cluster on GKE and to deploy the application on it whenever needed via a [single click](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/). \nOn our side, this same workflow is also started automatically on a recurring basis (at least weekly) via Github's cron facility (included in workflow yaml) \nto make sure that the deployment remains fully operational as underlying GKE infrastructure and implemented components evolve. You can access logs \nof previous runs in the [Actions Tab](actions/).\n\n## Access to application \u0026 load generator\n\nOn successful completion of the workflow, the Online Boutique is accessible from anywhere on the Internet at the public IP address \n(dynamically created and published by GKE) displayed in the final lines of workflow execution step \"Deploy application on GKE\". Indeed, it is the IP \naddress of the [K8s service](https://kubernetes.io/docs/concepts/services-networking/service/) 'frontend-external' defined by the deployment. Hence, \nyou can also get it at any time via *'kubectl get service 'frontend-external''* provided that you went through proper setup as described below.\n\nTo check the activity of the load generator, you can at any time run *'kubectl logs -l app=loadgenerator -c main'*\nYou should get something like the following describing how many requests were already triggered:\n\n```\nkubectl logs -l app=loadgenerator -c main\n GET /product/66VCHSJNUP                                          600     0(0.00%)      77      34    1048  |      41    0.10    0.00\n GET /product/6E92ZMYYFZ                                          563     0(0.00%)      77      34    1763  |      41    0.00    0.00\n GET /product/9SIQT8TOJO                                          593     0(0.00%)      73      34    1013  |      41    0.30    0.00\n GET /product/L9ECAV7KIM                                          631     0(0.00%)      82      34    1349  |      42    0.20    0.00\n GET /product/LS4PSXUNUM                                          608     0(0.00%)      83      34     896  |      42    0.20    0.00\n GET /product/OLJCESPC7Z                                          623     0(0.00%)      69      34    1079  |      41    0.10    0.00\n POST /setCurrency                                                808     0(0.00%)      82      44    1089  |      51    0.20    0.00\n--------------------------------------------------------------------------------------------------------------------------------------------\n Aggregated                                                      9517     0(0.00%)                                       1.80    0.00\n```\n \nIf you want to easily inject more traffic, you can additionally use the [hey](https://github.com/rakyll/hey) or [fortio](https://github.com/fortio/fortio) \nutilities as we did in our [Knative project](https://github.com/didier-durand/knative-on-cloud-kubernetes) : see correponding [workflow script](https://github.com/didier-durand/knative-on-cloud-kubernetes/blob/master/.github/workflows/gcloud-gke-knative.yml).\n\n## Access to deployed tools \u0026 dashboards\n\nYou have first to implement the requirements of the [Setup section](#setup-for-forks) \nbefore trying to access the dashboards.\n\nTo keep things simple, we access all tools and dashboards via the proxy functions available in Kubernetes: either directly via *'kubectl proxy'* \nor indirectly via *'istioctl dashboard xxx'*. Only limited additional definitions are then required: it's just fine for a demo and initial tests. \nOf course, the laptop running the proxies must be authentified to gcloud via SDK with proper credentials giving rights to cluster administration.\n\n__**Available dashboards:**__\n\n(click on pictures to enlarge them - also use the hyperlinks provided with each dashboard description to have a good overview of the \nfeatures of each tool from its official documentation)\n\n\u003cimg src=\"img/screenshots/k8s_ui.jpg\" height=\"200\"\u003e\n\n1. **Standard K8s UI**: our workflow deploys first [this standard Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) \nas a tool that should anyway be included in any installation. It gives a good overview of the deployed cluster with static (configuration) and dynamic (metrics) information \nabout the active objects. When *'kubectl proxy'* is active, the dashboard is available at [this url](http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/). \nThe security check at login is most easily satisfied by selection the login option of config file (see Prereqs to obtain it in Setup section). \n\n\u003cimg src=\"img/screenshots/polaris-dashboard.jpg\" height=\"200\"\u003e\n\n2. **Polaris dashboard**: [Polaris](https://github.com/FairwindsOps/polaris) is an interesting tool, even in its OSS version (the paid-for version \nprovides more checks) used here. After installation, it [scans the definitions](https://www.fairwinds.com/polaris) of various kinds of objects and applies sanity checking rules to \nvalidate their proper configuration. For example, in the case of Online Boutique, it will warn that containers have no resource constraints (cpu, \nmemory, etc.) imposed on them or that their security credentials are too wide compared to what they do. The hyperlinks provided on the unsatisfactory \nchecks document the reason(s) of the alert as well as the possible remedies to apply. So, a quite useful tool to incrementally increase the quality of the \nconfiguration of a given cluster: new versions of yaml object manifests can be gradually deployed to minimize the issue notifications.\n\n\u003cimg src=\"img/screenshots/kiali-dashboard.jpg\" height=\"200\"\u003e\n\n3. **Kiali Dashboard**: [Kiali](https://kiali.io/) claims itself the *\"Service mesh management for Istio\"*. it allows an interactive discovery of \n the defined relations between the services. Then, it provides [detailed insights and metrics](https://kiali.io/documentation/latest/features/) about the \n health of those services and the requests between them.  The live traffic animation in the UI is extremely useful: it allows to spot very quickly \n where the activity happens to focus on those hot spots during root cause analysis for an issue. You can also go back in time with the replay \n feature to see the traffic and the interactions that happened in the past to understand why and how you reached current situation. This dashboard \n is accessed via *'istioctl dashboard kiali'* that will open the corresponding UI into your web browser.\n\n\u003cimg src=\"img/screenshots/grafana-dashboard.jpg\" height=\"200\"\u003e\n\n4. **Grafana dashboard**: [Grafana](https://grafana.com/oss/grafana/) which *'allows you to query, visualize, alert on metrics and logs, no \n matter where they are stored'* provides very nice charts about the activity of the cluster as the whole (usual metrics about resource consumption: \n cpu, memory, etc. for nodes. But, more specifically in this context it provides interesting additional [dashboards specific to the Istio service mesh](https://istio.io/latest/docs/tasks/observability/metrics/using-istio-dashboard/) \n related to the traffic between the pods. Those dashboards are accessed via *'istioctl dashboard grafana'* that will open the corresponding UI into \n your web browser.\n\n\u003cimg src=\"img/screenshots/jaeger-dashboard.jpg\" height=\"200\"\u003e\n\n5. **Jaeger dashboard**: the Open Boutique is instrumented via [OpenCensus](https://opencensus.io/), now merged into \n[OpenTelemetry](https://opentelemetry.io/), component of the CNCF Cloud Trail Map. Jaeger - in CNCF Cloud Trail Map - is the tracing backend \nimplemented here. It centralizes and ingests the distributed traces produced by the various microservices. So, the Jaeger dashboard will allow the \ndetailed examination of those aggregated distributed traces also known as [\"spans\"](https://opentracing.io/docs/overview/spans/#what-is-a-span). This \ndashboard is accessed via *'istioctl dashboard jaeger'* that will open the corresponding UI into your web browser.\n \n\u003cimg src=\"img/screenshots/prometheus-dashboard.jpg\" height=\"200\"\u003e\n \n6. **Prometheus dashboard**: [Prometheus](https://prometheus.io/) (also member of CNCF Cloud Trail Map) is the cornerstone component for metrics \ncollection. The collected data is used by Grafana, Kiali \u0026 Jaeger for their specific purposes. The Prometheus dashboard can be used as the \"source \nof truth\": for example, it can be used to verify if some metrics claimed as missing by a downstream component using this value is really collected \nor not and to compare the graph produced by Prometheus itself to the graph produced downstream user to spot potential discrepancies. This dashboard \nis accessed via *'istioctl dashboard prometheus'* that will open the corresponding UI into your web browser.\n\n\u003cimg src=\"img/screenshots/envoy-dashboard.jpg\" height=\"200\"\u003e\n\n7. **Envoy dashboard**: an improved version of [Envoy Proxy](https://www.envoyproxy.io/docs/envoy/latest/intro/what_is_envoy), also part of the CNCF Cloud Trail Map,  is the [sidecar\n  container](https://istio.io/latest/docs/concepts/what-is-istio/#why-use-istio) used by Istio. Envoy provides a (somewhat rough) dashboard to go into the \n  nitty-gritty details of a given pod: it is usually used for low-level introspection into the traffic between pods in unexpected situations. It is \n  more a debugging tool at very low-level: most cluster administrators shouldn't need to use it. This dashboard is accessed via *'istioctl \n  dashboard envoy podname[.namespace]'*: it will open the corresponding UI for the chosen pod sidecar into your web browser.\n   \n## Workflow steps\n\nThe workflow has following steps:\n\n1. setup of workflow parameters via environment variables\n2. checkout of project artefacts\n3. Setup of gcloud SDK for authentication to Google Cloud through secrets defined in repository via setup.(subsequent steps in shell script)\n5. create cluster\n6, import cluster config \u0026 credentials for kubectl \u0026 istioctl\n7. deploy standard K8s dashboard and check its proper deployment\n8. deploy Polaris dashboard and check its proper deployment\n9. deploy Istio and check its proper deployment\n10. deploy Istio-based addons and check their proper deployment\n11. label application namespace to ensure automatic sidecar injection (see [Istio architecture](https://istio.io/latest/docs/ops/deployment/architecture/) uses an improved version of Envoy for sidecars) in microservice pods\n12. deploy proper application manifests for Istio\n13. deploy Online Boutique and check its proper deployment\n14. obtain application public IP address and check accessibility from GitHub CI/CD\n15. validate expected activity of the traffic generator for Online Boutique\n16. check proper service mesh config for the microservices via istioctl\n\nApplication can now be accessed as described [above](https://github.com/didier-durand/microservices-on-cloud-kubernetes#access-to-application--load-generator)\n\n\n## Application features \u0026 service mesh\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"img/application/online-boutique-architecture.png\" height=\"300\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003cins\u003eapplication service mesh\u003cins\u003e\u003c/p\u003e\n\n\nThis demo application contains an interesting service mesh to give some substance to demos and tests: its schema is given above. This mesh \nwhich is thoroughly used by a traffic generator - also part of the demo package - which generates constant solid traffic to make the implementation \nof monitoring tools.\n\nInteresting points of Online Boutique:\n\n1. **Multi-language:** the microservices corresponding to the various application features were on purpose written on purpose by the authors in \nnumerous languages (Go, NodeJS, Java, C#, Python) to demonstrate a key strength of container-based applications: many microservices collaborate in \na \"polyglot\" environment where each team (or individual) can program in its language of choice while ad hoc frameworks for each language make sure \nthat all Kubernetes standards (probes, etc.) and architecture can be easily respected with minimal effort to obtain a coherent and compliant global \nsystem, manageable by the standard palette of Kubernetes tools. This polyglot aspect is reinforced by the mixed use of http and gRpc, which are both\n understood by the monitoring tools.\n \n2. **Service Mesh:** the application graph shows relationships between the various services and the front-end. Indeed, the application \nis made of13 pods. This high-level of granularity is the accepted Kubernetes pattern for application architecture: it brings numerous advantages like continuous \ndelivery, exhaustive unit testing, higher resilience, optimal scalability, etc. But, it also requires the use of a thorough set of tools to \nmaximize the observability of the system. If \"divide and conquer\" is the motto of cloud-native architectures, the motto of their operations is probably \n\"observe to sustain\": when working with Kubernetes application, one feels very quickly the need for (very) solid tools monitoring automatically \nthe myriad of objects (services, pods, ingress, volumes, etc.) composing the system.   \n\n3. **GCP Tooling:** the application is instrumented for [Stackdriver (profiling, logging, debugging)](https://en.wikipedia.org/wiki/Stackdriver). So, \nthe source code of this application provides the right guidance to see how to code in order to obtain the right leverage on tools directly available \nfrom the GCP service portfolio.\n\n## Setup for forks\n\nTo start with, you need a Google Cloud account with a project in it where the GKE APIs have been enabled. Obtain the id of your project from \nGCP dashboard. Additionally, you need to create in this project a service account and give it proper GKE credentials: right to create, administer \nand delete a cluster. Save its private key in json format.\n\nThen, fork our repository and define the required [Github Secrets](https://docs.github.com/en/actions/reference/encrypted-secrets) in your forked \nrepository: \n1. your GCP project id will be {{ secrets.GCP_PROJECT }}\n2. The private key of your service account in json format will be ${{ secrets.GCP_SA_KEY }} \n\nTo easily launch the workflow, you can launch it with the [manual dispatch feature of Github](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/) that you can see as a launch button in the Action tab of your project for \nthe \"Deploy Online Boutique\" workflow. Similarly, you can stop it via similar button in \"Terminate Online Boutique\" workflow.\n\nWhen the deployment workflow completes successfully, you should be able to access the Online Boutique from anywhere on the Internet at the pubic IP \naddress displayed in the final lines of step \"Deploy application on GKE\" (or via \n\nTo get access to the cluster via kubectl and to the dashboards via istioctl, you need to install on your machine the gcloud SDK, connect to GCP \nwith your userid (having at least same credentials as service account above). Then, use *'gcloud container clusters get-credentials \u003cCLUSTER-NAME\u003e --zone \u003cGCP-ZONE\u003e --project=\u003cPROJECT-ID\u003e'* \nwith your own values. It will prepare and install on your machine the proper config and credentials files - usually located in ~/.kube - to give \nyou access to your cluster via kubectl and istioctl.\n\nFinally, you should [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) and \n[install istioctl](https://istio.io/latest/docs/setup/install/istioctl/) if not present on your laptop yet.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdidier-durand%2Fmicroservices-on-cloud-kubernetes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdidier-durand%2Fmicroservices-on-cloud-kubernetes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdidier-durand%2Fmicroservices-on-cloud-kubernetes/lists"}