{"id":14987733,"url":"https://github.com/felixanna/web-service-dlw","last_synced_at":"2025-04-12T00:17:19.330Z","repository":{"id":37479932,"uuid":"440916907","full_name":"FelixAnna/web-service-dlw","owner":"FelixAnna","description":"Restful Golang Microservice","archived":false,"fork":false,"pushed_at":"2024-12-11T23:57:54.000Z","size":5371,"stargazers_count":4,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-12T00:17:10.902Z","etag":null,"topics":["application-gateway","azure-kubernetes-service","consul","docker","gin-gonic","go-micro","golang","gorm","helm","horizontal-pod-autoscaler","kms","kong","kubernetes","mockery","mongodb","nginx","oauth2","parameter-store","testify","wire"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FelixAnna.png","metadata":{"files":{"readme":"docs/readme_old.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-22T16:18:55.000Z","updated_at":"2024-04-22T14:26:28.000Z","dependencies_parsed_at":"2025-02-20T10:42:21.590Z","dependency_job_id":null,"html_url":"https://github.com/FelixAnna/web-service-dlw","commit_stats":{"total_commits":422,"total_committers":4,"mean_commits":105.5,"dds":"0.17535545023696686","last_synced_commit":"63aff95b263d23861d0ff92c7533cb6c074d5484"},"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FelixAnna%2Fweb-service-dlw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FelixAnna%2Fweb-service-dlw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FelixAnna%2Fweb-service-dlw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FelixAnna%2Fweb-service-dlw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FelixAnna","download_url":"https://codeload.github.com/FelixAnna/web-service-dlw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248497817,"owners_count":21113984,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["application-gateway","azure-kubernetes-service","consul","docker","gin-gonic","go-micro","golang","gorm","helm","horizontal-pod-autoscaler","kms","kong","kubernetes","mockery","mongodb","nginx","oauth2","parameter-store","testify","wire"],"created_at":"2024-09-24T14:15:17.391Z","updated_at":"2025-04-12T00:17:19.317Z","avatar_url":"https://github.com/FelixAnna.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dlw - daily life web microservices (RESTful)\n\n[![Build Status](https://github.com/FelixAnna/web-service-dlw/workflows/Run%20Tests/badge.svg?branch=master)](https://github.com/FelixAnna/web-service-dlw/actions?query=branch%3Amaster)\n[![codecov](https://codecov.io/gh/FelixAnna/web-service-dlw/branch/master/graph/badge.svg)](https://codecov.io/gh/FelixAnna/web-service-dlw)\n[![Go Report Card](https://goreportcard.com/badge/github.com/FelixAnna/web-service-dlw/common)](https://goreportcard.com/report/github.com/FelixAnna/web-service-dlw/common)\n\n# Table of Contents\n\n- 1. [Prepare](#prepare)\n- 2. [Switch Context](#swith-kubectl-context)\n\n- 3. [Components](#components)\n    - 3.1 [Microservices](#microservices)\n    - 3.2 [Ingress controller](#ingress-controller)\n    - 3.3 [Metric Server](#metric-server)\n    - 3.4 [Dashboard](#dashboard)\n\n- 4. [Deployment](#deployments)\n    - 4.1 [Helm](#helm-deployments)\n    - 4.2 [Kubectl](#kubectl-deployments)\n    - 4.3 [Docker](#docker-deployments-with-consul-and-sd-solution)\n\n- 5. [Target](#target)\n    - 5.1 [Kind](#kind)\n    - 5.2 [AKS](#aks)\n    - 5.3 [MiniKube](#minikube)\n    - 5.4 [Docker-Desktop](#docker-desktop)\n    \n- 6. [Front-end](#front-end)\n\n## Prepare \n1. Register OAuth Apps in https://github.com/settings/developers\n   \n   the Authorization callback URL should be： {baseApiUrl}/user/oauth2/github/redirect\n   \n   keep the ClientID and ClientSecret\n\n2. Add parameters in aws parameter store: \n   \n   https://ap-southeast-1.console.aws.amazon.com/systems-manager/parameters/?region=ap-southeast-1\u0026tab=Table , \n   \n   use KMS customer managed keys if necessary.\n\n3. Create Tables in aws DynamoDB:\n\n   dlf.Memos, dlf.Users\n   \n4. Prepare an SQL Server instance to store data for finance api, table will be automatic migrated\n5. Prepare database in MongoDB atlas (free forever for first 500MB)\n\t\n\t* database: dlw_mathematicals\n\t* collections： answers， questions\n\n## Switch kubectl context\n\nafter you connected to aks, you context is attached to aks by default, if you want to check your local kubernetes status, you need switch context:\n\n```bash\nkubectl config view\nkubectl config use-context kind-dlw-cluster\n```\n\n## Components\n### Microservices\n\n1. User api service: [user api service](/user-api/readme.md) [![Go Report Card](https://goreportcard.com/badge/github.com/FelixAnna/web-service-dlw/user-api)](https://goreportcard.com/report/github.com/FelixAnna/web-service-dlw/user-api)\n2. Memo api service: [memo api service](/memo-api/readme.md) [![Go Report Card](https://goreportcard.com/badge/github.com/FelixAnna/web-service-dlw/memo-api)](https://goreportcard.com/report/github.com/FelixAnna/web-service-dlw/memo-api)\n3. Date api service: [date api service](/date-api/readme.md) [![Go Report Card](https://goreportcard.com/badge/github.com/FelixAnna/web-service-dlw/date-api)](https://goreportcard.com/report/github.com/FelixAnna/web-service-dlw/date-api)\n4. Finance api service: [finance api service](/finance-api/readme.md) [![Go Report Card](https://goreportcard.com/badge/github.com/FelixAnna/web-service-dlw/finance-api)](https://goreportcard.com/report/github.com/FelixAnna/web-service-dlw/finance-api)\n\n#### kubectl deployment templetes\n`devops/*.yaml`: native kubernetes deployments templements, include microservices and nginx ingress.\n\n#### helm deployment templetes (autoscaling)\n\n`devops/dlw-chart`: include autoscaling configurations which only supported by kubectl 1.23+ or above.\n\n#### helm deployment templetes(no autoscaling)\n`devops/dlw-helm`: no autoscaling configurations in the deployments templements\n\n\n### Ingress Controller\nfollowing: [install nginx](https://kubernetes.github.io/ingress-nginx/deploy/#docker-desktop) to install ingress for certain env.\n\n#### by kubectl: \n    \n```bash\nkubectl apply -f ingress_deployment.yml\n```\n\n#### by helm: \n\n```bash\nhelm upgrade --install ingress-nginx ingress-nginx \\\n--repo https://kubernetes.github.io/ingress-nginx \\\n--namespace ingress-nginx --create-namespace\n```\n\n\n### Metric Server\n`devops/metrics/*.yaml`: enable metrics server which is necessary for horizontalautoscaler or veticalautoscaler if metric server not deployed by default, --kubelet-insecure-tls args is used for local, --metric-resolution can be set to longer if use docker-desktop\n\ncloud based kubernetes already include metric server by default.\n\n### Dashboard\n`devops/dashboard`: follow the instructions to enable dashboard.\n\n## Deployments\n### Helm Deployments\n\n#### setup\n1. download and unzip helm, add folder to env PATH, following: https://helm.sh/   https://github.com/helm/helm/releases\n\n2. add helm chart repo: https://helm.sh/docs/intro/quickstart/\n\t```bash\n\thelm repo add bitnami https://charts.bitnami.com/bitnami\n\t```\n#### deploy\n1. update the *awsKeyId* and *awsSecretKey* to correct value in: `deployment\\kubernetes\\dlw-chartvalues_*.yaml`\n2. cd to `deployment\\kubernetes` folder, run:\n\t```bash\n\thelm install dlw ./dlw-chart/ --namespace dlw-dev --create-namespace  --values ./dlw-chart/values_dev.yaml\n\t```\n3. after all resources installed (include ingress controller), access test api from local browser: http://localhost/date/status\n4. update by running:\n\t```bash\n\thelm upgrade --install dlw ./dlw-chart/ --namespace dlw-dev --values ./dlw-chart/values_dev.yaml\n\t```\n5. remove all by running:\n\t```bash\n\thelm uninstall dlw -n dlw-dev\n\t```\n\n### Kubectl Deployments\n1. cd to `deployment\\kubernetes` folder, update the *AWS_ACCESS_KEY_ID* and *AWS_SECRET_ACCESS_KEY* to correct value in \"namespace_config_secret_dev.yaml\"\n2. build docker images for the 4 api services, and tag them as xxx-api:1.0.0\n3. start deployment from deployment folder:\n    ```bash\n    kubectl apply -f namespace_config_secret_dev.yaml -n dlw-dev\n    kubectl apply -f deployment_dev.yaml -n dlw-dev\n    kubectl apply -f ingress_dev.yaml -n dlw-dev\n    kubectl apply -f auto_scaler.yaml -n dlw-dev\n    ```\n\n4. waiting for the ingress resource ready\n    ```bash\n    kubectl config set-context --current --namespace=dlw-dev\n\n    kubectl describe ingress\n    ```\n5. access from local browser: http://localhost/date/status\n\n6. clean resource when you finished the local test:\n     ```bash\n    kubectl delete -f auto_scaler.yaml -n dlw-dev\n    kubectl delete -f ingress_dev.yaml -n dlw-dev\n    kubectl delete -f deployment_dev.yaml -n dlw-dev\n    kubectl delete -f namespace_config_secret_dev.yaml -n dlw-dev\n    ```\n\n### Docker Deployments (with consul and sd solution)\n\n1. start consul service and client: [start consul](https://learn.hashicorp.com/tutorials/consul/docker-container-agents?in=consul/docker)\n\n    a. start server\n\n```bash\ndocker run \\\n-d \\\n-p 8500:8500 \\\n-p 8600:8600/udp \\\n--name=shifu \\\nconsul agent -server -ui -node=server-1 -bootstrap-expect=1 -client=0.0.0.0\n\n```\n\n    check ip of the consul server by exec command inside badger server\n\n```bash\ndocker exec badger consul members\n```\n\n    b. register a client\n\n```bash\ndocker run \\\n--name=tuer \\\nconsul agent -node=client-1 -join=172.17.0.2\n```\n2. store your aws credentials in place: [configuring aws sdk](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html)\n\n    i. for local recommand: Shared Credentials File\n\n    ii. for docker container not in ecs/eks/ec2: use env passed to container (in yaml or from docker run command)\n\n    iii. for docker container in ecs/eks/ec2: use aws role assume\n\n3. start user-api on port 8181:\n\n    i. run in local system: \n        \n        ```bash\n        cd user-api\n        profile=dev go run .\n        ```\n\n    ii. run from local docker container: \n        \n        ```bash\n        docker run -d -e AWS_ACCESS_KEY_ID=xyz -e AWS_SECRET_ACCESS_KEY=abc -e AWS_REGION=ap-southeast-1 -e profile=dev  --publish 8383:8383 date-api:1.0.0\n        ```\n4. start memo-api/date-api/finance-api in the same way\n5. test your api works\n\n    a. get github redirectUrl: \n    ```bash\n    curl --location --request GET 'http://localhost:8181/oauth2/github/authorize/url'\n    ```\n    b. copy to browser, and keep the token returned\n    c. login to user api with the github token\n    ```bash\n    curl --location --request GET 'http://localhost:8181/oauth2/github/user?access_code=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'\n    ```\n    d. add one user memo\n    ```bash\n    curl --location --request PUT 'http://localhost:8282/memos/' \\\n    --header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiIxNjM4MzYzMDY1MDgxIiwiZW1haWwiOiJ5dWVjbnVAaG90bWFpbC5jb20iLCJleHAiOjE2NDAzMjk1MjB9.uOvsu9mLS95Wc9uWONGR-DZx6WPfGxChrHJ6dPaAsag' \\\n    --header 'Content-Type: application/json' \\\n    --data-raw '{\n        \"Subject\": \"Test Item 8\",\n        \"Description\":\"Test Description 8\",\n        \"MonthDay\":1208,\n        \"StartYear\": 1999,\n        \"Lunar\":true\n    }'\n    ```\n    e. find the memo just added, and see the previous and next memo date distance\n    ```bash\n    curl --location --request GET 'http://localhost:8282/memos/recent?start=1124\u0026end=1227' \\\n    --header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiIxNjM4MzYzMDY1MDgxIiwiZW1haWwiOiJ5dWVjbnVAaG90bWFpbC5jb20iLCJleHAiOjE2NDAzMjk1MjB9.uOvsu9mLS95Wc9uWONGR-DZx6WPfGxChrHJ6dPaAsag'\n    ```\n\n    ```json\n    [\n        {\n            \"Id\": \"1639581432081\",\n            \"UserId\": \"1638363065081\",\n            \"Subject\": \"Test Item 7\",\n            \"Description\": \"Test Description 7\",\n            \"MonthDay\": 1208,\n            \"StartYear\": 1999,\n            \"Lunar\": false,\n            \"Distance\": [\n                -28,\n                337\n            ],\n            \"CreateTime\": \"1639581432\"\n        },\n        {\n            \"Id\": \"1639581443887\",\n            \"UserId\": \"1638363065081\",\n            \"Subject\": \"Test Item 8\",\n            \"Description\": \"Test Description 8\",\n            \"MonthDay\": 1208,\n            \"StartYear\": 1999,\n            \"Lunar\": true,\n            \"Distance\": [\n                -32,\n                323\n            ],\n            \"CreateTime\": \"1639581443\"\n        }\n    ]\n    ```\n\n\n## Target\n### Kind\n`devops/kind/*.yml`: set up kubernetes cluster by using kind, which can run multiple control panel and work nodes by using docker containers in local.\n\nsee details in: [kind/readme.md](devops/kind/readme.md)\n\n### AKS\n1. create acr, like: dlwcr\n2. push local images to the acr like below:\n\n    ```bash\n    docker tag memo-api:1.0.0 dlwcr.azurecr.io/memo-api:1.0.0\n    docker push  dlwcr.azurecr.io/date-api:1.0.0\n    ```\n\n3. create aks cluster, 1 node is ok, select kubeneters \u003e=1.23\n4. connect your local kubectl to aks cluster\n\n\t`az aks get-credentials --resource-group dlw-cluste_group --name dlw-cluster`\n\n    `az ml computetarget detach -n dlw-cluster -g dlw-cluste_group -w myworkspace`\n    `az aks get-credentials --resource-group dlw-cluste_group --name dlw-cluster`\n\n5. install nginx-controller: [install nginx for aks](https://docs.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli)\n\n\tadd `--set controller.service.externalTrafficPolicy=Local` for enable access to the dynamic assigned public ip of nginx controller\n\n\t```bash\n\tNAMESPACE=ingress-basic\n\n\thelm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx\n\thelm repo update\n\n\thelm install ingress-nginx ingress-nginx/ingress-nginx \\\n\t  --create-namespace \\\n\t  --namespace $NAMESPACE \\\n\t  --set controller.service.annotations.\"service\\.beta\\.kubernetes\\.io/azure-load-balancer-health-probe-request-path\"=/healthz \\\n\t  --set controller.service.externalTrafficPolicy=Local\n\t  ```\n6. deploy/upgrade/uninstall by：\n\t\n\t```bash\n\thelm upgrade --install dlw ./dlw-chart/ --namespace dlw-dev --create-namespace --values ./dlw-chart/values_aks.yaml\n\n\thelm upgrade dlw ./dlw-chart/ --namespace dlw-dev --values ./dlw-chart/values_aks.yaml --set controller.service.externalTrafficPolicy=Local\n\n\thelm uninstall dlw -n dlw-dev\n\t```\n\n7. user external ip of ingress to access the api services\n\n### Minikube\n\na. start minikube and enable ingress: \n\n    ```bash\n    minikube start\n    minikube addons enable ingress\n    ```\n\nb. eval $(minikube -p minikube docker-env)  ***force to use minikube docker deamon in current shell***\n\nc. docker build -t xxx-api .  ***build image use minikube docker deamon so it visible to minikube***\n\nd. after deployed, ssh to minikube container to test the api after installed.\n\n### Docker desktop\n start your docker-desktop service, enable kubernetes feature (with wsl 2 enbled together).\n\n deploy ingress, metric server, microservices use either helm or kubectl.\n\n known issue is the metric server need longer `--metric-resolution`, refer [metric server](#metric-server)\n \n## Front-end\nimplemented by [ReactJs + Redux](https://github.com/FelixAnna/keep-hands-on/tree/master/important/dlw-app)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelixanna%2Fweb-service-dlw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffelixanna%2Fweb-service-dlw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelixanna%2Fweb-service-dlw/lists"}