{"id":22713929,"url":"https://github.com/rsachdeva/illuminatingdeposits-rest","last_synced_at":"2025-06-28T16:38:12.463Z","repository":{"id":57559397,"uuid":"280040308","full_name":"rsachdeva/illuminatingdeposits-rest","owner":"rsachdeva","description":"Illuminating Deposits Rest Server Services - Go(Golang); Docker; Kubernetes Ingress Deployment with TLS; Postgres SQL;  TLS requests with Sanity test Client; JWT generation for authentication;JWT Authentication for Interest Calculations. With Tests/ Coverage. With Observability - Tracing.","archived":false,"fork":false,"pushed_at":"2021-02-24T23:08:54.000Z","size":7424,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-05T00:28:01.295Z","etag":null,"topics":["docker","docker-compose","dockertest","go","go-test","go116","golang","helm","illuminatingdeposits-rest","ingress-nginx","integration-testing","json","jwt-authentication","kubernetes","observability","postgresql","rest-api","test-coverage","tls","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsachdeva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-16T03:12:28.000Z","updated_at":"2021-02-25T00:52:38.000Z","dependencies_parsed_at":"2022-08-28T14:03:17.920Z","dependency_job_id":null,"html_url":"https://github.com/rsachdeva/illuminatingdeposits-rest","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-rest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-rest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-rest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-rest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsachdeva","download_url":"https://codeload.github.com/rsachdeva/illuminatingdeposits-rest/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246254099,"owners_count":20747948,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","dockertest","go","go-test","go116","golang","helm","illuminatingdeposits-rest","ingress-nginx","integration-testing","json","jwt-authentication","kubernetes","observability","postgresql","rest-api","test-coverage","tls","unit-testing"],"created_at":"2024-12-10T14:10:10.904Z","updated_at":"2025-03-29T22:44:24.427Z","avatar_url":"https://github.com/rsachdeva.png","language":"Go","readme":"# Illuminating Deposits - Rest http json\n\n###### All commands should be executed from the root directory (illuminatingdeposits-rest) of the project \n(Development is WIP)\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./logo.png\" alt=\"Illuminating Deposits Project Logo\" title=\"Illuminating Deposits Project Logo\" /\u003e\n\u003c/p\u003e\n\n# REST API using JSON for Messages\n# Features include:\n- Golang (Go)  REST Http Service requests with json for Messages\n- TLS for all requests\n- Integration and Unit tests; run in parallel using dockertest for faster feedback\n- Coverage Result for key packages\n- Postgres DB health check service\n- User Management service with Postgres for user creation\n- JWT generation for Authentication\n- JWT Authentication for Interest Calculations\n- 30daysInterest for a deposit is called Delta\n- Delta is for\n    - each deposit\n    - each bank with all deposits\n    - all banks!\n- Sanity test client included for settings for each deployment\n- Dockering and using it for both Docker Compose and Kubernetes\n- Docker compose deployment for development\n- Kuberenets Deployment with Ingress; Helm\n- Running from Editor/IDE directly included\n- Tracing enabled using Zipkin for Observability\n  \n# Docker Compose Deployment\n\n### TLS files\n```shell\nexport DEPOSITS_REST_SERVICE_ADDRESS=localhost\ndocker build -t tlscert-rest:v0.1 -f ./build/Dockerfile.openssl ./conf/tls \u0026\u0026 \\\ndocker run --env DEPOSITS_REST_SERVICE_ADDRESS=$DEPOSITS_REST_SERVICE_ADDRESS -v $PWD/conf/tls:/tls tlscert-rest:v0.1\n```\n\n### Start postgres and tracing services\n```shell\nexport COMPOSE_IGNORE_ORPHANS=True \u0026\u0026 \\\ndocker-compose -f ./deploy/compose/docker-compose.external-db-trace-only.yml up \n```\n\n### Then Migrate and set up seed data:\n```shell\nexport COMPOSE_IGNORE_ORPHANS=True \u0026\u0026 \\\ndocker-compose -f ./deploy/compose/docker-compose.seed.yml up --build\n```\n\n### To start all services without TLS:\nMake sure DEPOSITS_REST_SERVICE_TLS=false in docker-compose.rest.server.yml\n### To start all services with TLS:\nMake sure DEPOSITS_REST_SERVICE_TLS=true in docker-compose.rest.server.yml\n### And then execute:\n```shell\nexport COMPOSE_IGNORE_ORPHANS=True \u0026\u0026 \\\ndocker-compose -f ./deploy/compose/docker-compose.rest.server.yml up --build\n``` \n\nThe --build option is there for any code changes.\n\nCOMPOSE_IGNORE_ORPHANS is there for \ndocker compose [setting](https://docs.docker.com/compose/reference/envvars/#compose_ignore_orphans).\n\n### Logs of running services (in a separate terminal):\ndocker-compose -f ./deploy/compose/docker-compose.rest.server.yml logs -f --tail 1\n\n### Sanity test Client -REST HTTP Services Endpoints Invoked Externally:\nThe server side DEPOSITS_REST_SERVICE_TLS should be consistent and set for client also.\n```shell\nexport GODEBUG=x509ignoreCN=0\nexport DEPOSITS_REST_SERVICE_TLS=true\nexport DEPOSITS_REST_SERVICE_ADDRESS=localhost\ngo run ./cmd/sanitytestclient\n```\n\n### Server Tracing\nAccess [zipkin](https://zipkin.io/) service at [http://localhost:9411/zipkin/](http://localhost:9411/zipkin/)\n\n### Profiling\n[http://localhost:4000/debug/pprof/](http://localhost:4000/debug/pprof/)\n\n### Metrics\n[http://localhost:4000/debug/vars](http://localhost:4000/debug/vars)\n\n### Shutdown \n```shell\ndocker-compose -f ./deploy/compose/docker-compose.external-db-trace-only.yml down\ndocker-compose -f ./deploy/compose/docker-compose.rest.server.yml down\n```\n\n### Quick calculations with Same JSON output without actually invoking REST Http Method\nRun at terminal:\n\n```shell\ndocker build -f ./build/Dockerfile.calculate -t illumcalculate  . \u0026\u0026 \\\ndocker run illumcalculate \n```\n\n# Runing from Editor/IDE\n\n### TLS files -same as in Docker compose\n```shell\nexport DEPOSITS_REST_SERVICE_ADDRESS=localhost\ndocker build -t tlscert-rest:v0.1 -f ./build/Dockerfile.openssl ./conf/tls \u0026\u0026 \\\ndocker run --env DEPOSITS_REST_SERVICE_ADDRESS=$DEPOSITS_REST_SERVICE_ADDRESS -v $PWD/conf/tls:/tls tlscert-rest:v0.1\n```\n### Start DB:\nTo start only external db and trace service for working with local machine:  \nStart postgres and tracing as usual\n```shell\nexport COMPOSE_IGNORE_ORPHANS=True \u0026\u0026 \\\ndocker-compose -f ./deploy/compose/docker-compose.external-db-trace-only.yml up\n```\n\n##### Then Migrate and set up seed data:\n```shell\nexport COMPOSE_IGNORE_ORPHANS=True \u0026\u0026 \\\ndocker-compose -f ./deploy/compose/docker-compose.seed.yml up --build\n````\n\nThen Set the following env variables when starting directly running server: change as needed\nAnd per your Editor/IDE:\n```shell\nexport DEPOSITS_REST_SERVICE_TLS=true\nexport DEPOSITS_DB_DISABLE_TLS=true\nexport DEPOSITS_DB_HOST=127.0.0.1\nexport DEPOSITS_TRACE_URL=http://127.0.0.1:9411/api/v2/spans\ngo run ./cmd/server\n```\n### Sanity test Client -REST HTTP Services Endpoints Invoked Externally:\nThe server side DEPOSITS_REST_SERVICE_TLS should be consistent and set for client also.\n```shell\nexport GODEBUG=x509ignoreCN=0\nexport DEPOSITS_REST_SERVICE_TLS=true\nexport DEPOSITS_REST_SERVICE_ADDRESS=localhost\ngo run ./cmd/sanitytestclient\n```\n\n# Kubernetes Deployment\n(for Better control; For Local Setup tested with Docker Desktop latest version with Kubernetes Enabled)\n\n### TLS files\n```shell\nexport DEPOSITS_REST_SERVICE_ADDRESS=restserversvc.127.0.0.1.nip.io\ndocker build -t tlscert:v0.1 -f ./build/Dockerfile.openssl ./conf/tls \u0026\u0026 \\\ndocker run --env DEPOSITS_REST_SERVICE_ADDRESS=$DEPOSITS_REST_SERVICE_ADDRESS -v $PWD/conf/tls:/tls tlscert:v0.1\n```\nAs a side note, For any troubleshooting, To see openssl version being used in Docker:\n```shell\ndocker build -t tlscert:v0.1 -f ./build/Dockerfile.openssl ./conf/tls \u0026\u0026 \\\ndocker run -ti -v $PWD/conf/tls:/tls tlscert:v0.1 sh\n```\nYou get a prompt\n/tls\nCheck version using command:\n```shell\nopenssl version\n```\n\n### Installing Ingress controller\nUsing helm to install nginx ingress controller\n```shell\nbrew install helm\nhelm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx\nhelm repo update\nhelm repo list\n```\nand then use\n```shell\nhelm install ingress-nginx -f ./deploy/kubernetes/nginx-ingress-controller/helm-values.yaml ingress-nginx/ingress-nginx\n```\nto install ingress controller\nTo see logs for nginx ingress controller:\n```shell\nkubectl logs -l app.kubernetes.io/name=ingress-nginx -f\n```\n\n### Make docker images and Push Images to Docker Hub\n\n```shell\ndocker build -t rsachdeva/illuminatingdeposits.rest.server:v1.4.0 -f ./build/Dockerfile.rest.server .  \ndocker build -t rsachdeva/illuminatingdeposits.seed:v1.4.0 -f ./build/Dockerfile.seed .  \n\ndocker push rsachdeva/illuminatingdeposits.rest.server:v1.4.0\ndocker push rsachdeva/illuminatingdeposits.seed:v1.4.0\n``` \n\n### Quick deploy for all resources\nWe only need to set secrets once after tls files have been generated\n```shell\nkubectl delete secret illuminatingdeposits-rest-secret-tls\nkubectl create --dry-run=client secret tls illuminatingdeposits-rest-secret-tls --key conf/tls/serverkeyto.pem --cert conf/tls/servercrtto.pem -o yaml \u003e ./deploy/kubernetes/tls-secret-ingress.yaml\n```\n```shell\nkubectl apply -f deploy/kubernetes/.\n```\nIf status for ```kubectl get pod -l job-name=seed | grep \"Completed\"```\nshows completed for seed pod, optionally can be deleted:\n```shell\nkubectl delete -f deploy/kubernetes/seed.yaml\n```\nAllows connecting Postgres UI using NodePort at 30007 from outside cluster locally to view data.\n\n### Sanity test Client -REST HTTP Services Endpoints Invoked Externally:\nThe server side DEPOSITS_REST_SERVICE_TLS should be consistent and set for client also.\nThe DEPOSITS_REST_SERVICE_TLS for client is true when Ingress is used with tls.\n```export DEPOSITS_REST_SERVICE_TLS=true```\n\n```shell\nexport GODEBUG=x509ignoreCN=0\nexport DEPOSITS_REST_SERVICE_TLS=true\nexport DEPOSITS_REST_SERVICE_ADDRESS=restserversvc.127.0.0.1.nip.io\ngo run ./cmd/sanitytestclient\n```\n\nWith this Sanity test client, you will be able to:\n- get status of Prostres DB\n- add a new user\n- JWT generation for Authentication\n- JWT Authentication for Interest Delta Calculations for each deposit; each bank with all deposits and all banks\n  Quickly confirms Sanity check for set up with Kubernetes/Docker.\n  There are also separate Integration and Unit tests.\n\n### Server Tracing\nAccess [zipkin](https://zipkin.io/) service at [http://zipkin.127.0.0.1.nip.io](http://zipkin.127.0.0.1.nip.io)\n\n### Detailed - Step by Step\n\n##### Start postgres service\n\n```shell\nkubectl apply -f deploy/kubernetes/postgres-env.yaml \nkubectl apply -f deploy/kubernetes/postgres.yaml\n```\n\n#### Then Migrate and set up seed data manually for more control initially:\nFirst should see in logs\ndatabase system is ready to accept connections\n```kubectl logs pod/postgres-deposits-0```\nAnd then execute migration/seed data for manual control when getting started:\n```shell\nkubectl apply -f deploy/kubernetes/seed.yaml\n```\nAnd if status for ```kubectl get pod -l job-name=seed | grep \"Completed\"``` \nshows completed for seed pod, optionally can be deleted:\n```shell\nkubectl delete -f deploy/kubernetes/seed.yaml\n```\nTo connect external tool with postgres to see database internals use:\nUse a connection string similar to:\njdbc:postgresql://127.0.0.1:30007/postgres\nIf still an issue you can try\nkubectl port-forward service/postgres 5432:postgres\nNow can easily connect using\njdbc:postgresql://localhost:5432/postgres\n\n\n#### Start tracing service\n```shell\nkubectl apply -f deploy/kubernetes/zipkin.yaml\n```\nAccess [zipkin](https://zipkin.io/) service at [http://zipkin.127.0.0.1.nip.io](http://zipkin.127.0.0.1.nip.io)\nSort Newest First and Click Find Traces\n\n#### Set up secret\n\n```shell\nkubectl delete secret illuminatingdeposits-rest-secret-tls\nkubectl create --dry-run=client secret tls illuminatingdeposits-rest-secret-tls --key conf/tls/serverkeyto.pem --cert conf/tls/servercrtto.pem -o yaml \u003e ./deploy/kubernetes/tls-secret-ingress.yaml\nkubectl apply -f deploy/kubernetes/tls-secret-ingress.yaml\n```\n\n#### Illuminating deposists Rest server in Kubernetes!\n```shell\nkubectl apply -f deploy/kubernetes/rest-server.yaml\n```\nAnd see logs using \n```kubectl logs -l app=restserversvc -f```\n\n### Remove all resources / Shutdown\n\n```shell\nkubectl delete -f ./deploy/kubernetes/.\nhelm uninstall ingress-nginx\n```\n\n  \n# Running Integration/Unit tests\nTests are designed to run in parallel with its own test server and docker based postgres db using dockertest.\nTo run all tests with coverages reports for focussed packages:\nRun following only once as tests use this image; so faster:\n```shell \ndocker pull postgres:13-alpine\n``` \ngo test -v -count=1 -covermode=count -coverpkg=./postgreshealth/... -coverprofile cover.out ./postgreshealth -run TestServiceServer_HealthOk \u0026\u0026 go tool cover -func cover.out\nAnd then run the following with coverages for key packages concerned:\n```shell\ngo test -v -count=1 -covermode=count -coverpkg=./userauthn/...,./usermgmt/...,./postgreshealth/...,./interestcal/... -coverprofile cover.out ./... \u0026\u0026 go tool cover -func cover.out\ngo test -v -count=1 -covermode=count -coverpkg=./userauthn/...,./usermgmt/...,./postgreshealth/...,./interestcal/... -coverprofile cover.out ./... \u0026\u0026 go tool cover -html cover.out\n```\nCoverage Result for key packages:  \n**total:\t(statements)\t96.3%**  \n\u003cp align=\"center\"\u003e\n\u003cimg src=\"./coverageresults.png\" alt=\"Illuminating Deposits REST Test Coverage\" title=\"lluminating Deposits REST Test Coverage\" /\u003e\n\u003c/p\u003e\n\nTo run a single test - no coverage:\n```shell \ngo test -v -count=1 -run=TestServiceServer_CreateUser ./usermgmt/...\n```\nTo run a single test - with coverage:\n```shell \ngo test -v -count=1 -covermode=count -coverpkg=./usermgmt -coverprofile cover.out -run=TestServiceServer_CreateUser ./usermgmt/... \u0026\u0026 go tool cover -func cover.out\n```\nThe -v is for Verbose output: log all tests as they are run. Search \"FAIL:\" in parallel test output here to see reason for failure\nin case any test fails.\nJust to run all easily with verbose ouput:\n```shell\ngo test -v ./... \n```\nThe -count=1 is mainly to not use caching and can be added as follows if needed for\nany go test command:\n```shell \ngo test -v -count=1 ./...\n```\nSee Editor specifcs to see Covered Parts in the Editor.\n#### Test Docker containers for Postgresdb\nDocker containers are mostly auto removed. This is done by passing true to testserver.InitRestServer(ctx, t, false)\nin your test.\nIf you want to examine postgresdb data for a particular test, you can temporarily\nset allowPurge as false in testserver.InitRestHttpServer(ctx, t, false) for your test.\nThen after running specific failed test connect to postgres db in the docker container using any db ui.\nAs an example, if you want coverage on a specific package and run a single test in a package with verbose output:\n```shell \ngo test -v -count=1 -covermode=count -coverpkg=./usermgmt -coverprofile cover.out -run=TestServiceServer_CreateUser ./usermgmt/... \u0026\u0026 go tool cover -func cover.out\n```\nAny docker containers still running after tests should be manually removed:\n```shell \ndocker ps\ndocker stop $(docker ps -qa)\ndocker rm -f $(docker ps -qa)\ndocker volume rm $(docker volume ls -qf dangling=true)\n```\n\n# Troubleshooting\nIf for any reason no connection is happening from client to server or client hangs or server start up issues:\nRun\n```\nps aux | grep \"go run\"\nps aux | grep \"go_build\" \n```\n\nto confirm is something else is already running\nMake sure to follow above TLS set up according to Kubernetes deployment, Docker compose deployment or Running from Editor.\nMake sure to follow Ingress controller installation for Kubernetes deployment.\n\n# Version\nv1.4.0","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsachdeva%2Filluminatingdeposits-rest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsachdeva%2Filluminatingdeposits-rest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsachdeva%2Filluminatingdeposits-rest/lists"}