Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tristcoil/docker-stock-analytics-kubernetes-deployment

deploys dockerized stock analytics app into AWS kubernetes cluster
https://github.com/tristcoil/docker-stock-analytics-kubernetes-deployment

Last synced: 28 days ago
JSON representation

deploys dockerized stock analytics app into AWS kubernetes cluster

Awesome Lists containing this project

README 

        
App is providing current/last available stock price for given stock.

Data can be delayed by one trading day depending on when we query Yahoo! Finance API since it is providing daily close data.



The app runs in Docker container and is built by Jenkins server in AWS Virtual Machine.

Jenkins is monitoring related github repo and triggers build pipeline with every commit to master.



Build pipeline is performing rolling update to kubernetes cluster running on AWS.

Kubernetes cluster creation in AWS is managed by separate Jenkins pipeline, this one is triggered manually to prevent unexpected AWS charges.



## How to run:

repo contains 2 Jenkinsfiles

- ./Jenkinsfile ... creates pipeline for linting, container building and rolling deploymento to Kubernetes cluster



- /cluster_creation/Jenkinsfile ... takes care of Kubernetes cluster creation



- ./get_stock_price_cluster.sh ... queries the cluster for given ticker and provides stock price



## Prerequisites:

Jenkins server has to have these installed

- docker

- eksctl

- kubectl

- hadolint

- awscli

- pipeline-aws Jenkins plugin to authenticate with AWS Cloud



## Sources:



#### original udacity project repos:

- https://github.com/udacity/DevOps_Microservices

- https://github.com/udacity/nd9991-c3-Build-CI-CD-Pipelines-Monitoring-and-Logging-v1

- https://github.com/udacity/nd9991-c2-Infrastructure-as-Code-v1



#### install docker:

- https://docs.docker.com/engine/install/ubuntu/

- https://www.digitalocean.com/community/questions/how-to-fix-docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket



#### installing minikube:

- https://minikube.sigs.k8s.io/docs/start/



#### hadolint:

- https://github.com/hadolint/hadolint/

- https://www.haskell.org/platform/#linux-ubuntu

- https://stackoverflow.com/questions/62369711/how-to-install-hadolint-on-ubuntu



#### Docker commands:

- https://docs.docker.com/engine/reference/commandline/image_rm/

- https://docs.docker.com/engine/reference/commandline/image_push/

- https://docs.docker.com/docker-hub/repos/

- https://docs.docker.com/engine/reference/commandline/login/



#### Kubernetes:

- https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive/

- https://kubernetes.io/docs/tutorials/kubernetes-basics/explore/explore-interactive/

- https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-interactive/



#### creating Kubernetes cluster on AWS:

- https://logz.io/blog/amazon-eks-cluster/

- https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html

- https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-eks-cluster.html

- https://docs.aws.amazon.com/eks/latest/userguide/clusters.html

- https://www.ironin.it/blog/a-simple-kubernetes-cluster-on-aws.html

- https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateCluster.html

- https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html

- https://rtfm.co.ua/en/aws-elastic-kubernetes-service-a-cluster-creation-automation-part-1-cloudformation/

- https://www.bluematador.com/blog/my-first-kubernetes-cluster-a-review-of-amazon-eks

- https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html

- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-eks-nodegroup.html

- https://aws.amazon.com/blogs/storage/deploying-jenkins-on-amazon-eks-with-amazon-efs/



#### rolling deployment

- https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-intro/

- https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-interactive/

- https://medium.com/platformer-blog/enable-rolling-updates-in-kubernetes-with-zero-downtime-31d7ec388c81

- https://www.bluematador.com/blog/kubernetes-deployments-rolling-update-configuration

- https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/



#### knowledge base:

- https://knowledge.udacity.com/questions/303115

- https://knowledge.udacity.com/questions/329275

- https://knowledge.udacity.com/questions/351451

- https://knowledge.udacity.com/questions/332693

- https://knowledge.udacity.com/questions/330414

- https://knowledge.udacity.com/questions/321217



- https://knowledge.udacity.com/questions/279390

- https://knowledge.udacity.com/questions/293833

- https://knowledge.udacity.com/questions/311884

- https://knowledge.udacity.com/questions/311887

- https://knowledge.udacity.com/questions/313832

- https://knowledge.udacity.com/questions/324445

- https://knowledge.udacity.com/questions/329228



- https://hub.udacity.com/rooms/community:nd9991:840625-project-639-smg-2?contextType=room



#### git:

- used commands generated by github.com when creating new repository



#### to modify the app:

- https://www.learndatasci.com/tutorials/python-finance-part-yahoo-finance-api-pandas-matplotlib/

- https://aroussi.com/post/python-yahoo-finance

- https://stackoverflow.com/questions/32490629/getting-todays-date-in-yyyy-mm-dd-in-python

- https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes



#### troubleshooting jenkins:

- https://askubuntu.com/questions/1139046/jenkins-error-incorrect-java-version-for-java11-after-removing-java11-and-ins

- https://stackoverflow.com/questions/19457867/problems-installing-latest-version-of-jenkins

actual jenkins config fix:

- https://askubuntu.com/questions/1139046/jenkins-error-incorrect-java-version-for-java11-after-removing-java11-and-ins

- https://www.edureka.co/community/39390/how-to-give-jenkins-super-user-permission

- https://stackoverflow.com/questions/11880070/how-to-run-a-script-as-root-in-jenkins



#### increasing swap on my jenkins box

- https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-18-04



#### creating config for app deployment on kubernetes:

- https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/examples/2048/2048_full.yaml



#### pushing to dockerhub with Jenkins:

- https://tutorials.releaseworksacademy.com/learn/building-your-first-docker-image-with-jenkins-2-guide-for-developers

- https://support.cloudbees.com/hc/en-us/articles/203802500-Injecting-Secrets-into-Jenkins-Build-Jobs

- https://plugins.jenkins.io/credentials-binding/

- https://stackoverflow.com/questions/35736377/reference-global-credentials-in-bash-script-in-jenkins-job



#### some docker commands:

- https://docs.docker.com/engine/reference/commandline/system_prune/



#### additional notes:

docker image that will go to Kubernetes cluster:

coil/stock_analytics