https://github.com/cr21/k8s-canary-deployment
https://github.com/cr21/k8s-canary-deployment
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/cr21/k8s-canary-deployment
- Owner: cr21
- License: apache-2.0
- Created: 2025-02-15T21:39:35.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-02-17T00:34:30.000Z (3 months ago)
- Last Synced: 2025-02-17T01:20:10.924Z (3 months ago)
- Language: Python
- Size: 11.3 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# k8s-Canary-Deployment
## Create EKS Cluster
```yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
iam:
withOIDC: true
metadata:
name: basic-cluster
region: us-east-1
version: "1.30"
managedNodeGroups:
- name: ng-dedicated-1 # for pods with volumes
instanceType: t3a.xlarge
desiredCapacity: 1
ssh:
allow: true
publicKeyPath: ~/.ssh/id_ed25519.pub
iam:
withAddonPolicies:
autoScaler: true
awsLoadBalancerController: true
certManager: true
externalDNS: true
ebs: true
- name: ng-spot-3
# instanceSelector:
# memory: "16"
# vCPUs: 4
instanceTypes:
- t2.xlarge
- t3.xlarge
- t3a.xlarge
desiredCapacity: 6
spot: true
labels:
role: spot
ssh:
allow: true
publicKeyPath: ~/.ssh/id_ed25519.pub
iam:
withAddonPolicies:
autoScaler: true
awsLoadBalancerController: true
certManager: true
externalDNS: true
ebs: true
```
## Create EKS Cluster
```bash
eksctl create cluster -f eks-cluster.yaml
```
## KNative
### Install Knative and related components
```bash
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.16.0/serving-crds.yaml
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.16.0/serving-core.yaml
kubectl apply -l knative.dev/crd-install=true -f https://github.com/knative/net-istio/releases/download/knative-v1.16.0/istio.yaml
kubectl apply -f https://github.com/knative/net-istio/releases/download/knative-v1.16.0/istio.yaml
kubectl apply -f https://github.com/knative/net-istio/releases/download/knative-v1.16.0/net-istio.yaml
kubectl --namespace istio-system get service istio-ingressgateway
kubectl get pods -n knative-serving
kubectl patch configmap/config-domain \
--namespace knative-serving \
--type merge \
--patch '{"data":{"emlo.tsai":""}}'
kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.16.0/serving-hpa.yaml
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml
kubectl apply --server-side -f https://github.com/kserve/kserve/releases/download/v0.14.1/kserve.yaml
kubectl apply --server-side -f https://github.com/kserve/kserve/releases/download/v0.14.1/kserve.yaml
kubectl apply --server-side -f https://github.com/kserve/kserve/releases/download/v0.14.1/kserve-cluster-resources.yaml
eksctl create iamserviceaccount \
--cluster=basic-cluster \
--name=s3-read-only \
--attach-policy-arn=arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
--override-existing-serviceaccounts \
--region us-east-1 \
--approve
kubectl apply -f s3-secret.yaml
kubectl patch serviceaccount s3-read-only -p '{"secrets": [{"name": "s3-secret"}]}'
```
```bash
docker run -it --rm --shm-size=1g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
-p 8085:8085 \
--mount type=bind,source=/home/ubuntu/session-17/model-store/cat-classifier/model-store,target=/mnt/models/model-store \
--mount type=bind,source=/home/ubuntu/session-17/model-store/cat-classifier/config,target=/mnt/models/config \
-e TS_CONFIG_FILE=/mnt/models/config/config.properties \
-e TS_DISABLE_TOKEN_AUTHORIZATION=true \
pytorch/torchserve-kfs:0.12.0 \
torchserve
```
```bash
kubectl apply -f vit-classifier.yaml
kubectl get pods
kubectl get inferenceservice
kubectl get pods
kubectl describe pod imagenet-vit-predictor-00001-deployment-cd85dc487-5zkm5
kubectl logs imagenet-vit-predictor-00001-deployment-cd85dc487-5zkm5 -c storage-initializer
kubectl describe pod imagenet-vit-predictor-00001-deployment-cd85dc487-5zkm5
❯ kubectl get inferenceservice
NAME URL READY PREV LATEST PREVROLLEDOUTREVISION LATESTREADYREVISION AGE
imagenet-vit http://imagenet-vit.default.emlo.tsai True 100 imagenet-vit-predictor-00001 4m57s
❯ kubectl get services -n istio-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.100.196.88 a7352fc5c5fd84b64a4381b964dcc123-1841539688.us-east-1.elb.amazonaws.com 15021:32538/TCP,80:32198/TCP,443:32150/TCP 133m
istiod ClusterIP 10.100.166.174 15010/TCP,15012/TCP,443/TCP,15014/TCP 133m
knative-local-gateway ClusterIP 10.100.107.184 80/TCP,443/TCP
```
```bash
❯ python3 send.py
{'content-length': '78', 'content-type': 'application/json', 'date': 'Sun, 16 Feb 2025 21:11:33 GMT', 'server': 'istio-envoy', 'x-envoy-upstream-service-time': '321'}
200
{'predictions': [{'class': 'tabby, tabby cat', 'probability': 0.484485000371933}]}
```
## Prometheus
```bash
git clone --branch release-0.14