https://github.com/jescalada/kubernetes-argo-ml-integration
Messing around with integrating Kubernetes and ArgoCD with a ML model.
https://github.com/jescalada/kubernetes-argo-ml-integration
Last synced: 3 months ago
JSON representation
Messing around with integrating Kubernetes and ArgoCD with a ML model.
- Host: GitHub
- URL: https://github.com/jescalada/kubernetes-argo-ml-integration
- Owner: jescalada
- Created: 2024-12-27T03:45:11.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-01-17T12:17:32.000Z (5 months ago)
- Last Synced: 2025-02-03T02:09:40.267Z (4 months ago)
- Language: Python
- Size: 11.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# kubernetes-argo-ml-integration
Demo for integrating Kubernetes and ArgoCD with a ML model.
## How to run the demo
Running the demo requires Docker and Kubernetes. First, we need to set the prediction service in a Docker container. Then, we can deploy the model to Kubernetes.
## Setting up Docker
1. Run `python model_training.py` to train the model.
- This will create a model directory with the trained model, which will be used by the prediction service.
2. Run `docker build -t my-tf-model:latest .` to build the Docker image.
3. Run `docker run -p 8501:8501 my-tf-model:latest` to run the Docker container based on the image. This will start a TensorFlow Serving server to easily use the model.
You can test the model as follows. Note that this will take a while the first time you run it, as the model is being loaded into memory.
### Unix-based systems
```bash
curl -X POST http://localhost:8501/v1/models/my_model:predict -d '{"instances": [[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]]}'
```
### Windows
```powershell
Invoke-WebRequest -Uri http://localhost:8501/v1/models/my_model:predict -Method POST -Body '{"instances": [[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]]}' -ContentType "application/json"
```
## Setting up the Kubernetes deployment
1. Push the Docker Image to a container registry such as Docker Hub. This example uses Docker Hub. You must have a Docker Hub account and be logged in.
```bash
docker tag my-tf-model:latest /my-tf-model:latest
docker push /my-tf-model:latest
```
2. Modify the `deployment.yaml` file to use your Docker Hub username.
```yaml
spec:
containers:
- name: tf-model
image: /my-tf-model:latest
ports:
- containerPort: 8501
```
3. Make sure you have Kubernetes running. You can use Docker Desktop Kubernetes or MiniKube for local development.
4. Run `kubectl apply -f deployment.yaml` to deploy the model to Kubernetes.
5. Run `kubectl apply -f service.yaml` to expose the model to the outside world.
You can verify the deployment by running `kubectl get pods` and `kubectl get services`.
Now, you can test the model by sending a request to the Kubernetes service. In this example, we used `NodePort` to expose the service, so the service will be available at `http://localhost:`. You can find the node port by running `kubectl get services`.
You may also use port forwarding to access the service directly by running `kubectl port-forward svc/tf-model-service 8080:8501`. This will forward the service from kubernetes port 8501 to localhost port 8080. If you get an error, you may need to double check the port in the `service.yaml` file.
Test the model as follows, and replace `` with the actual node port (8080 if you used the port forwarding command from above):
### Unix-based systems
```bash
curl -X POST http://localhost:/v1/models/my_model:predict -d '{"instances": [[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]]}'
```
### Windows
```powershell
Invoke-WebRequest -Uri http://localhost:/v1/models/my_model:predict -Method POST -Body '{"instances": [[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]]}' -ContentType "application/json"
```
## Setting up ArgoCD
1. Install ArgoCD by executing the following commands:
```bash
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
```
Verify the installation by running `kubectl get pods -n argocd`.
2. Expose the ArgoCD service by running `kubectl port-forward svc/argocd-server -n argocd 8080:443`. You can now access the ArgoCD UI at `http://localhost:8080`.
3. Log in to the ArgoCD UI with the default username `admin`.
You can obtain the default password by running the following command:
### Unix-based
```bash
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d`
```
### Windows
```powershell
kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}" | ForEach-Object { [System.Text.Encoding]::UTF8.GetString([Convert]::FromBase64String($_)) }
```
4. If using Docker Desktop Kubernetes, you'll have to register the cluster with ArgoCD.
First, get the Kubernetes API server URL by running `kubectl cluster-info`. You can use the URL under `Kubernetes control plane` as the cluster URL.
Then, run the following command to register the cluster:
```bash
argocd cluster add
```
Note that this requires the `argocd` CLI to be installed. You can install it by running `brew install argocd` on MacOS, or by following the instructions at https://argo-cd.readthedocs.io/en/stable/cli_installation/.
5. Create a new application in ArgoCD by clicking on the `New App` button. Fill in the following details and then click `Create`:
- Application Name: `tf-model`
- Project: `default`
- Repository URL: `https://github.com/jescalada/kubernetes-argo-ml-integration.git`
- Path: `manifests`
- Cluster: ``
6. You can now sync the application by clicking on the `Sync` button. This will deploy the model to Kubernetes and keep it in sync with the repo.
If everything is set up correctly, you should see the status of the application as "Synced" and "Healthy" in the ArgoCD UI.
7. Due to how ArgoCD authentication works, you will have to use OAuth to access the API:
> For Single Sign-On users, the user completes an OAuth2 login flow to the configured OIDC identity provider (either delegated through the bundled Dex provider, or directly to a self-managed OIDC provider). This JWT is signed & issued by the IDP, and expiration and revocation is handled by the provider. Dex tokens expire after 24 hours.
In this case, you'll have to set up an OIDC provider and configure ArgoCD to use it. You can find more details at https://argo-cd.readthedocs.io/en/stable/operator-manual/user-management/.
Once you set up the OIDC provider along with the RBAC settings, you should see any previously created applications in the ArgoCD UI (as long as you have the necessary permissions). You can test the OIDC setup by trying to add an application to ArgoCD. An error will be displayed if the setup is incorrect.