Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/j0hn-b/hera_argo
Hera is a Python framework for constructing and submitting Argo Workflows.
https://github.com/j0hn-b/hera_argo
argo-workflows devbox developer-tools hera kubernetes machine-learning mlops mlops-workflow nixos nixpkgs python
Last synced: 9 days ago
JSON representation
Hera is a Python framework for constructing and submitting Argo Workflows.
- Host: GitHub
- URL: https://github.com/j0hn-b/hera_argo
- Owner: J0hn-B
- License: mit
- Created: 2024-04-25T14:14:05.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2024-04-25T14:50:45.000Z (9 months ago)
- Last Synced: 2024-11-12T12:06:00.791Z (2 months ago)
- Topics: argo-workflows, devbox, developer-tools, hera, kubernetes, machine-learning, mlops, mlops-workflow, nixos, nixpkgs, python
- Language: Shell
- Homepage:
- Size: 23.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Argo Workflows with Hera
Hera is a Python framework for constructing and submitting Argo Workflows.
This is a tool for Machine Learning engineers to experiment,develop and orchestrate their workflows locally using `Python` and `Argo Workflows`
## Prerequisites
To run this project, you will need to have the following installed:
[Devbox](https://www.jetpack.io/devbox/docs/) || [Docker Desktop](https://www.docker.com/products/docker-desktop/)
---
> [!NOTE]
> Development environments created by Devbox are isolated and portable.
> No version conflicts, no need to install anything on your host machine.
---
## Quick Start
```bash
git clone https://github.com/J0hn-B/hera_argo.git && cd hera_argo
```
```bash
# Start a new shell to install the project dependencies
devbox shell
```
```bash
# Deploy and configure Argo Workflows on a local k3d cluster, and get the Argo Workflows UI URL
task deploy
```
---
## How to use
```bash
# Install the project dependencies
cd $PWD/src/mlops && poetry install
```
```bash
# Run the example workflows
poetry run python main.py
```
- Check the `src/mlops/main.py` file and `src/mlops/mlops` folder for the example workflows.
---
---
```bash
# Access the k3d cluster using kubectl from another terminal
cd hera_argo && devbox shell
```
```bash
# Deploy a yaml example workflow using argo cli
argo submit --watch src/mlops_yaml/first.yaml
```
- Permissions:
```dir
hera_argo/kubernetes/cluster/overlays/k3d/kustomization.yaml
```
```yaml
# Resources to apply
resources:
###? dir contains config files for a service account with custom permissions. Uncomment to apply/adjust/modify based on your needs.
#- ../../base
###? file contains a role binding for the default service account in the argo namespace with admin permissions. NOT recommended for production use.
- default_admin.yaml
```
---
## Clean up
```bash
# Delete the cluster
task delete
```
```bash
# Exit the shell
exit
```
```bash
# Clean up packages installed by devbox
nix-collect-garbage
```