https://github.com/jlsalvador/simple-cicd

Simple CI/CD operator for Kubernetes. Create workflows that can be trigged by webhooks.
https://github.com/jlsalvador/simple-cicd

cd ci ci-cd cicd controller k8s k8s-controller k8s-operator kubebuilder kubernetes kubernetes-controller kubernetes-operator operator pipeline webhook webhook-server

Last synced: 28 days ago
JSON representation

Simple CI/CD operator for Kubernetes. Create workflows that can be trigged by webhooks.

• Host: GitHub
• URL: https://github.com/jlsalvador/simple-cicd
• Owner: jlsalvador
• License: apache-2.0
• Created: 2023-09-21T05:59:27.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-03-05T18:36:20.000Z (10 months ago)
• Last Synced: 2024-11-15T01:30:58.548Z (about 2 months ago)
• Topics: cd, ci, ci-cd, cicd, controller, k8s, k8s-controller, k8s-operator, kubebuilder, kubernetes, kubernetes-controller, kubernetes-operator, operator, pipeline, webhook, webhook-server
• Language: Go
• Homepage:
• Size: 275 KB
• Stars: 6
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Simple CI/CD Operator for Kubernetes

![Simple CI/CD Logo](img/logo.png)

The Simple CI/CD operator for Kubernetes empowers users to create workflows triggered by webhooks, facilitating the orchestration of Kubernetes Jobs according to specific requirements.

## Table of Contents

1. [Getting Started](#getting-started)

2. [Description](#description)

3. [Contributing](#contributing)

4. [License](#license)

## Getting Started

### Installation by Manifest

To install Simple CI/CD directly from the internet, use the following command:

```sh

kubectl apply -k 'github.com/jlsalvador/simple-cicd/config/default?ref=stable'

```

If you have forked the Simple CI/CD repository and want to install it from your local copy, use the following command:

```sh

kubectl apply -k config/default

```

### Installation by Helm

```sh

helm upgrade --install --create-namespace --namespace simple-cicd --repo https://jlsalvador.github.io/simple-cicd simple-cicd simple-cicd

```

## Description

Simple CI/CD offers users the ability to trigger Kubernetes Jobs through webhooks, providing control over when and how multiple Jobs and their dependencies are executed.

### Features

- Easy to understand

- Uses standard Kubernetes resources

- No external cloud dependencies

- Low resource usage

### Custom Resource Definitions

- **Workflow**: Responsible for cloning Jobs and triggering subsequent Workflows based on the exit status of completed Jobs.

- **WorkflowWebhook**: Serves as the trigger for Workflows, allowing the Simple CI/CD operator to listen for HTTP requests on a path named as the WorkflowWebhook (`/namespace/name`).

- **WorkflowWebhookRequest**: Initiates WorkflowWebhooks.

### Example:

In this example, we demonstrate the Simple CI/CD system's functionality:

1. **Receive HTTP Request**: The Simple CI/CD operator receives an HTTP request.

2. **Create a Random Exit Pod**: A Kubernetes pod is created, which generates a random exit code (either 0 or 1) upon execution.

3. **Error Handling**: If the previous pod's exit status indicates a failure (1), the system initiates another pod that echoes the content of the original HTTP request.

```mermaid

sequenceDiagram

  participant User as User

  participant Ingress as Ingress Controller

  participant SimpleCI/CD as Simple CI/CD Operator

  participant Pod1 as "Random Exit" Pod

  participant Pod2 as "Error Handling" Pod

  User->>Ingress: HTTP Request (Outside Cluster)

  Ingress->>SimpleCI/CD: HTTP Request

  SimpleCI/CD->>Pod1: Create Random Exit Pod

  Note right of Pod1: Generates Random Exit Code (0 or 1)

  Pod1-->>SimpleCI/CD: Exit Status (0 or 1)

  alt Exit Status = 1 (KO)

    SimpleCI/CD->>Pod2: Create Error Handling Pod

    Pod2-->>SimpleCI/CD: Echo HTTP Request

  end

  SimpleCI/CD-->>Ingress: HTTP Response

  Ingress-->>User: HTTP Response (Outside Cluster)

```

```yaml

# Job that randomly exits with code 0 or 1 (OK or Error)

apiVersion: batch/v1

kind: Job

metadata:

  name: job-example-random-exit

spec:

  suspend: true # Required to be true to disallow Kubernetes to start this job when it will be created

  backoffLimit: 0 # If this Job fail do not try to run it again

  template:

    spec:

      containers:

        - name: random-exit

          image: bash

          command: ["sh", "-c", "exit $$(($RANDOM % 2))"] # Sometimes will fails

      restartPolicy: Never # Do not re-run the pod if something fails

---

# Job that echoes "ERROR"

apiVersion: batch/v1

kind: Job

metadata:

  name: job-example-error

  namespace: default # Job namespace. Optional

spec:

  suspend: true # Required to be true to disallow Kubernetes to start this job when it will be created

  template:

    spec:

      containers:

        - name: error

          image: bash

          command: ["echo", "ERROR"]

        - name: echo-request

          image: bash

          command:

            - cat # Simple CI/CD will mounts the next request payloads inside all Pods

            - /var/run/secrets/kubernetes.io/request/body

            - /var/run/secrets/kubernetes.io/request/headers

            - /var/run/secrets/kubernetes.io/request/host

            - /var/run/secrets/kubernetes.io/request/method

            - /var/run/secrets/kubernetes.io/request/url

      restartPolicy: Never # Do not re-run the pod if something fails

---

# Workflow that will clones the "job-example-error" Job

apiVersion: simple-cicd.jlsalvador.online/v1alpha1

kind: Workflow

metadata:

  name: workflow-example-some-failures

  namespace: default # Workflow namespace. Optional.

spec:

  jobsToBeCloned:

    - name: job-example-error # Job name that will cloned

      namespace: default # Job namespace. Optional.

---

# Workflow that will clones the "job-example-random-exit" Job and

# triggers the "workflow-example-some-failures" Workflow on any Job failure

apiVersion: simple-cicd.jlsalvador.online/v1alpha1

kind: Workflow

metadata:

  name: workflow-example

spec:

  jobsToBeCloned:

    - name: job-example-random-exit # Job name that will cloned

  next:

    - name: workflow-example-some-failures

      namespace: default # Workflow namespace. Optional.

      when: OnAnyFailure # Only run this Workflow if some of the jobsToBeCloned fails

---

# WorkflowWebhook set the simple-cicd operator to listen for requests on the path "$namespace/$name"

apiVersion: simple-cicd.jlsalvador.online/v1alpha1

kind: WorkflowWebhook

metadata:

  name: workflowwebhook-example

spec:

  workflows:

    - name: workflow-example # Workflow that will be initiated

---

# Optional, recommended for public requests.

# Secret for the next Ingress, recommended for preventing undesired requests.

apiVersion: v1

kind: Secret

metadata:

  name: basic-auth-example

  namespace: simple-cicd-system # Namespace where the ingress-example is deployed

type: Opaque

stringData:

  auth: |

    # user:pass

    user:$apr1$j.P.ucaS$hHtkMN19glS9.ffLns2Eh/

---

# Optional, allows external requests from the cluster.

# Public Ingress listening at http://example.org/default/workflowwebhook-example with basic authorization.

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-example

  namespace: simple-cicd-system # Namespace where the simple-cicd-controller-manager is deployed

  annotations:

    nginx.ingress.kubernetes.io/auth-type: basic

    nginx.ingress.kubernetes.io/auth-secret: basic-auth-example

spec:

  ingressClassName: nginx

  rules:

    - host: example.org

      http:

        paths:

          - path: /default/workflowwebhook-example # The WorkflowWebhook namespace/name

            pathType: Prefix

            backend:

              service:

                name: simple-cicd-controller-manager # The simple-cicd operator service name

                port:

                  name: http

```

```sh

# To trigger the WorkflowWebhook from outside the cluster:

curl -u user:pass http://example.org/default/workflowwebhook-sample

# To trigger the WorkflowWebhook from inside the cluster:

curl http://simple-cicd-controller-manager.simple-cicd-system:9000/default/workflowwebhook-sample

```

### Motivation

The motivation behind developing Simple CI/CD arises from the need for a tool that aligns with specific requirements, as outlined in the table below. Existing solutions either impose excessive requirements or fail to meet desired expectations. The primary objective is to launch Jobs within the Kubernetes environment using webhooks to control timing and execution, without the need for virtual machines, Docker-in-Docker configurations, external dependencies, or components external to the Kubernetes ecosystem.

  Disclaimer: Based in my personal opinion. Please, do your own investigations.

  

    

      Alternative

      Advantages

      Disadvantages

    

  

  

    

      

        Github Actions

      

      

        


          
De-facto CI/CD for public GIT repositories at Github.

        

      

      

        


          
Supported by Microsoft.

          
Closed garden.

          
Actions requires to be public publishing.

          
Limited by a paywall.

          
Not suitable for air-gap environments or private clusters.

        

      

    

    

      

        Jenkins

      

      

        


          
Open-source.

          
Supported by the community and company foundations.

          
Stable and real-world tested.

        

      

      

        


          
Resource hungry.

          
Requires Groovy for advanced tasks.

          
Requires addons for Kubernetes.

          
Limited CLI support.

        

      

    

    

      

        Tekton

      

      

        


          
Open-source.

          
Supported by CD Foundation.

        

      

      

        


          
Can not use more than one PersistentVolume on the same Pod.

          
Deprecated tasks from Catalog because its new on-going API version.

        

      

    

    

      

        Drone

      

      

        


          
Open-source.

        

      

      

        


          
Supported by Harness.

          
Limited community pull requests support. Personal example: https://github.com/harness/gitness/pull/3030



        

      

    

    

      

        Woodpecker CI

      

      

        


          
Open-source (community fork of Drone).

          
Supported by the community.

        

      

      

        


          
Forget about namespaced PersistentVolume for multiples tasks.

          
Limited Kubernetes support.

        

      

    

    

      

        Gitea act_runner

      

      

        


          
Community effort for on-premise Github Actions.

        

      

      

        


          
Requires Virtual Machine-like setup, as Github Actions.

          
Lacks Kubernetes integrations.

        

      

    

    

      

        Simple CI/CD

      

      

        


          
Open-source.

          
Supported by the community.

          
Simple.

          
Kubernetes native component as an operator.

          
Platform agnostic (cloud, on-premise, hybrid).

          
Low resource usage (around 32Mb RAM).

        

      

      

        


          
Experimental and not yet real-world tested.

          
Maintained by a single individual.

          
No web interface (yet).

        

      

    

  


## Contributing

### Design rules

- Keep it Simple

- Be Explicit

- Embrace Minimalism

### How it works

This project aims to follow the Kubernetes [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/).

It uses [Controllers](https://kubernetes.io/docs/concepts/architecture/controller/),

which provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.

You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster.

**Note:** Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster `kubectl cluster-info` shows).

### Test It Out

1. Install the CRDs into your cluster:

```sh

make install

```

2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):

```sh

make run

```

**NOTE:** You can also run this in one step by running: `make install run`

### Running on the cluster

1. Build and push your image to the location specified by `IMG`:

```sh

make docker-build docker-push IMG=/simple-cicd:tag

```

2. Deploy the controller to the cluster with the image specified by `IMG`:

```sh

make deploy IMG=/simple-cicd:tag

```

### Uninstall CRDs

To delete the CRDs from the cluster:

```sh

make uninstall

```

### Undeploy controller

UnDeploy the controller from the cluster:

```sh

make undeploy

```

### Modifying the API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

```sh

make manifests

```

**NOTE:** Run `make --help` for more information on all potential `make` targets

More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)

## License

Copyright 2023 José Luis Salvador Rufo .

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.