{"id":38310602,"url":"https://github.com/bartvanbenthem/cdtarget-operator","last_synced_at":"2026-01-17T02:31:46.304Z","repository":{"id":62793171,"uuid":"557327153","full_name":"bartvanbenthem/cdtarget-operator","owner":"bartvanbenthem","description":"Automate the configuration \u0026 lifecycle of Azure self-hosted pipelines agents and enable self-service for adding agent egress targets, without the need of delegating full network policy permissions.","archived":false,"fork":false,"pushed_at":"2023-12-06T09:19:49.000Z","size":37428,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-06-21T16:44:34.531Z","etag":null,"topics":["azure-devops","azure-pipelines","cicd","kubernetes-operator","operator-framework","self-hosted-agent"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bartvanbenthem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-25T13:42:19.000Z","updated_at":"2024-01-12T18:43:14.000Z","dependencies_parsed_at":"2024-06-21T15:29:59.474Z","dependency_job_id":null,"html_url":"https://github.com/bartvanbenthem/cdtarget-operator","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/bartvanbenthem/cdtarget-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcdtarget-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcdtarget-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcdtarget-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcdtarget-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bartvanbenthem","download_url":"https://codeload.github.com/bartvanbenthem/cdtarget-operator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bartvanbenthem%2Fcdtarget-operator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28492318,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T00:50:05.742Z","status":"online","status_checked_at":"2026-01-17T02:00:07.808Z","response_time":85,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-devops","azure-pipelines","cicd","kubernetes-operator","operator-framework","self-hosted-agent"],"created_at":"2026-01-17T02:31:46.235Z","updated_at":"2026-01-17T02:31:46.289Z","avatar_url":"https://github.com/bartvanbenthem.png","language":"Go","readme":"# Continues Deployment Target - Operator\r\nAutomate the configuration \u0026 lifecycle of Azure self-hosted pipelines agents and enable self-service for adding egress targets, without the need of delegating full network policy permissions to the namespace administrator. Event driven autoscaling is automatically enabled trough KEDA and Azure pipelines integrations.\r\n\r\n## Operator Design\r\n\r\n###  Describing the problem\r\nFor us as namespace administrators (cluster users) the CRUD functionality on network policy objects are unauthorized by security design and can only be changed by the cluster administrators. To enable end tot end automation, we need the ability to add target IPs ourselves to a specified set of allowed egress ports through a Custom Resource, the ports are specified by the cluster administrators from centralized configuration. An Operator should automatically create or update a network policy containing the specified IPs defined in the CustomResource. The operator should als configure and manage the lifecycle of the self-hosted pipeline agents, be able to inject proxy configurations and CA certificates trough Kubernetes secrets and simplify the enablement of event driven autoscaling.  \r\n\r\n### Designing the API and a CRD\r\n\r\n#### K8s Network Policy NetworkPolicyPeer API spec\r\n```text\r\nNetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of fields are allowed\r\n\r\n   .  egress.to.ipBlock (IPBlock)\r\n\r\n    IPBlock defines policy on a particular IPBlock. If this field is set then neither of the other fields can be.\r\n\r\n    IPBlock describes a particular CIDR (Ex. \"192.168.1.1/24\",\"2001:db9::/64\") that is allowed to the pods matched by a NetworkPolicySpec's podSelector. The except entry describes CIDRs that should not be included within this rule.\r\n\r\n        . egress.to.ipBlock.cidr (string), required\r\n\r\n        CIDR is a string representing the IP Block Valid examples are \"192.168.1.1/24\" or \"2001:db9::/64\"\r\n\r\n        . egress.to.ipBlock.except ([]string)\r\n\r\n        Except is a slice of CIDRs that should not be included within an IP Block Valid examples are \"192.168.1.1/24\" or \"2001:db9::/64\" Except values will be rejected if they are outside the CIDR range\r\n```\r\n#### IPBLock type\r\n```Go\r\ntype IPBlock struct {\r\n\t// CIDR is a string representing the IP Block\r\n\t// Valid examples are \"192.168.1.1/24\" or \"2001:db9::/64\"\r\n\tCIDR string `json:\"cidr\" protobuf:\"bytes,1,name=cidr\"`\r\n\t// Except is a slice of CIDRs that should not be included within an IP Block\r\n\t// Valid examples are \"192.168.1.1/24\" or \"2001:db9::/64\"\r\n\t// Except values will be rejected if they are outside the CIDR range\r\n\t// +optional\r\n\tExcept []string `json:\"except,omitempty\" protobuf:\"bytes,2,rep,name=except\"`\r\n}\r\n```\r\n\r\n#### CDTarget types\r\n```Go\r\n// CDTargetSpec defines the desired state of CDTarget\r\ntype CDTargetSpec struct {\r\n\t// IP is a slice of string that contains all the CDTarget IPs\r\n\tIP []string `json:\"ip,omitempty\"`\r\n\t// specify the pod selector key value pair\r\n\tAdditionalSelector map[string]string `json:\"additionalSelector\"`\r\n\t// pipeline agent image\r\n\tAgentImage string `json:\"agentImage,omitempty\"`\r\n\t// +optional\r\n\tAgentResources corev1.ResourceRequirements `json:\"agentResources,omitempty\"`\r\n\t// image pull secrets\r\n\tImagePullSecrets []corev1.LocalObjectReference `json:\"imagePullSecrets,omitempty\"`\r\n\t// +optional\r\n\tMinReplicaCount *int32 `json:\"minReplicaCount,omitempty\"`\r\n\t// +optional\r\n\tMaxReplicaCount *int32 `json:\"maxReplicaCount,omitempty\"`\r\n\t// Inject additional environment variables to the deployment\r\n\tEnv []corev1.EnvVar `json:\"env,omitempty\"`\r\n\t// reference to secret that contains the the Proxy settings\r\n\tProxyRef string `json:\"proxyRef,omitempty\"`\r\n\t// reference to secret that contains the PAT\r\n\tTokenRef string `json:\"tokenRef\"`\r\n\t// reference to secret that contains the CA certificates\r\n\tCACertRef string `json:\"caCertRef,omitempty\"`\r\n\t// AzureDevPortal is configuring the Azure DevOps pool settings of the Agent\r\n\t// by using additional environment variables.\r\n\tConfig AgentConfig `json:\"config,omitempty\"`\r\n\t// set to add or override the default metadata for the\r\n\t// scaled object trigger metadata\r\n\tTriggerMeta map[string]string `json:\"triggerMeta,omitempty\"`\r\n}\r\n\r\n// CDTargetStatus defines the observed state of CDTarget\r\ntype CDTargetStatus struct {\r\n\t// Conditions lists the most recent status condition updates\r\n\tConditions []metav1.Condition `json:\"conditions\"`\r\n}\r\n\r\n// control the pool and agent work directory\r\ntype AgentConfig struct {\r\n\tURL       string `json:\"url\"`\r\n\tPoolName  string `json:\"poolName\"`\r\n\tAgentName string `json:\"agentName,omitempty\"`\r\n\tWorkDir   string `json:\"workDir,omitempty\"`\r\n\t// Allow specifying MTU value for networks used by container jobs\r\n\t// useful for docker-in-docker scenarios in k8s cluster\r\n\tMTUValue string `json:\"mtuValue,omitempty\"`\r\n}\r\n```\r\n\r\n#### Custom Resource schema\r\n```yaml\r\napiVersion: cnad.gofound.nl/v1alpha1\r\nkind: CDTarget\r\nmetadata:\r\n  name: \u003c\u003ccdtarget-sample\u003e\u003e\r\n  namespace: \u003c\u003ctest\u003e\u003e\r\nspec:\r\n  agentImage: ghcr.io/bartvanbenthem/azagent-keda-22:latest\r\n  imagePullSecrets:\r\n  - name: \u003c\u003ccdtarget-regcred\u003e\u003e\r\n  minReplicaCount: 1\r\n  maxReplicaCount: 3\r\n  agentResources:\r\n    requests:\r\n      cpu: 100m\r\n    limits:\r\n      cpu: 200m\r\n  config:\r\n    url: \u003c\u003chttps://dev.azure.com/ORGANIZATION\u003e\u003e\r\n    poolName: \u003c\u003cpool-name\u003e\u003e\r\n    workDir: \r\n    mtuValue:\r\n    agentName:\r\n  tokenRef: \u003c\u003ccdtarget-token\u003e\u003e\r\n  proxyRef: \u003c\u003ccdtarget-proxy\u003e\u003e\r\n  caCertRef: \u003c\u003ccdtarget-ca\u003e\u003e\r\n  dnsPolicy: \u003c\u003cNone\u003e\u003e\r\n  dnsConfig:\r\n    nameservers: \r\n    - \u003c\u003c8.8.8.8\u003e\u003e\r\n  triggerMeta:\r\n    demands: \"maven,docker\"\r\n  env:\r\n  - name: INJECTED_ADDITIONAL_ENV\r\n    value: example-env\r\n  additionalSelector:\r\n    app: cdtarget-agent \r\n  ip:\r\n  - \u003c\u003c10.0.0.1\u003e\u003e\r\n  - \u003c\u003c10.0.0.2\u003e\u003e\r\n    ...\r\n```\r\n\r\n### Required Resources \u0026 Permissions\r\nWhat other resources are required:\r\n```Go\r\n//+kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch;create;update;patch;delete\r\n//+kubebuilder:rbac:groups=\"\",resources=configmaps,verbs=get;list;watch;create;update;patch;delete\r\n//+kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch;create\r\n//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch\r\n//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete\r\n//+kubebuilder:rbac:groups=operators.coreos.com,resources=operatorconditions,verbs=get;list;watch\r\n//+kubebuilder:rbac:groups=keda.sh,resources=scaledobjects,verbs=get;list;watch;create;update;patch;delete\r\n//+kubebuilder:rbac:groups=keda.sh,resources=triggerauthentications,verbs=get;list;watch;create;update;patch;delete\r\n```\r\n\r\n### Target reconciliation loop design\r\n```Go\r\nfunc Reconcile:\r\n// Get the Operator's CRD, if it doesn't exist then return\r\n// an error so the user knows to create it:\r\noperatorCrd, error = getMyCRD()\r\n    if error != nil {\r\n    return error\r\n}\r\n// Get the related resources for the Operator (networkpolicy)\r\n// If they don't exist, create them:\r\nresources, error = getRelatedResources()\r\nif error == ResourcesNotFound {\r\n    createRelatedResources()\r\n}\r\n// Check that the related resources relevant values match\r\n// what is set in the Operator's CRD. If they don't match,\r\n// update the resource with the specified values:\r\nif resources.Spec != operatorCrd.Spec {\r\n    updateRelatedResources(operatorCrd.Spec)\r\n}\r\n```\r\n\r\n### Handling upgrades and downgrades\r\ntodo\r\n\r\n### Failure reporting\r\nLogs, Events + status updates\r\n\r\n``` yaml\r\nstatus:\r\n  conditions:\r\n  - lastTransitionTime: 2022-01-01T00:00:00Z\r\n    message: reconciling message\r\n    reason: event\r\n    status: \"False\"/\"True\"\r\n    type: ReconcileSuccess\r\n```\r\n\r\n# Pereqs\r\n\r\n## Install KEDA\r\n```bash\r\n# Deploying using the deployment YAML files\r\nkubectl apply --server-side -f \\\r\n  https://github.com/kedacore/keda/releases/download/v2.12.0/keda-2.12.0.yaml\r\n```\r\n\r\n# Scaffolding parameters\r\n```bash\r\noperator-sdk init --domain gofound.nl --repo github.com/bartvanbenthem/cdtarget-operator\r\noperator-sdk create api --group cnad --version v1alpha1 --kind CDTarget --resource --controller\r\n```\r\n\r\n```bash\r\n# always run make after changing *_types.go and *_controller.go\r\ngo mod tidy\r\nmake generate\r\nmake manifests\r\n```\r\n\r\n# Build Operator image\r\n```bash\r\n# docker and github repo username\r\nexport USERNAME='bartvanbenthem'\r\n# image and bundle version\r\nexport VERSION=1.8.0\r\n# operator repo and name\r\nexport OPERATOR_NAME='cdtarget-operator'\r\n\r\n#######################################################\r\nsource ../00-ENV/env.sh # personal setup to inject PAT\r\n# login to ghcr.io registry\r\necho $CR_PAT | docker login ghcr.io -u USERNAME --password-stdin\r\n# Build the operator image\r\nmake docker-build docker-push IMG=ghcr.io/$USERNAME/$OPERATOR_NAME:v$VERSION\r\n```\r\n\r\n### Manual Operator Deployment\r\n```bash\r\n#######################################################\r\n# test and deploy the operator\r\nmake deploy IMG=ghcr.io/$USERNAME/$OPERATOR_NAME:v$VERSION\r\n```\r\n\r\n### Test custom resource\r\n```bash\r\n#######################################################\r\n# test cdtarget CR \r\nkubectl create ns test\r\n# prestage the PAT (token) Secret for succesfull Azure AUTH\r\nkubectl -n test create secret generic cdtarget-token --from-literal=AZP_TOKEN=$PAT\r\n# apply cdtarget resource\r\n# for scaling \u003e1 replica don`t set the agentName field in the CR\r\nkubectl -n test apply -f config/samples/cnad_cdtarget_sample.yaml\r\nkubectl -n test describe cdtarget cdtarget-agent\r\n# test CDTarget created objects\r\nkubectl -n test describe secret cdtarget-token\r\nkubectl -n test get configmaps\r\nkubectl -n test get networkpolicies\r\nkubectl -n test describe networkpolicies cdtarget-agent\r\nkubectl -n test describe scaledobject cdtarget-agent-keda\r\nkubectl -n test describe deployment cdtarget-agent\r\n\r\n```\r\n\r\n### Create pull secret\r\n```bash\r\n# create regcred secret\r\nkubectl -n test create secret docker-registry cdtarget-regcred \\\r\n          --docker-server='https://ghcr.io' \\\r\n          --docker-username='bartvanbenthem' \\\r\n          --docker-password=$CR_PAT \\\r\n```\r\n### Create \u0026 Update Proxy config\r\n```bash\r\n# update secret containing proxy settings\r\nkubectl -n test create secret generic cdtarget-proxy --dry-run=client -o yaml \\\r\n                  --from-literal=PROXY_USER='' \\\r\n                  --from-literal=PROXY_PW='' \\\r\n                  --from-literal=PROXY_URL='' \\\r\n                  --from-literal=HTTP_PROXY='' \\\r\n                  --from-literal=HTTPS_PROXY='' \\\r\n                  --from-literal=FTP_PROXY='' \\\r\n                  --from-literal=NO_PROXY='10.0.0.0/8' | kubectl apply -f -\r\nkubectl -n test scale deployment cdtarget-agent-keda --replicas=0  \r\n```\r\n\r\n### Update allowed ports\r\n```bash\r\ncat \u003c\u003cEOF | kubectl apply -f -\r\napiVersion: v1\r\nkind: ConfigMap\r\nmetadata:\r\n  name: cdtarget-ports\r\n  namespace: cdtarget-operator\r\ndata:\r\n  ports: | \r\n    443\r\n    22\r\n    5986\r\nEOF\r\n\r\nkubectl -n test delete networkpolicies.networking.k8s.io cdtarget-agent-keda\r\n```\r\n\r\n### Update Personal Access Token\r\n```bash\r\n# update CDTarget PAT\r\nkubectl -n test create secret generic cdtarget-token --dry-run=client -o yaml \\\r\n                  --from-literal=AZP_TOKEN=$PAT | kubectl apply -f -\r\nkubectl -n test scale deployment cdtarget-agent-keda --replicas=0  \r\n```\r\n\r\n### Inject CA Certificates from file\r\n* Best practise is to have the ca certificate prestaged as a kubernetes secret \r\n* from the custom resource a reference is made to the prestaged secret\r\n```bash\r\n# inject CA Certificates to CDTarget agents\r\n# in /usr/local/share/ca-certificates/\r\n# trust store: /etc/ssl/certs/ca-certificates.crt\r\nkubectl -n test create secret generic cdtarget-ca --dry-run=client -o yaml \\\r\n                --from-file=\"config/samples/CERTIFICATE.crt\" | kubectl apply -f -\r\nkubectl -n test scale deployment cdtarget-agent-keda --replicas=0  \r\n```\r\n\r\n### Manual Remove Operator, CRD and CR\r\n```bash\r\n# cleanup test deployment\r\nkubectl -n test delete -f config/samples/cnad_cdtarget_sample.yaml\r\nkubectl delete ns test\r\n# cleanup test deployment\r\nmake undeploy\r\n```\r\n\r\n## Operator lifecycle manager \r\n(instead of manual deployment)\r\n\r\n### Operator lifecycle manager Installation\r\n```bash\r\n#######################################################\r\n# install OLM (if not already present)\r\noperator-sdk olm install\r\noperator-sdk olm status\r\n```\r\n\r\n### Operator lifecycle manager Deployment\r\n```bash\r\n#######################################################\r\n# Build the OLM bundle\r\nmake bundle IMG=ghcr.io/$USERNAME/$OPERATOR_NAME:v$VERSION   \r\nmake bundle-build bundle-push BUNDLE_IMG=ghcr.io/$USERNAME/$OPERATOR_NAME-bundle:v$VERSION\r\n```\r\n\r\n```bash\r\n# Deploy OLM bundle\r\nkubectl create ns 'cdtarget-operator'\r\noperator-sdk run bundle ghcr.io/$USERNAME/$OPERATOR_NAME-bundle:v$VERSION --namespace='cdtarget-operator'\r\n```\r\n\r\n### Remove CR, CRD \u0026 Operator Bundle\r\n```bash\r\n# cleanup test deployment\r\nkubectl -n test delete -f config/samples/cnad_cdtarget_sample.yaml\r\nkubectl delete ns test\r\n# cleanup OLM bundle \u0026 OLM installation\r\noperator-sdk cleanup operator --delete-all --namespace='cdtarget-operator'\r\nkubectl delete ns 'cdtarget-operator'\r\n```\r\n\r\n### Uninstall Operator Lifecycle Manager\r\n```bash\r\n# uninstall OLM\r\noperator-sdk olm uninstall\r\n```\r\n\r\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbartvanbenthem%2Fcdtarget-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbartvanbenthem%2Fcdtarget-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbartvanbenthem%2Fcdtarget-operator/lists"}