{"id":18258822,"url":"https://github.com/apicurio/apicurio-3scale-gitops","last_synced_at":"2025-04-04T19:32:09.781Z","repository":{"id":206066758,"uuid":"715661119","full_name":"Apicurio/apicurio-3scale-gitops","owner":"Apicurio","description":"GitOps repository used for deploying applications into the 3scale clusters.","archived":false,"fork":false,"pushed_at":"2025-04-03T18:24:36.000Z","size":711,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-03T19:30:29.076Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Apicurio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-07T15:27:25.000Z","updated_at":"2025-04-03T18:24:39.000Z","dependencies_parsed_at":"2024-01-22T12:05:05.657Z","dependency_job_id":"efc39215-17ae-41ca-b7d2-2d6f0bd6e862","html_url":"https://github.com/Apicurio/apicurio-3scale-gitops","commit_stats":null,"previous_names":["apicurio/apicurio-3scale-gitops"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apicurio%2Fapicurio-3scale-gitops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apicurio%2Fapicurio-3scale-gitops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apicurio%2Fapicurio-3scale-gitops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Apicurio%2Fapicurio-3scale-gitops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Apicurio","download_url":"https://codeload.github.com/Apicurio/apicurio-3scale-gitops/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247237707,"owners_count":20906333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T10:34:50.772Z","updated_at":"2025-04-04T19:32:04.770Z","avatar_url":"https://github.com/Apicurio.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Table Of Contents\n- [Table Of Contents](#table-of-contents)\n- [3scale DEV](#3scale-dev)\n- [Useful links](#useful-links)\n  - [Infrastructure](#infrastructure)\n  - [Applications](#applications)\n- [Cluster and Namespace](#cluster-and-namespace)\n- [This Repository](#this-repository)\n  - [Manifests](#manifests)\n  - [Local Verification](#local-verification)\n  - [Continuous Deployment](#continuous-deployment)\n- [RDS Database](#rds-database)\n  - [RDS Specifics](#rds-specifics)\n  - [Connection Details](#connection-details)\n  - [How you should use the master credentials?](#how-you-should-use-the-master-credentials)\n  - [How to create new user accounts and DB?](#how-to-create-new-user-accounts-and-db)\n- [Manage Credentials](#manage-credentials)\n  - [Vault Access](#vault-access)\n  - [Adding Secrets](#adding-secrets)\n  - [Creating External Secrets](#creating-external-secrets)\n- [Automatic Deployments](#automatic-deployments)\n\n\n\n# 3scale DEV\nThe 3scale DEV environment is an ArgoCD controlled OpenShift cluster provided by the 3scale\nSRE team.  We have been granted an `apicurio` namespace within this cluster so that we can \ndeploy DEV versions of our Apicurio applications.\n\n# Useful links\n\n## Infrastructure\n  \n- [OpenShift Console](https://console-openshift-console.apps.dev-eng-ocp4-mas.dev.3sca.net/)\n\n- [ArgoCd Console](https://gitops.apps.dev-eng-ocp4-mas.dev.3sca.net/applications)\n\n- [Vault](https://vault.apps.dev-eng-ocp4-mas.dev.3sca.net/)\n\n## Applications\n\n- [Keycloak](https://sso.dev.apicur.io/)\n\n- [Backstage](https://backstage.dev.apicur.io/)\n  \n- [Apicurio Registry](https://registry.dev.apicur.io/)\n\n- [Apicurio Studio](https://studio.dev.apicur.io/)\n\n# Cluster and Namespace\n\nThe projects are presently deployed on the 3scale OpenShift Cluster and overseen by the 3Scale \nplatform's SRE team. If you encounter any challenges or have inquiries regarding the infrastructure, \nthere's the [#list-3scale-sre-mas](https://redhat-internal.slack.com/archives/C0586GTTXH9) Slack channel \navailable for you to get in touch with them.\n\nWe currently have a single namespace allocated for our use, namely `apicurio` with the resource \nquotas listed below.\n\n``` \nrequests.cpu: '4'\nrequests.memory: 10Gi\nservices.loadbalancers: '0'\n```\n\n\n\u003e Please note that this is a development cluster equipped with only three OpenShift Container Platform (OCP) worker nodes. Each node has 4 CPUs and 15GB of memory available, serving the needs of OCP, 3Scale and Apicurio components. If additional capacity is required, we can consider a **MODEST INCREASE** in the quota only after optimizing resource requests to match actual usage. However, any adjustments will be made conservatively.\n\nFor more details on the cluster and available services, [click here](https://github.com/3scale/platform/blob/master/docs/projects/engineering/dev-ocp-mas.md). \n\n\n# This Repository\n\nThis repository holds the manifest files that are used for deploying the apicurio projects on the 3scale cluster. \n\nThe project uses [kustomize](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/) and \n[Argo CD](https://argo-cd.readthedocs.io/en/stable/) to enable GitOps and continuous deployments. Hence, any \nchanges made to the manifest files will automatically be applied to the project deployments.\n\n## Manifests\nThe kubernetes/OpenShift manifest files for each project are organized under the `applications` folder.\n\nEach subfolder under the `applications` folder is an application monitored and deployed by ArgoCD.  The \nArgoCD applications are configured in the [3scale-platform git](https://github.com/3scale/platform/tree/master/manifests/apps/openshift-gitops/overlays/dev-eng-mas/apps) repository.\n\n\n## Continuous Deployment\n\nThis project leverages Argo CD for seamless and automated continuous deployment. Any modifications made to\nthe manifest files trigger an automatic deployment through these [Argo CD applications](https://gitops.apps.dev-eng-ocp4-mas.dev.3sca.net/applications?showFavorites=false\u0026proj=\u0026sync=\u0026autoSync=\u0026health=\u0026namespace=\u0026cluster=\u0026labels=).\n\nThe application is set to synchronize every 5 minutes for regular updates. However, you also have the option \nto manually initiate a synchronization by clicking on the **refresh** button. This ensures that changes are \npromptly reflected in the deployed environment.\n\n**TODO:** As of now, our permissions are limited to read-only access for the Argo applications. Any attempts \nto execute actions are met with denial, stemming from permission constraints. To address this, we could have \nthe assignment of the following admin role to our [AppProject](https://github.com/3scale/platform/blob/master/manifests/apps/openshift-gitops/overlays/dev-eng-mas/apps/apicurio-registry/appproject.yaml).\n\n```\nroles:\n    - name: project-admin\n      description: Read/Write access to this project only\n      policies:\n        - p, proj:apicurio-registry:project-admin, applications, get, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, create, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, update, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, delete, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, sync, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, override, apicurio-registry/*, allow\n        - p, proj:apicurio-registry:project-admin, applications, action/*, apicurio-registry/*, allow\n      groups:\n        - rhaf-dev\n```\n\n\u003e Note: The ArgoCD deployments are configured and managed by the SRE team in the 3scale platform repository [here](https://github.com/3scale/platform/tree/master/manifests/apps/openshift-gitops)\n\n\n# RDS Database\nThe Apicurio 3scale cluster setup leverages a private Amazon RDS (Relational Database Service) instance \nas its primary data storage solution.\n\n## RDS Specifics\n\n- DB Type: PostgreSQL\n- DB engine version: 15.4\n- Instance Type: db.t4g.small (2 vCPU and 2GB memory)\n- Instance Storage: 30 GB\n\n\u003e  Being a development AWS RDS instance, features typically needed for production workloads like MultiAZ, detailed monitoring, long backup retention, etc are not enabled.\n\n\n## Connection Details\n\nConnections string (master user, master password, RDS host...) can be found at Vault path:\n\nhttps://vault.apps.dev-eng-ocp4-mas.dev.3sca.net/ui/vault/secrets/secret/show/kubernetes/dev-eng-mas/rhaf-apicurio-registry/dev-eng-mas-apicurio-rds-pgsql-master-credentials\n\n\n## How you should use the master credentials?\n\nThe master username and password should be used exclusively for the creation of individual user accounts, \nassociated databases, and the granting of relevant privileges. It is imperative not to utilize them directly \nwithin your application.\n\nThis approach ensures that each application is associated with a dedicated user account, limited to accessing \na specific database. Consequently, in the unfortunate event of an application compromise, any potential attacker \nwould only gain access to the specific database associated with that particular application.\n\n## How to create new user accounts and DB?\n\nTo connect to the RDS database, a PostgreSQL client is required. However, you won't be able to connect to the \nRDS instance directly from your laptop, because the RDS is confined to the cluster network's access due to \nsecurity measures in place.\n\nAs a solution, a PostgreSQL client (pg-client) has been provisioned within the cluster. You can utilize it to \nestablish a connection with the database by following the steps below:\n\n1. **Login to the openshift cluster.**\n2. **Exec into the pg-client pod running in the cluster.**\n    ```\n    oc exec -it pg-client -n rhaf-apicurio -- /bin/bash\n    ```\n3. **Connect to the RDS**\n    ```\n    PGPASSWORD='\u003cpassword\u003e' psql -h \u003chostname\u003e -U \u003cuser\u003e -d postgres\n    ```\n4. **Create the new user account and DB.** [*Usually, the database name and the username are kept the same.. The example below illustrates the process of generating a new user and database for the apicurio application. Please make sure to substitute the database name and username according to your specific requirements.*]\n    ```\n    CREATE USER apicurio WITH PASSWORD '\u003cpassword\u003e';\n    CREATE DATABASE apicurio;\n    GRANT ALL PRIVILEGES ON DATABASE apicurio TO apicurio;\n    ```\n\nIf your application encounters difficulties executing certain database instructions, you may need to\ndesignate the user as the owner of the database using the following command:\n\n    ```\n    ALTER DATABASE apicurio OWNER TO apicurio;\n    ```\n\n\u003e Note: To confirm the successful creation of the user and database, exit the session and attempt to log in using the newly created user and database credentials.\n\n```\nPGPASSWORD='\u003cpassword\u003e' psql -h \u003chostname\u003e -U apicurio -d apicurio\n```\n\n# Manage Credentials\n\nWe employ HashiCorp Vault as a trusted solution for securely managing and storing sensitive information, such\nas secrets, credentials, and other confidential data. Vault ensures the highest level of security by providing\na centralized platform for secret management, encryption, and access control. By leveraging Vault's robust \nfeatures and integrations, we can securely store, retrieve, and distribute secrets to our applications and \nservices while maintaining strict access controls and audit trails. This approach not only enhances the \nsecurity posture of our deployments but also streamlines the management of secrets, making it an essential \ncomponent of our infrastructure.\n\n## Vault Access\n\n1. Obtain a GitHub personal authentication token by following these steps:\n   - Within your GitHub account, access your \"Personal settings\" by clicking on your profile picture in the upper right corner of the screen and then selecting \"settings\".\n\n   - Proceed to \"Developer settings\", followed by \"Personal access tokens\", and click on \"Generate new token\". Make sure to grant at least the `read:org` permission.\n2. Visit the [Vault Application](https://vault.apps.dev-eng-ocp4-mas.dev.3sca.net/ui/)\n3. Authenticate using your GitHub Token.\n\n## Adding Secrets\n\n1. Navigate to `secrets \u003e secret \u003e kubernetes \u003e dev-eng-mas \u003e apicurio`\n2. From the WebApplication Secrets page, click Add new secret.\n3. Enter secretname in the `Name`` field and secret-value in the Value field, and click Save.\n\n\n## Creating External Secrets\n\nExternal Secrets is a Kubernetes extension that enables the integration of external secret management \nsystems, such as HashiCorp Vault, with your Kubernetes clusters. By leveraging External Secrets, you \ncan centralize the management of secrets in established external systems while benefiting from Kubernetes' \nnative secret controller capabilities.\n\nAfter successfully adding the credentials within the vault, you can proceed to define the ExternalSecret \nCustom Resource. Below is an illustrative YAML example pertaining to the secret named\ndev-eng-mas-apicurio-rds-pgsql-apicurio-credentials, which has been incorporated into the vault.\n\n```\napiVersion: external-secrets.io/v1beta1\nkind: ExternalSecret\nmetadata:\n  name: rds-apicurio-credentials\nspec:\n  target:\n    name: rds-apicurio-credentials\n  secretStoreRef:\n    kind: ClusterSecretStore\n    name: vault-mas  \n  refreshInterval: 1m0s\n  data:\n    - secretKey: database\n      remoteRef:\n        key: kubernetes/dev-eng-mas/apicurio/dev-eng-mas-apicurio-rds-pgsql-apicurio-credentials\n        property: database\n    - secretKey: host\n      remoteRef:\n        key: kubernetes/dev-eng-mas/apicurio/dev-eng-mas-apicurio-rds-pgsql-apicurio-credentials\n        property: host\n    - secretKey: port\n      remoteRef:\n        key: kubernetes/dev-eng-mas/apicurio/dev-eng-mas-apicurio-rds-pgsql-apicurio-credentials\n        property: port\n    - secretKey: user\n      remoteRef:\n        key: kubernetes/dev-eng-mas/apicurio/dev-eng-mas-apicurio-rds-pgsql-apicurio-credentials\n        property: user\n    - secretKey: password\n      remoteRef:\n        key: kubernetes/dev-eng-mas/apicurio/dev-eng-mas-apicurio-rds-pgsql-apicurio-credentials\n        property: password\n```\n\n# Automatic Deployments\n\nThe Registry and Studio applications are each configured to automatically deploy the latest snapshot\nversion whenever that version is built.  This is accomplished through GitHub workflows configured in\nthe following locations:\n\n* [apicurio-registry/.github/workflows/verify.yaml](https://github.com/Apicurio/apicurio-registry/blob/main/.github/workflows/verify.yaml)\n* [apicurio-studio/.github/workflows/verify.yaml](https://github.com/Apicurio/apicurio-studio/blob/main/.github/workflows/verify.yaml)\n* [apicurio-3scale-gitops/.github/workflows/deploy_latest_registry.yml](https://github.com/Apicurio/apicurio-3scale-gitops/blob/main/.github/workflows/deploy_latest_registry.yml)\n* [apicurio-3scale-gitops/.github/workflows/deploy_latest_studio.yml](https://github.com/Apicurio/apicurio-3scale-gitops/blob/main/.github/workflows/deploy_latest_studio.yml)\n\nWhenever a change is made to `main` for either apicurio-registry or apicurio-studio, the\nstandard Verify workflow is run.  That workflow builds and tests the software, and then it\nbuilds and pushes docker containers tagged with `latest-snapshot`.  Each of those workflows\nhas a job called `trigger-3scale-deploy` which will trigger the appropriate `deploy_latest_*.yaml`\nworkflow from the 3scale gitops repo.\n\nWhen triggered, the `deploy_latest_*.yaml` perform the following steps:\n\n1. `docker pull` on the appropriate images tagged by `latest-snapshot`\n2. `docker inspect` on the images to determine their SHA/digest\n3. Update the appropriate deployment yaml config with the new SHA/digest for the image\n4. Push changes to the repo\n\nThis will then cause ArgoCD to redeploy the application with the new docker image.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapicurio%2Fapicurio-3scale-gitops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapicurio%2Fapicurio-3scale-gitops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapicurio%2Fapicurio-3scale-gitops/lists"}