{"id":18291144,"url":"https://github.com/rhecosystemappeng/cluster-iq","last_synced_at":"2025-04-05T10:30:58.595Z","repository":{"id":189942054,"uuid":"681566400","full_name":"RHEcosystemAppEng/cluster-iq","owner":"RHEcosystemAppEng","description":"Cluster IQ Backend repo","archived":false,"fork":false,"pushed_at":"2025-04-01T12:47:00.000Z","size":8346,"stargazers_count":1,"open_issues_count":18,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-01T13:38:57.023Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RHEcosystemAppEng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-22T09:37:57.000Z","updated_at":"2025-03-26T17:19:16.000Z","dependencies_parsed_at":"2023-08-22T15:46:14.774Z","dependency_job_id":"6ede1a86-44d8-447e-bb57-1eea4df394c2","html_url":"https://github.com/RHEcosystemAppEng/cluster-iq","commit_stats":null,"previous_names":["rhecosystemappeng/cluster-iq"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fcluster-iq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fcluster-iq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fcluster-iq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fcluster-iq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RHEcosystemAppEng","download_url":"https://codeload.github.com/RHEcosystemAppEng/cluster-iq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247324497,"owners_count":20920658,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T14:13:21.161Z","updated_at":"2025-04-05T10:30:58.574Z","avatar_url":"https://github.com/RHEcosystemAppEng.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cluster IQ\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/RHEcosystemAppEng/cluster-iq)](https://goreportcard.com/report/github.com/RHEcosystemAppEng/cluster-iq)\n[![Go Reference](https://pkg.go.dev/badge/github.com/RHEcosystemAppEng/cluster-iq.svg)](https://pkg.go.dev/github.com/RHEcosystemAppEng/cluster-iq)\n\n\nCluster IQ is a tool for making stock of the Openshift Clusters and its\nresources running on the most common cloud providers and collects relevant\ninformation about the compute resources, access routes and billing.\n\nMetrics and monitoring is not part of the scope of this project, the main\npurpose is to maintain and updated inventory of the clusters and offer a easier\nway to identify, manage, and estimate costs.\n\n## Supported cloud providers\n\nThe scope of the project is to cover make stock on the most common public cloud\nproviders, but as the component dedicated to scrape data is decoupled, more\nproviders could be included in the future.\n\nThe following table shows the compatibility matrix and which features are\navailable for every cloud provider:\n\n| Cloud Provider | Compute Resources | Billing | Actions | Scheduled Actions |\n| -------------- | ----------------- | ------- | ------- | ----------------- |\n| AWS            | Yes               | Yes     | Yes     | Yes               |\n| Azure          | No                | No      | No      | No                |\n| GCP            | No                | No      | No      | No                |\n\n\n## Architecture\n\nThe following graph shows the architecture of this project:\n![ClusterIQ architecture diagram](./doc/architecture.png)\n\n## Documentation\n\nThe following documentation is available:\n\n- [Events Documentation](doc/events/README.md) - Event flows and sequence diagrams\n- [Development Setup](doc/development-setup.md) - Local development guide\n\n## Installation\nThis section explains how to deploy ClusterIQ and ClusterIQ Console.\n\n\n### Prerequisites:\n#### Cloud provider RBAC configuration\nBefore configuring credentials for ClusterIQ, it is recommended to access the\nuser and permission management service and create a dedicated user exclusively\nfor ClusterIQ. This user should have the minimum necessary permissions to\nfunction properly. This approach enhances the security of your public cloud\nprovider accounts by enforcing the principle of least privilege.\n\nEach Cloud Provider has a different way for configuring users and permissions.\nBefore continuing, check and follow the steps for each Cloud Provider you want\nto configure:\n\n* **[Amazon Web Services (AWS)](./doc/aws-user-permissions-config.md)**\n* **Microsoft Azure** Not available.\n* **Google cloud Platform:** Not available.\n\n#### Accounts Configuration\n1. Create a folder called `secrets` for saving the cloud credentials. This folder is ignored on this repo to keep your\n   credentials safe.\n    ```text\n    mkdir secrets\n    export CLUSTER_IQ_CREDENTIALS_FILE=\"./secrets/credentials\"\n    ```\n    :warning: Please take care and don't include them on the repo.\n\n2. Create your credentials file with the AWS credentials of the accounts you\n   want to scrape. The file must follow the following format:\n    ```text\n    echo \"\n    [ACCOUNT_NAME]\n    provider = {aws/gcp/azure}\n    user = XXXXXXX\n    key = YYYYYYY\n    billing_enabled = {true/false}\n    \" \u003e\u003e $CLUSTER_IQ_CREDENTIALS_FILE\n    ```\n    :warning: The values for `provider` are: `aws`, `gcp` and `azure`, but the\n    scraping is only supported for `aws` by the moment.  The credentials file\n    should be placed on the path `secrets/*` to work with\n    `docker/podman-compose`.\n\n    :exclamation: This file structure was design to be generic, but it works\n    differently depending on the cloud provider. For AWS, `user` refers to the\n    `ACCESS_KEY`, and `key` refers to `SECRET_ACCESS_KEY`.\n\n    :exclamation: Some Cloud Providers has extra costs when querying the Billing\n    APIs (like AWS Cost Explorer). Be careful when enable this module. Check your\n    account before enabling it.\n\n### Openshift Deployment\nSince version 0.3, ClusterIQ includes its own Helm Chart placed on\n`./deployments/helm/cluster-iq`.\nFor more information about the\n   supported parameters, check the [Configuration Section](#configuration).\n1. Prepare your cluster and CLI\n    ```sh\n    oc login ...\n\n    export NAMESPACE=\"cluster-iq\"\n    oc new-project $NAMESPACE\n    ```\n\n2. Create a secret containing this information is needed. To create the secret,\n   use the following command:\n    ```shell\n    oc create secret generic credentials -n $NAMESPACE \\\n      --from-file=credentials=$CLUSTER_IQ_CREDENTIALS_FILE\n    ```\n\n3. Configure your cluster-iq deployment by modifying the\n   `./deployments/helm/cluster-iq/values.yaml` file.\n\n4. Deploy the Helm Chart\n    ```sh\n    helm upgrade cluster-iq ./deployments/helm/cluster-iq/ \\\n      --install \\\n      --namespace $NAMESPACE \\\n      -f ./deployments/helm/cluster-iq/values.yaml\n    ```\n\n5. Monitor every resource was created correctly:\n    ```sh\n    oc get pods -w -n $NAMESPACE\n    helm list\n    ```\n\n6. Once every pod is up and running, trigger the scanner manually for\n   initializing the inventory\n   ```sh\n   oc create job --from=cronjob/scanner scanner-init -n $NAMESPACE\n   ```\n\n\n## Local Deployment (for development)\nFor deploying ClusterIQ in local for development purposes, check the following\n[document](./doc/development-setup.md)\n\n\n\n### Configuration\nAvailable configuration via Env Vars:\n| Key                                  | Value                                                 | Description                               |\n| ------------------------------------ | ----------------------------------------------------- | ----------------------------------------- |\n| CIQ_AGENT_API_URL                    | string (Default: \"http://api:8080/api/v1\")            | ClusterIQ Agent URL for accessing the API |\n| CIQ_AGENT_INSTANT_SERVICE_LISTEN_URL | string (Default: \"0.0.0.0:50051\")                     | ClusterIQ Agent gRPC listen URL           |\n| CIQ_AGENT_POLLING_SECONDS_INTERVAL   | integer (Default: 30)                                 | ClusterIQ Agent polling time (seconds)    |\n| CIQ_AGENT_URL                        | string (Default: \"agent:50051\")                       | ClusterIQ Agent listen URL                |\n| CIQ_API_LISTEN_URL                   | string (Default: \"0.0.0.0:8080\")                      | ClusterIQ API listen URL                  |\n| CIQ_API_URL                          | string (Default: \"\")                                  | ClusterIQ API public endpoint             |\n| CIQ_AGENT_LISTEN_URL                 | string (Default: \"0.0.0.0:50051\")                     | ClusterIQ Agent listen URL                |\n| CIQ_AGENT_URL                        | string (Default: \"\")                                  | ClusterIQ Agent public endpoint           |\n| CIQ_DB_URL                           | string (Default: \"postgresql://pgsql:5432/clusteriq\") | ClusterIQ DB URL                          |\n| CIQ_CREDS_FILE                       | string (Default: \"\")                                  | Cloud providers accounts credentials file |\n| CIQ_LOG_LEVEL                        | string (Default: \"INFO\")                              | ClusterIQ Logs verbosity mode             |\n\n\n### Scanner\nThe scanner searches each region for instances (servers) that are part of an\nOpenshift cluster. As each provider and each service has different\nspecifications, the Scanner includes a specific module dedicated to each of\nthem. These modules are automatically activated or deactivated depending on the\nconfigured accounts and their configuration.\n```shell\n# Building in a container\nmake build-scanner\n\n# Building in local\nmake local-build-scanner\n```\n\n## API Server\nThe API server interacts between the UI and the DB.\n\n```shell\n# Building in a container\nmake build-api\n\n# Building in local\nmake local-build-api\n```\n\n## Agent (gRPC)\nThe Agent performs actions over the selected cloud resources. It only accepts\nincoming requests from the API.\n\nCurrently, on release `v0.4`, the agent only supports Power On/Off clusters on AWS.\n\n```shell\n# Building in a container\nmake build-agent\n\n# Building in local\nmake local-build-agent\n```\n\n---\n[![Cluster IQ Container image building](https://github.com/RHEcosystemAppEng/cluster-iq/actions/workflows/container-image-building.yaml/badge.svg)](https://github.com/RHEcosystemAppEng/cluster-iq/actions/workflows/container-image-building.yaml)\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fcluster-iq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhecosystemappeng%2Fcluster-iq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fcluster-iq/lists"}