{"id":28409982,"url":"https://github.com/crowdstrike/logscale-azure-aks-deployment","last_synced_at":"2026-01-27T07:35:27.176Z","repository":{"id":177364133,"uuid":"659840301","full_name":"CrowdStrike/logscale-azure-aks-deployment","owner":"CrowdStrike","description":"LogScale Cluster Deployment in Azure AKS","archived":false,"fork":false,"pushed_at":"2024-04-09T15:41:36.000Z","size":444,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-10T06:21:52.105Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CrowdStrike.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-06-28T17:13:57.000Z","updated_at":"2023-07-25T20:17:55.000Z","dependencies_parsed_at":"2025-06-22T23:40:46.944Z","dependency_job_id":null,"html_url":"https://github.com/CrowdStrike/logscale-azure-aks-deployment","commit_stats":null,"previous_names":["crowdstrike/logscale-azure-aks-deployment"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CrowdStrike/logscale-azure-aks-deployment","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdStrike%2Flogscale-azure-aks-deployment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdStrike%2Flogscale-azure-aks-deployment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdStrike%2Flogscale-azure-aks-deployment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdStrike%2Flogscale-azure-aks-deployment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CrowdStrike","download_url":"https://codeload.github.com/CrowdStrike/logscale-azure-aks-deployment/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrowdStrike%2Flogscale-azure-aks-deployment/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28808102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T07:14:39.408Z","status":"ssl_error","status_checked_at":"2026-01-27T07:14:39.098Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-02T11:09:34.464Z","updated_at":"2026-01-27T07:35:27.167Z","avatar_url":"https://github.com/CrowdStrike.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# LogScale Cluster Deployment in Azure AKS\n\n## Overview :\n\nThis document is a guide to provision a self-hosted LogScale cluster on Azure Cloud using Azure AKS kubernetes, with Azure object store for event repositories.\n\nThe sequence of steps and associated tasks required to provision LogScale are split into sections for planning, deployment, and validation. Sections should be processed in top-down order to ensure that prerequisites are met as provisioning steps are executed.\n\nNotes:\n\n- This cluster deployment utilizes an independent Kafka service.\n- This document assumes at least intermediate level knowledge of Azure Cloud.\n\n## Architectural Diagrams:\n\n**Deployment Overview:**\n\n![dep-overview](/docs/asset/dep-overview.png) \n\n**Functional Overview:**\n\n![fun-overview](/docs/asset/fun-overview.png) \n\n## Prerequisites :\n\n- Falcon LogScale License Key\n- LogScale Instance sizing guidelines:\n  - [Instance Sizing](https://library.humio.com/falcon-logscale-self-hosted/installation-provisioning-sizing.html)\n  - [Recommended Instance Sizing](https://library.humio.com/falcon-logscale-self-hosted/installation-prep-rec.html)\n- Azure Portal account with an active subscription\n- Azure Storage account\n  - [Create a storage account](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-create?tabs=azure-portal)\n- Azure Blob Container (i.e. storage bucket)\n  - [Create a blob container](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container)\n- Good understanding of [Azure Resource groups](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal)\n- Good understanding Kubernetes\n\n## Deployment:\n\n**1. Create an AKS cluster using any of the following methods:**\n\n  - [Azure CLI](https://learn.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster?tabs=azure-cli)\n  - [Azure Portal](https://learn.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-portal?tabs=azure-cli#create-an-aks-cluster)\n  - [Terraform](https://learn.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-terraform?tabs=azure-cli#implement-the-terraform-code)\n\nValidate the cluster is up and running by checking kubernetes service status:\n\n- kubectl get svc\n\n**2. Deploy Kafka/ZooKeeper to the AKS cluster using the** [strimzi-kafka-operator](https://github.com/strimzi/strimzi-kafka-operator)\n  - Note: Default partition count is 24. For details, refer to the [LogScale Digest Rules](https://library.humio.com/falcon-logscale-self-hosted/cluster-management-digest-rules.html)\n  - Deploy the strimzi-kafka-operator (kafka/k8s orchestration facility - [strimzi key features](https://strimzi.io/docs/operators/latest/overview.html#key-features-product_str))\n    - Execute following commands in K8s environment\n      - helm repo add strimzi [https://strimzi.io/charts/](https://strimzi.io/charts/)\n      - helm repo update\n      - helm install strimzi-kafka strimzi/strimzi-kafka-operator\n\n  - Provision the Kafka+zookeeper cluster\n    - Configure Kafka/zookeeper parameters (see sample kafka-zookeeper.yaml)\n      - replicas\n      - offsets.topic.replication.factor\n      - transaction.state.log.replication.factor\n      - transaction.state.log.min.isr\n      - default.replication.factor\n      - min.insync.replicas\n      - Inter.broker.protocol.version\n      - storage.size\n    - Apply Kafka yaml\n  - Validate installation by checking pods and services\n\n- kubectl get pods\n- kubectl get svc\n\n**3. Deploy S3Proxy**\n\n  - LogScale supports S3 compatible storage using S3Proxy service. Azure Blob Storage is one of them. For details, refer to the [S3Proxy documentation](https://github.com/gaul/s3proxy).\n  - Provision S3Proxy node(s)\n    - Configure parameters as required (see sample s3proxy.yaml)\n      | Name | Value |\n      | :-- | :-- |\n      | S3PROXY\\_AUTHORIZATION | `none` |\n      | JCLOUDS\\_PROVIDER | `azureblob` |\n      | JCLOUDS\\_ENDPOINT | \\\u003cAzure storage a/c endpoint\\\u003e |\n      | JCLOUDS\\_IDENTITY | \\\u003cAzure storage a/c name\\\u003e |\n      | JCLOUDS\\_CREDENTIAL | \\\u003cAzure storage a/c access key\\\u003e |\n      | JCLOUDS\\_AZYREBLOB\\_AUTH | `azureKey` |\n      | LOG\\_LEVEL | `debug` |\n    - Apply S3Proxy \u0026 S3Proxy service (see sample s3proxy-service.yaml)\n  - Verify that s3proxy pod and service are running.\n\n**4. Deploy LogScale**\n  - Note: This deployment uses NODE\\_ROLES=\"all\". For details, refer to the [NODE_ROLES documentation](https://library.humio.com/falcon-logscale-self-hosted/envar-node-roles.html). \n  - Deploy LogScale Operator (see setup\\_logscale.sh)\n    - Obtain the latest stable version from [LogScale Operator Releases](https://github.com/humio/humio-operator/releases).\n    - Apply LogScale CRDs \u0026 Operator\n  - Configure LogScale\n    - Create a license key secret\n      - kubectl create secret generic example-humiocluster-license --from-literal=data=\\\u003cLogScale License\\\u003e\n    - Obtain the latest stable version from [LogScale Releases](https://library.humio.com/release-notes/release-notes-stable.html).\n    - In HumioCluster yaml, set the parameters as required (see sample logscale-cluster.yaml)\n      - Set the parameters under spec\n        - targetReplicationFactor\n        - storagePartitionsCount\n        - digestPartitionsCount\n      - Set the Env. variables under spec.environmentVariables which are required to enable the S3 compatible storage\n        | Name | Value |\n        | :-- | :-- |\n        | USING\\_EPHEMERAL\\_DISKS | `true` |\n        | NODE\\_ROLES | `all\\|ingestonly\\|httponly`|\n        | S3\\_STORAGE\\_ENDPOINT\\_BASE | \\\u003cs3proxy endpoint\\\u003e |\n        | S3\\_STORAGE\\_ACCESSKEY | \\\u003cAzure Storage a/c accesskey\\\u003e |\n        | S3\\_STORAGE\\_SECRETKEY | \\\u003cAzure Storage a/c accesskey\\\u003e |\n        | LOCAL\\_STORAGE\\_PERCENTAGE | \"80\" |\n        | S3\\_STORAGE\\_PATH\\_STYLE\\_ACCESS | `true` |\n        | S3\\_STORAGE\\_IBM\\_COMPAT | `true` |\n        | BUCKET\\_STORAGE\\_IGNORE\\_ETAG\\_UPLOAD | `true` |\n        | BUCKET\\_STORAGE\\_IGNORE\\_ETAG\\_AFTER\\_UPLOAD | `false` |\n        | BUCKET\\_STORAGE\\_SSE\\_COMPATIBLE | `true` |\n        | S3\\_STORAGE\\_ENCRYPTION\\_KEY | `off` |\n        | S3\\_STORAGE\\_BUCKET | \"\\\u003cblob container\\\u003e\" |\n        | S3\\_ARCHIVING\\_PATH\\_STYLE\\_ACCESS | `true` |\n        | S3\\_EXPORT\\_PATH\\_STYLE\\_ACCESS | `true` |\n        | S3\\_STORAGE\\_PREFERRED\\_COPY\\_SOURCE | `true`\n\n        More information on environment variables : [Falcon LogScale - Configuration Parameters](https://library.humio.com/falcon-logscale-self-hosted/envar.html)\n      - Apply HumioCluster yaml\n    - Status of each container can be checked by running following command:\n      - kubectl get pods\n      - kubectl describe pod \\\u003chumio-cluster-name\\\u003e\n\n**5. Test and Verify**\n\n- Check availability of pods and services\n  - kubectl get pods\n  - kubectl get svc\n- Test connectivity and availability of LogScale cluster forwarding the port locally\n  - kubectl port-forward svc/example-humiocluster 8888:8080\n\n**6. Deploy Application gateway and enable AGIC**\n- Deploy Application Gateway Ingress Controller Add-on to access LogScale App from the Internet\n  - [Create and configure Application Gateway](https://learn.microsoft.com/en-us/azure/application-gateway/quick-create-portal)\n  - [Enable the ingress controller add-on for a new AKS cluster](https://learn.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-new)\n\n\n\n**Useful Reference Links:**\n  - [LogScale K8s Reference Architecture](https://library.humio.com/falcon-logscale-self-hosted/installation-k8s-ref-arch.html)\n  - [K8s core concept for AKS](https://learn.microsoft.com/en-us/azure/aks/concepts-clusters-workloads)\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"docs/asset/cs-logo-footer.png\"\u003e\u003cBR/\u003e\u003cimg width=\"150px\" src=\"docs/asset/adversary-red-eyes.png\"\u003e\u003c/P\u003e\n\u003ch3\u003e\u003cP align=\"center\"\u003eWE STOP BREACHES\u003c/P\u003e\u003c/h3\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrowdstrike%2Flogscale-azure-aks-deployment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrowdstrike%2Flogscale-azure-aks-deployment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrowdstrike%2Flogscale-azure-aks-deployment/lists"}