{"id":13679204,"url":"https://github.com/jfrog/log-analytics-prometheus","last_synced_at":"2025-10-10T02:55:48.824Z","repository":{"id":37761934,"uuid":"292628854","full_name":"jfrog/log-analytics-prometheus","owner":"jfrog","description":"JFrog Prometheus Log Analytics Integration","archived":false,"fork":false,"pushed_at":"2025-03-19T00:45:20.000Z","size":7329,"stargazers_count":29,"open_issues_count":1,"forks_count":34,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-09-26T13:25:31.395Z","etag":null,"topics":["jfrog","jfrog-log-analytics","log-analysis"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jfrog.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-09-03T16:55:05.000Z","updated_at":"2025-06-19T07:36:08.000Z","dependencies_parsed_at":"2023-11-11T00:23:11.858Z","dependency_job_id":"c3c7f739-426c-43e1-bd73-938967e68716","html_url":"https://github.com/jfrog/log-analytics-prometheus","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/jfrog/log-analytics-prometheus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jfrog%2Flog-analytics-prometheus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jfrog%2Flog-analytics-prometheus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jfrog%2Flog-analytics-prometheus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jfrog%2Flog-analytics-prometheus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jfrog","download_url":"https://codeload.github.com/jfrog/log-analytics-prometheus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jfrog%2Flog-analytics-prometheus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002527,"owners_count":26083403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jfrog","jfrog-log-analytics","log-analysis"],"created_at":"2024-08-02T13:01:03.127Z","updated_at":"2025-10-10T02:55:48.818Z","avatar_url":"https://github.com/jfrog.png","language":null,"funding_links":[],"categories":["Others"],"sub_categories":[],"readme":"# Prometheus, Loki and Grafana Log Collection and Metrics for JFrog Artifactory and Xray\n\nThe JFrog Log Analytics and Metrics solution using Prometheus consists of three segments,\n\n1. Prometheus - the component where metrics data gets ingested\n2. Loki - the component where log data gets ingested\n3. Grafana - the component where data visualization is achieved via prebuilt dashboards\n\n## Pre-Requisites\n\n1. A Kubernetes Cluster - Amazon EKS / Google GKE / Azure AKS / Docker Desktop / Minikube\n   1. Recommended Kubernetes Version 1.25.2 and above\n   2. For Google GKE, refer [GKE Guide](https://cloud.google.com/kubernetes-engine/docs/how-to)\n   3. For Amazon EKS, refer [EKS Guide](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)\n   4. For Azure AKS, refer [AKS Guide](https://docs.microsoft.com/en-us/azure/aks/)\n   5. For Docker Desktop and Kubernetes, refer [Docker Guide](https://docs.docker.com/desktop/kubernetes/)\n\n2. `kubectl` configured to the Kubernetes cluster\n   1. For Installation and usage refer [kubectl setup](https://kubernetes.io/docs/tasks/tools/#kubectl)\n\n3. `helm` v3\n   1. For Installation and usage refer [helm setup](https://helm.sh/docs/intro/install/)\n\n4. Versions supported and Tested:\n   1. Artifactory: 7.117.x\n   2. Xray: 3.124.x\n   3. Prometheus: 3.5.x\n   4. Grafana: 12.0.x\n   5. Loki: 3.5.x\n\n## Known Limitations\n\nSome known limitations we are aware of\n- The stack does not install well on a [GKE Autopilot](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview) due to permissions needed\n\n## Read This Before Installing\n\n### Important Note: This version replaces all previous implementations. This version is not an in-place upgrade to the existing solution from JFrog but is a full reinstall. Any dashboard customizations done on previous versions will need to be redone.\n\n```\nThis guide assumes the implementer is performing new setup. Changes to handle install in an existing setup will be highlighted where applicable.\nIf Prometheus is already installed and configured, we recommend to have the existing Prometheus release name handy.\nIf Loki is already installed and configured, we recommend to have its service URL handy.\n```\n\nIf Prometheus and Loki are already available you can skip the installation section and proceed to [Configuration Section](#Configuration).\n\n\u003e [!WARNING]\n\u003e\n\u003e The old docker registry `partnership-pts-observability.jfrog.io`, which contains older versions of this integration is now deprecated. We'll keep the existing docker images on this old registry until August 1st, 2024. After that date, this registry will no longer be available. Please `helm upgrade` your JFrog kubernetes deployment in order to pull images as specified on the above helm value files, from the new `releases-pts-observability-fluentd.jfrog.io` registry. Please do so in order to avoid `ImagePullBackOff` errors in your deployment once this registry is gone.\n\n# Installation\n\n## Installing Prometheus, Grafana and Loki\n\nThe Prometheus Community [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) helm chart allows the creation of Prometheus instances and includes Grafana.\nThe Grafana Community [grafana](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) helm chart allows the creation of Loki instances and includes Grafana which can link to prometheus.\n\nOnce the Pre-Requisites are met, to install Prometheus Kubernetes stack:\n\n1. Create the namespaces required for the Kubernetes deployments\n   1. We use the `jfrog` namespace for the JFrog applications\n   2. We use the `monitoring` namespace for the observability tools\n\n```shell\nexport JFROG_NAMESPACE=jfrog\nkubectl create namespace ${JFROG_NAMESPACE}\n\nexport OBS_NAMESPACE=monitoring\nkubectl create namespace ${OBS_NAMESPACE}\n```\n\nNote: The `monitoring` namespace is also used in the Loki configuration in [artifactory-values.yaml](helm/artifactory-values.yaml) and [xray-values.yaml](helm/xray-values.yaml). If you decide to change it, make sure to update these files (the `LOKI_URL` variable).\n\n2. Install Prometheus and Grafana\n\n```shell\n# Add the required Helm repository\nhelm repo add prometheus-community https://prometheus-community.github.io/helm-charts\nhelm repo update\n```\n\n```shell\n# Install the kube-prometheus-stack chart\nhelm upgrade --install prometheus --values helm/prometheus-grafana-values.yaml prometheus-community/kube-prometheus-stack -n ${OBS_NAMESPACE}\n\n# Might need to add --set prometheus.prometheusSpec.maximumStartupDurationSeconds=600 to avoid an error (bug?)\n```\n\n3. For Docker Desktop\n\n   Run this additional command to correct the mount path propagation for prometheus node-exporter component.\n\n   An error event will be appearing as follows \"Error: failed to start container \"node-exporter\": Error response from daemon: path / is mounted on / but it is not a shared or slave mount\"\n\n```shell\nkubectl patch ds prometheus-prometheus-node-exporter --type json -p '[{\"op\": \"remove\", \"path\" : \"/spec/template/spec/containers/0/volumeMounts/2/mountPropagation\"}]' -n ${OBS_NAMESPACE}\n```\n\n4. Install Loki\n\n```shell\n# Add the required Helm repository\nhelm repo add grafana https://grafana.github.io/helm-charts\nhelm repo update\n```\n\n```shell\n# Install the Loki chart\nhelm upgrade --install loki --values helm/loki-values.yaml grafana/loki -n ${OBS_NAMESPACE}\n```\n\n## Install Artifactory with Open Metrics\n\n### Artifactory\nInstalling Artifactory using the official [Helm Chart](https://github.com/jfrog/charts/tree/master/stable/artifactory)\n\n1. Before starting the Artifactory installation generate a join and master keys\n\n```shell\nexport JOIN_KEY=$(openssl rand -hex 32)\nexport MASTER_KEY=$(openssl rand -hex 32)\n```\n\n2. Install Artifactory (using the generated join and master keys)\n\n```shell\n# Install Artifactory\nhelm upgrade --install artifactory jfrog/artifactory \\\n     --set artifactory.masterKey=${MASTER_KEY} \\\n     --set artifactory.joinKey=${JOIN_KEY} \\\n     --set artifactory.metrics.enabled=true \\\n     -n ${JFROG_NAMESPACE}\n```\n\n:bulb: Open Metrics is disabled by default in Artifactory. It's enabled by setting `artifactory.metrics.enabled=true`.\n\n3. Follow the instructions how to get your new Artifactory URL from the helm install output\n\n```shell\nexport SERVICE_IP=$(kubectl get svc --namespace ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n\necho ${SERVICE_IP}\necho \"http://${SERVICE_IP}/\"\n```\n   OR\n```shell\nexport SERVICE_IP=$(kubectl get svc --namespace ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')\n\necho ${SERVICE_IP}\necho \"http://${SERVICE_IP}/\"\n```\n\n4. Browse to the URL above and login to Artifactory with the default credentials: `admin`/`password`\n   1. Follow initial setup wizard\n   2. You will need to enter a valid Artifactory license. If needed, get a free trial license from [here](https://jfrog.com/start-free/)\n\n5. In the Artifactory UI, go to \"Administration\" -\u003e \"User Management\" -\u003e \"Access Tokens\" and generate an [admin access token](https://jfrog.com/help/r/how-to-generate-an-access-token-video/artifactory-creating-access-tokens-in-artifactory). Using the generated token, create a Kubernetes generic secret for the token - using one of the following methods\n\n```shell\nkubectl create secret generic jfrog-admin-token --from-file=token=\u003cpath_to_token_file\u003e -n ${JFROG_NAMESPACE}\n```\nOR\n```shell\nkubectl create secret generic jfrog-admin-token --from-literal=token=\u003cJFROG_ADMIN_TOKEN\u003e -n ${JFROG_NAMESPACE}\n```\n\n6. The PostgreSQL password is required for Artifactory upgrade. Run the following command to get the current PostgreSQL password\n```shell\nexport POSTGRES_PASSWORD=$(kubectl get secret -n ${JFROG_NAMESPACE} artifactory-postgresql -o jsonpath=\"{.data.postgres-password}\" | base64 --decode)\necho ${POSTGRES_PASSWORD}\n```\n\n7. Upgrade Artifactory with the custom values in [helm/artifactory-values.yaml](helm/artifactory-values.yaml) to create additional Kubernetes resources, which are required for the Prometheus service discovery process.\n\n```shell\n# Upgrade Artifactory\nhelm upgrade --install artifactory jfrog/artifactory \\\n     --set artifactory.joinKey=${JOIN_KEY} \\\n     --set databaseUpgradeReady=true --set postgresql.auth.password=${POSTGRES_PASSWORD} \\\n     -f helm/artifactory-values.yaml \\\n     -n ${JFROG_NAMESPACE}\n```\n\nThis will complete the necessary configuration for Artifactory and expose new service monitors `servicemonitor-artifactory` and `servicemonitor-observability` to expose metrics to Prometheus\n\n## Install Xray with Open Metrics\n\nTo configure and install Xray with Prometheus metrics being exposed use our file `helm/xray-values.yaml` to expose a metrics and new service monitor to Prometheus.\n\n### Xray\n\n1. Generate a master key for the Xray installation:\n\n```shell\nexport XRAY_MASTER_KEY=$(openssl rand -hex 32)\n```\n\n2. Use the same `JOIN_KEY` from the Artifactory installation, in order to connect Xray to Artifactory. You'll also be using the `jfrog-admin-token` kubernetes secret, that was created earlier as part of Artifactory installation\n\n   Getting the Artifactory URL:\n\n```shell\nexport JFROG_URL=$(kubectl get svc -n ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n\necho \"http://${JFROG_URL}\"\n```\n\n   OR\n```shell\nexport JFROG_URL=$(kubectl get svc -n ${JFROG_NAMESPACE} artifactory-artifactory-nginx -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')\n\necho \"http://${JFROG_URL}\"\n```\n\n   Install Xray\n```shell\nhelm upgrade --install xray jfrog/xray --set xray.jfrogUrl=http://${JFROG_URL} \\\n     --set xray.masterKey=${XRAY_MASTER_KEY} \\\n     --set xray.joinKey=${JOIN_KEY} \\\n     -f helm/xray-values.yaml \\\n     -n ${JFROG_NAMESPACE}\n```\n\n# Configuration\n\n## Access the Prometheus UI\n\nUse `kubectl port-forward` as mentioned below in a separate terminal window\n\n```shell\nkubectl port-forward service/prometheus-operated 9090:9090 -n ${OBS_NAMESPACE}\n```\n\nGo to the web UI of the Prometheus instance http://localhost:9090 and verify \"Status -\u003e Service Discovery\", the list shows all the `serviceMonitor`s.\n\nSearch for `servicemonitor-artifactory` and `servicemonitor-xray` to confirm they are successfully picked up by Prometheus.\n\n## Complete the Grafana Setup\n\nUse `kubectl port-forward` as mentioned below in a separate terminal window\n\n```shell\nkubectl port-forward service/prometheus-grafana 3000:80 -n ${OBS_NAMESPACE}\n```\n\n1. Open your Grafana on a browser at http://localhost:3000. Grafana default credentials are `admin/prom-operator` (set in [prometheus-grafana-values.yaml](helm/prometheus-grafana-values.yaml)).\n\n2. Go to \"Data sources\" on the sidebar menu\n\n3. Click `Add new data source`\n   1. Add your Prometheus as datasources (if not already configured): Set \"Prometheus server URL\" to `http://prometheus-kube-prometheus-prometheus:9090/`\n   2. Add your Loki as datasources: Set \"URL\" to `http://loki:3100`\n\n4. When adding the `Loki` and `Prometheus` datasources, click `Save \u0026 Test` button at the bottom to validate connection to services is successful\n\n## Artifactory and Xray Grafana Dashboards\n\nExample dashboards are included in the [grafana](grafana) directory. These dashboards need to be imported to Grafana. These include:\n\n- Artifactory Application Metrics (Open Metrics) Dashboard [Download Here](grafana/ArtifactoryMetrics.json)\n- Xray Application Metrics (Open Metrics) Dashboard [Download Here](grafana/XrayMetrics.json)\n\n1. After downloading the dashboards go to \"Dashboards\" -\u003e \"New\" -\u003e \"Import\"\n\n2. Pick `Upload dashboard JSON file` and upload Artifactory and Xray dashboards files that you downloaded in the previous step\n\n## Artifactory and Xray Logs in Grafana\n\nIf you have Loki configured as a Grafana datasource, you will see a `Logs` link on the sidebar menu. Click it and you should see the Artifactory and Xray services with a snippet of their logs.\n\nClick the `Show logs` on any of these services, and you can now see all the service (Artifactory or Xray) logs in Grafana and start searching and filtering through them.\n\n## References\n\n* [Grafana Dashboards](https://grafana.com/docs/grafana/latest/features/dashboard/dashboards/)\n* [Prometheus Queries](https://prometheus.io/docs/prometheus/latest/querying/basics/)\n* [Loki Queries](https://grafana.com/docs/loki/latest/query/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjfrog%2Flog-analytics-prometheus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjfrog%2Flog-analytics-prometheus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjfrog%2Flog-analytics-prometheus/lists"}