{"id":13564283,"url":"https://github.com/okp4/github-exporter","last_synced_at":"2025-04-03T21:30:37.628Z","repository":{"id":61740689,"uuid":"525695874","full_name":"okp4/github-exporter","owner":"okp4","description":"Prometheus GitHub exporter with a focus on Pull Request/Issue/Milestone metrics","archived":false,"fork":true,"pushed_at":"2023-06-26T20:11:46.000Z","size":419,"stargazers_count":3,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-04T17:47:19.602Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"xrstf/github_exporter","license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/okp4.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-17T08:05:34.000Z","updated_at":"2022-11-23T16:42:44.000Z","dependencies_parsed_at":"2023-02-10T08:30:39.979Z","dependency_job_id":null,"html_url":"https://github.com/okp4/github-exporter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okp4%2Fgithub-exporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okp4%2Fgithub-exporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okp4%2Fgithub-exporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okp4%2Fgithub-exporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/okp4","download_url":"https://codeload.github.com/okp4/github-exporter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247082900,"owners_count":20880738,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:29.167Z","updated_at":"2025-04-03T21:30:36.937Z","avatar_url":"https://github.com/okp4.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# GitHub Exporter for Prometheus\n\n\u003e This project was forked from [xrstf's work](https://github.com/xrstf/github_exporter) ❤️. This project was originally MIT licensed.\n\n[![version](https://img.shields.io/github/v/release/okp4/github-exporter?style=for-the-badge\u0026logo=github)](https://github.com/okp4/github-exporter/releases)\n[![lint](https://img.shields.io/github/actions/workflow/status/okp4/github-exporter/lint.yml?label=lint\u0026style=for-the-badge\u0026logo=github)](https://github.com/okp4/github-exporter/actions/workflows/lint.yml)\n[![build](https://img.shields.io/github/actions/workflow/status/okp4/github-exporter/build.yml?label=build\u0026style=for-the-badge\u0026logo=github)](https://github.com/okp4/github-exporter/actions/workflows/build.yml)\n[![conventional commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg?style=for-the-badge\u0026logo=conventionalcommits)](https://conventionalcommits.org)\n[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg?style=for-the-badge)](https://github.com/okp4/.github/blob/main/CODE_OF_CONDUCT.md)\n[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg?style=for-the-badge)](https://opensource.org/licenses/BSD-3-Clause)\n\nIt exposes Prometheus metrics for a list of pre-configured GitHub repositories.\nThe focus is on providing more insights about issues, pull requests and milestones.\n\n![Grafana Screenshot](https://github.com/okp4/github-exporter/blob/main/contrib/grafana/screenshot.png?raw=true)\n\nIt uses GitHub's API v4 and tries its best to not exceed the request quotas, but for large\nrepositories (5k+ PRs) it's recommended to tweak the settings a bit.\n\n## Operation\n\nThe goal of this particular exporter is to provide metrics for **all** pull requests, issues\nand milestones (collectively called \"items\" from here on) within a given set of repositories.\nAt the same time, **open** items should be refreshed much more often and quickly than older\ndata.\n\nTo achieve this, the exporter upon startup scans all repositories for all items. After\nthis is complete, it will\n\n- fetch the most recently updated 100 items (to detect new elements and elements\n  whose status has changed),\n- re-fetch all open items frequently (every 5 minutes by default) and\n- re-fetch **all** items every 12 hours by default.\n\nWhile the scheduling for the re-fetches happens concurrently in multiple go routines,\nthe fetching itself is done sequentially to avoid triggering GitHub's anti-abuse system.\n\nFetching open items has higher priority, so that even large amounts of old items\ncannot interfere with the freshness of open items.\n\nIt is possible to limit the initial scan (using `-pr-depth`, `-issue-depth` and `-milestone-depth`),\nso that for very large repositories not all items are fetched. But this only limits the\ninitial scan, over time the exporter will learn about new items and not forget the old ones\n(and since it always keeps all items up-to-date, the number of items fetched will slooooowly\nover time grow).\n\nJobs are always removed from the queue, even if they failed. The exporter relies on the\ngoroutines to re-schedule them later anyway, and this prevents flooding GitHub when the\nAPI has issues or misconfiguration occurs. Job queues can only contain one job per kind,\nso even if the API is down for an hour, the queue will not fill up with the re-fetch job.\n\n## Installation\n\nYou need Go 1.19 installed on your machine.\n\n```sh\nmake install\n```\n\nA Docker image is available as [`okp4/github-exporter`](https://hub.docker.com/r/okp4/github-exporter).\n\n## Usage\n\nYou need an OAuth2 token to authenticate against the API. Make it available\nas the `GITHUB_TOKEN` environment variable.\n\nBy default, the exporter listens on `0.0.0.0:9612`.\n\nAll configuration happens via command line arguments. At the bare minimum, you need to\nspecify a single repository to scrape:\n\n```sh\n./github-exporter -repo myself/my-repository\n```\n\nYou can configure multiple `-repo` (which is also recommended over running the exporter\nmultiple times in parallel, so a single exporter can serialize all API requests) and\ntweak the exporter further using the available flags:\n\n```sh\nUsage of ./github-exporter:\n  -debug\n        enable more verbose logging\n  -issue-depth int\n        max number of issues to fetch per repository upon startup (-1 disables the limit, 0 disables issue fetching entirely) (default -1)\n  -issue-refresh-interval duration\n        time in between issue refreshes (default 5m0s)\n  -issue-resync-interval duration\n        time in between full issue re-syncs (default 12h0m0s)\n  -listen string\n        address and port to listen on (default \":9612\")\n  -milestone-depth int\n        max number of milestones to fetch per repository upon startup (-1 disables the limit, 0 disables milestone fetching entirely) (default -1)\n  -milestone-refresh-interval duration\n        time in between milestone refreshes (default 5m0s)\n  -milestone-resync-interval duration\n        time in between full milestone re-syncs (default 12h0m0s)\n  -pr-depth int\n        max number of pull requests to fetch per repository upon startup (-1 disables the limit, 0 disables PR fetching entirely) (default -1)\n  -pr-refresh-interval duration\n        time in between PR refreshes (default 5m0s)\n  -pr-resync-interval duration\n        time in between full PR re-syncs (default 12h0m0s)\n  -realnames\n        use usernames instead of internal IDs for author labels (this will make metrics contain personally identifiable information)\n  -repo value\n        repository (owner/name format) to include, can be given multiple times\n  -owner string\n        github login (username or organization) of the owner of the repositories that will be included. Excludes forked and locked repo, includes 100 first private \u0026 public repos\n```\n\n## Metrics\n\n**All** metrics are labelled with `repo=(full repo name)`, for example\n`repo=\"xrstf/github_exporter\"`.\n\nFor each repository, the following metrics are available:\n\n- `github_exporter_repo_disk_usage_bytes`\n- `github_exporter_repo_forks`\n- `github_exporter_repo_stargazers`\n- `github_exporter_repo_watchers`\n- `github_exporter_repo_is_private`\n- `github_exporter_repo_is_archived`\n- `github_exporter_repo_is_disabled`\n- `github_exporter_repo_is_fork`\n- `github_exporter_repo_is_locked`\n- `github_exporter_repo_is_mirror`\n- `github_exporter_repo_is_template`\n- `github_exporter_repo_language_size_bytes` is additionally labelled with `language`.\n- `github_exporter_repo_commits_count`\n\nFor pull requests, these metrics are available:\n\n- `github_exporter_pr_info` contains lots of metadata labels and always has a constant\n  value of `1`. Labels are:\n\n  - `number` is the PR's number.\n  - `state` is one of `open`, `closed` or `merged`.\n  - `author` is the author ID (or username if `-realnames` is configured).\n\n  In addition, the exporter recognizes a few common label conventions, namely:\n\n  - `size/*` is reflected as a `size` label (e.g. the `size/xs` label on GitHub becomes\n    a `size=\"xs\"` label on the Prometheus metric).\n  - `team/*` is reflected as a `team` label.\n  - `kind/*` is reflected as a `kind` label.\n  - `priority/*` is reflected as a `priority` label.\n  - `approved` is reflected as a boolean `approved` label.\n  - `lgtm` is reflected as a boolean `lgtm` label.\n  - `do-no-merge/*` is reflected as a boolean `pending` label.\n\n- `github_exporter_pr_label_count` is the number of PRs that have a given label\n  and state. This counts all labels individually, not just those recognized for\n  the `_info` metric.\n\n- `github_exporter_pr_created_at` is the UNIX timestamp of when the PR was\n  created on GitHub. This metric only has `repo` and `number` labels.\n\n- `github_exporter_pr_updated_at` is the UNIX timestamp of when the PR was\n  last updated on GitHub. This metric only has `repo` and `number` labels.\n\n- `github_exporter_pr_fetched_at` is the UNIX timestamp of when the PR was\n  last fetched from the GitHub API. This metric only has `repo` and `number` labels.\n\nThe PR metrics are mirrored for issues:\n\n- `github_exporter_issue_info`\n- `github_exporter_issue_label_count`\n- `github_exporter_issue_created_at`\n- `github_exporter_issue_updated_at`\n- `github_exporter_issue_fetched_at`\n\nThe metrics for milestones are similar:\n\n- `github_exporter_milestone_info` has `repo`, `number`, `title` and `state` labels.\n- `github_exporter_milestone_issues` counts the number of open/closed issues/PRs\n  for a given milestone, so it has `repo`, `number`, `kind` (issue or pullrequest)\n  and `state` labels.\n- `github_exporter_milestone_created_at`\n- `github_exporter_milestone_updated_at`\n- `github_exporter_milestone_fetched_at`\n- `github_exporter_milestone_closed_at` is optional and 0 if the milestone is open.\n- `github_exporter_milestone_due_on` is optional and 0 if no due date is set.\n\nAnd a few more metrics for monitoring the exporter itself are available as well:\n\n- `github_exporter_pr_queue_size` is the number of PRs currently queued for\n  being fetched from the API. This is split via the `queue` label into `priority`\n  (open PRs) and `regular` (older PRs).\n- `github_exporter_issue_queue_size` is the same as for the PR queue.\n- `github_exporter_milestone_queue_size` is the same as for the PR queue.\n- `github_exporter_api_requests_total` counts the number of API requests per\n  repository.\n- `github_exporter_api_costs_total` is the sum of costs (in API points) that have\n  been used, grouped by `repo`.\n- `github_exporter_api_points_remaining` is a gauge representing the remaining\n  API points. 5k points can be consumed per hour, with resets after 1 hour.\n\n## Long-term storage\n\nIf you plan on performing long-term analysis over repositories, make sure to put proper\nrecording rules into place so that queries can be performed quickly. The exporter\nintentionally does not pre-aggregate most things, as to not spam Prometheus or restrict\nthe available information.\n\nA few example rules can be found in `contrib/prometheus/rules.yaml`.\n\n## You want to get involved? 😍\n\nPlease check out OKP4 health files :\n\n- [Contributing](https://github.com/okp4/.github/blob/main/CONTRIBUTING.md)\n- [Code of conduct](https://github.com/okp4/.github/blob/main/CODE_OF_CONDUCT.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokp4%2Fgithub-exporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fokp4%2Fgithub-exporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokp4%2Fgithub-exporter/lists"}