{"id":16584908,"url":"https://github.com/mikaelvesavuori/github-dora-metrics","last_synced_at":"2025-03-23T14:31:06.544Z","repository":{"id":158767890,"uuid":"634230207","full_name":"mikaelvesavuori/github-dora-metrics","owner":"mikaelvesavuori","description":"Instant, badge-ready DORA metrics for your GitHub repository.","archived":false,"fork":false,"pushed_at":"2023-05-07T08:04:22.000Z","size":222,"stargazers_count":39,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-18T21:08:20.290Z","etag":null,"topics":["badge","dora","dora-metrics","github","github-metrics","metrics"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mikaelvesavuori.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"mikaelvesavuori"}},"created_at":"2023-04-29T13:13:36.000Z","updated_at":"2025-01-09T08:18:28.000Z","dependencies_parsed_at":"2023-05-20T14:15:29.207Z","dependency_job_id":null,"html_url":"https://github.com/mikaelvesavuori/github-dora-metrics","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikaelvesavuori%2Fgithub-dora-metrics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikaelvesavuori%2Fgithub-dora-metrics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikaelvesavuori%2Fgithub-dora-metrics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikaelvesavuori%2Fgithub-dora-metrics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mikaelvesavuori","download_url":"https://codeload.github.com/mikaelvesavuori/github-dora-metrics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245115704,"owners_count":20563214,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["badge","dora","dora-metrics","github","github-metrics","metrics"],"created_at":"2024-10-11T22:46:11.036Z","updated_at":"2025-03-23T14:31:06.518Z","avatar_url":"https://github.com/mikaelvesavuori.png","language":"TypeScript","funding_links":["https://github.com/sponsors/mikaelvesavuori"],"categories":[],"sub_categories":[],"readme":"# DORA metrics for GitHub repositories\n\nThis repository contains a ready-to-use solution for setting up an API-based service on AWS that will calculate the four (well, [there are 5 these days](https://www.getunleash.io/blog/dora-metrics-in-2023-5-ways-to-measure-devops-performance), but who's counting?) DORA metrics completely from processing data that GitHub holds on a given repository:\n\n- [Change failure rate (CFR)](#change-failure-rate)\n- [Deployment frequency (DF)](#deployment-frequency)\n- [Lead time to change (LTC)](#lead-time-for-changes)\n- [Mean time to repair (MTTR)](#mean-time-to-repair)\n\nThis solution also supports returning the response in a format that works for those sweet [shields.io](https://shields.io/endpoint) badges.\n\n_Please note that it's kind of messy to sometimes calculate these values strictly to their intended use. Read more about the details of these in the [Calculations](#calculations) section and how this solution adapts some shortcomings in the available data._\n\n## Prerequisites\n\n- Creating deployments in GitHub and using GitHub issues with `bug` or `incident` labels\n- Recent [Node.js](https://nodejs.org/en/) (ideally 18+) installed.\n- Amazon Web Services (AWS) account with sufficient permissions so that you can deploy infrastructure. A naive but simple policy would be full rights for CloudWatch, Lambda, API Gateway, and S3.\n- Ideally some experience with [Serverless Framework](https://www.serverless.com) as that's what we will use to deploy the service and infrastructure.\n\n## Configuration\n\n### GitHub personal access tokens\n\nTo make use of this project, you'll need a GitHub personal access token with `repo` and `issues` access and use this in any calls to GitHub.\n\nYou need to use one of these options to actually use such a token:\n\n- Let the caller pass in their token.\n- Pre-bake a token:\n  - Passing it in as an option during CI `npx sls deploy --pat YOUR_PAT`\n  - Hardcode the value in `serverless.yml`\n  - Get the value from AWS Secrets Manager when building/compiling\n\n_See [this guide for how to create a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)._\n\nThe statement at the top—that this is a ready-to-use solution—is therefore technically true since consumers of the API _may_ pass in their own GitHub personal access token (PAT) to make calls, for example to private/internal repositories. However, you'll probably want to set up and bake in a public access PAT to make the requests towards GitHub for any case in which no token is directly passed in.\n\n#### Setting the PAT value\n\nYou can simply pass it in as an option during CI; for example with `npx sls deploy --pat YOUR_PAT`.\n\nFor hardcoding it, you'll set the PAT value in `serverless.yml` under `custom.config.gitHubPersonalAccessToken`. The recommended way, however, would be to use [AWS Secrets Manager or similar](https://www.serverless.com/blog/aws-secrets-management/). Both ways are present in `serverless.yml` but the Secrets Manager solution is commented out. Please refer to the comments there for more information.\n\n### GitHub behaviors that drive the calculations\n\n- Use GitHub Issues with `bug` or `incident` labels\n  - Open and close times are used\n- Use GitHub deployments, deploying to one of the named environments `prd`, `prod`, `production`, or `live`\n  - The commit time (prompting the deployment) and start-of-deployment times are used\n\n## Installation\n\nClone, fork, or download the repo as you normally would. Run `npm install`.\n\n## Commands\n\n- `npm start`: Run application locally\n- `npm test`: Test the business/application logic with AVA\n- `npm run build`: Package application with Serverless Framework\n- `npm run deploy`: Deploy application to AWS with Serverless Framework\n- `npm run teardown`: Remove stack from AWS\n\n## Running locally\n\nUsing `npm start` you can start using the local endpoint `http://localhost:3000/metrics` to call the service. See example calls further down.\n\n## Calculations\n\n_Quotes from [a blog post on Google Cloud](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance)._\n\nThe period that is taken into account is the one provided in the `serverless.yml` configuration, under `custom.config.maxPeriodInDays`. It's set to 30 (days) by default.\n\n### Deployment frequency\n\n\u003e How **often** an organization **successfully** releases to **production**.\n\n#### Calculation\n\nThis is really straight-forward but assumes/requires you to use [GitHub \"deployments\"](https://docs.github.com/en/actions/deployment/about-deployments/about-continuous-deployment) to express these deployments.\n\n`{number of deployments in period}` which use an [environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment) named one of the following:\n\n- `prd`\n- `prod`\n- `production`\n- `live`\n\n### Lead Time for Changes\n\n\u003e The amount of **time** it takes a **commit** to get into **production**.\n\n#### Calculation\n\nThis reflects the time from a commit being made to a deployment being started.\n\nThere is no crystal-clear way to get the _time of deployment_ as the `updatedAt` value of the deployment may express other changes as well – this is why the start of the deployment will have to be the best proxy value I could come up with.\n\n`{time of deploy - time committed}` as full seconds, using the median of all processed values.\n\n### Change Failure Rate\n\n\u003e The **percentage** of **deployments** causing a **failure** in production.\n\n#### Calculation\n\nThis one is hard as doing it orthodoxly would mean mapping _each_ deployment to a _specific_ failure, which is obviously very hard, sometimes not even possible, and is definitely something that would explode the scope far out from just having to do with GitHub. While it's indeed possible to check for _strictly failed_ deployments, I don't find that to be a true indicator of real issues as they can be (given you don't do canary releases etc.) attributed to things happening before the code even reaches actual users.\n\nThe solution used here is to accept a somewhat wider perspective by simply dividing the number of closed issues tagged `bug` or `incident` with the number of deployments.\n\n`{number of issues / number of deployments} * 100` as a rounded percentage.\n\n_If there are no deployments in the period, this will be represented as zero percent._\n\n### Mean Time To Repair\n\n\u003e How **long** it takes an organization to **recover** from a **failure** in production.\n\n#### Calculation\n\nSame as above, this can be tricky to have an opinion on and getting it working completely inside of GitHub.\n\nThe solution is to calculate the median time to close issues marked `bug` or `incident` within the period. While it won't _only_ reflect handling production failures, it will at least give a bug resolution time.\n\n`{issue close time - issue open time}`.\n\n## Example API calls\n\n### Get metrics\n\n#### Request\n\n```bash\ncurl {ENDPOINT}/metrics?repo={OWNER}/{REPO}\n```\n\n#### Response example\n\n```json\n{\n  \"changeFailureRate\": \"66.67%\",\n  \"deploymentFrequency\": \"0.10/day\",\n  \"leadTimeForChange\": \"00:00:00:20\",\n  \"meanTimeToRepair\": \"00:06:21:15\"\n}\n```\n\n### Get metrics for a shields.io badge\n\n```bash\ncurl {ENDPOINT}/metrics?repo={OWNER}/{REPO}\u0026badge=true\n```\n\n#### Response example\n\n```json\n{\n  \"schemaVersion\": 1,\n  \"label\": \"DORA metrics\",\n  \"message\": \"CFR: 66.67% | DF: 0.10/day | LTC: 00:00:00:20 | MTTR: 00:06:21:15\",\n  \"color\": \"black\",\n  \"labelColor\": \"blue\",\n  \"style\": \"for-the-badge\"\n}\n```\n\n### Get metrics and pass in custom GitHub PAT\n\nEither pass it in your request:\n\n```bash\ncurl {ENDPOINT}/metrics?repo={OWNER}/{REPO}\u0026token={YOUR_GH_PAT}\n```\n\nOr use a header for this:\n\n`Authorization: {YOUR_GH_PAT}`\n\n## Known issues\n\n- There doesn't seem to be a (reliable) way to get pagination for GitHub deployments, meaning only the last 100 deployments (maximum) will be used to calculate deployment-related metrics.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikaelvesavuori%2Fgithub-dora-metrics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmikaelvesavuori%2Fgithub-dora-metrics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikaelvesavuori%2Fgithub-dora-metrics/lists"}