{"id":30732621,"url":"https://github.com/prayerslayer/gh-license-scraper","last_synced_at":"2025-09-03T17:10:51.406Z","repository":{"id":28665298,"uuid":"32184889","full_name":"prayerslayer/gh-license-scraper","owner":"prayerslayer","description":"Go through public repositories and fetch their licenses.","archived":false,"fork":false,"pushed_at":"2015-03-14T20:17:35.000Z","size":156,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-15T02:57:48.008Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prayerslayer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-03-13T22:51:32.000Z","updated_at":"2015-03-14T20:17:35.000Z","dependencies_parsed_at":"2022-08-21T15:00:36.164Z","dependency_job_id":null,"html_url":"https://github.com/prayerslayer/gh-license-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/prayerslayer/gh-license-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prayerslayer%2Fgh-license-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prayerslayer%2Fgh-license-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prayerslayer%2Fgh-license-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prayerslayer%2Fgh-license-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prayerslayer","download_url":"https://codeload.github.com/prayerslayer/gh-license-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prayerslayer%2Fgh-license-scraper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273477209,"owners_count":25112636,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-03T02:00:09.631Z","response_time":76,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-03T17:10:34.841Z","updated_at":"2025-09-03T17:10:51.371Z","avatar_url":"https://github.com/prayerslayer.png","language":"JavaScript","readme":"# Github License Scraper\n\nA tool to fetch licenses from Github repositories.\n\n## What does it do?\n\n1. Get repositories from Github using one of two available strategies\n2. Put subset of information into CSV file\n\n## Prerequisites\n\n* Node 0.10.x or better\n* A Github [Access Token](https://help.github.com/articles/creating-an-access-token-for-command-line-use/) that can access private repositories. I.e. the `repo` scope is sufficient.\n\n## Installation and usage\n\nClone this repository. Then run `node app.js` with the necessary parameters. Stop it using SIGINT, i.e. `ctrl+c`.\n\n## Parameters\n\n* **`token`**: The Access Token. Mandatory.\n* `out`: The file to write to. Defaults to `repos.csv`.\n* `timeout`: Timeout (in ms) between calls, you only have 5000 per hour. Defaults to 20 seconds.\n* `strategy`: Either `popular` or `sample`. Defaults to `popular`.\n* `size`: How many repositories do you want to have? Defaults to 10K. Only applicable with `strategy=sample`.\n* `pool`: How many repositories to consider? Defaults to roughly 32M (the id of this repo). Only applicable with `strategy=sample`.\n* `page`: The page to start from. Only applicable with `strategy=popular`. Defaults to 0.\n* `before`: Consider repositores created before `before`. Only applicable with `strategy=popular`. Defaults to `2015-01-01`.\n\n## Strategies\n\nThere are two strategies available:\n\n### Popular\n\nUses the Search API to get the most poular (= most starred) repositories created before a certain date (see parameters above). As of March 2015 the Github Search API will return maximum 1000 results.\n\n### Sample\n\nTake a random sample of repositories (defaults to 10K out of 32M).\n\n## Example\n\n    node app.js --token abcdefgh --before 2015-01-03 --out data.csv","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprayerslayer%2Fgh-license-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprayerslayer%2Fgh-license-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprayerslayer%2Fgh-license-scraper/lists"}