{"id":22778992,"url":"https://github.com/sinipelto/repo-license-crawler","last_synced_at":"2026-05-09T01:04:51.092Z","repository":{"id":163362191,"uuid":"447680765","full_name":"sinipelto/repo-license-crawler","owner":"sinipelto","description":"Collects and summarizes license information on Python and NPM packages into output files.","archived":false,"fork":false,"pushed_at":"2023-08-15T13:43:34.000Z","size":24,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T14:13:36.774Z","etag":null,"topics":["crawler","crawler-python","license","license-checker","license-checking","license-crawler","license-management","licenses","licensing","nodejs","npm","npm-license-crawler","npm-license-tracker","npm-licenses","python","python-script","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sinipelto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-01-13T16:57:46.000Z","updated_at":"2024-04-28T03:26:24.000Z","dependencies_parsed_at":"2023-08-15T15:00:53.990Z","dependency_job_id":"a78dd0ee-b5a8-4ce7-9bb7-01e61d42c46b","html_url":"https://github.com/sinipelto/repo-license-crawler","commit_stats":null,"previous_names":["sinipelto/repo-license-crawler"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sinipelto/repo-license-crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinipelto%2Frepo-license-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinipelto%2Frepo-license-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinipelto%2Frepo-license-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinipelto%2Frepo-license-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sinipelto","download_url":"https://codeload.github.com/sinipelto/repo-license-crawler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sinipelto%2Frepo-license-crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32803626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"ssl_error","status_checked_at":"2026-05-08T08:22:45.650Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","crawler-python","license","license-checker","license-checking","license-crawler","license-management","licenses","licensing","nodejs","npm","npm-license-crawler","npm-license-tracker","npm-licenses","python","python-script","python3"],"created_at":"2024-12-11T20:07:29.780Z","updated_at":"2026-05-09T01:04:51.076Z","avatar_url":"https://github.com/sinipelto.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# (Universal) Repository License Crawler\n\nLicense collector tool for Python PIP and NodeJS NPM package licenses.\n\nThe aim is to get a complete collection of what kind of licenses are being used in the given repository (path), in any form, as parts of the application or individual libraries.\n\nThis helps identify the legitimacy of the current software project, especially in larger repos with dozens of libraries or dependencies.\n\nThis tool collects the licenses into human and machine-readable JSON files. From there it should be relatively easy to detect any conflicting licenses and take actions based on that information.\n\n# Getting Started\n\nInstall / ensure installed the following:\n- [Python 3.8+](https://www.python.org/downloads/)\n- [Node.js](https://nodejs.org/en/) (or Node Package Manager (NPM) at minimum)\n\nAfter installation, ensure both `python` and `npm` are available through `PATH` directly.\n\nEnsure pip is up-to-date:\n\n```bash\npip -V\npip install -U pip\npip -V\n```\n\nOr through python if pip is not globally available\n\n```bash\npython -m pip -V\npython -m pip install -U pip\npython -m pip -V\n```\n\nEnsure NPM is globally up-to-date:\n\n```bash\nnpm -v\nnpm install -g npm@latest\nnpm -v\n```\n\nCreate an empty Python Virtual Environment (VENV) for the tool:\n\n```bash\npython -m venv C:\\path\\to\\venv\n```\n\nActivate the newly created VENV:\n\n```bash\n.\\venv\\Scripts\\activate\n```\n\nEnsure pip is up-to-date (inside VENV):\n\n```bash\npip -V\npip install -U pip\npip -V\n```\n\nOr through python if pip is not globally available (inside VENV):\n\n```bash\npython -m pip -V\npython -m pip install -U pip\npython -m pip -V\n```\n\n\nInstall the required dependencies for the tool into the VENV (if applicable):\n\n```bash\npip install -r requirements.txt\n```\nOR\n```bash\npython -m pip install -r requirements.txt\n```\n\nRun the main file\n\n```bash\npython main.py\n```\n\nOnce successfully finished, check the output file for results (default: `./out/output.json`)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsinipelto%2Frepo-license-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsinipelto%2Frepo-license-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsinipelto%2Frepo-license-crawler/lists"}