{"id":15653130,"url":"https://github.com/zkoppert/innersource-crawler","last_synced_at":"2026-02-26T16:34:50.020Z","repository":{"id":40471769,"uuid":"327443948","full_name":"zkoppert/innersource-crawler","owner":"zkoppert","description":"This project creates a repos.json that can be utilized by the SAP InnerSource Portal.","archived":false,"fork":false,"pushed_at":"2025-04-28T17:20:50.000Z","size":140,"stargazers_count":32,"open_issues_count":2,"forks_count":21,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-30T17:09:04.523Z","etag":null,"topics":["actions","github-actions","hacktoberfest","innersource","innersource-commons","innersource-portal","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zkoppert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-01-06T22:31:23.000Z","updated_at":"2025-04-28T17:20:53.000Z","dependencies_parsed_at":"2023-02-14T04:31:34.872Z","dependency_job_id":"f438fc80-8040-4a1f-889d-7e4443865c7b","html_url":"https://github.com/zkoppert/innersource-crawler","commit_stats":{"total_commits":84,"total_committers":7,"mean_commits":12.0,"dds":"0.29761904761904767","last_synced_commit":"0e5e7ec5214e52ebe19ef5d971ceb92e16f3e51d"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkoppert%2Finnersource-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkoppert%2Finnersource-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkoppert%2Finnersource-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zkoppert%2Finnersource-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zkoppert","download_url":"https://codeload.github.com/zkoppert/innersource-crawler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251748950,"owners_count":21637418,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actions","github-actions","hacktoberfest","innersource","innersource-commons","innersource-portal","python"],"created_at":"2024-10-03T12:44:46.498Z","updated_at":"2026-02-26T16:34:44.961Z","avatar_url":"https://github.com/zkoppert.png","language":"Python","readme":"# InnerSource Crawler\n\n![CodeQL](https://github.com/zkoppert/innersource-crawler/actions/workflows/codeql-analysis.yml/badge.svg)\n\nThis project creates a `repos.json` that can be utilized by the [SAP InnerSource Portal][SAP-InnerSource-Portal]. The current approach assumes that the repos that you want to show in the portal are available in a GitHub organization, and that they all are tagged with a certain _topic_.\n\n## Support\n\nIf you need support using this project or have questions about it, please [open up an issue in this repository](https://github.com/zkoppert/innersource-crawler/issues). Requests made directly to GitHub staff or support team will be redirected here to open an issue. GitHub SLA's and support/services contracts do not apply to this repository.\n\n## Use as a GitHub Action\n\n1. Create a repository to host this GitHub Action or select an existing repository\n1. Create the env values from the sample workflow below (GH_TOKEN, ORGANIZATION) with your information as repository secrets. More info on creating secrets can be found [here](https://docs.github.com/en/actions/security-guides/encrypted-secrets).\nNote: Your GitHub token will need to have read/write access to all the repositories in the organization\n1. Copy the below example workflow to your repository and put it in the `.github/workflows/` directory with the file extension `.yml` (ie. `.github/workflows/crawler.yml`)\n1. Don't forget to do something with the resulting `repos.json` file. You can [move it to another repository](https://github.com/marketplace/actions/push-a-file-to-another-repository) if needed or [save it as a build artifact](https://github.com/actions/upload-artifact). This will all depend on what you are doing with it and what repository you are running this action out of.\n\n### Example workflow\n\n```yaml\nname: InnerSource repo crawler\n\non:\n  workflow_dispatch:\n  schedule:\n    - cron: '00 5 * * *'\n\njobs:\n  build:\n    name: InnerSource repo crawler\n    runs-on: ubuntu-latest\n\n    steps:\n    - name: Checkout code\n      uses: actions/checkout@v2\n    \n    - name: Run crawler tool\n      uses: docker://ghcr.io/zkoppert/innersource-crawler:v1\n      env:\n        GH_TOKEN: ${{ secrets.GH_TOKEN }}\n        ORGANIZATION: ${{ secrets.ORGANIZATION }}\n        # for multiple topics, add them after a comma eg:\n        # TOPIC: inner-source,actions,security,python\n        TOPIC: inner-source\n```\n\n## Local usage without Docker\n\n1. Copy `.env-example` to `.env`\n1. Fill out the `.env` file with a _token_ from a user that has access to the organization to scan (listed below). Tokens should have admin:org or read:org access.\n1. Fill out the `.env` file with the exact _topic_ name you are searching for\n1. Fill out the `.env` file with the exact _organization_ that you want to search in\n1. (Optional) Fill out the `.env` file with the exact _URL_ of the GitHub Enterprise that you want to search in. Keep empty if you want to search in the  public `github.com`.\n1. `python -m pip install -r requirements.txt`\n1. Run `python3 ./crawler.py`, which will create a `repos.json` file containing the relevant metadata for the GitHub repos for the given _topic_\n1. Copy `repos.json` to your instance of the [SAP-InnerSource-Portal][SAP-InnerSource-Portal] and launch the portal as outlined in their installation instructions\n\n[SAP-InnerSource-Portal]: https://github.com/sap/project-portal-for-InnerSource\n\n## License\n\n[MIT](LICENSE)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkoppert%2Finnersource-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzkoppert%2Finnersource-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzkoppert%2Finnersource-crawler/lists"}