{"id":23663682,"url":"https://github.com/bervproject/crawler","last_synced_at":"2025-12-06T11:30:17.991Z","repository":{"id":81762892,"uuid":"556262409","full_name":"bervProject/crawler","owner":"bervProject","description":"Crawler Service","archived":false,"fork":false,"pushed_at":"2024-03-18T09:23:09.000Z","size":292,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-12-29T05:32:25.107Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bervProject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"berviantoleo","ko_fi":"berviantoleo","patreon":"berviantoleo","liberapay":"berviantoleo","open_collective":"berviantoleo","issuehunt":"berviantoleo"}},"created_at":"2022-10-23T12:58:37.000Z","updated_at":"2022-10-23T13:45:32.000Z","dependencies_parsed_at":"2024-12-29T05:31:10.358Z","dependency_job_id":"93b8e2bf-f86e-48f6-8671-5da09fadf4b8","html_url":"https://github.com/bervProject/crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bervProject%2Fcrawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bervProject%2Fcrawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bervProject%2Fcrawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bervProject%2Fcrawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bervProject","download_url":"https://codeload.github.com/bervProject/crawler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239662732,"owners_count":19676435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-29T05:30:58.116Z","updated_at":"2025-12-06T11:30:17.949Z","avatar_url":"https://github.com/bervProject.png","language":"JavaScript","funding_links":["https://github.com/sponsors/berviantoleo","https://ko-fi.com/berviantoleo","https://patreon.com/berviantoleo","https://liberapay.com/berviantoleo","https://opencollective.com/berviantoleo","https://issuehunt.io/r/berviantoleo"],"categories":[],"sub_categories":[],"readme":"# Crawler Service\n\n## Current Implementation\n\n1. Read Badges Data from Credly to DynamoDB\n\n## Sample Data\n\n![Sample](./samples/data.png)\n\n## Columns\n\n### Badge\n\n```json\n{\n  \"id\": \"\u003cbadge id\u003e\",\n  \"name\": \"\u003cbadge title\u003e\",\n  \"src\": \"\u003cimage link\u003e\",\n  \"provider\": \"\u003cprovider name\u003e\",\n  \"expires\": \"\u003cexpires date time\u003e\"\n}\n```\n\n## Required Environment Variables\n\n```bash\nexport AWS_ACCESS_KEY_ID=\nexport AWS_SECRET_ACCESS_KEY=\nexport AWS_REGION=\nexport DYNAMODB_TABLE_NAME=\nexport CREDLY_USERNAME=\n```\n\n## TODO\n\n- [ ] Multiple Data Store Options (Extendable)\n- [ ] Custom Mapper Columns\n- [ ] Multiple Providers\n\n## License\n\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbervproject%2Fcrawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbervproject%2Fcrawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbervproject%2Fcrawler/lists"}