{"id":31565393,"url":"https://github.com/jakubgania/github-api-scraper-producer-consumer","last_synced_at":"2026-04-07T21:31:05.376Z","repository":{"id":309815578,"uuid":"1037623242","full_name":"jakubgania/github-api-scraper-producer-consumer","owner":"jakubgania","description":null,"archived":false,"fork":false,"pushed_at":"2025-10-26T22:53:53.000Z","size":81,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-27T00:34:19.736Z","etag":null,"topics":["api","aws","container","docker","graphql","http","postgresql","producer-consumer","pulumi","python3","redis","sql","uv"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jakubgania.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-13T21:33:32.000Z","updated_at":"2025-10-26T22:53:56.000Z","dependencies_parsed_at":"2025-08-14T01:06:30.270Z","dependency_job_id":"be4efe03-55b0-4e7c-91c3-bfad9ce7fac3","html_url":"https://github.com/jakubgania/github-api-scraper-producer-consumer","commit_stats":null,"previous_names":["jakubgania/github-api-scraper-producer-consumer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jakubgania/github-api-scraper-producer-consumer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubgania%2Fgithub-api-scraper-producer-consumer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubgania%2Fgithub-api-scraper-producer-consumer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubgania%2Fgithub-api-scraper-producer-consumer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubgania%2Fgithub-api-scraper-producer-consumer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jakubgania","download_url":"https://codeload.github.com/jakubgania/github-api-scraper-producer-consumer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jakubgania%2Fgithub-api-scraper-producer-consumer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31530640,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"ssl_error","status_checked_at":"2026-04-07T16:28:06.951Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","aws","container","docker","graphql","http","postgresql","producer-consumer","pulumi","python3","redis","sql","uv"],"created_at":"2025-10-05T07:05:28.849Z","updated_at":"2026-04-07T21:31:04.902Z","avatar_url":"https://github.com/jakubgania.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# github-api-scraper-producer-consumer\n\nThis is a scraper project for the GitHub GraphQL API, built with a producer–consumer architecture. \n\nImportant Information ⚠️\n\nIf you want to run this app on your MacBook for several hours or days, be sure to keep the following in mind. Connect you MacBook to a power source and then go to:\n\nSystem settings -\u003e Options\n\nAnd activate the following option:\n\n\"Prevent automatic sleeping on power adapter when the display is off\"\n\n### If you want to view or test the dashboard locally\n\nGo to this folder:\n\n```bash\ndashboard-analytics/aws\n```\n\nThen run this command:\n\n```bash\npython3 -m http.server 8000\n```\n\nThen open in your browser: `http://localhost:8000/index.html`\n\n### Docker commands\n\n\n```bash\ndocker build -t api-github-scraper .\n```  \n\n```bash\ndocker-compose up -d\n```  \n\n```bash\ndocker-compose down -v\n```  \n\n```bash\ndocker run \\\n--name github-app-1 \\\n--label type=worker \\\n--network api-github-scraper \\\n-e GITHUB_API_TOKEN=your-github-api-token \\\n-e POSTGRES_DSN=\"postgresql://postgres:postgres@github_postgres:5432/postgres\" \\\n-e REDIS_HOST=\"redis-container\" \\\n-e CONTAINER_ID=github-app-1\n--memory 128m \\\napi-github-scraper\n```  \n\n```bash\ndocker ps --filter \"label=type=worker\"\n```  \n\n```bash\ndocker run -d --name redisinsight -p 5540:5540 redis/redisinsight:latest\n```  \n\n```bash\ndocker ps -a\n```  \n\n```bash\ndocker rm container-name\n```  \n\n### Resources\n\n[https://github.com/astral-sh/uv](https://github.com/astral-sh/uv)  \n[https://formulae.brew.sh/formula/pulumi](https://formulae.brew.sh/formula/pulumi)  \n[https://formulae.brew.sh/formula/awscli](https://formulae.brew.sh/formula/awscli)  \n[https://hub.docker.com/r/redis/redisinsight](https://hub.docker.com/r/redis/redisinsight)  \n[https://hub.docker.com/_/postgres](https://hub.docker.com/_/postgres)  \n[https://docs.github.com/en/graphql](https://docs.github.com/en/graphql)  \n[https://github.com/agronholm/apscheduler](https://github.com/agronholm/apscheduler)  ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakubgania%2Fgithub-api-scraper-producer-consumer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjakubgania%2Fgithub-api-scraper-producer-consumer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjakubgania%2Fgithub-api-scraper-producer-consumer/lists"}