{"id":19001618,"url":"https://github.com/kevcodez/intheloop-worker","last_synced_at":"2025-10-07T17:15:07.108Z","repository":{"id":91719505,"uuid":"586549848","full_name":"kevcodez/intheloop-worker","owner":"kevcodez","description":null,"archived":false,"fork":false,"pushed_at":"2023-01-29T22:51:42.000Z","size":305,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-01T18:33:05.696Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kevcodez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-08T14:40:00.000Z","updated_at":"2023-01-08T16:01:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"50451a63-8ca6-422d-8c7f-a3eb47706f91","html_url":"https://github.com/kevcodez/intheloop-worker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevcodez%2Fintheloop-worker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevcodez%2Fintheloop-worker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevcodez%2Fintheloop-worker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kevcodez%2Fintheloop-worker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kevcodez","download_url":"https://codeload.github.com/kevcodez/intheloop-worker/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240021016,"owners_count":19735272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T18:12:06.654Z","updated_at":"2025-10-07T17:15:02.071Z","avatar_url":"https://github.com/kevcodez.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# intheloop worker\n\nResponsible for determining and executing scraping tasks. Built on Nestjs+BullMQ+Redis.\n\n## Blog posts\n\nA blog *may* have a RSS feed url.\nIf it does, the RSS feed is consumed regularly and unsaved blog posts will be saved.\nThe Google Translate API is used to determine the language of the blog post as we don't want to show non-english blog posts to every user.\n\n## Popular tweets\n\nThe V1 Twitter API is used to fetch popular tweets.\nThe V2 Twitter API does not allow filtering by minimum favorites or replies and consumes *a lot* of quota as one has to loop through all relevant tweets.\nWith the V1 API, we can simply include those filters in the search query.\n\n## Releases\n\nReleases are fetched through NPM or Github, depending on the project.\nEven though there might be relases on Github, it does not mean that the API will return all those releases, thus, we have to be careful when selecting the source for releases.\n\n## Repositories\n\n- [Web](https://github.com/kevcodez/intheloop) - Nuxt.js web app that powers [Intheloop](https://intheloop.dev)\n- [Functions](https://github.com/kevcodez/intheloop-functions) - Firebase functions for API endpoints\n- [Worker](https://github.com/kevcodez/intheloop-worker) - Nest.js+BullMQ worker for scraping data\n- [Admin](https://github.com/kevcodez/intheloop-admin) - Nuxt.js admin interface for adding/editing topics\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevcodez%2Fintheloop-worker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkevcodez%2Fintheloop-worker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkevcodez%2Fintheloop-worker/lists"}