{"id":19339030,"url":"https://github.com/librariesio/depper","last_synced_at":"2025-04-23T01:31:32.718Z","repository":{"id":42369421,"uuid":"323422996","full_name":"librariesio/depper","owner":"librariesio","description":"Consumer of ecosystem APIs.","archived":false,"fork":false,"pushed_at":"2025-02-24T21:40:29.000Z","size":176,"stargazers_count":5,"open_issues_count":0,"forks_count":7,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-04-02T06:22:17.738Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/librariesio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-21T18:56:44.000Z","updated_at":"2025-02-24T21:40:30.000Z","dependencies_parsed_at":"2023-01-22T22:46:02.268Z","dependency_job_id":"f0f2d67d-fbea-4fef-96f0-5cb568091d7a","html_url":"https://github.com/librariesio/depper","commit_stats":{"total_commits":102,"total_committers":11,"mean_commits":9.272727272727273,"dds":0.4607843137254902,"last_synced_commit":"e0bfee2be22cca8aa22a75aba39f61b188a06555"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/librariesio%2Fdepper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/librariesio%2Fdepper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/librariesio%2Fdepper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/librariesio%2Fdepper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/librariesio","download_url":"https://codeload.github.com/librariesio/depper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250352321,"owners_count":21416470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T03:19:34.133Z","updated_at":"2025-04-23T01:31:32.706Z","avatar_url":"https://github.com/librariesio.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# depper\n\n\u003e A dapper consumer of ecosystem APIs.\n\nDepper is an ingestor of package releases from multiple ecosystems (each ecosystem is found in [ingestors/](ingestors/)).\n\nWhen new package releases are found, they are pushed to a shared redis queue for [Libraries.io](https://libraries.io) to process.\n\n## Types of Ingestors\n\nIngestors must satisfy one of these interfaces:\n\n* `ingestors.PollingIngestor`: these are scheduled to ingest new versions at specific intervals (`ingestor.Schedule()`).\n* `ingestors.StreamingIngestor`: these are always running in a goroutine, ingesting new releases via a channel\n\n## Ingestor Cursor Patterns\n\nDepper has to know where to pick up once it restarts, so there are several methods for storing such a cursor:\n\n* `ingestors.setBookmarkTime()` + `ingestor.getBookmarkTime()` [RECOMMENDED] : reads/sets a `time.Time` to redis (persistent)\n* `ingestors.setBookmark()` + `ingestor.getBookmark()`: reads/sets an arbitrary string to redis (persistent)\n* `LatestRun`: reads/sets a `time.Time` on the ingestor instance (non-persistent)\n\n## Running Locally\n\n`go run main.go`\n\n## Running Tests\n\n`go test -v ./...`\n\n## Running the Linter\n\nYou'll need the same version of our linter as CI, so reference the `\".circleci/config.yml\"` for the installation command.\n\n`golangci-lint run`: this will run the linter.\n\n`golangci-lint run --fix`: this will run the linter and autofix any autofix-able linter errors.\n\n## Deploying\n\n1. merge PR into `main` branch\n2. `tl setenv libraries`\n3. `./bin/deploy.sh`\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibrariesio%2Fdepper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flibrariesio%2Fdepper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flibrariesio%2Fdepper/lists"}