{"id":31898286,"url":"https://github.com/marcw/dgtools","last_synced_at":"2026-05-15T22:04:18.678Z","repository":{"id":312709666,"uuid":"1048357715","full_name":"marcw/dgtools","owner":"marcw","description":"Tools to work with Discogs data dumps","archived":false,"fork":false,"pushed_at":"2025-09-12T15:52:33.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-12T18:25:40.198Z","etag":null,"topics":["discogs","discogs-dump","parquet","postgresql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-01T10:15:46.000Z","updated_at":"2025-09-12T15:52:34.000Z","dependencies_parsed_at":"2025-09-01T15:16:51.838Z","dependency_job_id":"cf7af0b0-5753-47d9-be9c-4a0c3cae3fbd","html_url":"https://github.com/marcw/dgtools","commit_stats":null,"previous_names":["marcw/dgtools"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/marcw/dgtools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcw%2Fdgtools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcw%2Fdgtools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcw%2Fdgtools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcw%2Fdgtools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcw","download_url":"https://codeload.github.com/marcw/dgtools/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcw%2Fdgtools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279014820,"owners_count":26085596,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["discogs","discogs-dump","parquet","postgresql"],"created_at":"2025-10-13T11:57:41.310Z","updated_at":"2025-10-13T11:57:42.656Z","avatar_url":"https://github.com/marcw.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dgtools\n\nA command line utility to work with the Discogs data dumps.\n\nIt makes it super easy to:\n\n- List data dumps\n- Download a specific dumps\n- Convert dumps to ndjson or parquet\n- Import a dump into a PostgreSQL database\n\n## Usage\n\n```\ndgtools [global options] command [command options] [arguments...]\n```\n\n### Global Options\n\n- `--discogs-bucket` - The URL of the Discogs data dumps (default: \"https://discogs-data-dumps.s3.us-west-2.amazonaws.com\")\n\n## Commands\n\n### dump\n\nWork with Discogs data dump files.\n\n#### dump list\n\nList the files in the Discogs data dumps.\n\n```\ndgtools dump list [options]\n```\n\n**Options:**\n- `--year` - Filter by year\n- `--month` - Filter by month  \n- `--type` - Filter by data type\n- `--no-table` - Don't print the table (output filenames only)\n\n#### dump structure\n\nDump the structure of an XML file.\n\n```\ndgtools dump structure \u003cfile\u003e [options]\n```\n\n**Arguments:**\n- `file` - The file to dump the structure of\n\n**Options:**\n- `--stop-after X` - Stops analysis after X records\n\n\n#### dump download\n\nDownload a Discogs data dump.\n\n```\ndgtools dump download [options] \u003cname\u003e\n```\n\n**Arguments:**\n- `name` - The file to download\n\n\n**Options:**\n- `--out-dir` - The output directory (default: \".\")\n- `--overwrite` - Force the download even if the file already exists\n- `--checksum` - Check the checksum of the file after downloading (default: true)\n\n#### dump convert\n\nConvert a dump to a different format\n\n```\ndgtools dump convert \u003cname\u003e --out \u003cname\u003e [options]\n```\n\n**Arguments:**\n- `name` - The file to convert\n\n**Options:**\n- `--out` - The output file\n- `--stop-after X` - Stop conversion after X records\n\n\n### db\n\nWork with a database.\n\n**Options:**\n- `--database-url` - The URL of the database to connect to (default: \"postgres://$USER@localhost:5432/dgtools\", can be set via DATABASE_URL environment variable)\n\n#### db prepare\n\nPrepare the database for import by running migrations.\n\n```\ndgtools db prepare\n```\n\n#### db import\n\nImport data from a dump file to the database.\n\n```\ndgtools db import \u003cfile\u003e\n```\n\n**Arguments:**\n- `file` - The file to import the data from\n\n#### db nuke\n\nNuke the database by rolling back all migrations.\n\n```\ndgtools db nuke\n```\n\n## LICENSE\n\nPlease see [LICENSE.md](LICENSE.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcw%2Fdgtools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcw%2Fdgtools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcw%2Fdgtools/lists"}