{"id":16953160,"url":"https://github.com/c4milo/gofetch","last_synced_at":"2025-10-09T06:40:13.965Z","repository":{"id":137877707,"uuid":"43556402","full_name":"c4milo/gofetch","owner":"c4milo","description":"Go library for downloading content from internet, in parallel.","archived":false,"fork":false,"pushed_at":"2017-07-04T18:05:41.000Z","size":10509,"stargazers_count":26,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-08-24T20:44:38.424Z","etag":null,"topics":["downloader","etags","fetcher","gofetch","golang","golang-library","http","parallel"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/c4milo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-02T14:09:45.000Z","updated_at":"2024-10-15T08:18:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"f8320c17-d6ed-4680-8aa9-69075af36795","html_url":"https://github.com/c4milo/gofetch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/c4milo/gofetch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c4milo%2Fgofetch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c4milo%2Fgofetch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c4milo%2Fgofetch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c4milo%2Fgofetch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/c4milo","download_url":"https://codeload.github.com/c4milo/gofetch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c4milo%2Fgofetch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000841,"owners_count":26082951,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["downloader","etags","fetcher","gofetch","golang","golang-library","http","parallel"],"created_at":"2024-10-13T22:06:30.550Z","updated_at":"2025-10-09T06:40:13.925Z","avatar_url":"https://github.com/c4milo.png","language":"Go","readme":"# Gofetch\n[![Build Status](https://travis-ci.org/c4milo/gofetch.svg?branch=master)](https://travis-ci.org/c4milo/gofetch)\n[![GoDoc](https://godoc.org/github.com/c4milo/gofetch?status.svg)](https://godoc.org/github.com/c4milo/gofetch)\n\nGo library to download files from the internerds using Go 1.7 or greater.\n\n## Features\n\n* Resumes downloads if interrupted.\n* Allows parallel downloading of a single file by requesting multiple data chunks at once over HTTP.\n* Reports download progress through a Go channel if indicated to do so.\n* Supports file integrity verification if a checksum is provided.\n* Supports ETags, skipping downloading a file if it hasn't changed on the server.\n* Can be combined with https://github.com/cenkalti/backoff to support retrying with exponential back-off\n\n## Gotchas\nWhen downloading file chunks concurrently, you may encounter some issues:\n\n* Servers may limit the number of concurrent connections you have open against them\n* Servers may accept the connections but will not send anything, in this case the default HTTP client will timeout for you.\nIf you provide your own client, make sure it has proper timeouts or your connection will block for several seconds, depending\non your operating system network timeouts.\n\n## Example\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/c4milo/gofetch\"\n)\n\nfunc main() {\n\tgf := gofetch.New(\n\t\tgofetch.WithDestDir(os.TempDir()),\n\t\tgofetch.WithConcurrency(10),\n\t\tgofetch.WithETag(),\n\t)\n\n\tprogressCh := make(chan gofetch.ProgressReport)\n\n\tvar myFile *os.File\n\tgo func() {\n\t\tvar err error\n\t\tmyFile, err = gf.Fetch(\n\t\t\t\"http://releases.ubuntu.com/15.10/ubuntu-15.10-server-amd64.iso\",\n\t\t\tprogressCh)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}()\n\n\t// pogressCh is close by gofetch once a download finishes\n\tvar totalWritten int64\n\tfor p := range progressCh {\n\t\t// p.WrittenBytes does not accumulate, it represents the chunk size written\n\t\t// in the current operation.\n\t\ttotalWritten += p.WrittenBytes\n\t\tfmt.Printf(\"%d of %d\\n\", totalWritten, p.Total)\n\t}\n\n\tdestFile, err := os.Create(\"/tmp/ubuntu-15.10-server-amd64.iso\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tdefer func() {\n\t\tif err := destFile.Close(); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}()\n\n\tif _, err := io.Copy(destFile, myFile); err != nil {\n\t\tpanic(err)\n\t}\n}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc4milo%2Fgofetch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fc4milo%2Fgofetch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc4milo%2Fgofetch/lists"}