{"id":13509783,"url":"https://github.com/etix/mirrorbits","last_synced_at":"2026-01-11T03:43:10.164Z","repository":{"id":18349992,"uuid":"21529845","full_name":"etix/mirrorbits","owner":"etix","description":"Mirrorbits is a geographical download redirector written in Go for distributing files efficiently across a set of mirrors.","archived":false,"fork":false,"pushed_at":"2025-03-28T00:12:25.000Z","size":2000,"stargazers_count":528,"open_issues_count":49,"forks_count":96,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-03-28T01:23:49.383Z","etag":null,"topics":["cdn","checksum","cli","distribution","dockerfile","ftp","go","http","https","ipv6","mirror","open-source","realtime","rsync"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/etix.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-07-05T23:15:44.000Z","updated_at":"2025-03-28T00:12:31.000Z","dependencies_parsed_at":"2023-01-11T20:28:46.611Z","dependency_job_id":"d15c8a30-2da8-4855-a916-b85d87b36a03","html_url":"https://github.com/etix/mirrorbits","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/etix%2Fmirrorbits","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/etix%2Fmirrorbits/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/etix%2Fmirrorbits/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/etix%2Fmirrorbits/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/etix","download_url":"https://codeload.github.com/etix/mirrorbits/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246332210,"owners_count":20760438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdn","checksum","cli","distribution","dockerfile","ftp","go","http","https","ipv6","mirror","open-source","realtime","rsync"],"created_at":"2024-08-01T02:01:13.030Z","updated_at":"2026-01-11T03:43:10.118Z","avatar_url":"https://github.com/etix.png","language":"Go","funding_links":[],"categories":["Go","cli"],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/etix/mirrorbits.svg?branch=master)](https://travis-ci.org/etix/mirrorbits)\n[![Go Report Card](https://goreportcard.com/badge/github.com/etix/mirrorbits)](https://goreportcard.com/report/github.com/etix/mirrorbits)\n\nMirrorbits\n===========\n\nMirrorbits is a geographical download redirector written in [Go](https://golang.org) for distributing files efficiently across a set of mirrors. It offers a simple and economic way to create a Content Delivery Network layer using a pure software stack. It is primarily designed for the distribution of large-scale Open-Source projects with a lot of traffic.\n\n![mirrorbits_screenshot](https://cloud.githubusercontent.com/assets/38853/3636687/ab6bba38-0fd8-11e4-9d69-01543ed2531a.png)\n\n## Main Features\n\n* Blazing fast, can reach 8K QPS on a single laptop\n* Easy to deploy and maintain, everything is packed in a single binary\n* Automatic synchronization with the mirrors over **rsync** or **FTP**\n* Response can be either JSON or HTTP redirect\n* Support partial repositories\n* Complete checksum / size control\n* Realtime monitoring and reports\n* Disable misbehaving mirrors without human intervention\n* Realtime decision making based on location, AS number and defined rules\n* Smart load-balancing over multiple mirrors in the same area to avoid hotspots\n* Ability to adjust the weight of each mirror\n* Limit access to a country, region or ASN for any mirror\n* Clustering (multiple mirrorbits instances)\n* High-availability using redis-sentinel\n* Automatically fix timezone offsets for broken mirrors\n* Realtime statistics per file / mirror / date\n* Realtime reconfiguration\n* Seamless binary upgrade (aka zero downtime upgrade)\n* [Mirmon](http://www.staff.science.uu.nl/~penni101/mirmon/) support\n* Full **IPv6** support\n* If-Modified-Since (RFC-7232) support\n* more...\n\n## Is it production ready?\n\n**Yes!** Mirrorbits has served **billions** of files already and is known to be running in production at:\n* [CarbonROM](https://carbonrom.org/)\n* [Chaos Computer Club](https://media.ccc.de/) to distribute media\n* [Jellyfin](https://jellyfin.org/) since [April 2021](https://jellyfin.org/posts/mirrorbits-cdn/)\n* [Jenkins](https://www.jenkins.io/) to distribute Jenkins releases since [February 2020](https://github.com/jenkins-infra/docker-mirrorbits)\n* [Kali Linux](https://www.kali.org/) to distribute packages and images since [December 2023](https://www.kali.org/blog/kali-linux-2023-4-release/#enters-mirrorbits)\n* [Kodi](http://kodi.tv/) (previously XBMC) since [July 2015](https://forum.kodi.tv/showthread.php?tid=233824)\n* [LineageOS](http://lineageos.org/) (previously CyanogenMod) since January 2017\n* [MariaDB](https://mariadb.org/) to distribute packages (deb/rpm) for Linux distributions since [December 2021](https://mariadb.org/mirrorbits/)\n* [OSMC](https://osmc.tv)\n* [VideoLAN](http://www.videolan.org/) to distribute [VLC media player](http://www.videolan.org/vlc/) since [April 2014](https://blog.l0cal.com/2014/07/11/mirrorbits-is-now-on-github/)\n* [Endless OS](https://endlessos.org/os)\n* [MSYS2](https://www.msys2.org/) to distribute packages since [June 2021](https://github.com/msys2/msys2-main-server/commit/6a212b9ac76913f96)\n\nYet some things might change before the 1.0 release. If you intend to deploy Mirrorbits in a production system it is advised to notify the author first so we can help you to make any transition as seamless as possible!\n\n_Previous projects which have used Mirrorbits:_\n* [Parrot OS](https://www.parrotsec.org)\n* [Popcorn Time](https://popcorntime.io)\n* [SuperRepo](https://superrepo.org)\n\n# Quick start\n\n## Prerequisites\n\n* Go 1.11 or later\n* Protobuf (protoc)\n* Redis 3.2 or later (with [persistence](https://redis.io/topics/persistence) enabled)\n* GeoIP2 databases from [Maxmind](https://dev.maxmind.com/geoip/geoip2/geolite2/) (preferably updated regularly)\n\n:warning: **GeoIP-legacy is not supported anymore, please use the new GeoIP2 mmdb databases!**\n\n**Optional:**\n\n* redis-sentinel (for high-availability support)\n\n## Upgrading\n\nBefore upgrading to the latest version, please check [this guide](https://github.com/etix/mirrorbits/wiki/Upgrade-Guide).\n\n## Installation\n\nYou can either get a [prebuilt version](https://github.com/etix/mirrorbits/releases) or choose to build it yourself.\n\n### Docker\n\nA docker \"quick start\" can be found [on the wiki](https://github.com/etix/mirrorbits/wiki/Running-within-Docker).\n\n### Manual build\n\n\nGo \u003e= 1.11:\n```\n$ git clone https://github.com/etix/mirrorbits.git\n$ cd mirrorbits\n$ sudo make install\n```\n\nGo \u003c 1.11:\n```\n$ go get -u github.com/etix/mirrorbits\n$ cd $GOPATH/src/github.com/etix/mirrorbits\n$ sudo make install\n```\n\nThe resulting executable should now live in your */usr/local/bin* directory. You can also specify a `PREFIX` or `DESTDIR` if necessary:\n```\nsudo make install PREFIX=/usr\n```\n\n## Configuration\n\nA sample configuration file can be found [here](mirrorbits.conf).\n\n## Running\n\nMirrorbits is a self-contained application and can act, at the same time, as the server and the cli.\n\nTo run the server:\n```\nmirrorbits daemon\n```\nAdditional options can be found with ```mirrorbits -help```.\n\nTo run the cli:\n```\nmirrorbits help\n```\n\nAdd a mirror:\n```\nmirrorbits add -ftp=\"ftp://ftp.mirrors.example/myproject/\" -http=\"http://ftp.mirrors.example/myproject/\" mirrors.example\n```\n\nEnable the mirror:\n```\nmirrorbits enable mirrors.example\n```\n\n### Realtime file availability\n\nBy appending `?mirrorlist` to any file served by mirrorbits, you'll be able to get some useful realtime informations about the given file. You can see a [live example here](https://get.videolan.org/vlc/2.2.4/win32/vlc-2.2.4-win32.exe?mirrorlist).\n\n### Realtime mirrors statistics\n\nMirror statistics are available by querying mirrorbits with the `?mirrorstats` argument. You can see a [live example here](https://get.videolan.org/?mirrorstats).\n\n## Clustering / High availability\n\nMultiple instances of mirrorbits can be started simultaneously on different servers, discovery of other nodes should be automatic as long as all the instances are connected to the same redis server. In addition to the clustering it is advised to use redis-sentinel to monitor the database and gracefully handle failover.\n\n## Upgrading\n\nMirrorbits has a mode called *seamless binary upgrade* to upgrade the server executable at runtime without service disruption. Once the binary has been replaced on the filesystem just issue the following command in the cli:\n```\nmirrorbits upgrade\n```\n\n## Considerations\n\n* When configured in redirect mode, Mirrorbits can easily serve client requests directly but it is usually recommended to set it behind a reverse proxy like nginx. In this case take care to pass the IP address of the client within a X-Forwarded-For header:\n```\nproxy_set_header X-Forwarded-For $remote_addr;\n```\n* It is advised to never cache requests intended for Mirrorbits since each request is supposed to be unique, caching the result might have unexpected consequences.\n\n# We're social!\n\nThe best place to discuss about mirrorbits is to join the [#VideoLAN IRC channel on Libera.chat](https://www.videolan.org/webirc/).\nFor the latest news, you can follow [@mirrorbits](http://twitter.com/mirrorbits) on Twitter.\n\n# License MIT\n\n\u003e Permission is hereby granted, free of charge, to any person obtaining a copy\n\u003e of this software and associated documentation files (the \"Software\"), to deal\n\u003e in the Software without restriction, including without limitation the rights\n\u003e to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n\u003e copies of the Software, and to permit persons to whom the Software is\n\u003e furnished to do so, subject to the following conditions:\n\u003e\n\u003e The above copyright notice and this permission notice shall be included in\n\u003e all copies or substantial portions of the Software.\n\u003e\n\u003e THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n\u003e IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n\u003e FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n\u003e AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n\u003e LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n\u003e OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n\u003e THE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fetix%2Fmirrorbits","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fetix%2Fmirrorbits","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fetix%2Fmirrorbits/lists"}