{"id":14235585,"url":"https://github.com/nroi/flexo","last_synced_at":"2025-12-27T12:59:18.841Z","repository":{"id":43074289,"uuid":"230621137","full_name":"nroi/flexo","owner":"nroi","description":"a central pacman cache","archived":false,"fork":false,"pushed_at":"2024-10-17T18:33:57.000Z","size":12755,"stargazers_count":180,"open_issues_count":19,"forks_count":10,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-10-20T04:08:11.634Z","etag":null,"topics":["arch-linux","arch-linux-packages","archlinux","bandwidth","cache","http","mirror","pacman"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nroi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-28T14:31:56.000Z","updated_at":"2024-10-18T10:53:30.000Z","dependencies_parsed_at":"2024-10-19T23:34:12.035Z","dependency_job_id":"581c085b-86e2-4944-94ff-624b976d61f9","html_url":"https://github.com/nroi/flexo","commit_stats":null,"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nroi%2Fflexo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nroi%2Fflexo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nroi%2Fflexo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nroi%2Fflexo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nroi","download_url":"https://codeload.github.com/nroi/flexo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229478679,"owners_count":18079372,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arch-linux","arch-linux-packages","archlinux","bandwidth","cache","http","mirror","pacman"],"created_at":"2024-08-20T21:02:07.465Z","updated_at":"2025-12-27T12:59:18.828Z","avatar_url":"https://github.com/nroi.png","language":"Rust","readme":"# Flexo \u003cimg align=\"right\" width=\"80\" src=\"flexo_beard.svg\" alt=\"Flexo logo\"\u003e\n\nFlexo is a caching proxy for pacman, the package manager of Arch Linux.\n\n## Why should I use it?\n\n* If you're bothered by slow mirrors: Instead of manually maintaining a `mirrorlist`, Flexo automatically chooses\na low-latency mirror for you and switches to another mirror if the selected mirror turns out to be slow.\nIn addition, Flexo uses multiple mirrors for parallel downloads, which can increase download speeds substantially\n  if you use Pacman's `ParallelDownloads`   setting.\n* If you have multiple machines running ArchLinux, and you don't want each machine to download\nand store the packages: You can just set Flexo as your new ArchLinux mirror so that no file needs\nto be downloaded more often than once.\n* If you run ArchLinux inside Docker, you may be annoyed when packages have to be downloaded and installed on the container even though they have already been downloaded on the host: Just install Flexo on the host and run this command on the Docker container:\n    ````\n    echo 'Server = http://172.17.0.1:7878/$repo/os/$arch' \u003e /etc/pacman.d/mirrorlist\n    ````\n  so that packages that have already been downloaded will be fetched from the cache.\n\n## Installation\nA package for Arch Linux is available on [AUR](https://aur.archlinux.org/packages/flexo-git/).\nAlternatively, you can use the [docker image](https://hub.docker.com/r/nroi/flexo) instead.\nFlexo needs to be installed on a single machine (the server) so that it can be accessed by\nmultiple clients.\nOnce you have installed Flexo on the server, start and enable the systemd service:\n```\nsystemctl enable --now flexo.service\n```\nNext, set the new mirror in `/etc/pacman.d/mirrorlist` on all clients.\nIn most cases, the server that runs Flexo will also be a client that uses Flexo, so\nadd the following entry to the top of your mirrorlist:\n```bash\nServer = http://localhost:7878/$repo/os/$arch\n```\nIf you have additional ArchLinux clients in your LAN, proceed as follows to download packages from Flexo:\n1. Set `listen_ip_address` to `0.0.0.0` in `/etc/flexo/flexo.toml` (alternatively, you can put a reverse proxy in front of Flexo).\n2. Modify the mirrorlist files of all clients on your LAN.\n    Instead of referring to localhost, use the appropriate IP address or hostname:\n    ```bash\n    Server = http://\u003cFLEXO_SERVER_IP_ADDRESS\u003e:7878/$repo/os/$arch\n    ```\n\nNotice that if you start Flexo for the first time, it will run latency tests to select\nfast mirrors, which will take half a minute or so. During that time, Flexo is not available\nto serve any requests. Subsequent starts will be faster.\n\n## Features\n\n* Concurrent downloads: You can have multiple clients downloading files from Flexo without one client having to wait.\n* Efficient bandwidth sharing for concurrent downloads: Flexo does not require a new connection to the remote mirror\n  when the same file is downloaded by multiple clients. For instance, suppose a client starts downloading a given file.\n  After 5 seconds have elapsed, 100 MB have been downloaded. Now, a second client requests the same file. The second\n  client will receive the first 100 MB immediately from the local file system. Then, both clients continue to download\n  the file, while only a single connection to the remote mirror exists. This means that your bandwidth is not split for\n  the two clients, both clients will be able to download the file with the full download speed provided by your ISP.\n* Persistent connections: This is especially useful when many small files are downloaded, since no new TLS negotiation\n  is required for each file.\n* The package cache is cleaned automatically: No need to set up cron jobs or systemd timers to clean the cache\n  regularly, Flexo will automatically ensure that only the 3 most recent versions of a package are kept in your cache\n  (this parameter can be changed).\n\n## Configuration\n\nThe AUR package will install the configuration file in `/etc/flexo/flexo.toml`.\nIt includes many comments and should be self-explanatory (open an issue in case you disagree).\nIf you use Docker, the settings can be modified with environment variables. Environment variables\nare prefixed with `FLEXO_`, for example, `listen_ip_address` corresponds to the `FLEXO_LISTEN_IP_ADDRESS`\nenvironment variable. More details about configuring Flexo are listed [in the wiki](https://github.com/nroi/flexo/wiki/Flexo-with-Docker).\n\nIn most cases, you will want to leave all settings unchanged, with the following exceptions:\n\n1. The setting `low_speed_limit` is commented by default, which means that Flexo will *not* attempt\nto switch to a faster mirror if a download is extremely slow. To make use of this feature,\nuncomment the setting and enter an appropriate value. If you use Docker, use the `FLEXO_LOW_SPEED_LIMIT`\nenvironment variable.\n\n2. The setting `allowed_countries` is set to the empty list by default, which means that at the first start and at\n   regular intervals, Flexo will run latency tests on all official mirrors from all continents. Add the ISO code\n   of your own country (and perhaps a few neighboring countries) to improve the startup time of Flexo.\n   If you use Docker, use the `FLEXO_MIRRORS_AUTO_ALLOWED_COUNTRIES` environment variable. Multiple countries can be separated by\n   comma, for example, `FLEXO_MIRRORS_AUTO_ALLOWED_COUNTRIES=DE,AT,CZ`\n\n3. If you have additional ArchLinux clients in your LAN, you probably want to change the\n   `listen_ip_address` setting. It's set to `127.0.0.1` by default for security reasons, change it\n   to `0.0.0.0` to make Flexo accessible to all clients in your LAN.\n\nIn addition, if you have a high-bandwidth connection, you may want to consider enabling Pacman's\n[ParallelDownloads](https://wiki.archlinux.org/title/Pacman#Enabling_parallel_downloads)\nsetting.\nWith `ParallelDownloads` enabled, Flexo will receive multiple requests concurrently and therefore\nfetch the packages from multiple mirrors in parallel, thus making it more likely that your entire bandwidth\nis utilized.\n\n## Supported Platforms\n\nFlexo obviously runs on a standard Arch Linux x86_64 platform. In addition, it runs on many ARM\nplatforms (more details [here](#arm-support)).\n\nIf you intend to compile and run Flexo on any operating system other than ArchLinux (and without\nFlexo's Docker image), please note that:\n* Flexo requires either [paccache](https://man.archlinux.org/man/paccache.8) or\n  [scruffy](https://github.com/nroi/scruffy) to clean the cache. On any system other than ArchLinux,\n  installing scruffy will probably be easier than installing paccache.\n* Linux is the only supported operating system at the moment.\n\n## Troubleshooting\n\nIf Flexo does not start at all or crashes, check the logs first:\n```bash\njournalctl --unit=flexo\n```\nIf that does not help you, please open an issue. The following information may be helpful to troubleshoot your issue:\n1. An excerpt of that log, if Flexo has crashed or did not start.\n2. Your installation method (Docker or AUR).\n3. The version you are using (either the output of `pacman -Qi flexo`, or the tag if you are using Docker).\n4. Your settings, if you have changed them\n   (either the `/etc/flexo/flexo.toml` file, or the environment variables if you use Docker).\n5. If the issue is related to the mirror selection, it might also help if you include the country you are located in\nand the `/var/cache/flexo/state/latency_test_results.json` file, if it exists.\n\nFor issues related to the mirror selection, also see [this page](./mirror_selection.md) for more details.\n\n## Cleaning the package cache\n\nThe default configuration of Flexo will keep 3 versions of a package in cache: After a 4th version of a package has been\ndownloaded, the oldest version will be automatically removed. This setting can be changed with the `num_versions_retain`\nparameter. See the [configuration example](./flexo/conf/flexo.toml) for more details.\n\nIf you use Docker, the default behavior can be changed with the `FLEXO_NUM_VERSIONS_RETAIN` environment variable.\n\nIf you want to disable this setting and never purge the cache, set the parameter to `0`.\n\n## Using Unofficial User Repositories\n\nIf you are using [unofficial user repositories](https://wiki.archlinux.org/index.php/Unofficial_user_repositories)\nand you want Flexo to cache packages from those repositories, both `pacman.conf` and `flexo.toml`\nneed to include the custom repository. For example, suppose that Flexo is running on localhost, port 7878,\nand you want to add two custom repositories: archzfs and eschwartz. First, adapt your `/etc/pacman.conf` to include\nboth repositories. Notice that the path must start with `custom_repo/\u003crepo-name\u003e`:\n\n```\n[archzfs]\nServer = http://localhost:7878/custom_repo/archzfs/$repo/$arch\n\n[eschwartz]\nServer = http://localhost:7878/custom_repo/eschwartz/~eschwartz/repo/$arch\n```\nNext, add the corresponding entries to your `/etc/flexo/flexo.toml` before the `[mirrors_auto]` section:\n\n```toml\n[[custom_repo]]\nname = \"archzfs\"\nurl = \"https://archzfs.com\"\n\n[[custom_repo]]\nname = \"eschwartz\"\nurl = \"https://pkgbuild.com\"\n```\n\nNotice that the names (in this case `archzfs` and `eschwartz`) must match the path component right after\nthe `/custom_repo` in `pacman.conf`: So if your `pacman.conf` includes a repo with the path `/custom_repo/foo`,\nthen your `flexo.toml` must include a matching `[[custom_repo]]` entry with `name = \"foo\"`.\n\nAlternatively, if you use Docker, set the environment variable instead of modifying the `flexo.toml` file:\n```bash\nFLEXO_CUSTOM_REPO=\"eschwartz@https://pkgbuild.com archzfs@https://archzfs.com\"\n```\n\n## ARM support\n\n### Running Flexo on ARM devices\nFlexo can be built on various ARM platforms, including the Raspberry Pi. So far, no problems have been reported with\nbuilding and running Flexo on ARM. If you run into problems, please open an issue.\n\nOptions to use Flexo on ARM devices include:\n* Build and install the [AUR](https://aur.archlinux.org/packages/flexo-git/) package on your ARM device (recommended).\n* Build and run the Docker image on your ARM device.\n* Cross-compile Flexo on an x86-device and move the binary to the ARM device. Requires [cross](https://github.com/rust-embedded/cross). Also requires adding openssl as a vendored dependency: Add `openssl = { version = \"0.10\", features = [\"vendored\"] }` to the `[dependencies]` section in `Cargo.toml`.\n\n### Serving packages for ARM clients\nWith its default configuration, Flexo only serves packages from the official ArchLinux mirrors, which means packages\nbuilt for x86. However, we can configure an ARM mirror as a `custom_repo` in order to fetch ARM packages from Flexo.\n\nFirst, visit https://archlinuxarm.org/about/mirrors and choose a mirror. Once you have decided for an ARM mirror,\nconfigure it as a `custom_repo` in your `/etc/flexo.toml`. In this example, we have chosen the mirror\n`de3.mirror.archlinuxarm.org` and we have given it the name `arm`:\n\n```toml\n[[custom_repo]]\nname = \"arm\"\nurl = \"https://de3.mirror.archlinuxarm.org\"\n```\n\nNext, configure the mirrorlist on all clients that are going to fetch ARM packages from this server. For example,\nif the server that runs Flexo should fetch the package from Flexo, configure your `/etc/pacman.d/mirrorlist` as follows:\n```\nServer = http://localhost:7878/custom_repo/arm/$arch/$repo\n```\n\n## Attributes \u0026 Design Goals\n* Lightweight: Flexo is a single binary with less than 3 MB and a low memory footprint.\n* Robust: As long as *most* mirrors work fine, Flexo should be able to handle the download process\n  without the client noticing any issues or interruptions, even if remote mirrors are slow or connections\n  are unexpectedly dropped.\n* Simple: Users should not require more than a few minutes to set up Flexo and understand what it does.\n\n\n## Contribute\n\nIf you know Rust, feel free to dive into the code base and send a PR. Smaller improvements\nto make the code base cleaner, more idiomatic or efficient are always welcome. Before submitting\nlarger changes, including new features or design changes, you should first open an issue to see\nif that feature is desired and if it fits into the design goals of Flexo.\n\nOther than code, you can contribute by submitting feedback. One aspect of Flexo where feedback is particularly\nvaluable is the mirror selection process. If you notice that downloads are too slow because the selected mirrors\nare not fast, please open an issue. You can determine the primary mirror chosen by Flexo with the journal:\n\n```bash\njournalctl --since '7 days ago' --unit=flexo | grep 'Primary mirror'\n```\n\n## Development\n\nDetails about design decisions, and the terminology used in the code,\nare described [here](flexo/terminology.md).\n\nBefore submitting a PR, please run `cargo test` inside the `flexo` directory to make sure that all tests pass.\n","funding_links":[],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnroi%2Fflexo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnroi%2Fflexo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnroi%2Fflexo/lists"}