{"id":13581707,"url":"https://github.com/gokrazy/rsync","last_synced_at":"2025-05-15T12:05:15.684Z","repository":{"id":38042930,"uuid":"398850671","full_name":"gokrazy/rsync","owner":"gokrazy","description":"rsync in Go! implements client and server, which can send or receive files (upload, download, all directions supported)","archived":false,"fork":false,"pushed_at":"2025-05-05T14:55:56.000Z","size":372,"stargazers_count":619,"open_issues_count":10,"forks_count":43,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-05T15:59:32.279Z","etag":null,"topics":["client","daemon","download","file-transfer","golang","network","protocol","receiver","rsync","sender","server","upload"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gokrazy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-08-22T16:47:40.000Z","updated_at":"2025-05-05T14:55:59.000Z","dependencies_parsed_at":"2024-06-18T18:40:48.156Z","dependency_job_id":"f1d9a257-ca63-4850-92ca-045a8bbea839","html_url":"https://github.com/gokrazy/rsync","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokrazy%2Frsync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokrazy%2Frsync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokrazy%2Frsync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gokrazy%2Frsync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gokrazy","download_url":"https://codeload.github.com/gokrazy/rsync/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337613,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["client","daemon","download","file-transfer","golang","network","protocol","receiver","rsync","sender","server","upload"],"created_at":"2024-08-01T15:02:11.655Z","updated_at":"2025-05-15T12:05:13.394Z","avatar_url":"https://github.com/gokrazy.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# gokrazy rsync\n\n[![tests](https://github.com/gokrazy/rsync/actions/workflows/main.yml/badge.svg)](https://github.com/gokrazy/rsync/actions/workflows/main.yml)\n[![Sourcegraph](https://sourcegraph.com/github.com/gokrazy/rsync/-/badge.svg)](https://sourcegraph.com/github.com/gokrazy/rsync??badge)\n\nThis repository contains a native Go rsync implementation: the `gokr-rsync`\ncommand implements an rsync client and server, which can send or receive files\n(all directions supported). Daemon mode is supported, meaning you can deploy\n`gokr-rsync` behind SSH (anonymous or authorized), as command or daemon, or\nlistening directly on the network (on port 873/tcp by default).\n\nThis project accepts contributions as time permits to merge them (best effort).\n\n## How do I know this project won’t eat my data?\n\nThis rsync implementation is not as well-tested as the original “tridge”\nimplementation from the Samba project. gokrazy/rsync was started in 2021 and\ndoesn’t have many users yet.\n\nWith that warning out of the way, the rsync protocol uses MD4 checksums over\nfile contents, so at least your file contents should never be able to be\ncorrupted.\n\nThere is enough other functionality (delta transfers, file metadata, special\nfiles like symlinks or devices, directory structures, etc.) in the rsync\nprotocol that provides opportunities for bugs to hide.\n\nI recommend you carefully check that your transfers work, and please do report\nany issues you run into!\n\n## Existing rsync implementation survey\n\n| Language | URL                                                                                 | Note                                                                                                                                  | Max Protocol                                                                                                        | Server mode? |\n|----------|-------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|--------------|\n| C        | [RsyncProject/rsync](https://github.com/RsyncProject/rsync) (formerly WayneD/rsync) | original “tridge” implementation; I found [older versions](https://github.com/WayneD/rsync/tree/v2.6.1pre2) easier to study           | [32](https://github.com/RsyncProject/rsync/blob/v3.4.1/rsync.h#L114)                                                | ✔ yes        |\n| C        | [kristapsdz/openrsync](https://github.com/kristapsdz/openrsync)                     | OpenBSD, good docs                                                                                                                    | [27](https://github.com/kristapsdz/openrsync/blob/e54d57f7572381da2b549d39c7968fc79dac8e1d/extern.h#L30)            | ✔ yes        |\n| **Go**   | [gokrazy/rsync](https://github.com/gokrazy/rsync)                                   | → you are here ←                                                                                                                      | [27](https://github.com/gokrazy/rsync/blob/b3b58770b864613551036a2ef2827b74ace77749/internal/rsyncd/rsyncd.go#L317) | ✔ yes 🎉     |\n| **Go**   | [jbreiding/rsync-go](https://github.com/jbreiding/rsync-go)                         | rsync algorithm                                                                                                                       |                                                                                                                     | ❌ no        |\n| **Go**   | [kaiakz/rsync-os](https://github.com/kaiakz/rsync-os)                               | only client/receiver                                                                                                                  | [27](https://github.com/kaiakz/rsync-os/blob/64e84daeabb1fa4d2c7cf766c196306adfba6cb2/rsync/const.go#L4)            | ❌ no        |\n| **Go**   | [knight42](https://gist.github.com/knight42/6ad35ce6fbf96519259b43a8c3f37478)       | proxy                                                                                                                                 |                                                                                                                     | ❌ no        |\n| **Go**   | [c4milo/gsync](https://github.com/c4milo/gsync)                                     |                                                                                                                                       |                                                                                                                     | ❌ no        |\n| Java     | [APNIC-net/repositoryd](https://github.com/APNIC-net/repositoryd)                   | archived                                                                                                                              |                                                                                                                     | ✔ yes        |\n| Java     | [JohannesBuchner/Jarsync](https://github.com/JohannesBuchner/Jarsync/)              | archived, [internet draft RFC “The rsync Network Protocol”](https://github.com/JohannesBuchner/Jarsync/blob/master/jarsync/rsync.txt) |                                                                                                                     | ✔ yes        |\n| Java     | [perlundq/yajsync](https://github.com/perlundq/yajsync#example)                     |                                                                                                                                       |                                                                                                                     | ✔ yes        |\n| C++      | [gilbertchen/acrosync-library](https://github.com/gilbertchen/acrosync-library)     | commercial                                                                                                                            |                                                                                                                     | ❌ no        |\n| Rust     | [sourcefrog/rsyn](https://github.com/sourcefrog/rsyn#why-do-this)                   | archived, client, “rsyn is rsync with no c”                                                                                                     | [27](https://github.com/sourcefrog/rsyn/blob/2ebbfcfe999fdf2d1a434d8614d07aa93873461b/src/connection.rs#L38)        | ❌ no        |\n\n## Getting started\n\nTo serve the `/usr/share/man` directory via rsync on `localhost:8730`, use:\n\n```\ngo install github.com/gokrazy/rsync/cmd/gokr-rsync@latest\ngokr-rsync --daemon --gokr.listen=localhost:8730 --gokr.modulemap=man=/usr/share/man\n```\n\nYou can then copy the contents of the current directory with clients such as\n`rsync(1)`:\n\n```\n% rsync -v --archive --port 8730 rsync://localhost/man/ man\nreceiving file list ... done\ncreated directory man\n./\nar/\nar/man1/\n[…]\nzh_TW/man8/userdel.8.gz\nzh_TW/man8/usermod.8.gz\n\nsent 658.973 bytes  received 88.012.067 bytes  3.940.935,11 bytes/sec\ntotal size is 84.504.170  speedup is 0,95\n```\n\n…or [`openrsync(1)`](https://github.com/kristapsdz/openrsync), shown doing a\ndifferential update:\n\n```\n% openrsync -v --archive --port 8730 rsync://localhost/man/ man\nopenrsync: warning: connect refused: ::1, localhost\nTransfer starting: 40202 files\n[…]\nzh_TW/man8/userdel.8.gz (732 B, 100.0% downloaded)\nzh_TW/man8/usermod.8.gz (1.8 KB, 100.0% downloaded)\nTransfer complete: 83.93 MB sent, 643.5 KB read, 80.59 MB file size\n```\n\n## Usage / Setup\n\n | setup                                   | encrypted | authenticated      | private files?         | privileges                                                      | protocol version | config required                       |\n |-----------------------------------------|-----------|--------------------|------------------------|-----------------------------------------------------------------|------------------|---------------------------------------|\n | 1. rsync daemon protocol (TCP port 873) | ❌ no     | ⚠ rsync (insecure) | ❌ only world-readable | ✔ dropped + [namespace](#privileged-linux-including-gokrazyorg) | ✔ negotiated     | config required                       |\n | 2. anon SSH (daemon)                    | ✔ yes     | ✔ rsync            | ❌ only world-readable | ✔ dropped + [namespace](#privileged-linux-including-gokrazyorg) | ✔ negotiated     | config required                       |\n | 3. SSH (command)                        | ✔ yes     | ✔ SSH              | ✔ yes                  | ⚠ full user                                                     | ⚠ assumed       | no config                             |\n | 4. SSH (daemon)                         | ✔ yes     | ✔ SSH (+ rsync)    | ✔ yes                  | ⚠ full user                                                     | ✔ negotiated     | `~/.config/gokr-rsyncd.toml` required |\n\nRegarding protocol version “assumed”: the flags to send over the network are\ncomputed *before* starting SSH and hence the remote rsync process. You might\nneed to specify `--protocol=27` explicitly on the client. Once the connection is\nestablished, both sides *do* negotiate the protocol, though.\n\n### Setup 1: rsync daemon protocol (TCP port 873)\n\nServing rsync daemon protocol on TCP port 873 is only safe where the network\nlayer ensures trusted communication, e.g. in a local network (LAN), or when\nusing [Tailscale](https://tailscale.com/) or similar. In untrusted networks,\nattackers can eavesdrop on file transfers and possibly even modify file\ncontents.\n\nPrefer setup 2 instead.\n\nExample:\n* Server: `gokr-rsync --daemon --gokr.modulemap=module=/srv/rsync-module`\n* Client: `rsync rsync://webserver/module/path`\n\n### Setup 2: anon SSH (daemon)\n\nThis setup is well suited for serving world-readable files without\nauthentication.\n\nExample:\n* Server: `gokr-rsync --daemon --gokr.modulemap=module=/srv/rsync-module --gokr.anonssh_listen=:22873`\n* Client: `rsync -e ssh rsync://webserver/module/path`\n\n\n### Setup 3: SSH (command)\n\nThis setup is well suited for interactive one-off transfers or regular backups,\nand uses SSH for both encryption and authentication.\n\nNote that because `gokr-rsync` is invoked with user privileges (not root\nprivileges), it cannot do [namespacing](#privileged-linux-including-gokrazyorg)\nand hence retains more privileges. When serving public data, it is generally\npreferable to use setup 2 instead.\n\nNote that `rsync(1)` assumes the server process understands all flags that it\nsends, i.e. is running the same version on client and server, or at least a\ncompatible-enough version. You can either specify `--protocol=27` on the client,\nor use setup 4, which negotiates the protocol version, side-stepping possible\ncompatibility gaps between rsync clients and `gokr-rsync`.\n\nExample:\n* Server will be started via SSH\n* Client: `rsync --rsync-path=gokr-rsync webserver:path`\n\n### Setup 4: SSH (daemon)\n\nThis setup is more reliable than setup 3 because the rsync protocol version will\nbe negotiated between client and server. This setup is slightly inconvenient\nbecause it requires a config file to be present on the server in\n`~/.config/gokr-rsyncd.toml`.\n\nNote that this mode of operation is only implemented by the original “trigde”\nrsync, not in openrsync. Apple started shipping openrsync with macOS 15 Sequoia,\nso you might need to explicitly start /usr/libexec/rsync/rsync.samba on Macs.\n\nExample:\n* Server will be started via SSH\n* Client: `rsync -e ssh --rsync-path=gokr-rsync rsync://webserver/module/path`\n\n## Limitations\n\n### Bandwidth\n\nIn my tests, `gokr-rsync` can easily transfer data at \u003e 6 Gbit/s. The current\nbottleneck is the MD4 algorithm itself (not sure whether in the “tridge” rsync\nclient, or in `gokr-rsync`). Implementing support for more recent protocol\nversions would help here, as these include hash algorithm negotiation with more\nrecent choices.\n\n### Protocol related limitations\n\n* xattrs (including acls) was introduced in rsync protocol 30, so is currently\n  not supported.\n\n## Supported environments and privilege dropping\n\nSupported environments:\n\n1. systemd (Linux)\n1. Docker (Linux)\n1. privileged Linux\n1. privileged non-Linux\n\nIn all environments, the default instructions will take care that:\n\n* (On Linux only) Only configured rsync modules from the host file system are\n  mounted **read-only** into a Linux mount namespace for `gokr-rsync`, to guard\n  against data modification and data exfiltration.\n* (On Linux only) File system access is restricted using the\n  [Landlock](https://docs.kernel.org/userspace-api/landlock.html) Linux kernel\n  security module, which works similar to OpenBSD’s\n  [`unveil(2)`](https://man.openbsd.org/unveil.2) API.\n* `gokr-rsync` is running without privileges, as user `nobody`, to limit the\n  scope of what an attacker can do when exploiting a vulnerability.\n\nKnown gaps:\n\n* `gokr-rsync` does not guard against denial of service attacks, i.e. consuming\n  too many resources (connections, bandwidth, CPU, …).\n  * See also [Per-IP rate limiting with\n    iptables](https://making.pusher.com/per-ip-rate-limiting-with-iptables/).\n\n\n### systemd (unprivileged)\n\nWe provide [a `gokr-rsyncd.socket` and `gokr-rsyncd.service`\nfile](https://github.com/gokrazy/rsync/tree/main/systemd/) for systemd. These\nfiles enables most of systemd’s security features. You can check by running\n`systemd-analyze security gokr-rsyncd.service`, which should result in an\nexposure level of “0.2 SAFE” as of systemd 249 (September 2021).\n\nFirst, configure your server flags by creating a systemd service override file:\n\n```shell\nsystemctl edit gokr-rsyncd.service\n```\n\nIn the opened editor, change the file to:\n```\n[Service]\nExecStart=\nExecStart=/usr/bin/gokr-rsync --daemon --gokr.modulemap=pwd=/etc/tmpfiles.d\n```\n\nClose the editor and install the service using:\n\n```shell\nsystemctl enable --now gokr-rsyncd.socket\n```\n\nAdditional hardening recommendations:\n\n* Restrict which IP addresses are allowed to connect to your rsync server, for example:\n  * using iptables or nftables on your host system\n  * using [`gokr-rsync`’s built-in IP allow/deny mechanism](https://github.com/gokrazy/rsync/issues/4) (once implemented)\n  * using [systemd’s `IPAddressDeny` and `IPAddressAllow`](https://manpages.debian.org/systemd.resource-control.5) in `gokr-rsyncd.socket`\n* To reduce the impact of Denial Of Service attacks, you can restrict resources\n  with systemd, see [Managing\n  Resources](http://0pointer.de/blog/projects/resources.html).\n* To hide system directories not relevant to any rsync module, use [systemd’s\n  `TemporaryFileSystem=` and\n  `BindReadOnlyPaths=`](https://manpages.debian.org/systemd.exec.5) directives\n  as described in [Use TemporaryFileSystem to hide files or directories from\n  systemd\n  services](https://www.sherbers.de/use-temporaryfilesystem-to-hide-files-or-directories-from-systemd-services/). Note\n  that you [may need to disable `ProtectSystem=strict` due to a\n  bug](https://github.com/systemd/systemd/issues/18999).\n\n### Docker (unprivileged)\n\nWe provide [a `Dockerfile` for\n`gokr-rsyncd`](https://github.com/gokrazy/rsync/tree/main/docker/).\n\n```shell\ndocker run \\\n  --read-only \\\n  -p 127.0.0.1:8730:8730 \\\n  -v /etc/tmpfiles.d:/srv/rsync:ro \\\n  stapelberg/gokrazy-rsync:latest \\\n    --gokr.modulemap=pwd=/srv/rsync\n```\n\nAdditional hardening recommendations:\n\n* Restrict which IP addresses are allowed to connect to your rsync server, for example:\n  * using iptables or nftables on your host system\n  * using [`gokr-rsync`’s built-in IP allow/deny mechanism](https://github.com/gokrazy/rsync/issues/4) (once implemented)\n    * Be sure to set up Docker such that the remote IPv4 or IPv6 address is available inside the container, see https://michael.stapelberg.ch/posts/2018-12-12-docker-ipv6/\n\n### privileged Linux (including gokrazy.org)\n\nWhen started as `root` on Linux, `gokr-rsync` will create a [Linux mount\nnamespace](https://manpages.debian.org/mount_namespaces.7), mount all configured\nrsync modules read-only into the namespace, then change into the namespace using\n[`chroot(2)`](https://manpages.debian.org/chroot.2) and drop privileges using\n[`setuid(2)`](https://manpages.debian.org/setuid.2).\n\n**Tip:** you can verify which file system objects the daemon process can see by\nusing `ls -l /proc/$(pidof gokr-rsync)/root/`.\n\nAdditional hardening recommendations:\n\n* Restrict which IP addresses are allowed to connect to your rsync server, for example:\n  * using iptables or nftables on your host system\n  * using [`gokr-rsync`’s built-in IP allow/deny mechanism](https://github.com/gokrazy/rsync/issues/4) (once implemented)\n\n### privileged non-Linux (e.g. Mac)\n\nWhen started as `root` on non-Linux (e.g. Mac), `gokr-rsync` will drop\nprivileges using [`setuid(2)`](https://manpages.debian.org/setuid.2).\n\n### unprivileged with write permission (e.g. from a shell)\n\nTo prevent accidental misconfiguration, `gokr-rsync` refuses to start when it\ndetects that it has write permission in any configured rsync module.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgokrazy%2Frsync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgokrazy%2Frsync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgokrazy%2Frsync/lists"}