{"id":37177289,"url":"https://github.com/aledbf/clusterf","last_synced_at":"2026-01-14T20:40:37.909Z","repository":{"id":144291350,"uuid":"48038298","full_name":"aledbf/clusterf","owner":"aledbf","description":"Clustered IPVS load balancer control plane with Docker integration","archived":false,"fork":true,"pushed_at":"2015-12-03T13:07:21.000Z","size":117,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-14T16:08:59.118Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"qmsk/clusterf","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aledbf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-12-15T11:20:08.000Z","updated_at":"2015-12-15T11:20:09.000Z","dependencies_parsed_at":"2023-03-21T22:14:12.151Z","dependency_job_id":null,"html_url":"https://github.com/aledbf/clusterf","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/aledbf/clusterf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aledbf%2Fclusterf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aledbf%2Fclusterf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aledbf%2Fclusterf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aledbf%2Fclusterf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aledbf","download_url":"https://codeload.github.com/aledbf/clusterf/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aledbf%2Fclusterf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28434488,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T18:57:19.464Z","status":"ssl_error","status_checked_at":"2026-01-14T18:52:48.501Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T20:40:37.210Z","updated_at":"2026-01-14T20:40:37.898Z","avatar_url":"https://github.com/aledbf.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cluster Frontend\n\n`clusterf` is a clustered L3 loadbalancer control plane.\nIt uses [CoreOS etcd](https://github.com/coreos/etcd) as a configuration backend, supports the Linux built-in [IPVS](http://www.linuxvirtualserver.org/software/ipvs.html) TCP/UDP load balancer, and provides [Docker](https://www.docker.com/) integration.\n\nThe `clusterf-docker` daemon runs on the docker hosts, and enumerates the Docker API for running containers to synchronizes any labeled services into the etcd `/clusterf` configuration store. The daemon continues to listen for Docker events to update any container state changes to the backend configurations, adding and removing backends as existing containers go away or new containers are started.\n\nThe `clusterf-ipvs` daemon runs on the cluster frontend hosts with external connectivity, and enumerates configured service frontend+backends from the etcd `/clusterf` configuration store to synchronizes the in-kernel IPVS configuration. The daemon continues to watch for etcd changes to update the live IPVS service state.\n\nThe system essentially acts as a L4-aware L3 routed network, routing packets at the L3 layer based on L4 information.\n\n## Highlights\n\nThe use of `etcd` as a distributed share configuration backend allows the seamless operation of multiple `clusterf-docker` hosts and multiple `clusterf-ipvs` hosts, with changes to service state on backend nodes being immediately propagated to all frontend nodes.\n\nIn terms of performance, the `clusterf` daemons act as a control-plane only: the actual packet-handling data plane is implemented by the IPVS code inside the Linux kernel, and forwarded packets do not need to pass through user-space.\n\n## Example\n\n    $ sudo ipvsadm\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n      \n    $ etcdctl set /clusterf/services/test/frontend '{\"ipv4\": \"10.107.107.107\", \"tcp\": 1337}'\n    {\"ipv4\": \"10.107.107.107\", \"tcp\": 1337}\n    $ sudo ipvsadm\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n    TCP  10.107.107.107:1337 wlc\n\n    $ etcdctl set /clusterf/services/test/backends/test3-1 '{\"ipv4\": \"10.3.107.1\", \"tcp\": 1337}'\n    {\"ipv4\": \"10.3.107.1\", \"tcp\": 1337}\n    $ sudo ipvsadm\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n    TCP  10.107.107.107:1337 wlc\n      -\u003e 10.3.107.1:1337              Masq    10     0          0         \n\n    $ etcdctl set /clusterf/services/test/backends/test3-2 '{\"ipv4\": \"10.3.107.2\", \"tcp\": 1337}'\n    {\"ipv4\": \"10.3.107.2\", \"tcp\": 1337}\n    $ sudo ipvsadm -L -n\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n    TCP  10.107.107.107:1337 wlc\n      -\u003e 10.3.107.1:1337              Masq    10     0          0         \n      -\u003e 10.3.107.2:1337              Masq    10     0          0         \n\n    $ etcdctl set /clusterf/services/test/backends/test3-2 '{\"ipv4\": \"10.3.107.2\", \"tcp\": 1338}'\n    {\"ipv4\": \"10.3.107.2\", \"tcp\": 1338}\n    $ sudo ipvsadm -L -n\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n    TCP  10.107.107.107:1337 wlc\n      -\u003e 10.3.107.1:1337              Masq    10     0          0         \n      -\u003e 10.3.107.2:1338              Masq    10     0          0         \n\n    $ etcdctl rm --recursive /clusterf/services/test\n    $ sudo ipvsadm -L -n\n    IP Virtual Server version 1.2.1 (size=4096)\n    Prot LocalAddress:Port Scheduler Flags\n      -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\n\n## Docker integration\n\nThe docker integration uses container (image) labels:\n\n    net.qmsk.clusterf.service=$service\n    net.qmsk.clusterf.backend.tcp=$port\n    net.qmsk.clusterf.backend.udp=$port\n\nA container can also be a backend in multiple different services:\n\n    net.qmsk.clusterf.service=\"$service1 $service2\"\n    net.qmsk.clusterf.backend:$service.tcp=$port\n    net.qmsk.clusterf.backend:$service.udp=$port\n\nAs an example:\n\n    docker run --rm -it --expose 8080 -l net.qmsk.clusterf.service=test -l net.qmsk.clusterf.backend.tcp=8080 ...\n\nThe ports must be EXPOSE'd on the container, but do not necessarily need to be published. The backend will be configured using the internal address of the container.\n\n## Additional features\n\n### Local configuration\n\nThe `clusterf-ipvs` daemon supports a local filesystem `-config-path=` configuration tree which is loaded in addition to the configuration in etcd.\n\n### Forwarding configuration\n\nThe forwarding method for IPVS destinations can be configured in aggregate for different sets of backends via `/clusterf/routes/...`, using IPv4 address *prefix* information to represent the network topology:\n\n    $ etcdctl get /clusterf/routes/test3\n    {\"Prefix4\":\"10.3.107.0/24\",\"IpvsMethod\":\"masq\"}\n\nThis means that any backends configured under `10.3.107.0/24` will be configured with an IPVS *masq* forwarding-method.\n\n\n### Routed backends\n\nThe `clusterf` code additionally supports the use of *routed backends*, to redirect traffic to a set of backends via some intermediate *gateway*:\n\n    {\"Prefix4\":\"10.6.107.0/24\",Gateway4\":\"10.107.107.6\",\"IpvsMethod\":\"droute\"}\n\nThe backend's IPVS dest will be added using the given *gateway* address (retaining the service's frontend port) in place of the dest's *host:port* address.\n\nThis feature enables the separaration of the IPVS traffic handling into two tiers: a scaleable and fault-tolerant stateless frontend tier using IPVS `droute` forwarding, plus a simple-to-configure stateful intermediate tier using IPVS `masq` forwarding.\n\nThe `-filter-etcd-routes` can be used to override any routes in etcd on the intermediate tier, which can be used to limit IPVS destinations to local backends only.\n\nThe `-advertise-route-*` flags can be used to advertise a route for local backends into etcd for the frontend tier.\n\n### Weighted backends\n\nEach backend can define its own weight, which can be updated at runtime. Backends with a higher weight will recieve proportionally more connections.\n\nA backend weight of zero will prevent new connections being scheduled for the backend, allowing existing connections to continue.\n\n### Backend merging\n\nOverlapping backends are merged. This will happen if multiple backends for a given service resolve to the same IPVS host:port, typically as a result of a route aggregating a set of backends to an intermediate frontend.\n\nThe merging is based on the backend weight. The IPVS weight of the merged destination is calculated from the weights of all merged backends, and updated as backends are added/removed/reweighted.\n\n## Known issues\n\n*   Dead service backends are not cleaned up.\n    The `clusterf-docker` daemon will remove any containers that are stopped, but a dead `clusterf-docker` daemon or docker host will result in\n    ghost backends in etcd.\n*   Route updates are not propagated to backends.\n    Adding/Updating/Removing a route will not be reflected in the IPVS state until the service backends are updated.\n*   The `clusterf-docker` daemon is limited in terms of the policy configuration available.\n    It needs support for different network topologies, such as Docker's traditional \"published\" NAT ports.\n*   Hairpinning to allow access to local backends from docker containers on the same host requires some work to deal with asymmetric routing on the docker host bridge.\n*   IPv6 configuration is partially supported in the `clusterf-ipvs` code, but untested. The `clusterf-docker` code is lacking IPv6 configuration support.\n\n## Future ideas\n\n*   Implement a docker networking extension to configure the public VIP directly within the docker container.\n    Removes the need for DNAT on the docker host, as forwaded traffic can be routed directly to the container.\n\n## Acknowledgments\n\nThis work was supported by the Academy of Finland project [\"Cloud Security Services\" (CloSe)](https://wiki.aalto.fi/display/CloSeProject/CloSe+Project+Public+Homepage) at Aalto University Department of Communications and Networking.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faledbf%2Fclusterf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faledbf%2Fclusterf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faledbf%2Fclusterf/lists"}