{"id":13508487,"url":"https://github.com/facebookincubator/katran","last_synced_at":"2025-05-13T16:08:13.120Z","repository":{"id":37493003,"uuid":"131347954","full_name":"facebookincubator/katran","owner":"facebookincubator","description":"A high performance layer 4 load balancer","archived":false,"fork":false,"pushed_at":"2025-05-09T13:13:04.000Z","size":9225,"stargazers_count":4940,"open_issues_count":0,"forks_count":512,"subscribers_count":202,"default_branch":"main","last_synced_at":"2025-05-09T14:29:01.540Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookincubator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-04-27T21:49:35.000Z","updated_at":"2025-05-09T13:13:08.000Z","dependencies_parsed_at":"2023-10-23T19:31:16.793Z","dependency_job_id":"b4928189-7431-4d0c-b095-25df9b581726","html_url":"https://github.com/facebookincubator/katran","commit_stats":{"total_commits":4966,"total_committers":214,"mean_commits":"23.205607476635514","dds":0.441401530406766,"last_synced_commit":"c71c0455ebedb054051633edeb7689e168debd34"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookincubator%2Fkatran","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookincubator%2Fkatran/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookincubator%2Fkatran/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookincubator%2Fkatran/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookincubator","download_url":"https://codeload.github.com/facebookincubator/katran/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253980052,"owners_count":21994042,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:00:53.795Z","updated_at":"2025-05-13T16:08:13.078Z","avatar_url":"https://github.com/facebookincubator.png","language":"C","funding_links":[],"categories":["C","others","Projects","C++","NetWork","Load Balancing \u0026 Ingress"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"logo.png\" alt=\"logo\" width=\"50%\"/\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://opensource.fb.com/support-ukraine\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Support-Ukraine-FFD500?style=flat\u0026labelColor=005BBB\" alt=\"Support Ukraine\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/facebookincubator/katran/actions?workflow=CI\"\u003e\n    \u003cimg src=\"https://github.com/facebookincubator/katran/actions/workflows/getdeps_linux.yml/badge.svg\" alt=\"CI Status\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\nKatran is a C++ library and [`BPF`](https://en.wikipedia.org/wiki/Berkeley_Packet_Filter) program to build high-performance\nlayer 4 load balancing forwarding plane. Katran leverages [`XDP infrastructure`](https://www.iovisor.org/technology/xdp)\nfrom the kernel to provide an in-kernel facility for fast packet's processing.\n\n## Katran's key features\n\n1. Blazing fast (especially w/ XDP in driver mode).\n2. Performance scaling linearly with a number of NIC's RX queues.\n3. RSS friendly encapsulation.\n\nSee the detailed features' description below :\n\n## Documentation's sections\n\n1. [`README`](README.md) (this file) - a generic overview of katran.\n2. [`USAGE`](USAGE.md) - the detailed description of how katran's library can be used, configured or changed.\n3. [`DEVELOPING`](DEVELOPING.md) - description of how developing process looks like: how to build and test katran.\n4. [`EXAMPLE`](EXAMPLE.md) - output of how to run provided examples (thrift and [`gRPC services`](https://grpc.io/docs/), which leverage katran library)\n\n## Examples of usage\n\nWe provide simple examples of katran library usage w/ thrift and gRPC endpoints.\nPlease refer to [`Examples`](EXAMPLE.md) for more detailed information.\n\n## Installation\n\nCurrent distribution tested: Ubuntu 20.04\n\nRequirements:\n  - Recent linux kernel (5.6+)\n  - Recent version of clang compiler (6.0+)\n      For ubuntu if unsure run `sudo apt install build-essential`\n\n### build_katran.sh\n\nTo build and install katran library and thrift/gRPC examples - you need to run `build_katran.sh` script.\n\nIt should take care of all the required dependencies.\nIf you need to build it for other Linux distributions, you need to make sure that you have installed:\n    - [`folly`](https://github.com/facebook/folly)\n    - recent version of clang compiler (6.0+)\n    - glog/gtest/gflags/elf libraries\n\nAdditionally, if you want to build examples, [`fbthrift`](https://github.com/facebook/fbthrift) and [`gRPC`](https://github.com/grpc/grpc) must be installed, as well!\n\nWhile we run Meta's CI on CentOS, we do our best to support OSS build on recent Ubuntu versions. If you have any issues with build or need older Ubuntu versions - open github issue or better send us PR :)\n\n## Motivation behind layer 4 load balancing\n\nLayer 4 load balancer (lb) enables to easily scale out Layer7 load balancers (the\nones which terminate TCP sessions). Benefits of L4 lb over other\ntechniques for scaling L7 lb is that it is :\n\n1. compared to DNS it doesn't need to wait for TTL to redirect traffic from failed\nL7 lb.\n\n2. compared to anycast-based solution, L4 lbs are more resilient to the networking\nrelated issues (which triggers mass ECMP reshuffle)/ enable better handling\nof adding/removing of L7 lbs from the pool as well as provide better support for\nunequal load balancing.\n\n## Environment requirements for katran to run\n\nThere are special requirements for katran to work properly. However, we do\nbelieve that most of the potential users of the library can easily satisfy\nthem:\n\n1. katran works only in DSR (direct service response) mode.\n\n2. Network topology should be L3 based (everything above the top of the rack switch should be routed).\nThis is because we are 'offloading' routing decision for sending packets to the real server to first\nrouting device (by unconditionally sending all packets from katran there.)\n\n3. katran doesn't support fragmentation (it cannot forward the fragmented packet, nor it can fragment\nthem by itself if resulting packet size is bigger then MTU). This could be mitigated either by increasing MTU inside your\nnetwork or by changing advertising TCP MSS from L7 lbs (this is recommended even if you have\nincreased MTU, as it will prevent fragmentation related issues towards some of the client.\nFor example, if instead of default TCP MSS 1460 (for ipv4) you will advertise 1450 - it will help clients behind [`PPPoE`](https://en.wikipedia.org/wiki/Point-to-Point_Protocol_over_Ethernet) connection).\n\n4. katran doesn't support packets w/ IP options set.\n\n5. Maximum packet size cannot be bigger than 3.5k (and 1.5k by default).\n\n6. katran is built with the assumption that it's going to be used in a\n\"load balancer on a stick\" scenario: where single interface would be used both for\ntraffic \"from user to L4 lb (ingress)\" and \"from L4 lb to L7 lb (egress).\"\n\n\u003ch3 align=\"center\"\u003e L4 load balancing network topology \u003c/h3\u003e\n\n![alt text](imgs/katran_pktflow.png \"Network Topology\")\n\n__Steps:__\n\n1. katran receives packet\n\n2. Checks if the destination of the packet is configured as a\nVIP (virtual IP address - IP address of the service).\n\n3. For an incoming packet toward a VIP - katran is checking if it saw packet from the same\nsession before, and if it has - it sends the packet to the same real (actual server/l7 lb\nwhich then processes/terminates the TCP session).\n\n4. If it's a new session - from 5 tuples in the packet, calculate a hash value.\n\n5. Using this hash value - pick a real server.\n\n6. Update session table with this lookup information so that katran can simply\nlookup this information for the next packet in the session and not calculate the hash again.\n\n7. Encapsulate packet in another IP packet and send to the real.\n\n\u003ch3 align=\"center\"\u003e L4 load balancing failure scenario \u003c/h3\u003e\n\n![alt text](imgs/katran_consistency.png \"Failure Scenario\")\n\nAs we use only the data from the packet's headers to calculate a hash value,\nwhich is then used to pick a real server, different L4 lbs are consistent in real server selection,\neven w/o explicit state sharing amongst each other. This feature allows us to restart/drain single\nL4 lb w/o affecting TCP sessions, going to the L7 lbs.\n\n## katran's features description\n\n1. __Fast :__ katran uses XDP for packet forwarding, which allows\nto run packet handling routines right after packet has been received by\nnetwork interface card (NIC) and before kernel had any chance to run (when XDP\nis working in \"driver mode\", katran supports \"generic XDP\" mode of operation\nas well (with some performance degradation compare to \"driver mode\")).\n\n2. __Performance scales linearly with a number of NIC's RX queues :__ The way XDP\nworks is that it invokes BPF program on every received packet, and if your\nNIC has multiple queues, for each of them BPF program will be invoked\nindependently. As katran is completely lockless and uses per-cpu\nversions of BPF maps - it scales linearly.\n\n3. __RSS friendly encapsulation :__ katran is uses ipip encapsulation for packet\nforwarding from L4 lb to L7 lb. However, to be able to work in conjunction with\nRSS on L7 lb receive side, instead of using the same source for every ipip\npacket, katran crafts a special one, in such a way, that different flows will have\ndifferent outer (ipip) source IP, but packets in same flow will always have\nthe same.\n\n4. Fixed size (size is configurable on start) connection tracking table w/ LRU\nstrategy for eviction of old entries.\n\n5. __Modified Maglev hashing for connections :__ It provides us good resiliency in\ncase of failure and excellent load balancing features. The hashing was modified\nto be able to support unequal weights for backend (L7 lbs) servers\n\n6. __No need for busylooping on receive path :__ Your load balancer will barely\nconsume any CPU if there is no traffic to serve.\n\n7. katran (and XDP in general) allows you to run any application w/o any\nperformance penalties on the same server (compare to some of other\n\"kernel bypass\" technologies)\n\n## Trivia\n\nkatran is named (it's actually an alternative name) after __[spiny dogfish shark](https://en.wikipedia.org/wiki/Spiny_dogfish)__.\n\n## License\n\nkatran is GPL-2.0 licensed, as found in the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookincubator%2Fkatran","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookincubator%2Fkatran","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookincubator%2Fkatran/lists"}