{"id":13575577,"url":"https://github.com/oracle/bpftune","last_synced_at":"2025-12-12T06:02:41.817Z","repository":{"id":164049369,"uuid":"638533777","full_name":"oracle/bpftune","owner":"oracle","description":"bpftune uses BPF to auto-tune Linux systems","archived":false,"fork":false,"pushed_at":"2025-04-08T17:41:26.000Z","size":3054,"stargazers_count":1583,"open_issues_count":12,"forks_count":85,"subscribers_count":30,"default_branch":"main","last_synced_at":"2025-04-12T16:45:11.924Z","etag":null,"topics":["auto-tuning","bpf","ebpf","linux"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oracle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-09T14:56:10.000Z","updated_at":"2025-04-08T17:41:30.000Z","dependencies_parsed_at":"2024-12-27T01:01:53.546Z","dependency_job_id":"92734003-196c-498e-b35c-7737d1c9fa9a","html_url":"https://github.com/oracle/bpftune","commit_stats":{"total_commits":504,"total_committers":6,"mean_commits":84.0,"dds":"0.029761904761904767","last_synced_commit":"39dbe541044a4cfc51c7e452db96f28cb22d704a"},"previous_names":["oracle/bpftune"],"tags_count":0,"template":false,"template_full_name":"oracle/template-repo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fbpftune","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fbpftune/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fbpftune/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oracle%2Fbpftune/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oracle","download_url":"https://codeload.github.com/oracle/bpftune/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254160037,"owners_count":22024567,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-tuning","bpf","ebpf","linux"],"created_at":"2024-08-01T15:01:02.274Z","updated_at":"2025-12-12T06:02:41.808Z","avatar_url":"https://github.com/oracle.png","language":"C","readme":"# bpftune - BPF driven auto-tuning\n\nbpftune aims to provide lightweight, always-on auto-tuning of system\nbehaviour.  The key benefit it provides are\n\n- by using BPF observability features, we can continuously monitor\n  and adjust system behaviour\n- because we can observe system behaviour at a fine grain (rather\n  than using coarse system-wide stats), we can tune at a finer grain\n  too (individual socket policies, individual device policies etc)\n\n## The problem\n\nThe Linux kernel contains a large number of tunables; these\noften take the form of sysctl(8) parameters, and are usually\nintroduced for situations where there is no one \"right\" answer\nfor a configuration choice.  The number of tunables available\nis quite daunting.  On a 6.2 kernel we see\n\n```\n# sysctl --all 2\u003e/dev/null|wc -l\n1624\n```\n\n[See here for an excellent writeup on network-related tunables.](https://github.com/leandromoreira/linux-network-performance-parameters).\n\nAt the same time, individual systems get a lot less care\nand adminstrator attention than they used to; phrases like\n\"cattle not pets\" exemplify this.  Given the modern cloud\narchitectures used for most deployments, most systems never\nhave any human adminstrator interaction after initial\nprovisioning; in fact given the scale requirements, this\nis often an explicit design goal - \"no ssh'ing in!\".\n\nThese two observations are not unrelated; in an earlier\nera of fewer, larger systems, tuning by administrators was\nmore feasible.\n\nThese trends - system complexity combined with minimal\nadmin interaction suggest a rethink in terms of tunable\nmanagement.\n\nA lot of lore accumulates around these tunables, and to help\nclarify why we developed bpftune, we will use a straw-man\nversion of the approach taken with tunables:\n\n\"find the set of magic numbers that will work for the\n system forever\"\n\nThis is obviously a caricature of how administrators\napproach the problem, but it does highlight a critical\nimplicit assumption - that systems are static.\n\nAnd that gets to the \"BPF\" in bpftune; BPF provides means\nto carry out low-overhead observability of systems. So\nnot only can we observe the system and tune appropriately,\nwe can also observe the effect of that tuning and re-tune\nif necessary.\n\n## Key design principles\n\n- Minimize overhead.  Use observability features sparingly; do not\n  trace very high frequency events.\n- Be explicit about policy changes providing both a \"what\" - what\n  change was made - and a \"why\" - how does it help? syslog logging\n  makes policy actions explicit with explanations\n- Get out of the way of the administrator.  We can use BPF\n  observability to see if the admin sets tunable values that we\n  are auto-tuning; if they do, we need to get out of the way and\n  disable auto-tuning of the related feature set.\n- Don't replace tunables with more tunables! bpftune is designed to\n  be zero configuration; there are no options, and we try to avoid\n  magic numbers where possible.\n- Use push-pull approaches. For example, with tcp buffer sizing,\n  we often want to get out of the way of applications and bump\n  up tcp sndbuf and rcvbuf, but at a certain point we run the\n  risk of exhausting TCP memory.  We can however monitor if we\n  are approaching TCP memory pressure and if so we can tune down\n  values that we've tuned up.  In this way, we can let the system\n  find a balance between providing resources and exhausting them.\n  In some cases, we won't need to tune up values; they may be fine\n  as they are. But in other cases these limits block optimal performance,\n  and if they are raised safely - with awareness of global memory\n  limits - we can get out the way of improved performance.  Another\n  concern is that increasing buffer size leads to latency - to\n  handle that, we correlate buffer size changes and TCP smoothed\n  round-trip time; if the correlation between these exceeds a\n  threshold (0.7) we stop increasing buffer size.\n\n## Concepts\n\nThe key components are\n\n- tuners: each tuner manages tunables and handles events sent\n  from BPF programs to userspace via the shared ring buffer.\n  Each tuner has an associated set of tunables that it manages.\n\n- optional strategies: a tuner can specify multiple strategies;\n  after running for a while a strategy times out and we assess\n  if a better strategy is available.  Each strategy specifies a\n\t- name\n\t- description\n\t- timeout\t\n\t- evaluation function\n\t- set of BPF program names in tuner associated with strategy\n\n  Strategies are optional and should be set in the tuner init()\n  method via bpftune_strategies_add().  See test/strategy\n  for a coded example.  When a strategy times out, the various\n  evaluation functions are called and the highest-value evaluation\n  dictates the next stratgey.\n\n  Strategies provide a way of providing multiple schemes for\n  auto-tuning the same set of tunables, where the choice is\n  guided by an evaluation of the effectiveness of the strategies.\n\n- events specify a\n\t- tuner id: which tuner the event is destined for\n\t- a scenario: what happened\n\t- an associated netns (if supported)\n\t- information about the event (IP address etc)\n\n- the tuner then responds to the event guided by the active strategy;\n  increase or decrease a tunable value, etc.  Describing the event\n  in the log is key; this allows an admin to understand what\n  changed and why.\n\n## Architecture\n\n- bpftune is a daemon which manages a set of .so plugin tuners;\n  each of these is a shared object that is loaded on start-up.\n- tuners can be enabled or disabled; a tuner is automatically\n  disabled if the admin changes associated tunables manually.\n- tuners share a global BPF ring buffer which allows posting of\n  events from BPF programs to userspace.  For example, if the\n  sysctl tuner sees a systl being set, it posts an event.\n- each tuner has an associated id (set when it is loaded),\n  and events posted contain the tuner id.\n- each tuner has a BPF component (built using a BPF skeleton)\n  and a userspace component.  The latter has init(), fini()\n  and event_handler() entrypoints.  When an event is\n  received, the tuner id is used to identify the appropriate\n  event handler and its event_handler() callback function is run.\n- init, fini and event_handler functions are loaded from the\n  tuner .so object.\n- BPF components should include bpftune.bpf.h; it contains\n  the common map definitions (ringbuf, etc) and shared variables\n  such as learning rate and tuner ids that each tuner needs.\n\n## Supported tuners\n\n- TCP connection tuner: auto-tune choice of congestion control algorithm.\n  See [bpftune-tcp-conn (8)](./docs/bpftune-tcp-conn.rst)\n- IP fragmentation tuner: auto-tune IP fragmentation memory limits\n  to support fragment reassembly.  See [bpftune-ip-frag (8)](./docs/bpftune-ip-frag.rst)\n- neighbour table tuner: auto-tune neighbour table sizes by growing\n  tables when approaching full. See [bpftune-neigh (8)](./docs/bpftune-neigh.rst)\n- sysctl tuner: monitor sysctl setting and if it collides with an\n  auto-tuned sysctl value, disable the associated tuner.  See\n  [bpftune-sysctl (8)](./docs/bpftune-sysctl.rst)\n- TCP buffer tuner: auto-tune max and initial buffer sizes.  See\n  [bpftune-tcp-buffer (8)](./docs/bpftune-tcp-buffer.rst)\n- net buffer tuner: auto-tune tunables related to core networking.\n  See [bpftune-net-buffer (8)](./docs/bpftune-net-buffer.rst)\n- netns tuner: notices addition and removal of network namespaces,\n  which helps power namespace awareness for bpftune as a whole.\n  Namespace awareness is important as we want to be able to auto-tune\n  containers also.  See [bpftune-netns (8)](./docs/bpftune-netns.rst)\n- UDP buffer tuner: auto-tune buffers relating to UDP. See\n  [bpftune-udp-buffer (8)](./docs/bpftune-udp-buffer.rst)\n\n## Code organization\n\nBoth core bpftune.c and individual tuners use the libbpftune library.\nIt handles logging, tuner init/fini, and BPF init/fini.\n\nEach tuner shared object defines an init(), fini() and event_handler()\nfunction. These respectively set up and clean up BPF and handle events\nthat originate from the BPF code.\n\n## Getting Started\n\nIf building the repository manually, simply run\n\n```\n$ make ; sudo make install\n```\nat the top-level of the repository.  bpftune also supports a\n\n```\n$ make pkg\n```\n\ntarget, which will make a bpftune RPM.  See ./buildrpm/bpftune.spec\n\nBy default, we disable openrc script installation to avoid unneeded\npackage dependencies; to enable this add\n\n```\n--with openrc\n```\n\nto the rpmbuild command.\n\nWe can also build with non-standard libdir for distros which do not\nuse /usr/lib64 like CachyOS; in this case to install to /usr/lib\ninstead\n\n```\n$ make libdir=lib\n$ sudo make install libdir=lib\n```\n\nTo build the following packages are needed (names may vary by distro);\n\n- libbpf, libbpf-devel \u003e= 0.6\n- libcap-devel\n- bpftool \u003e= 4.18\n- libnl3-devel (on some distros like Debian libnl-route-3-dev is needed)\n- clang \u003e= 11\n- llvm \u003e= 11\n- python3-docutils\n\nThe bpf components in bpftune can be built via GCC BPF support.\nSee https://gcc.gnu.org/wiki/BPFBackEnd for details on the BPF backend.\nTo build with gcc bpf, specify\n\n```\n$ GCC_BPF=bpf-unknown-none-gcc make\n```\n\nFrom the kernel side, the kernel needs to support BPF ring buffer\n(around the 5.6 kernel, though 5.4 is supported on Oracle Linux\nas ring buffer support was backported), and kernel BTF is\nrequired (CONFIG_DEBUG_INFO_BTF=y).  Verify /sys/kernel/btf/vmlinux\nis present.\n\nTo enable bpftune as a service\n\n```\n$ sudo service bpftune start\n```\n\n...and to enable it by default\n\n```\n$ sudo systemctl enable bpftune\n```\n\nbpftune logs to syslog so /var/log/messages will contain details\nof any tuning carried out.\n\nbpftune can also be run in the foreground as a program; to redirect\noutput to stdout/stderr, run\n\n```\n$ sudo bpftune -s\n```\n\nOn exit, bpftune will summarize any tuning done.\n\nQueries of bpftune state can be done via `bpftune -q`.\n\n## Performance Co-Pilot (PCP) Support\n\nSupport has been added to export bpftune tunable values to Performance\nCo-Pilot via a PMDA (Performance Metric Domain Agent).  It uses\nbpftune quereies to populate metrics in PCP via a python-based\nPMDA.  See src/pcp/pmdabpftune.python.\n\nTo install the PMDA (ensuring pcp and python3-pcp packages are installed\nfirst), simply run\n\n```\n$ sudo make install\n$ cd /var/lib/pcp/pmdas/bpftune\n$ sudo ./Install\n\n```\n\nOnce the above has been done, PCP metrics will be available for bpftune\ntunables.  To see these:\n\n```\n$ pminfo -f bpftune\n\nbpftune.udp_buffer.net.core.rmem_default\n    value 212992\n\nbpftune.udp_buffer.net.core.rmem_max\n    value 3099438\n...\n```\n\nThe PMDA is also packaged in bpftune-pcp-pmda; see buildrpm/bpftune.spec.\n\nTo create grafana dashboards using bpftune tunables, see the\n[blog entry here](https://blogs.oracle.com/linux/post/visualising-pcp-metrics-using-grafana)\nfor details on setting up grafana to handle PCP metric visualization.\n\nFrom there it is necessary to ensure that pmlogger is logging the metrics\nregularly; add something like the following to\n\n/var/lib/pcp/config/pmlogger/config.default\n\n```\nlog advisory on default {\n        bpftune.tcp_buffer.net.ipv4.tcp_rmem\n        bpftune.tcp_buffer.net.ipv4.tcp_wmem\n        bpftune.tcp_buffer.net.ipv4.tcp_mem\n}\n```\n\n...and restart pmlogger\n\n```\n$ sudo service pmlogger restart\n```\n\nOnce that is done, it should be possible to create dashboards with\nqueries of bpftune metrics.  For example here is a simple dashboard\ncreated for the tcp_buffer tuner:\n\n![alt text](bpftune.tcp_buffer.png \"bpftune grafana dashboard for tcp_buffer tuner\")\n\n## Ansible Install Play\n\nInformation: If you are using an Fedora Upstream based Distribution you have to enable the correct repository based on the system you are using, because the libbpf-devel package is getting shipped on additional repository, based on the Distribution. You can look it up here: https://pkgs.org/search/?q=libbpf-devel\n\n```\n- name: bpftune download, build, install and start service\n  hosts: all\n  become: true\n  vars:\n    repo_url: \"https://github.com/oracle/bpftune\"\n    repo_dest: \"/root/bpftune/\"\n  tasks:\n    - name: Install git and system independent build requirements if not present\n      ansible.builtin.package:\n        name:\n          - git\n          - clang            # build requirement\n          - llvm             # build requirement\n          - bpftool          # build requirement\n          - iperf3           # build requirement\n          - python3-docutils # build requirement\n        state: present\n\n    - name: Gather package manager fact\n      ansible.builtin.setup:\n        filter: ansible_pkg_mgr\n      register: setup_info\n\n    - name: install run and build requirements for bpftune on dnf based systems\n      ansible.builtin.dnf:\n        name:\n          - libbpf       # run requirement\n          - libnl3       # run requirement\n          - libcap       # run requirement\n          - libbpf-devel # build requirement\n          - libnl3-devel # build requirement\n          - libcap-devel # build requirement\n          - clang-libs   # build requirement\n          - llvm-libs    # build requirement\n        state: present\n        # enablerepo: \u003crepository\u003e # !ATTENTION! based on the system you use you have to enable the correct repository based on the system you can look it up here: https://pkgs.org/search/?q=libbpf-devel\n      when: setup_info.ansible_facts.ansible_pkg_mgr == \"dnf\"\n\n    - name: install build requirements for bpftune on apt based systems\n      ansible.builtin.apt:\n        name:\n          - libbpf-dev        # build requirement\n          - libcap-dev        # build requirement\n          - libnl-3-dev       # build requirement\n          - libnl-route-3-dev # build requirement\n        state: present\n        install_recommends: false\n      when: setup_info.ansible_facts.ansible_pkg_mgr == \"apt\"\n\n    - name: Clone or update the bpftune repository if a new version is available\n      ansible.builtin.git:\n        repo: \"{{ repo_url }}\"\n        dest: \"{{ repo_dest }}\"\n        update: true\n      register: git_result\n\n    - name: if repository changes rebuild software\n      block:\n        - name: Run make with taget 'all' for bpftune\n          community.general.make:\n            chdir: \"{{ repo_dest }}\"\n            target: all\n\n        - name: Run make with taget 'install' target for bpftune\n          community.general.make:\n            chdir: \"{{ repo_dest }}\"\n            target: install\n\n        - name: Check bpftune status\n          command: bpftune -S\n          register: bpftune_status\n          changed_when: false\n          failed_when: \"'bpftune works fully' not in bpftune_status.stderr\"\n\n        - name: restart and enable bpftune service\n          ansible.builtin.service:\n            name: bpftune.service\n            state: started\n            enabled: yes\n      when: git_result.changed\n```\n\n## Tests\n\nTests are supplied for each tuner in the tests/ subdirectory.\n\"make test\" runs all the tests.  Tests use network namespaces\nto simulate interactions with remote hosts. See ./TESTING.md\nfor more details.\n\n## Does my system support bpftune?\n\nSimply run \"bpftune -S\" to see:\n\n```\n$ bpftune -S\nbpftune works fully\nbpftune supports per-netns policy (via netns cookie)\n```\n\nTwo aspects are important here\n\n- does the system support fentry/fexit etc? If so full support\n  is likely.\n- does the system support network namespace cookies? If so\n  per-network-namespace policy is supported.\n\n## Demo\n\nSimply starting bpftune and observing changes made via /var/log/messages\ncan be instructive.  For example, on a standard VM with sysctl defaults,\nI ran\n\n```\n$ service bpftune start\n```\n\n...and went about normal development activities such as cloning git\ntrees from upstream, building kernels, etc.  From the log we see\nsome of the adjustments bpftune made to accommodate these activities\n\n```\n$ sudo grep bpftune /var/log/messages\n...\nApr 19 16:14:59 bpftest bpftune[2778]: bpftune works fully\nApr 19 16:14:59 bpftest bpftune[2778]: bpftune supports per-netns policy (via netns cookie)\nApr 19 16:18:40 bpftest bpftune[2778]: Scenario 'specify bbr congestion control' occurred for tunable 'TCP congestion control' in global ns. Because loss rate has exceeded 1 percent for a connection, use bbr congestion control algorithm instead of default\nApr 19 16:18:40 bpftest bpftune[2778]: due to loss events for 145.40.68.75, specify 'bbr' congestion control algorithm\nApr 19 16:26:53 bpftest bpftune[2778]: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput\nApr 19 16:26:53 bpftest bpftune[2778]: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 6291456) -\u003e (4096 131072 7864320)\nApr 19 16:26:53 bpftest bpftune[2778]: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput\nApr 19 16:26:53 bpftest bpftune[2778]: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 7864320) -\u003e (4096 131072 9830400)\nApr 19 16:29:04 bpftest bpftune[2778]: Scenario 'specify bbr congestion control' occurred for tunable 'TCP congestion control' in global ns. Because loss rate has exceeded 1 percent for a connection, use bbr congestion control algorithm instead of default\nApr 19 16:29:04 bpftest bpftune[2778]: due to loss events for 140.91.12.81, specify 'bbr' congestion control algorithm\n```\n\nTo deterministically trigger bpftune behaviour, one approach we can\ntake is to download a large file with inappropriate settings.\n\nIn one window, set tcp rmem max to a too-low value, and run bpftune\nas a program logging to stdout/stderr (-s):\n\n```\n$ sudo sysctl -w net.ipv4.tcp_rmem=\"4096 131072 1310720\"\nnet.ipv4.tcp_rmem = 4096 131072 1310720\n$ sudo bpftune -s\n```\n\nIn another window, wget a large file:\n\n```\n$ wget https://yum.oracle.com/ISOS/OracleLinux/OL8/u7/x86_64/OracleLinux-R8-U7-x86_64-dvd.iso\n```\n\nIn the first window, we see bpftune tuning up rmem:\n\n```\nbpftune: bpftune works in legacy mode\nbpftune: bpftune does not support per-netns policy (via netns cookie)\nbpftune: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput\nbpftune: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 1310720) -\u003e (4096 131072 1638400)\n```\n\nThis occurs multiple times, and on exit (Ctrl+C) we see\nthe summary of changes made:\n\n```\nbpftune: Summary: scenario 'need to increase TCP buffer size(s)' occurred 9 times for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput\nbpftune: sysctl 'net.ipv4.tcp_rmem' changed from (4096 131072 1310720 ) -\u003e (4096 131072 9765625 )\n```\n\n## For more info\n\nSee the docs/ subdirectory for manual pages covering bpftune\nand associated tuners.\n\nbpftune was presented at the eBPF summit; [video here](https://www.youtube.com/watch?v=X0TvfH8hrQE\u0026t=420s).\n\nbpftune [was also discussed on Liz Rice's excellent eCHO eBPF podcast](https://www.youtube.com/watch?v=3ylmGE6sW8w), specifically in the context of using reinforcement learning in BPF \n## Contributing\n\nThis project welcomes contributions from the community. Before submitting a pull request, please [review our contribution guide](./CONTRIBUTING.md)\n\n## Security\n\nPlease consult the [security guide](./SECURITY.md) for our responsible security vulnerability disclosure process\n\n## License\n\nCopyright (c) 2023 Oracle and/or its affiliates.\n\nThis software is available to you under\n\nSPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note\n\nBeing under the terms of the GNU General Public License version 2.\n\nSPDX-URL: https://spdx.org/licenses/GPL-2.0.html\n\nSee [the license file](./LICENSE.txt) for more details.\n","funding_links":[],"categories":["C"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle%2Fbpftune","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foracle%2Fbpftune","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foracle%2Fbpftune/lists"}