Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-ebpf

A curated list of awesome projects related to eBPF.
https://github.com/zoidyzoidzoid/awesome-ebpf

Last synced: about 17 hours ago
JSON representation

Reference Documentation
- eBPF Essentials
  - ebpf.io - A gateway to discover all the basics of eBPF, including a listing of the main related projects and of community resources.
- Kernel Documentation
  - BPF Documentation - Index for BPF-related documentation coming with the Linux kernel.
  - linux/Documentation/networking/filter.rst - eBPF specification (somewhat outdated; information should still be valid, but not exhaustive).
  - BPF Design Q&A - Frequently Asked Questions on the decisions behind the BPF infrastructure.
  - HOWTO interact with BPF subsystem - Frequently Asked Questions about contributing to eBPF development.
- Manual Pages
  - `bpf(2)` - Manual page about the `bpf()` system call, used to manage BPF programs and maps from userspace.
  - `tc-bpf(8)` - Manual page about using BPF with tc, including example commands and samples of code.
  - `bpf-helpers(7)` man page - Description of the in-kernel helper functions forming the BPF standard library.
- Other
  - IO Visor's Unofficial eBPF spec - Summary of eBPF syntax and operation codes.
  - Jesper Dangaard Brouer's documentation - Work in progress, contributions welcome.
  - bpf.h and you...
  - Contextually speaking...
  - BPF Verifier Overview
  - List of BPF features per kernel version
Articles and Presentations
- Generic eBPF Presentations and Articles
  - A brief introduction to XDP and eBPF - An accessible introduction providing context, history, and details about the functioning of eBPF.
  - Part 1: Introduction
  - Part 2: Machine & Bytecode
  - Ferris Ellis's blog posts about eBPF - They have a few posts about eBPF:
  - Part 1: Past, Present, and Future
  - Part 2: Syscall and Map Types
  - Making the Kernel's Networking Data Path Programmable with BPF and XDP - A set of slides covering all the basics about eBPF and XDP (mostly for network processing).
  - The BSD Packet Filter - An introduction mostly covering the tracing aspects.
  - BPF: tracing and more - An introduction mostly covering the tracing aspects.
  - Linux BPF Superpowers - An introduction mostly covering the tracing aspects, first part with flame graphs.
  - IO Visor - Also introduces [IO Visor project](https://www.iovisor.org/).
  - BPF -- in-kernel virtual machine - Presentation by the author of eBPF.
  - Extending extended BPF - A blog post from 2014 on the development of BPF and demonstrating what can be done with it, using an example of stateful socket filtering by attaching an eBPF program to a socket.
  - A Tour of Program Types - A description of all existing hooks for BPF program types, and of their interest.
  - BPF helper functions - A review of the kernel functions that can be called from within eBPF programs.
  - Communicating with Userspace - How BPF communicates with userspace - BPF maps, perf events, bpf_trace_printk.
  - Building BPF Programs - Setting up your environment to build BPF programs.
  - The BPF Bytecode and the BPF Verifier - How does BPF ensure that programs are safe?
  - Using BPF to do Packet Transformation - One eBPF usage about packet transformation.
  - Linux Kernel Observability through eBPF - A blog post covering the basics of eBPF as well as code samples in Go on how to build and load a minimal eBPF program into the kernel.
  - eBPF - From a Programmer's Perspective - A short paper describing the fundamentals of eBPF and how to get started with writing eBPF programs.
  - Cloudflare's blog posts on eBPF - Different blog posts about networking use cases and low-level aspects of eBPF.
  - Linux Extended BPF (eBPF) Tracing Tools - An in-depth collection of information around examples of performance analysis tools using eBPF. Contains also a section at the end of the page about other resources.
  - Cloudflare's blog posts on eBPF - Different blog posts about networking use cases and low-level aspects of eBPF.
  - BPF: tracing and more - An introduction mostly covering the tracing aspects.
- BPF Internals
  - eBPF and XDP walkthrough and recent (2017) updates
  - Advanced programmability and recent updates with tc's cls_bpf - Details on eBPF, its use for tunneling and encapsulation, direct packet access, and more.
  - cls_bpf/eBPF updates since netdev 1.1 - Part of [this tc workshop](http://netdevconf.org/1.2/session.html?jamal-tc-workshop).
  - On getting tc classifier fully programmable with cls_bpf - Introduction to eBPF, including several features (map management, tail calls, verifier). The full paper [is also available here](http://www.netdevconf.org/1.1/proceedings/papers/On-getting-tc-classifier-fully-programmable-with-cls-bpf.pdf).
  - Linux tc and eBPF
  - IO Visor blog
  - Linux Networking Explained - Linux networking internals, with a part about eBPF.
- Kernel Tracing
  - Full-system dynamic tracing on Linux using eBPF and bpftrace - A detailed introduction to tracing with eBPF, from listing the available trace points to running bpftrace programs.
  - Meet-cute between eBPF and Kernel Tracing - Kprobes, uprobes, ftrace.
  - Linux Kernel Tracing - Systemtap, Kernelshark, trace-cmd, LTTng, perf-tool, ftrace, hist-trigger, perf, function tracer, tracepoint, kprobe/uprobe, and more.
  - Linux BPF Superpowers
  - Linux Kernel Tracing - Systemtap, Kernelshark, trace-cmd, LTTng, perf-tool, ftrace, hist-trigger, perf, function tracer, tracepoint, kprobe/uprobe, and more.
- XDP
  - The eXpress Data Path - A very accessible introduction to XDP, providing sample code to show how to process packets.
  - Work-in-progress documentation for XDP
  - XDP Project overview
  - eXpress Data Path (XDP) - The first presentation about XDP.
  - BoF - What Can BPF Do For You?
  - eXpress Data Path - Contains some benchmark results obtained with the mlx4 driver.
  - XDP − eXpress Data Path, Intro and future use-cases - Linux Kernel's fight against DPDK. Future plans (as of this writing) for XDP and comparison with DPDK.
  - Network Performance Workshop - Additional hints about XDP internals and expected evolution.
  - XDP – eXpress Data Path, Used for DDoS protection - Details and use cases about XDP, with benchmark results, and code snippets for benchmarking as well as for basic DDoS protection with eBPF/XDP (based on an IP blacklisting scheme).
  - Memory vs. Networking, Provoking and fixing memory bottlenecks - Advanced details about current memory issues faced by XDP developers.
  - XDP now with REDIRECT - Update on XDP, and in particular on the redirect actions.
  - XDP workshop -- Introduction, experience, and future development (Video)
  - High Speed Packet Filtering on Linux - About packet filtering on Linux, DDoS protection, packet processing in the kernel, kernel bypass, XDP and eBPF.
  - How to drop 10 million packets per second - Cloudflare's blog post talking about their move to using XDP for packet filtering.
  - BPF and XDP Reference Guide - Guide from the Cilium project.
  - How to drop 10 million packets per second - Cloudflare's blog post talking about their move to using XDP for packet filtering.
- AF_XDP
  - AF_XDP - Kernel documentation on the AF_XDP address family.
  - Fast Packet Processing in Linux with AF_XDP
- bpfilter
  - Why is the kernel community replacing iptables with BPF? - A blog post by Cilium on the the motivations behind eBPF and bpfilter, with a couple examples and links to other projects using eBPF and bpfilter.
  - bpfilter: Linux firewall with eBPF sauce - Slides from a talk by Quentin Monnet with a background on eBPF and comparing bpfilter to iptables.
- BTF
  - BPF Type Format (BTF) - Kernel documentation about BTF, explaining how to use it.
  - Enhancing the Linux kernel with BTF type information - A description of the work done with BTF to provide debugging information for BPF programs.
- cBPF
  - The BSD Packet Filter: A New Architecture for User-level Packet Capture - The original paper about (classic) BPF.
  - The FreeBSD manual page about BPF
  - Linux' packet mmap(2), BPF, and Netsniff-NG
  - tc and cls bpf: lightweight packet classifying with BPF
  - Introducing Cloudflare's BPF Tools - Usage of BPF bytecode with the `xt_bpf` module for iptables.
  - Libpcap filters syntax
  - Introducing Cloudflare's BPF Tools - Usage of BPF bytecode with the `xt_bpf` module for iptables.
- Hardware Offload
  - eBPF/XDP hardware offload to SmartNICs - Hardware offload for eBPF with TC or XDP (Linux kernel 4.9+), introduced by Netronome.
  - Comprehensive XDP offload---Handling the edge cases - An update on the topic above.
  - OpenCSD eBPF SSD offloading - Computational Storage simulation (QEMU) platform with FUSE LFS filesystem for Zoned Namespaces NVMe SSDs using uBPF for compute kernel offloading, all in userspace.
Tutorials
- Hardware Offload
  - bcc Python Developer Tutorial - Comes with bcc, but targets the Python bits across seventeen "lessons".
  - Building BPF applications with libbpf-bootstrap - Helps generate minimal or advanced templates to bootstrap your own applications (kernel side and user space management for maps and programs) with features like CO-RE, global variables, and ring buffer.
  - Tracing a packet journey using Linux tracepoints, perf and eBPF - Troubleshooting ping requests and replies with perf and bcc programs.
  - Open NFP platform - Operated by Netronome: some tutorials for network-related eBPF use cases, including an eBPF Offload Starting Guide.
  - XDP for the Rest of Us - Second edition, with new contents.
  - Load XDP programs using the ip (iproute2) command
  - All your tracing are belong to BPF - A step-by-step walkthrough to integrate tracing capabilities in your C++ applications with the LLVM libraries.
  - Firewalling with BPF/XDP: Examples and Deep Dive - A simple guide to build basic firewalls with TC and XDP.
  - A Deep Dive into eBPF: Writing an Efficient DNS Monitoring. - A detailed explanation of methods used to capture DNS requests at the socket filter layer.
  - eBPF Developer Tutorial - Learn eBPF by examples - Start with eBPF basics and progress to advanced topics using 20+ hands-on tutorials and examples. Covers performance, networking, and security with libbpf and CO-RE. Available in Chinese and English.
  - XDP for the Rest of Us - First edition of a workshop to get started with XDP.
  - bcc Reference Guide - Many incremental steps to start using bcc and eBPF, mostly centered on tracing and monitoring.
  - Catch Performance Regressions in eBPF - A step-by-step guide to benchmarking both the client and kernel eBPF code written in Rust.
Examples
- Hardware Offload
  - linux/samples/bpf/ - In the kernel tree: some sample eBPF programs.
  - linux/tools/testing/selftests/bpf - In the kernel tree: Linux BPF selftests, with many eBPF programs.
  - prototype-kernel/kernel/samples/bpf - Jesper Dangaard Brouer's prototype-kernel repository contains some additional examples that can be compiled outside of kernel infrastructure.
  - iproute2/examples/bpf/ - Some networking programs to attach to the TC interface.
  - bcc/examples - Examples coming along with the bcc tools, mostly about tracing.
  - bcc/tools - These tools themselves can be seen as example use cases for BPF programs, mostly for tracing and monitoring. bcc tools have been packaged for some Linux distributions.
  - redbpf examples - Example programs for using RedBPF to write eBPF programs in Rust.
eBPF Workflow: Tools and Utilities
- bcc
  - Lua front-end for BCC - Another alternative to C, and even to most of the Python code used in bcc.
- iproute2
  - iproute2 - Package containing tools for network management on Linux. In particular, it contains `tc`, used to manage eBPF filters and actions, and `ip`, used to manage XDP programs. Most of the code related to BPF is in lib/bpf.c.
  - iproute2-next - The development tree, synchronised with net-next.
- LLVM
  - LLVM - Contains several tools used in eBPF workflows. Snapshots of the latest versions for Ubuntu/Debian can be retrieved from [here](http://apt.llvm.org/).
  - this commit
- libbpf
  - libbpf - A C library used for handling BPF objects (programs and maps), and manipulating ELF object files containing them. It is shipped with the kernel and [mirrored on GitHub](https://github.com/libbpf/libbpf).
- User Space eBPF
  - A generic implementation - With support for FreeBSD kernel, FreeBSD user space, Linux kernel, Linux user space and macOS user space. Used for the [VALE software switch](https://www.unix.com/man-page/freebsd/4/vale/)'s [BPF extension module](https://github.com/YutaroHayakawa/vale-bpf).
  - oster - Written in Go. A tool for tracing execution of Go programs by attaching eBPF to uprobes.
- bpftool and Other Tools from the Kernel Tree
  - `bpf_asm` - A minimal cBPF assembler.
  - `bpf_dbg` - A small debugger for cBPF programs.
  - `bpf_jit_disasm` - A disassembler for both BPF flavors and could be highly useful for JIT debugging.
  - `bpftool` - A generic utility that can be used to interact with eBPF programs and maps from userspace, for example to show, dump, load, disassemble, pin programs, or to show, create, pin, update, delete maps, or to attach and detach programs to cgroups.
- Testing in Virtual Environments
  - A Vagrant setup - To easily test XDP. Less useful now that generic XDP (driver-independant, mostly for testing) exists.
Projects Related to eBPF
- Networking
  - P4 on the Edge - P4 with eBPF to create high-performance programmable switches.
  - OvS Orbit episode (#11), called P4 on the Edge - Related to the former item. Audio interview of John Fastabend by Ben Pfaff, one of the core maintainers of Open vSwitch.
  - P4, EBPF and Linux TC Offload - P4 with some elements related to eBPF hardware offload on Netronome's NFP (Network Flow Processor) architecture.
  - Old documentation for P4 usage with eBPF - From bcc repository; deprecated by the P4_16 backend linked below.
  - P4_16 backend for eBPF
  - Cilium - kernel networking and security policy enforcement for containers based on eBPF programs generated on the fly". Many presentations available (with overlap):
  - Cilium: Networking & Security for Containers with BPF & XDP - Also featuring a load balancer use case
  - Cilium: Networking & Security for Containers with BPF & XDP - [video](https://www.youtube.com/watch?v=TnJF7ht3ZYc&list=PLkA60AVN3hh8oPas3cq2VA9xB7WazcIgs)
  - Cilium: Fast IPv6 container Networking with BPF and XDP
  - Cilium: BPF & XDP for containers
  - OvS Orbit episode (#4) - Interview of Thomas Graf by Ben Pfaff.
  - A generic introduction to Cilium
  - A podcast interviewing Thomas Graf - Ivan Pepelnjak interviewing Thomas, October 2016, on eBPF, P4, XDP and Cilium.
  - Offloading OVS Flow Processing using eBPF
  - Coupling the Flexibility of OVN with the Efficiency of IOVisor
  - Katran - A layer 4 load-balancer based on XDP, open-sourced by Facebook.
  - XDP in practice: integrating XDP in our DDoS mitigation pipeline - Protection against DDoS with XDP at Cloudflare.
  - Droplet: DDoS countermeasures powered by BPF + XDP - Protection against DDoS with XDP at Facebook.
  - DPDK has a poll-mode driver (PMD) based on AF_XDP
  - CETH for XDP - Common Ethernet Driver Framework for faster network I/O, a technology initiated by Mellanox.
  - Enabling eBPF data plane with Calico
  - "eBPF and XDP" section of Suricata documentation
  - The adventures of a Suricate in eBPF land
  - eBPF and XDP seen from the eyes of a meerkat
  - Project Calico - Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico's eBPF data plane delivers a low latency, high throughput data plane with a rich network security policy model.
  - PcapPlusPlus - An open-source C++ library for capturing, parsing and crafting network packets. It features a C++ interface for creating AF_XDP sockets, making it easy to [send and receive packets through them](https://pcapplusplus.github.io/docs/next/features#af_xdp-support-beta).
  - A blog post introducing the feature
  - Cilium: Networking & Security for Containers with BPF & XDP - [video](https://www.youtube.com/watch?v=TnJF7ht3ZYc&list=PLkA60AVN3hh8oPas3cq2VA9xB7WazcIgs)
  - Cilium: Fast IPv6 container Networking with BPF and XDP
- Observability
  - InKeV: In-Kernel Distributed Network Virtualization for DCN
  - DEEP-mon - Helps with measuring power consumption for servers and uses eBPF programs for in-kernel aggregation of data.
- Security
  - Falco - A cloud-native runtime security project used as a Kubernetes threat detection engine.
  - Red Canary Linux Agent - Red Canary has started to incorporate eBPF to their Linux security sensor.
- Tools
  - bpftrace - A tool for tracing with its own high-level tracing language. It is flexible enough to be envisioned as a Linux replacement for DTrace and SystemTap.
  - bpftrace Cheat Sheet - Summary and cheat sheet for programming in bpftrace. Contains information about syntax, probe types, variables and functions.
  - FUSE - Considers using eBPF.
  - Embrace The Red: Offensive BPF! - A series of posts around the introduction into BPF with a focus to an offensive setting, and also how its misuse can be detected. Posts include discussions on the rootkit capabilities of eBPF, or on which tracing type is needed for different use cases.
  - eBPF: Block Linux Fileless Payload "Malware" Execution with BPF LSM - Blog post about how BPF can help detection and blocking fileless malware.
  - Blackhat 2021: With Friends Like eBPF, Who Needs Enemies? - Talk about an eBPF rootkit and how the capabilities of eBPF could be abused. The rootkit was also the object of a talk at Defcon, [eBPF, I thought we were friends !](https://defcon.org/html/defcon-29/dc-29-speakers.html#fournier).
The Code
- Tools
  - linux/include/linux/bpf.h - with [linux/include/uapi/bpf.h](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/bpf.h): definitions related to eBPF, to be used respectively in the kernel and to interface with userspace programs.
  - linux/include/linux/filter.h - with [linux/include/uapi/filter.h](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/filter.h): information used to run the BPF programs themselves.
  - linux/kernel/bpf/ - This directory contains most of BPF-related code. In particular, those files are worth of interest:
  - `syscall.c` - Different operations permitted by the system call, such as program loading or map management.
  - `core.c` - BPF interpreter.
  - `verifier.c` - BPF verifier.
  - linux/net/core/filter.c - Functions and eBPF helpers related to networking (TC, XDP etc.); also contains the code to migrate cBPF bytecode to eBPF (all cBPF programs are translated to eBPF in recent kernels).
  - linux/kernel/trace/bpf_trace.c - Functions and eBPF helpers related to tracing and monitoring (kprobes, tracepoints, etc.).
  - linux/arch/x86/net/bpf_jit_comp.c
  - linux/net/sched/ - and in particular in files `act_bpf.c` (action) and `cls_bpf.c` (filter): code related to BPF actions and filters with TC.
  - linux/kernel/seccomp.c
  - linux/net/core/dev.c - contains the function `dev_change_xdp_fd()` that is called through a Netlink command to hook a XDP program to a device, after is has been loaded into the kernel from user space. This function in turns uses a callback from the relevant driver.
Development and Community
- Tools
  - The bpf-next tree - BPF patches land in this tree. It is regularly merged into [net-next](https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git), which is itself merged for each release to Linus' tree.
  - Kernel documentation - About contributions to BPF.
  - The netdev mailing list - Mailing list for Linux kernel networking stack development. All patches are sent there for review and inclusion.
  - IO Visor mailing list - BPF is at the heart of the project, and is regularly discussed on the mailing list.
  - @IOVisor Twitter account
  - IO Visor mailing list - BPF is at the heart of the project, and is regularly discussed on the mailing list.
  - XDP-newbies - A mailing list specially dedicated to XDP programming (both for architecture or for asking for help).
Other Lists of Resources on eBPF
- Tools
  - IO Visor's bcc documentation
Acknowledgement
- Tools
  - Dive into BPF: A List of Reading Material

Ecosyste.ms: Awesome

awesome-ebpf

Reference Documentation

eBPF Essentials

Kernel Documentation

Manual Pages

Other

Articles and Presentations

Generic eBPF Presentations and Articles

BPF Internals

Kernel Tracing

XDP

AF_XDP

bpfilter

BTF

cBPF

Hardware Offload

Tutorials

Hardware Offload

Examples

Hardware Offload

eBPF Workflow: Tools and Utilities

bcc

iproute2

LLVM

libbpf

User Space eBPF

bpftool and Other Tools from the Kernel Tree

Testing in Virtual Environments

Projects Related to eBPF

Networking

Observability

Security

Tools

The Code

Tools

Development and Community

Tools

Other Lists of Resources on eBPF

Tools

Acknowledgement

Tools