{"id":13581319,"url":"https://github.com/ebpfdev/dev-agent","last_synced_at":"2025-04-11T18:31:28.425Z","repository":{"id":165253329,"uuid":"640602926","full_name":"ebpfdev/dev-agent","owner":"ebpfdev","description":"eBPF remote debug agent","archived":false,"fork":false,"pushed_at":"2023-06-18T11:24:32.000Z","size":962,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-25T16:55:34.001Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ebpfdev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-14T16:25:35.000Z","updated_at":"2024-12-09T16:25:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"d6602dde-089c-4ebb-ac04-e129909c8774","html_url":"https://github.com/ebpfdev/dev-agent","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebpfdev%2Fdev-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebpfdev%2Fdev-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebpfdev%2Fdev-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebpfdev%2Fdev-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ebpfdev","download_url":"https://codeload.github.com/ebpfdev/dev-agent/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248458471,"owners_count":21107085,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:02:00.296Z","updated_at":"2025-04-11T18:31:27.126Z","avatar_url":"https://github.com/ebpfdev.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# dev-agent\nThis agent provides access to system's eBPF-programs and maps to perform remote debugging.\n\nFeatures:\n* GraphQL model for maps and programs\n* Prometheus metrics exporter (including values of maps entries)\n\nSee also [CHANGELOG](./CHANGELOG.md).\n\n# Usage\n\n## Server\n\n```shell\nsudo ./phydev server [--help]\n```\n\nGraphQL interface: [http://localhost:8080/](http://localhost:8080/)\n\nPrometheus endpoint: [http://localhost:8080/metrics](http://localhost:8080/metrics)\n\nSchema: [pkg/graph/schema.graphqls](pkg/graph/schema.graphqls)\n\n![GraphQL interface example](docs/graphql-example.png)\n\n### Prometheus endpoint\n\nMetrics scrape endpoint for Prometheus: [http://localhost:8080/metrics](http://localhost:8080/metrics)\n\n* program metrics:\n  * `devagent_ebpf_prog_count` - number of eBPF programs by `type`\n  * runtime metrics only available with `sysctl -w kernel.bpf_stats_enabled=1`:\n    * `devagent_ebpf_prog_run_count` - number of times an eBPF program has been run (by `id`, `name`, `tag`, `type`)\n    * `devagent_ebpf_prog_run_time` - total time spent running eBPF programs (by `id`, `name`, `tag`, `type`)\n* map metrics:\n  * `devagent_ebpf_map_count` - number of eBPF maps by `type`\n  * if map export is configured (see below):\n    * `devagent_ebpf_map_entry_count` - number of entries in an eBPF map (by `id`, `name`, `type`)\n    * `devagent_ebpf_map_entry_value` - value of an eBPF map entry (by `key`, `cpu`, `id`, `name`, `type`)\n\nYou can find example of Grafana dashboard in [grafana-ebpf-dashboard.json](./grafana-ebpf-dashboard.json):\n![grafana dashboard with program metrics](docs/grafana-ebpf.png)\n\n#### Configuring map export\n\nAs an example, I'm running this [bpftrace](https://github.com/iovisor/bpftrace) program:\n```shell\nsudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @SYSCALLNUM[comm] = count(); }'\n```\n\nYou could see the name of created map - `AT_SYSCALLNUM`, and the map content in [ebpf-explorer](https://github.com/ebpfdev/explorer):\n![exbpf explorer showing AT_SYSCALNUM page](docs/explorer-syscallnum.png)\n\nBy default, dev-agent doesn't export map entries to Prometheus, as it may introduce some performance issues.\n\nInstead, you could set an option `--etm -:AT_SYSCALLNUM:string` when running server, which will suggest agent which map entries to expose in /metrics.\n\nFor this HASH_PER_CPU map, it will export 2 metrics:\n```text\n# HELP devagent_ebpf_map_entry_count Number of entries in an eBPF map\n# TYPE devagent_ebpf_map_entry_count gauge\ndevagent_ebpf_map_entry_count{id=\"25\",name=\"AT_SYSCALLNUM\",type=\"PerCPUHash\"} 764\n# HELP devagent_ebpf_map_entry_value Value of an eBPF map entry\n# TYPE devagent_ebpf_map_entry_value gauge\ndevagent_ebpf_map_entry_value{cpu=\"0\",id=\"25\",key=\"(anacron)\",name=\"AT_SYSCALLNUM\",type=\"PerCPUHash\"} 0\ndevagent_ebpf_map_entry_value{cpu=\"0\",id=\"25\",key=\"(fprintd)\",name=\"AT_SYSCALLNUM\",type=\"PerCPUHash\"} 0\n```\n\nThis is how it may look in Grafana (top 10 processes doing most of syscalls):\n![Grafana showing top 10 processes doing most of syscalls](docs/grafana-syscallnum.png)\n\nFull demo in terminal:\n```shell\nsudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @SYSCALLNUM[comm] = count(); }' \u0026\ndocker run -ti --rm --cap-add CAP_SYS_ADMIN --pid=host -v /sys/fs/bpf:/sys/fs/bpf --pid=host -p 8080:8080 ghcr.io/ebpfdev/dev-agent:v0.0.4 server --etm -:AT_SYSCALLNUM:string\ncurl http://localhost:8080/metrics | grep devagent_ebpf_map_entry_count\n```\n\nRun `./phydev server --help` for more details on this flag.\n\n## CLI commands\n\nThese are just for debugging purpose, use [bpftool](https://github.com/libbpf/bpftool) instead\n\nList loaded eBPF programs:\n\n```shell\nsudo ./phydev inspect progs list\n\u003e ID      Type    Tag     RunCount        RunTime AvgRunTime\n\u003e 3               CGroupDevice    e3dbd137be8d6168        0       0s      0s\n\u003e 4               CGroupSKB       6deef7357e7b4530        0       0s      0s\n\u003e 125     uprobe__BIO_new Kprobe  0d9ea14e5516f975        0       0s      0s\n\u003e 126     socket__http_fi SocketFilter    6b7ab673cb23d3f0        0       0s      0s\n\u003e 127     kretprobe__do_s Kprobe  154f35d6575c73f9        0       0s      0s\n\u003e 128     uretprobe__SSL_ Kprobe  8737d2e349595de3        0       0s      0s\n```\n\nList loaded eBPF maps:\n\n```shell\n% sudo ./phydev inspect maps list \nID      Name    FD      Type    Flags   IsPinned        KeySize ValueSize       MaxEntries\n1               3       Hash    0       false   9       1       500\n2               4       Hash    0       false   9       1       500\n44      do_sendfile_arg 10      Hash    0       false   8       8       1024\n61      http_in_flight  27      Hash    0       false   48      112     10000\n62      http_notificati 28      PerfEventArray  0       false   4       4       16\n63      open_at_args    29      Hash    0       false   8       128     1024\n```\n\n## Docker\n\nInstead of `./phydev server`, use docker command:\n\n```shell\ndocker run -ti --rm --cap-add CAP_SYS_ADMIN --pid=host -e BPF_DIR=/sys/fs/bpf -v /sys/fs/bpf:/sys/fs/bpf -p 8080:8080 ghcr.io/ebpfdev/dev-agent:v0.0.4 server\n```\n\n### Security options breakdown\n\nRequired:\n* `--cap-add CAP_SYS_ADMIN` is needed for access BPF maps and programs (CAP_BPF is not yet enough to list existing maps and programs)\n\nOptional:\n* `--pid=host` is needed to determine tracepoint/kprobe attachment,\n  if you skip it, you won't see what tracepoint/kprobe a program is attached to:\n  ![](docs/secopts-attachments.png)\n  \n* `-e BPF_DIR=/sys/fs/bpf -v /sys/fs/bpf:/sys/fs/bpf` is needed to determine pinned maps (they will be resolved relative to `BPF_DIR` tough, so it's better to mount it to the same path)\n  if you skip it, you won't see maps pinned path, and it won't be possible to pin new maps:\n  ![](docs/secopts-pins.png)\n\n\n\n# Development\n\n## Build\n```shell\ngo build -o phydev cmd/dev-agent/main.go\n```\n\n## Updated generated code\n\nThis will update GraphQL code and some other constants:\n\n```shell\ngo install golang.org/x/tools/cmd/stringer@latest\ngo generate ./...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febpfdev%2Fdev-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Febpfdev%2Fdev-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febpfdev%2Fdev-agent/lists"}