{"id":22393063,"url":"https://github.com/mkcms/bdx","last_synced_at":"2025-06-10T19:04:05.578Z","repository":{"id":262287381,"uuid":"880991283","full_name":"mkcms/bdx","owner":"mkcms","description":"ELF directory index + tools","archived":false,"fork":false,"pushed_at":"2025-01-14T12:11:19.000Z","size":266,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-29T05:57:02.326Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mkcms.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-30T18:15:47.000Z","updated_at":"2025-01-14T12:11:23.000Z","dependencies_parsed_at":"2024-11-29T18:24:46.327Z","dependency_job_id":"8b55a808-cb52-4311-a2a2-e58f56355f3e","html_url":"https://github.com/mkcms/bdx","commit_stats":null,"previous_names":["mkcms/bdx"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkcms%2Fbdx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkcms%2Fbdx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkcms%2Fbdx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mkcms%2Fbdx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mkcms","download_url":"https://codeload.github.com/mkcms/bdx/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236615525,"owners_count":19177567,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-05T04:22:32.414Z","updated_at":"2025-06-10T19:04:05.557Z","avatar_url":"https://github.com/mkcms.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bdx #\n\nAn indexer and graph generator for binary build directories.\n\nThis tool can be used to quickly search where an ELF symbol matching some\ncriteria is defined in a directory and generate graphs for various queries.\n\nFeatures:\n\n- Parallel, incremental indexing using sharded Xapian database\n- Indexes cross-references by analyzing ELF relocations\n- Query the database with a simple query language and custom output formats\n- Disassemble symbols matching a query\n- Generate symbol reference graphs in DOT format\n\n## Installation ##\n\nWith pip:\n\n    pip install .\n\nOr, for development:\n\n    pip install -e .[dev]\n\nFor optional graph generation (this installs `pygraphviz`):\n\n    pip install .[graphs]\n\n[xapian][xapian] is required to be installed on the system.\n\nTo install shell completion for the `bdx` command provided, put into shell init\nfile:\n\n    eval \"$(_BDX_COMPLETE=bash_source bdx)\" # or `zsh_source` on zsh\n\n### Getting Xapian ###\n\nYou need Xapian Python bindings, you can get them:\n\n1. By installing [**unofficial** Xapian bindings][xapian-bindings] Python\n   package with:\n\n        pip install xapian-bindings\n\n2. By running the provided [install_xapian_bindings.sh](./install_xapian_bindings.sh) script\n3. By manually downloading and installing them from [Xapian download page][xapian-downloads]\n\n## Usage ##\n\n### Indexing ###\n\nTo index a project that contains a `compile_commands.json` file:\n\n    bdx index -c\n\nOr you can specify the directory to index:\n\n    bdx index -d ./build\n\nThe indexer will only index files changed since last run.  With `--exclude`\noption you can choose directories/glob patterns to ignore.\n\nWhen indexing a large repository when a lot of files have changed, it's often\nbetter to use `--delete` option, to completely rebuild the index.  Removing\noutdated documents from the database is very slow.\n\nThe `index` command also accepts `-o`, `--opt` option which can be used to set\nsome indexing settings, e.g. to enable indexing relocations:\n\n    bdx index -d ./build --opt index_relocations=True\n\nAvailable options:\n\n- `num_processes` - number of parallel indexing processes (default=same as # of\n  CPUs).\n\n- `demangle_names` - if True (the default), then symbol names are demangled and\n  saved to the database.\n\n- `index_relocations` - if True, all relocations will be applied and indexed.\n  By default this is false.  Setting this to True will slow down indexing.\n\n- `min_symbol_size` - (default 1) only index symbols with size equal to or\n  greater than this.\n\n- `use_dwarfdump` - if True (the default), use `dwarfdump` program, if it's\n  available, to find the source file for a compiled file, if it can't be found\n  in any other way.\n\n- `save_filters` - if True (False by default), then exclusions provided with\n  `--exclude` option are saved for future runs.\n\n- `delete_saved_filters` - if True (False by default), then delete all previous\n  `--exclude` exclusions saved with `save_filters` option.\n\n### Disassembling ###\n\nAfter a directory is indexed, you can disassemble symbols matching a search\nquery.\n\nYou can set the command to disassemble with the `-D`, `--disassembler` option,\nwhich can contain `{}` placeholders for replacement.\n\n```\n$ bdx disass tree node defp source:./gcc/cp/parser* section:.text\n\n/src/gcc-12/build/gcc/cp/parser.o:     file format elf64-x86-64\n\n\nDisassembly of section .text:\n\n000000000000a1d0 \u003cdefparse_location(tree_node*)\u003e:\n    a1d0:\t48 8b 47 08          \tmov    0x8(%rdi),%rax\n    a1d4:\t48 8b 10             \tmov    (%rax),%rdx\n    a1d7:\t48 8b 40 08          \tmov    0x8(%rax),%rax\n    a1db:\t8b 7a 04             \tmov    0x4(%rdx),%edi\n    a1de:\t8b 50 04             \tmov    0x4(%rax),%edx\n    a1e1:\t89 fe                \tmov    %edi,%esi\n    a1e3:\te9 00 00 00 00       \tjmp    a1e8 \u003cdefparse_location(tree_node*)+0x18\u003e\n```\n\n### Searching ###\n\n`bdx search` and other commands accept a query string.  A simple query language\nis recognized.\n\n```\n$ bdx search -n 5 tree\ntree-eh.o: _ZL20outside_finally_tree8treempleP6gimple\nhooks.o: _Z14hook_void_treeP9tree_node\ntree-eh.o: _ZL22record_in_finally_tree8treempleP4gtry\nlanghooks.o: _Z20lhd_return_null_treeP9tree_node\nlanghooks.o: _Z23lhd_tree_dump_dump_treePvP9tree_node\n```\n\nThe `-n` option sets the maximum number of symbols to search for.\n\nThe `-f` option can be used to set output format (`json`, `sexp` or Python string format spec):\n\n```\n$ bdx search -n 5 -f json tree\n{\"path\": \"/src/gcc-12/build/stage1-gcc/tree-eh.o\", \"name\": \"_ZL20outside_finally_tree8treempleP6gimple\", \"section\": \".text\", \"address\": 12255, \"size\": 104, \"type\": \"FUNC\", \"relocations\": [\"\", \"_ZN10hash_tableI19finally_tree_hasherLb0E11xcallocatorE4findERKP17finally_tree_node\"], \"mtime\": 1652372105820280262, \"demangled\": \"outside_finally_tree(treemple, gimple*)\"}\n{\"path\": \"/src/gcc-12/build/prev-gcc/hooks.o\", \"name\": \"_Z14hook_void_treeP9tree_node\", \"section\": \".text\", \"address\": 560, \"size\": 1, \"type\": \"FUNC\", \"relocations\": [], \"mtime\": 1652375092039025278, \"demangled\": \"hook_void_tree(tree_node*)\"}\n{\"path\": \"/src/gcc-12/build/gcc/tree-eh.o\", \"name\": \"_ZL22record_in_finally_tree8treempleP4gtry\", \"section\": \".text\", \"address\": 13440, \"size\": 415, \"type\": \"FUNC\", \"relocations\": [\"\", \"_Z11fancy_abortPKciS0_\", \"_ZN10hash_tableI19finally_tree_hasherLb0E11xcallocatorE6expandEv\", \"prime_tab\", \"xmalloc\"], \"mtime\": 1652377778150208461, \"demangled\": \"record_in_finally_tree(treemple, gtry*)\"}\n{\"path\": \"/src/gcc-12/build/stage1-gcc/langhooks.o\", \"name\": \"_Z20lhd_return_null_treeP9tree_node\", \"section\": \".text\", \"address\": 278, \"size\": 15, \"type\": \"FUNC\", \"relocations\": [], \"mtime\": 1652372076295950259, \"demangled\": \"lhd_return_null_tree(tree_node*)\"}\n{\"path\": \"/src/gcc-12/build/stage1-gcc/langhooks.o\", \"name\": \"_Z23lhd_tree_dump_dump_treePvP9tree_node\", \"section\": \".text\", \"address\": 1692, \"size\": 19, \"type\": \"FUNC\", \"relocations\": [], \"mtime\": 1652372076295950259, \"demangled\": \"lhd_tree_dump_dump_tree(void*, tree_node*)\"}\n$ bdx search -n 5 -f '0x{address:0\u003e10x}|{section:\u003c10}|{type:8}|{demangled}' tree\n0x0000002fdf|.text     |FUNC    |outside_finally_tree(treemple, gimple*)\n0x0000000230|.text     |FUNC    |hook_void_tree(tree_node*)\n0x0000003480|.text     |FUNC    |record_in_finally_tree(treemple, gtry*)\n0x0000000116|.text     |FUNC    |lhd_return_null_tree(tree_node*)\n0x000000069c|.text     |FUNC    |lhd_tree_dump_dump_tree(void*, tree_node*)\n```\n\n\n#### Examples ####\n\n1. Search for symbols having `foo` AND `bar` somewhere in their name:\n\n        bdx search foo AND bar\n\n    or:\n\n        bdx search foo bar\n\n2. Search for symbols having either `foo` or `bar` in their name:\n\n        bdx search foo OR bar\n\n3. Search for symbols named _exactly_ `foo`:\n\n        bdx search fullname:foo\n\n4. Search for symbols where [Elf ST_INFO type][elf-manpage] is `STT_FUNC` or `STT_OBJECT`:\n\n        bdx search type:FUNC OR type:OBJECT\n\n5. Search for symbols `foo*` in binary files named `bar.o`:\n\n        bdx search 'name:foo*' path:bar.o\n\n6. Search for symbols in files compiled from source file named `file.c`:\n\n        bdx search source:file.c\n\n7. Search for symbols `foo` or `bar` that are not mangled (`_Z*` prefix):\n\n        bdx search '(foo OR bar)' AND NOT name:_Z*\n\n8. Search for symbols that reference/call `memset`:\n\n        bdx search relocations:memset\n\n9. Search for symbols that call `malloc`, but not `free`:\n\n        bdx search relocations:malloc NOT relocations:free\n\n10. Search for symbols with size in some range, where address is at least 0xfff0:\n\n        bdx search foo size:100..200 address:0xfff0..\n\n11. Search for symbols by relative path of the binary:\n\n        bdx search 'path:./build/module/*'\n\n12. Search for string literals:\n\n        bdx search 'path:\"/path/to/File With Spaces.o\"'\n\n13. Search for big symbols in some section:\n\n        bdx search section:.rodata AND size:1000..\n\n\n### Graph generation ###\n\nGenerate an SVG image showing at most 20 routes from symbol `main` in\n`main.o` to all symbols in section `.text` in files matching wildcard\n`Algorithms_*`:\n\n    bdx graph 'main path:main.o' 'section:\".text\" AND path:Algorithms*' -n 20 | dot -Tsvg \u003e graph.svg\n\nExample graphs: ![ASTAR](./examples/astar.svg) ![BFS](./examples/bfs.svg) ![DFS](./examples/dfs.svg)\n\nBy default this generates paths by using the ASTAR algorithm, the `--algorithm\nBFS` or `--algorithm DFS` options will use\nbreadth-first-search/depth-first-search algorithms which can generate different\ngraphs and can be slower/faster depending on the index and the queries\nprovided.\n\n## License ##\n\n```\nCopyright (C) 2024 Michał Krzywkowski\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n```\n\u003c!-- Local Variables: --\u003e\n\u003c!-- coding: utf-8 --\u003e\n\u003c!-- fill-column: 79 --\u003e\n\u003c!-- indent-tabs-mode: nil --\u003e\n\u003c!-- End: --\u003e\n\n[xapian]: https://xapian.org/\n[xapian-downloads]: https://xapian.org/download\n[xapian-bindings]: https://pypi.org/project/xapian-bindings/\n[elf-manpage]: https://manpages.ubuntu.com/manpages/oracular/en/man5/elf.5.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkcms%2Fbdx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmkcms%2Fbdx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmkcms%2Fbdx/lists"}