{"id":20280441,"url":"https://github.com/jrmadsen/compile-time-perf","last_synced_at":"2026-03-07T03:30:56.714Z","repository":{"id":51438507,"uuid":"347287416","full_name":"jrmadsen/compile-time-perf","owner":"jrmadsen","description":"Measures high-level timing and memory usage metrics during compilation","archived":false,"fork":false,"pushed_at":"2021-05-11T21:52:26.000Z","size":40,"stargazers_count":73,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-04T02:42:46.432Z","etag":null,"topics":["compile-time","performance-analysis","performance-metrics","performance-monitoring"],"latest_commit_sha":null,"homepage":"","language":"CMake","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jrmadsen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-13T06:00:32.000Z","updated_at":"2025-02-24T08:55:55.000Z","dependencies_parsed_at":"2022-08-26T17:01:27.823Z","dependency_job_id":null,"html_url":"https://github.com/jrmadsen/compile-time-perf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jrmadsen/compile-time-perf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrmadsen%2Fcompile-time-perf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrmadsen%2Fcompile-time-perf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrmadsen%2Fcompile-time-perf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrmadsen%2Fcompile-time-perf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jrmadsen","download_url":"https://codeload.github.com/jrmadsen/compile-time-perf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jrmadsen%2Fcompile-time-perf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30206562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T03:24:23.086Z","status":"ssl_error","status_checked_at":"2026-03-07T03:23:11.444Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compile-time","performance-analysis","performance-metrics","performance-monitoring"],"created_at":"2024-11-14T13:35:39.260Z","updated_at":"2026-03-07T03:30:56.692Z","avatar_url":"https://github.com/jrmadsen.png","language":"CMake","funding_links":[],"categories":[],"sub_categories":[],"readme":"# compile-time-perf\n\nI work with a lot of larger projects and do a lot of template meta-programming so while I absolutely love the flamegraphs from\nthe `-ftime-trace` compiler flag, I've had a hard time detecting when changes affect the total compilation time and which\nfiles saw the most dramatic increase/decrease in compile time.\n\nThus, I created `compile-time-perf` (CTP), which is designed to be a high-level \"profiler\" for compiling large projects.\n__*It is designed to be simple to install, compiler and language agnostic, and included as part of CI.*__\nIt is not intended to replace compiler flags like `-ftime-trace` but supplement them.\nThe main problem with `-ftime-trace` is that it provides no high-level data w.r.t. _which_ files to focus on, so in large projects,\nit can be difficult to locate which file should be focused on first. Another problem is that it only provides timing data --\nif the system has limited memory resources but has 4+ cores, your compile times can shoot up drastically if each core is compiling\nsomething requiring 2+ GB of memory because you'll start using swap.\n\nDown the road, I could see the \"analyzer\" script actually including support for detecting `-ftime-trace` in the compile command\nlogs, searching the folder structure for the JSON, and then using/providing something like\n[ClangBuildAnalyzer](https://github.com/aras-p/ClangBuildAnalyzer) to provide more in-depth details.\nBut I think the useful enough by itself right now.\n\nCTP essentially uses a UNIX `time`-like command-line tool to launch the compile commands, called\n[timem](https://github.com/NERSC/timemory/blob/develop/source/tools/timemory-timem/README.md)\n([docs](https://timemory.readthedocs.io/en/develop/tools/timemory-timem/README.html)),\nwhich I built using the [timemory toolkit](https://github.com/NERSC/timemory) --\na modular C++ template library for build profiling tools which, recursively, is one of the two primary places\nwhere I need this functionality (other is [Kokkos](https://github.com/kokkos/kokkos)).\n\nTimem doesn't do anything particularly fancy: it just forks and does a mix of deterministic phase measurements and\n(on Linux) some statistical sampling of a few `/proc/\u003cpid\u003e` files while the command executes. Then that data along\nwith the command executed are put into a JSON file whose name is generated from an md5sum of command executed\n(for uniqueness and reproducibility). Then the Python \"analyzer\" script just globs the files and directories it is\npassed and combines that data, does some sorting and filtering to make the commands more easily readable and thats it.\n\n## Building CTP\n\nStandard cmake build system without any project-specific options.\nCTP uses the [timem](https://github.com/NERSC/timemory/blob/develop/source/tools/timemory-timem/README.md)\nexecutable from the [timemory toolkit](https://github.com/NERSC/timemory) to do the measurements on the compile commands.\n[Timemory](https://timemory.readthedocs.io/en/develop/) is included as a git submodule and\ncmake will automatically run `git submodule update --init` if you don't.\n\n```console\n# clone\ngit clone https://github.com/jrmadsen/compile-time-perf.git\n# configure (it's not hanging, timemory can take a while here)\ncmake -B compile-time-perf-build -D CMAKE_INSTALL_PREFIX=/usr/local compile-time-perf\n# build\ncmake --build compile-time-perf-build --target all\n# install\ncmake --build compile-time-perf-build --target install\n```\n\nWhen configuring your project, just set append the CMAKE_INSTALL_PREFIX value to the CMAKE_PREFIX_PATH environment variable.\n\nMinimum requirements:\n\n- CMake (minimum: v3.13)\n- C++ compiler supporting C++14\n- Python interpreter\n\n\u003e __*CTP is not supported on Windows currently*__. It can be made available easily if someone with more access to a Windows\n\u003e machine than me wants to write the [_spawn](https://docs.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functions?view=msvc-160)\n\u003e implementation for the `timem` executable. It won't be hard to get a working prototype:\n\u003e - the timem source code is only about 1,200 lines of code\n\u003e - timemory already has Windows support for the timers, peak RSS, and page RSS on Windows -- the most relevant values\n\u003e - it is 100% trivial to locally disable any non-working measurement type for windows builds\n\u003e   - it's literally 1 LOC: `TIMEMORY_DEFINE_CONCRETE_TRAIT(is_available, \u003ccomponent\u003e, false_type)` and it will disappear from every template instantiation\n\u003e\n\u003e So if you would like to use this on Windows and have an afternoon to spare, feel free to\n\u003e [file an issue in timemory](https://github.com/NERSC/timemory/issues) and I can help you get started.\n\n## Quick Start\n\ncompile-time-perf (CTP) is designed primarily for CMake but if you want to use it manually, see the [Manual Usage](#manual-usage) section.\n\n### Setup\n\nAdd this to your main CMakeLists.txt somewhere after `project(...)`. I'd recommend replacing `foo` below with `${PROJECT_NAME}`.\n\n```cmake\nfind_package(compile-time-perf REQUIRED)\nenable_compile_time_perf(foo-ctp)\n```\n\nor, making it optional:\n\n```cmake\nfind_package(compile-time-perf)\nif(compile-time-perf_FOUND)\n    enable_compile_time_perf(foo-ctp)\nendif()\n```\n\nThe argument `foo-ctp` is just a \"NAME\" used to generate two \"helper\" targets: one for running the analysis (`analyze-${NAME}`) and another for cleaning up the generated files (`clean-${NAME}`).\n\n### Building and Analyzing\n\nOnce you've added the [cmake setup](#setup), configure and build your code normally, and then build the `analyze-${NAME}` target, e.g. `analyze-foo-ctp`.\n\n```console\n$ cmake --build . --target all\n\n[1/81] Building CXX object source/timemory/CMakeFiles/timemory-core-shared.dir/utility/popen.cpp.o\n[/opt/local/bin/clang++]\u003e Outputting '.compile-time-perf-timem-output/foo-ctp-33ce7f2719c9a3f0a9147cf3f1dfc242.json'...\n...\n\n$ cmake --build . --target analyze-foo-ctp\n\nwall_clock          74.947 sec  clang++ tools/timemory-avail/timemory-avail.cpp\nwall_clock          42.742 sec  clang++ tools/timemory-timem/timem.cpp\nwall_clock           4.092 sec  clang++ tools/timemory-pid/timemory-pid.cpp\n\ncpu_clock          74.150 sec  clang++ tools/timemory-avail/timemory-avail.cpp\ncpu_clock          41.970 sec  clang++ tools/timemory-timem/timem.cpp\ncpu_clock           3.680 sec  clang++ tools/timemory-pid/timemory-pid.cpp\n\npeak_rss        2040.410 MB   clang++ tools/timemory-avail/timemory-avail.cpp\npeak_rss        1040.617 MB   clang++ tools/timemory-timem/timem.cpp\npeak_rss         194.691 MB   clang++ tools/timemory-pid/timemory-pid.cpp\n```\n\n\u003e NOTE: The JSON filename (e.g. `33ce...c242.json` above) is just the md5sum of the compile command without spaces, e.g. for the command `g++ foo.cpp -o foo`, the md5sum is computed from `g++foo.cpp-ofoo`. This is used to ensure uniqueness and reproducibility.\n\n### Advanced Tutorial\n\nIn CMake, CTP uses what is called the `RULE_LAUNCH_COMPILE` property to prefix every compile command and `enable_compile_time_perf(...)` is a CMake macro that sets it.\nThis macro has quite a few features to enable getting the exact amount of information emitted during CI. For example, you may want to create a CTest\nwhich depends on the `all` target being built and you want the compile command to print an abbreviated path and include the optimization/arch flags\nso that you can set `FAIL_REGULAR_EXPRESSION` if the build time exceeded the range defined in the expression.\nIn other words, if the failure case is the wall-clock compile time exceeding 30 seconds with the clang compiler and `-O3 -march=native`,\ntransforming `clang++ source/tools/timemory-timem/timem.cpp` into rendering `clang++ -O3 -march=native timem.cpp` for\n`^wall_clock    [3-9][0-9].([0-9]+) sec  clang.. -O3 -march=native timem.cpp\"` is supported.\n\n- It can be applied globally, per-project, per-directory, and/or per-target\n  - `GLOBAL` (zero args)\n  - `PROJECT` (zero args)\n  - `DIRECTORY` (1+ args)\n  - `TARGET` (1+ args)\n- You can pass options to timem\n  - `TIMEM_OPTIONS` (1+ args)\n  - See `timem --help`\n- You can pass options to the python analysis script\n  - `ANALYZER_OPTIONS` (1+ args)\n  - See `compile-time-perf-analyzer --help`\n- You can prefix link commands as well as compile commands\n  - `LINK` (zero args)\n- You can customize the output directory\n  - `TIMEM_OUTPUT_DIRECTORY` (one arg)\n\n```cmake\nadd_library(foo ...)\n\nenable_compile_time_perf(foo-ctp\n    LINK                                    # include link command\n    TARGET\n        foo                                 # only apply to foo target\n    TIMEM_OPTIONS\n        --disable-sampling                  # disable timem sampling /proc/pid on Linux\n    ANALYZER_OPTIONS                        #\n        -m wall_clock peak_rss cpu_clock    # only report these metrics\n        -s \"${PROJECT_SOURCE_DIR}/\"         # remove this strings from prefix/suffix\n        -i \"^(-D).*\"                        # include the compile definitions in the labels\n        -n 5                                # only show first 5 entries\n    )\n\n# enable unity builds\nset(CMAKE_UNITY_BUILD ON)\n\nadd_library(bar ...)\n\nenable_compile_time_perf(bar-ctp\n    TARGET\n        bar\n    ANALYZER_OPTIONS\n        -r \".dir/Unity/unity_[0-9]_(cxx|cpp).(cxx|cpp)\" # remove unity build generated path\n        -i \"^(-).*\"             # include everything starting with hyphen in label\n        -e \"^(-D).*(_EXPORTS)$\" # except for definitions ending with _EXPORTS\n    )\n```\n\n## Manual Usage\n\nCTP installs a simple Python script called `compile-time-perf-analyzer`. Usage is fairly staight-forward.\n\n```console\n$ compile-time-perf-analyzer --help\nusage: compile-time-perf-analyzer [-h] [-v] [-n MAX_ENTRIES] [-i [INCLUDE_REGEX [INCLUDE_REGEX ...]]]\n                                  [-e [EXCLUDE_REGEX [EXCLUDE_REGEX ...]]] [-f [FILE_EXTENSIONS [FILE_EXTENSIONS ...]]]\n                                  [-s [STRIP [STRIP ...]]] [-r [REGEX_STRIP [REGEX_STRIP ...]]] [-m METRIC [METRIC ...]]\n                                  [-l]\n                                  [inputs [inputs ...]]\n\nMeasures high-level timing and memory usage metrics during compilation\n\npositional arguments:\n    inputs                                      List of JSON files or directory containing JSON files from timem\n\noptional arguments:\n    -h, --help                                  show this help message and exit\n    -v, --verbose, --debug                      Print out verbosity messages (i.e. debug messages)\n    -n MAX_ENTRIES                              Max number of entries to display\n    -i [INCLUDE_REGEX [INCLUDE_REGEX ...]]      List of regex expressions for including command-line arguments\n    -e [EXCLUDE_REGEX [EXCLUDE_REGEX ...]]      List of regex expressions for removing command-line arguments\n    -f [FILE_EXTENSIONS [FILE_EXTENSIONS ...]]  List of file extensions (w/o period) to include in label. Use 'lang-c',\n                                                'lang-cxx', 'lang-fortran' keywords to include common extensions. Use\n                                                'lang-all' to include all defaults. Use 'none' to disable all extension\n                                                filtering\n    -s [STRIP [STRIP ...]]                      List of string to strip from the start/end of the labels\n    -r [REGEX_STRIP [REGEX_STRIP ...]]          List of regular expressions to strip from the start/end of the labels\n    -m METRIC [METRIC ...]                      List of metrics to display\n    -l, --list-metrics                          List the metrics which can be potentially reported and exit\n\nAll arguments after standalone '--' are treated as input files/directories, e.g. ./foo \u003coptions\u003e -- bar.txt baz/\n```\n\nIn order to generate the JSON files for the Python script, just prefix every compile command with `timem -o \u003cDIR\u003e/%m -q --`\nwhere `\u003cDIR\u003e` is the common directory for the output files, `%m` instructs timem to generate an md5sum of everything after\nthe `--`, `-q` just instructs timem to not output to console.\n\n```shell\n# original command\ng++ foo.cpp -o foo\n\n# manual command\ntimem -o foo-ctp/%m -q -- g++ foo.cpp -o foo\n```\n\n## Example Usage\n\nIn timemory, the usage looks like this in the main CMakeLists.txt:\n\n```cmake\noption(TIMEMORY_USE_CTP \"Enable compile-time-perf\" OFF)\nmark_as_advanced(TIMEMORY_USE_CTP)\n\nif(TIMEMORY_BUILD_DEVELOPER OR TIMEMORY_USE_CTP)\n    find_package(compile-time-perf)\n    if(compile-time-perf_FOUND)\n        enable_compile_time_perf(timemory-compile-time\n            LINK\n            ANALYZER_OPTIONS\n                -s \"${PROJECT_BINARY_DIR}/\" \"${PROJECT_SOURCE_DIR}/\"\n                -f \"lang-all\" \"so\" \"a\" \"dylib\" \"dll\"\n                -i \".*(_tests)$\" \"^(ex_).*\"\n                -e \"^(@rpath).*\" \"^(/usr)\" \"^(/opt)\")\n        set(TIMEMORY_USE_CTP ON)\n    else()\n        set(TIMEMORY_USE_CTP OFF)\n    endif()\nendif()\n```\n\nLater, in `source/tests/CMakeLists.txt`, a ctest is created which executes during CI alongside all the tests\n(at which point, all the code has been built):\n\n```cmake\nif(TIMEMORY_USE_CTP AND compile-time-perf_ANALYZER_EXECUTABLE)\n    add_test(\n        NAME                compile-time-perf\n        COMMAND             ${CMAKE_COMMAND}\n                            --build ${PROJECT_BINARY_DIR}\n                            --target analyze-timemory-compile-time\n        WORKING_DIRECTORY   ${PROJECT_BINARY_DIR})\nendif()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjrmadsen%2Fcompile-time-perf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjrmadsen%2Fcompile-time-perf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjrmadsen%2Fcompile-time-perf/lists"}