Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aras-p/ClangBuildAnalyzer

Clang build analysis tool using -ftime-trace
https://github.com/aras-p/ClangBuildAnalyzer

build-analysis clang

Last synced: 3 months ago
JSON representation

Clang build analysis tool using -ftime-trace

Awesome Lists containing this project

README 

        
# Clang Build Analyzer [![Build Status](https://github.com/aras-p/ClangBuildAnalyzer/workflows/build_and_test/badge.svg)](https://github.com/aras-p/ClangBuildAnalyzer/actions)



Clang C/C++ build analysis tool when using Clang 9+ `-ftime-trace`. The `-ftime-trace` compiler flag

(see [blog post](https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/) or

[Clang 9 release notes](https://releases.llvm.org/9.0.0/tools/clang/docs/ReleaseNotes.html#new-compiler-flags)) can be useful

to figure out what takes time during compilation of *one* source file. This tool helps to aggregate time trace

reports from multiple compilations, and output "what took the most time" summary:



- Which files are slowest to parse? i.e. spend time in compiler lexer/parser front-end

- Which C++ templates took the most time to instantiate?

- Which files are slowest to generate code for? i.e. spend time in compiler backend doing codegen and optimizations

- Which functions are slowest to generate code for?

- Which header files are included the most in the whole build, how much time is spent parsing them, and what are the include chains of them?



### Usage



1. **Start the build capture**: `ClangBuildAnalyzer --start `


   This will write current timestamp in a `ClangBuildAnalyzerSession.txt` file under the given `artifacts_folder`. The artifacts

   folder is where the compiled object files (and time trace report files) are expected to be produced by your build.

1. **Do your build**. Does not matter how; an IDE build, a makefile, a shell script, whatever. As long as it invokes

   Clang and passes `-ftime-trace` flag to the compiler (**Clang 9.0 or later is required** for this).

1. **Stop the build capture**: `ClangBuildAnalyzer --stop  `


   This will load all Clang time trace compatible `*.json` files under the given `artifacts_folder` that were modified after

   `--start` step was done (Clang `-ftime-trace` produces one JSON file next to each object file), process them and store data file into

   a binary `capture_file`.

1. **Run the build analysis**: `ClangBuildAnalyzer --analyze `


   This will read the `capture_file` produced by `--stop` step, calculate the slowest things and print them. If a

   `ClangBuildAnalyzer.ini` file exists in the current folder, it will be read to control how many of various things to print.



Aternatively, instead of doing `--start` and `--stop` steps, you can do `ClangBuildAnalyzer --all  ` after your build; that will

include all the compatible `*.json` files for analysis, no matter when they were produced.



### Analysis Output



The analysis output will look something like this:



```

Analyzing build trace from 'artifacts/FullCapture.bin'...

**** Time summary:

Compilation (7664 times):

  Parsing (frontend):         2118.9 s

  Codegen & opts (backend):   1204.1 s



**** Files that took longest to parse (compiler frontend):

  5084 ms: cycles_scene.build/RelWithDebInfo/volume.o

  4471 ms: extern_ceres.build/RelWithDebInfo/covariance_impl.o

  4225 ms: bf_intern_libmv.build/RelWithDebInfo/resect.o

  4121 ms: bf_blenkernel.build/RelWithDebInfo/volume_to_mesh.o

 

**** Files that took longest to codegen (compiler backend):

 47123 ms: bf_blenkernel.build/RelWithDebInfo/volume.o

 39617 ms: bf_blenkernel.build/RelWithDebInfo/volume_to_mesh.o

 37488 ms: bf_modifiers.build/RelWithDebInfo/MOD_volume_displace.o

 30676 ms: bf_gpu.build/RelWithDebInfo/gpu_shader_create_info.o



**** Templates that took longest to instantiate:

 11172 ms: fmt::detail::vformat_to (142 times, avg 78 ms)

  6662 ms: std::__scalar_hash::operator() (3549 times, avg 1 ms)

  6281 ms: std::__murmur2_or_cityhash::operator() (3549 times, avg 1 ms)

  5757 ms: std::basic_string::basic_string (3597 times, avg 1 ms)

  5541 ms: blender::CPPType::to_static_type_tag (30461 times, avg 1 ms)

 30098 ms: Eigen::MatrixBase<$> (8639 times, avg 3 ms)

 27524 ms: Eigen::internal::call_assignment_no_alias<$> (2397 times, avg 11 ms)



**** Functions that took longest to compile:

 28359 ms: gpu_shader_create_info_init (source/blender/gpu/intern/gpu_shader_create_info.cc)

  4090 ms: ccl::GetConstantValues(ccl::KernelData const*) (intern/cycles/device/metal/kernel.mm)

  3996 ms: gpu_shader_dependency_init (source/blender/gpu/intern/gpu_shader_dependency.cc)



**** Function sets that took longest to compile / optimize:

 10606 ms: bool openvdb::v10_0::tree::NodeList<$>::initNodeChildren<$>(openvdb:... (470 times, avg 22 ms)

  9640 ms: void tbb::interface9::internal::dynamic_grainsize_mode<$>::work_bala... (919 times, avg 10 ms)

  9459 ms: void tbb::interface9::internal::dynamic_grainsize_mode<$>::work_bala... (715 times, avg 13 ms)

  7279 ms: blender::Vector<$>::realloc_to_at_least(long long) (1840 times, avg 3 ms)

 

**** Expensive headers:

261580 ms: /Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/algorithm (included 3389 times, avg 77 ms), included via:

  341x: BKE_context.h BLI_string_ref.hh string 

  180x: DNA_mesh_types.h BLI_math_vector_types.hh array 

  125x: DNA_space_types.h DNA_node_types.h DNA_node_tree_interface_types.h BLI_function_ref.hh BLI_memory_utils.hh 

  ...



188777 ms: /Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/string (included 3447 times, avg 54 ms), included via:

  353x: BKE_context.h BLI_string_ref.hh 

  184x: DNA_mesh_types.h BLI_offset_indices.hh BLI_index_mask.hh BLI_linear_allocator.hh BLI_string_ref.hh 

  131x: DNA_node_types.h DNA_node_tree_interface_types.h BLI_span.hh 

  ...



174792 ms: source/blender/makesdna/DNA_node_types.h (included 1653 times, avg 105 ms), included via:

  316x: ED_screen.hh DNA_space_types.h 

  181x: DNA_space_types.h 

  173x: 

  ...

```



Granularity and amount of most expensive things (files, functions, templates, includes) that are reported can be controlled by having an

`ClangBuildAnalyzer.ini` file in the working directory. Take a look at [`ClangBuildAnalyzer.ini`](/ClangBuildAnalyzer.ini) for an example.



### Building it



* Windows: Visual Studio 2019 solution at `projects/vs2019/ClangBuildAnalyzer.sln`.

* Mac: Xcode 10.x project at `projects/xcode/ClangBuildAnalyzer.xcodeproj`.

* Linux: Makefile for gcc (tested with 7.4), build with `make -f projects/make/Makefile`.

* You can also use provided `CMakeLists.txt`, if you want to build using `CMake`.



### Limitations



* Does not capture anything related to linking or LTO right now.

* May or may not scale to _huge_ builds (I haven't tried on something ginormous like a Chrome

  build). However I have tried it on Unity editor and Blender builds and it worked fine.



### License



License for the Clang Build Analyzer itself is [Unlicense](https://unlicense.org/), i.e. public domain. However, the source code

includes several external library source files (all under `src/external`), each with their own license:



* `cute_files.h` from [RandyGaul/cute_headers](https://github.com/RandyGaul/cute_headers): zlib or public domain,

* `cwalk` from [likle/cwalk](https://github.com/likle/cwalk): MIT,

* `enkiTS`, from [dougbinks/enkiTS](https://github.com/dougbinks/enkiTS): zlib,

* `flat_hash_map`, from [skarupke/flat_hash_map](https://github.com/skarupke/flat_hash_map): Boost 1.0,

* `inih`, from [benhoyt/inih](https://github.com/benhoyt/inih): BSD 3 clause,

* `llvm-Demangle`, part of [LLVM](https://llvm.org/): Apache-2.0 with LLVM-exception,

* `simdjson` from [lemire/simdjson](https://github.com/lemire/simdjson): Apache-2.0, version 3.2.2 (2023 Aug),

* `sokol_time.h` from [floooh/sokol](https://github.com/floooh/sokol): zlib/libpng,

* `xxHash` from [Cyan4973/xxHash](https://github.com/Cyan4973/xxHash): BSD 2 clause.