{"id":18711168,"url":"https://github.com/rocm/rocprofiler-systems","last_synced_at":"2026-03-10T14:34:27.054Z","repository":{"id":257851812,"uuid":"856526112","full_name":"ROCm/rocprofiler-systems","owner":"ROCm","description":"[DEPRECATED] Moved to ROCm/rocm-systems repo","archived":false,"fork":false,"pushed_at":"2026-02-26T14:53:34.000Z","size":12122,"stargazers_count":26,"open_issues_count":1,"forks_count":21,"subscribers_count":7,"default_branch":"develop_deprecated","last_synced_at":"2026-02-26T20:41:31.912Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/ROCm/rocm-systems","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ROCm.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-09-12T18:12:56.000Z","updated_at":"2026-01-20T07:55:51.000Z","dependencies_parsed_at":"2025-04-29T00:29:36.739Z","dependency_job_id":"cf627742-7055-4aca-b008-569e5afe714f","html_url":"https://github.com/ROCm/rocprofiler-systems","commit_stats":null,"previous_names":["rocm/rocprofiler-systems"],"tags_count":55,"template":false,"template_full_name":null,"purl":"pkg:github/ROCm/rocprofiler-systems","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Frocprofiler-systems","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Frocprofiler-systems/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Frocprofiler-systems/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Frocprofiler-systems/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ROCm","download_url":"https://codeload.github.com/ROCm/rocprofiler-systems/tar.gz/refs/heads/develop_deprecated","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Frocprofiler-systems/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30337302,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T12:41:07.687Z","status":"ssl_error","status_checked_at":"2026-03-10T12:41:06.728Z","response_time":106,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T12:37:32.775Z","updated_at":"2026-03-10T14:34:27.033Z","avatar_url":"https://github.com/ROCm.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ROCm Systems Profiler: Application profiling, tracing, and analysis\n\n\u003e [!CAUTION]\n\u003e The rocprofiler-systems repository is retired, please use the [ROCm/rocm-systems](https://github.com/ROCm/rocm-systems) repository\n\n[![Ubuntu 20.04 with GCC, ROCm, and MPI](https://github.com/ROCm/rocprofiler-systems/actions/workflows/ubuntu-focal.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/ubuntu-focal.yml)\n[![Ubuntu 22.04 (GCC, Python, ROCm)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/ubuntu-jammy.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/ubuntu-jammy.yml)\n[![OpenSUSE 15.x with GCC](https://github.com/ROCm/rocprofiler-systems/actions/workflows/opensuse.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/opensuse.yml)\n[![RedHat Linux (GCC, Python, ROCm)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/redhat.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/redhat.yml)\n[![Installer Packaging (CPack)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/cpack.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/cpack.yml)\n[![Documentation](https://github.com/ROCm/rocprofiler-systems/actions/workflows/docs.yml/badge.svg)](https://github.com/ROCm/rocprofiler-systems/actions/workflows/docs.yml)\n\n\u003e [!NOTE]\n\u003e If you are using a version of ROCm prior to ROCm 6.3.1 and are experiencing problems viewing your trace in the latest version of [Perfetto](http://ui.perfetto.dev), then try using [Perfetto UI v46.0](https://ui.perfetto.dev/v46.0-35b3d9845/#!/).\n\n## Overview\n\nROCm Systems Profiler (rocprofiler-systems), formerly Omnitrace, is a comprehensive profiling and tracing tool for parallel applications written in C, C++, Fortran, HIP, OpenCL, and Python which execute on the CPU or CPU+GPU.\nIt is capable of gathering the performance information of functions through any combination of binary instrumentation, call-stack sampling, user-defined regions, and Python interpreter hooks.\nROCm Systems Profiler supports interactive visualization of comprehensive traces in the web browser in addition to high-level summary profiles with mean/min/max/stddev statistics.\nIn addition to runtimes, ROCm Systems Profiler supports the collection of system-level metrics such as the CPU frequency, GPU temperature, and GPU utilization, process-level metrics\nsuch as the memory usage, page-faults, and context-switches, and thread-level metrics such as memory usage, CPU time, and numerous hardware counters.\n\n\u003e [!NOTE]\n\u003e Full documentation is available at [ROCm Systems Profiler documentation](https://rocm.docs.amd.com/projects/rocprofiler-systems/en/latest/index.html) in an organized, easy-to-read, searchable format.\nThe documentation source files reside in the [`/docs`](/docs) folder of this repository. For information on contributing to the documentation, see\n[Contribute to ROCm documentation](https://rocm.docs.amd.com/en/latest/contribute/contributing.html)\n\n### Data collection modes\n\n- Dynamic instrumentation\n  - Runtime instrumentation\n    - Instrument executable and shared libraries at runtime\n  - Binary rewriting\n    - Generate a new executable and/or library with instrumentation built-in\n- Statistical sampling\n  - Periodic software interrupts per-thread\n- Process-level sampling\n  - Background thread records process-, system- and device-level metrics while the application executes\n- Causal profiling\n  - Quantifies the potential impact of optimizations in parallel codes\n\n### Data analysis\n\n- High-level summary profiles with mean/min/max/stddev statistics\n  - Low overhead, memory efficient\n  - Ideal for running at scale\n- Comprehensive traces\n  - Every individual event/measurement\n- Application speedup predictions resulting from potential optimizations in functions and lines of code (causal profiling)\n\n### Parallelism API support\n\n- HIP\n- HSA\n- Pthreads\n- MPI\n- Kokkos-Tools (KokkosP)\n- OpenMP-Tools (OMPT)\n\n### GPU metrics\n\n- GPU hardware counters\n- HIP API tracing\n- HIP kernel tracing\n- HSA API tracing\n- HSA operation tracing\n- rocDecode API tracing\n- rocJPEG API tracing\n- System-level sampling (via AMD-SMI)\n  - Memory usage\n  - Power usage\n  - Temperature\n  - Utilization\n  - VCN Utilization\n  - JPEG Utilization\n\n\u003e [!NOTE]\n\u003e The availability of VCN and JPEG engine utilization depends on device support for different ASICs. If unsupported, all values for VCN_ACTIVITY and JPEG_ACTIVITY will be reported as N/A in the output of `amd-smi metric --usage`.\n\n### CPU metrics\n\n- CPU hardware counters sampling and profiles\n- CPU frequency sampling\n- Various timing metrics\n  - Wall time\n  - CPU time (process and/or thread)\n  - CPU utilization (process and/or thread)\n  - User CPU time\n  - Kernel CPU time\n- Various memory metrics\n  - High-water mark (sampling and profiles)\n  - Memory page allocation\n  - Virtual memory usage\n- Network statistics\n- I/O metrics\n- ... many more\n\n## Quick start\n\n### Installation\n\n- Visit [Releases](https://github.com/ROCm/rocprofiler-systems/releases) page\n- Select appropriate installer (recommendation: `.sh` scripts do not require super-user priviledges unlike the DEB/RPM installers)\n  - If targeting a ROCm application, find the installer script with the matching ROCm version\n  - If you are unsure about your Linux distro, check `/etc/os-release` or use the `rocprofiler-systems-install.py` script\n\nIf the above recommendation is not desired, download the `rocprofiler-systems-install.py` and specify `--prefix \u003cinstall-directory\u003e` when\nexecuting it. This script will attempt to auto-detect a compatible OS distribution and version.\nIf ROCm support is desired, specify `--rocm X.Y` where `X` is the ROCm major version and `Y`\nis the ROCm minor version, e.g. `--rocm 6.2`.\n\n```console\nwget https://github.com/ROCm/rocprofiler-systems/releases/latest/download/rocprofiler-systems-install.py\npython3 ./rocprofiler-systems-install.py --prefix /opt/rocprofiler-systems --rocm 6.2\n```\n\nSee the [ROCm Systems Profiler installation guide](https://rocm.docs.amd.com/projects/rocprofiler-systems/en/latest/install/install.html) for detailed information.\n\n### Setup\n\n\u003e [!NOTE]\n\u003e Replace `/opt/rocprofiler-systems` below with installation prefix as necessary.\n\n- **Option 1**: Source `setup-env.sh` script\n\n```bash\nsource /opt/rocprofiler-systems/share/rocprofiler-systems/setup-env.sh\n```\n\n- **Option 2**: Load modulefile\n\n```bash\nmodule use /opt/rocprofiler-systems/share/modulefiles\nmodule load rocprofiler-systems\n```\n\n- **Option 3**: Manual\n\n```bash\nexport PATH=/opt/rocprofiler-systems/bin:${PATH}\nexport LD_LIBRARY_PATH=/opt/rocprofiler-systems/lib:${LD_LIBRARY_PATH}\n```\n\n### Testing environment\n\nThe `build-docker` script can be used to create a testing environment. To see the available options, use the following commands:\n\n```shell\ncd docker\n./build-docker.sh --help\n```\n\n\u003e [!NOTE]\n\u003e The `-m` argument can be used to show supported OS + ROCm combinations.\n\n**Example:** To set up an Ubuntu 24.04 + ROCm 6.4 + Python 3.12 environment for building and testing, run the following commands:\n\n```shell\ncd docker\n./build-docker.sh --distro ubuntu --versions 24.04                               \\\n        --rocm-versions 6.4 --python-versions 12 --retry 1\ndocker run -v \"$(cd .. \u0026\u0026 pwd)\":/home/development                                \\\n        -it -w /home/development                                                 \\\n        --device /dev/kfd --device /dev/dri                                      \\\n        $(whoami)/rocprofiler-systems:release-base-ubuntu-24.04-rocm-6.4\n```\n\nInside the container, clean, build, and install the project with testing enabled using the following commands:\n\n```shell\nrm -rf rocprof-sys-build\ncmake -B rocprof-sys-build -S .                                                  \\\n       -D CMAKE_INSTALL_PREFIX=/opt/rocprofiler-systems                          \\\n       -D ROCPROFSYS_USE_PYTHON=ON      -D ROCPROFSYS_BUILD_DYNINST=ON           \\\n       -D ROCPROFSYS_BUILD_TBB=ON       -D ROCPROFSYS_BUILD_BOOST=ON             \\\n       -D ROCPROFSYS_BUILD_ELFUTILS=ON  -D ROCPROFSYS_BUILD_LIBIBERTY=ON         \\\n       -D ROCPROFSYS_BUILD_TESTING=ON\ncmake --build rocprof-sys-build --target all --parallel 8\ncmake --build rocprof-sys-build --target install\nsource /opt/rocprofiler-systems/share/rocprofiler-systems/setup-env.sh\n```\n\n\u003e [!NOTE]\n\u003e If you see \"dubious ownership\" Git errors when working in the container, run:\n\u003e\n\u003e ```shell\n\u003e git config --global --add safe.directory /home/development\n\u003e ```\n\u003e\n\u003e and\n\u003e\n\u003e ```shell\n\u003e git config --global --add safe.directory /home/development/external/timemory\n\u003e ```\n\nThen, use the following command to start automated testing:\n\n```shell\nctest --test-dir rocprof-sys-build --output-on-failure\n```\n\nTo enable MPI testing inside the container, set the following environment variables:\n\n```shell\nexport OMPI_ALLOW_RUN_AS_ROOT=1\nexport OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1\n```\n\nFor manual testing, you can find the executables in `rocprof-sys-build/bin`.\n\n### ROCm Systems Profiler settings\n\nGenerate a rocprofiler-systems configuration file using `rocprof-sys-avail -G rocprof-sys.cfg`. Optionally, use `rocprof-sys-avail -G rocprof-sys.cfg --all` for\na verbose configuration file with descriptions, categories, etc. Modify the configuration file as desired, e.g. enable\n[perfetto](https://perfetto.dev/), [timemory](https://github.com/ROCm/timemory), sampling, and process-level sampling by default\nand tweak some sampling default values:\n\n```console\n# ...\nROCPROFSYS_TRACE                = true\nROCPROFSYS_PROFILE              = true\nROCPROFSYS_USE_SAMPLING         = true\nROCPROFSYS_USE_PROCESS_SAMPLING = true\n# ...\nROCPROFSYS_SAMPLING_FREQ        = 50\nROCPROFSYS_SAMPLING_CPUS        = all\nROCPROFSYS_SAMPLING_GPUS        = $env:HIP_VISIBLE_DEVICES\n```\n\nOnce the configuration file is adjusted to your preferences, either export the path to this file via `ROCPROFSYS_CONFIG_FILE=/path/to/rocprof-sys.cfg`\nor place this file in `${HOME}/.rocprof-sys.cfg` to ensure these values are always read as the default. If you wish to change any of these settings,\nyou can override them via environment variables or by specifying an alternative `ROCPROFSYS_CONFIG_FILE`.\n\n### Call-Stack sampling\n\nThe `rocprof-sys-sample` executable is used to execute call-stack sampling on a target application without binary instrumentation.\nUse a double-hypen (`--`) to separate the command-line arguments for `rocprof-sys-sample` from the target application and it's arguments.\n\n```shell\nrocprof-sys-sample --help\nrocprof-sys-sample \u003crocprof-sys-options\u003e -- \u003cexe\u003e \u003cexe-options\u003e\nrocprof-sys-sample -f 1000 -- ls -la\n```\n\n### Binary instrumentation\n\nThe `rocprof-sys-instrument` executable is used to instrument an existing binary. Call-stack sampling can be enabled alongside\nthe execution an instrumented binary, to help \"fill in the gaps\" between the instrumentation via setting the `ROCPROFSYS_USE_SAMPLING`\nconfiguration variable to `ON`.\nSimilar to `rocprof-sys-sample`, use a double-hypen (`--`) to separate the command-line arguments for `rocprof-sys-instrument` from the target application and it's arguments.\n\n```shell\nrocprof-sys-instrument --help\nrocprof-sys-instrument \u003crocprof-sys-options\u003e -- \u003cexe-or-library\u003e \u003cexe-options\u003e\n```\n\n#### Binary rewrite\n\nRewrite the text section of an executable or library with instrumentation:\n\n```shell\nrocprof-sys-instrument -o app.inst -- /path/to/app\n```\n\nIn binary rewrite mode, if you also want instrumentation in the linked libraries, you must also rewrite those libraries.\nExample of rewriting the functions starting with `\"hip\"` with instrumentation in the amdhip64 library:\n\n```shell\nmkdir -p ./lib\nrocprof-sys-instrument -R '^hip' -o ./lib/libamdhip64.so.4 -- /opt/rocm/lib/libamdhip64.so.4\nexport LD_LIBRARY_PATH=${PWD}/lib:${LD_LIBRARY_PATH}\n```\n\n\u003e [!NOTE]\n\u003e Verify via `ldd` that your executable will load the instrumented library. If you built your executable with an RPATH to the original library's directory, then prefixing `LD_LIBRARY_PATH` will have no effect.\n\nOnce you have rewritten your executable and/or libraries with instrumentation, you can just run the (instrumented) executable\nor exectuable which loads the instrumented libraries normally, e.g.:\n\n```shell\nrocprof-sys-run -- ./app.inst\n```\n\nIf you want to re-define certain settings to new default in a binary rewrite, use the `--env` option. This `rocprof-sys` option\nwill set the environment variable to the given value but will not override it. E.g. the default value of `ROCPROFSYS_PERFETTO_BUFFER_SIZE_KB`\nis 1024000 KB (1 GiB):\n\n```shell\n# buffer size defaults to 1024000\nrocprof-sys-instrument -o app.inst -- /path/to/app\nrocprof-sys-run -- ./app.inst\n```\n\nPassing `--env ROCPROFSYS_PERFETTO_BUFFER_SIZE_KB=5120000` will change the default value in `app.inst` to 5120000 KiB (5 GiB):\n\n```shell\n# defaults to 5 GiB buffer size\nrocprof-sys-instrument -o app.inst --env ROCPROFSYS_PERFETTO_BUFFER_SIZE_KB=5120000 -- /path/to/app\nrocprof-sys-run -- ./app.inst\n```\n\n```shell\n# override default 5 GiB buffer size to 200 MB via command-line\nrocprof-sys-run --trace-buffer-size=200000 -- ./app.inst\n# override default 5 GiB buffer size to 200 MB via environment\nexport ROCPROFSYS_PERFETTO_BUFFER_SIZE_KB=200000\nrocprof-sys-run -- ./app.inst\n```\n\n#### Runtime instrumentation\n\nRuntime instrumentation will not only instrument the text section of the executable but also the text sections of the\nlinked libraries. Thus, it may be useful to exclude those libraries via the `-ME` (module exclude) regex option\nor exclude specific functions with the `-E` regex option.\n\n```shell\nrocprof-sys-instrument -- /path/to/app\nrocprof-sys-instrument -ME '^(libhsa-runtime64|libz\\\\.so)' -- /path/to/app\nrocprof-sys-instrument -E 'rocr::atomic|rocr::core|rocr::HSA' --  /path/to/app\n```\n\n### Python profiling and tracing\n\nUse the `rocprof-sys-python` script to profile/trace Python interpreter function calls.\nUse a double-hypen (`--`) to separate the command-line arguments for `rocprof-sys-python` from the target script and it's arguments.\n\n```shell\nrocprof-sys-python --help\nrocprof-sys-python \u003crocprof-sys-options\u003e -- \u003cpython-script\u003e \u003cscript-args\u003e\nrocprof-sys-python -- ./script.py\n```\n\n\u003e [!NOTE]\n\u003e The first argument after the double-hyphen must be a Python script, e.g. `rocprof-sys-python -- ./script.py`.\n\nIf you need to specify a specific python interpreter version, use `rocprof-sys-python-X.Y` where `X.Y` is the Python\nmajor and minor version:\n\n```shell\nrpcprof-sys-python-3.8 -- ./script.py\n```\n\nIf you need to specify the full path to a Python interpreter, set the `PYTHON_EXECUTABLE` environment variable:\n\n```shell\nPYTHON_EXECUTABLE=/opt/conda/bin/python rocprof-sys-python -- ./script.py\n```\n\nIf you want to restrict the data collection to specific function(s) and its callees, pass the `-b` / `--builtin` option after decorating the\nfunction(s) with `@profile`. Use the `@noprofile` decorator for excluding/ignoring function(s) and its callees:\n\n```python\ndef foo():\n    pass\n\n@noprofile\ndef bar():\n    foo()\n\n@profile\ndef spam():\n    foo()\n    bar()\n```\n\nEach time `spam` is called during profiling, the profiling results will include 1 entry for `spam` and 1 entry\nfor `foo` via the direct call within `spam`. There will be no entries for `bar` or the `foo` invocation within it.\n\n### Trace visualization\n\n- Visit [ui.perfetto.dev](https://ui.perfetto.dev) in the web-browser\n- Select \"Open trace file\" from panel on the left\n- Locate the rocprofiler-systems perfetto output (extension: `.proto`)\n\n![rocprof-sys-perfetto](docs/data/rocprof-sys-perfetto.png)\n\n![rocprof-sys-rocm](docs/data/rocprof-sys-rocm.png)\n\n![rocprof-sys-rocm-flow](docs/data/rocprof-sys-rocm-flow.png)\n\n![rocprof-sys-user-api](docs/data/rocprof-sys-user-api.png)\n\n## Using Perfetto tracing with system backend\n\nPerfetto tracing with the system backend supports multiple processes writing to the same\noutput file. Thus, it is a useful technique if rocprofiler-systems is built with partial MPI support\nbecause all the perfetto output will be coalesced into a single file. The\ninstallation docs for perfetto can be found [here](https://perfetto.dev/docs/contributing/build-instructions).\nIf you are building rocprofiler-systems from source, you can configure CMake with `ROCPROFSYS_INSTALL_PERFETTO_TOOLS=ON`\nand the `perfetto` and `traced` applications will be installed as part of the build process. However,\nit should be noted that to prevent this option from accidentally overwriting an existing perfetto install,\nall the perfetto executables installed by ROCm Systems Profiler are prefixed with `rocprof-sys-perfetto-`, except\nfor the `perfetto` executable, which is just renamed `rocprof-sys-perfetto`.\n\nEnable `traced` and `perfetto` in the background:\n\n```shell\npkill traced\ntraced --background\nperfetto --out ./rocprof-sys-perfetto.proto --txt -c ${ROCPROFSYS_ROOT}/share/perfetto.cfg --background\n```\n\n\u003e [!NOTE]\n\u003e If the perfetto tools were installed by rocprofiler-systems, replace `traced` with `rocprof-sys-perfetto-traced` and `perfetto` with `rocprof-sys-perfetto`.\n\nConfigure rocprofiler-systems to use the perfetto system backend via the `--perfetto-backend` option of `rocprof-sys-run`:\n\n```shell\n# enable sampling on the uninstrumented binary\nrocprof-sys-run --sample --trace --perfetto-backend=system -- ./myapp\n\n# trace the instrument the binary\nrocprof-sys-instrument -o ./myapp.inst -- ./myapp\nrocprof-sys-run --trace --perfetto-backend=system -- ./myapp.inst\n```\n\nor via the `--env` option of `rocprof-sys-instrument` + runtime instrumentation:\n\n```shell\nrocprof-sys-instrument --env ROCPROFSYS_PERFETTO_BACKEND=system -- ./myapp\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Frocprofiler-systems","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frocm%2Frocprofiler-systems","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Frocprofiler-systems/lists"}