{"id":13574510,"url":"https://github.com/amd/ZenDNN","last_synced_at":"2025-04-04T15:31:16.978Z","repository":{"id":50267099,"uuid":"394990533","full_name":"amd/ZenDNN","owner":"amd","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-04T16:28:34.000Z","size":4432,"stargazers_count":105,"open_issues_count":11,"forks_count":13,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-04T16:42:01.377Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-11T13:02:06.000Z","updated_at":"2025-03-04T15:54:29.000Z","dependencies_parsed_at":"2024-11-05T09:34:24.483Z","dependency_job_id":"72ea5aba-ccbb-4484-a86b-e28e2640b42b","html_url":"https://github.com/amd/ZenDNN","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amd%2FZenDNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amd%2FZenDNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amd%2FZenDNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amd%2FZenDNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amd","download_url":"https://codeload.github.com/amd/ZenDNN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247202715,"owners_count":20900829,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:00:52.218Z","updated_at":"2025-04-04T15:31:11.968Z","avatar_url":"https://github.com/amd.png","language":"C++","funding_links":[],"categories":["Table of Contents"],"sub_categories":["AI - Frameworks and Toolkits"],"readme":"\n\nZen Deep Neural Network Library (ZenDNN)\n========================================\nZenDNN (Zen Deep Neural Network) Library accelerates deep learning inference applications on AMD CPUs. This library, which includes APIs for basic neural network building blocks optimized for AMD CPUs, targets deep learning application and framework developers with the goal of improving inference performance on AMD CPUs across a variety of workloads, including computer vision, natural language processing (NLP), and recommender systems. ZenDNN leverages oneDNN/DNNL v2.6.3's basic infrastructure and APIs. ZenDNN optimizes several APIs and adds new APIs, which are currently integrated into TensorFlow, ONNXRT, and PyTorch.\n\nZenDNN depends on:\n- AOCL-BLAS is a high-performant implementation of the Basic Linear Algebra Subprograms (BLAS).\n- Composable Kernel for convolutions using an implicit GEMM algorithm\n\nAOCL-BLAS is required dependencies for ZenDNN, whereas AMD Composable Kernel is an optional dependency.\n\n# Table of Contents\n- [Zen Deep Neural Network Library (ZenDNN)](#zen-deep-neural-network-library-zendnn)\n- [Table of Contents](#table-of-contents)\n- [Scope](#scope)\n- [Release Highlights](#release-highlights)\n- [Supported OS and Compilers](#supported-os-and-compilers)\n  - [OS](#os)\n  - [Compilers](#compilers)\n- [Prerequisites](#prerequisites)\n- [AOCL-BLAS Library Installation](#aocl-blas-library-installation)\n  - [General Convention](#general-convention)\n  - [AOCL-BLAS Library Setup](#aocl-blas-library-setup)\n- [Composable Kernel Library Installation](#composable-kernel-library-installation)\n  - [Composable Kernel Library Setup](#composable-kernel-library-setup)\n    - [Prerequisites](#prerequisites-1)\n    - [Download code](#download-code)\n    - [Compile](#compile)\n    - [Link from ZenDNN](#link-from-zendnn)\n- [Runtime Dependencies](#runtime-dependencies)\n- [Build from Source](#build-from-source)\n  - [GCC compiler](#gcc-compiler)\n  - [Validate the build](#validate-the-build)\n- [Logs](#logs)\n- [License](#license)\n- [Technical Support](#technical-support)\n\n# Scope\nThe scope of ZenDNN is to support AMD EPYC\u003csup\u003eTM\u003c/sup\u003e CPUs on the Linux® platform. ZenDNN v4.2 offers optimized primitives, such as Convolution, MatMul, Elementwise, and Pool (Max and Average), Gelu, LayerNorm that improve performance of many convolutional neural networks, recurrent neural networks, transformer-based models, and recommender system models. For the primitives not supported by ZenDNN, execution will fall back to the  native path of the framework.\n\n\n# Release Highlights\nFollowing are the highlights of this release:\n* ZenDNN library is integrated with TensorFlow v2.16 (Plugin), ONNXRT v1.17.0, and PyTorch v2.1 (Plugin).\n* Python v3.9-v3.12 has been used to generate the TensorFlow v2.16 (Plugin) wheel files (*.whl).\n* Python v3.8-v3.11 has been used to generate the ONNXRT v1.17.0 wheel files (*.whl).\n* Python v3.8-v3.11 has been used to generate the PyTorch v2.1 (Plugin) wheel files (*.whl).\n* NHWC (default format), NHWC_BLOCKED and Blocked Format are supported.\n\nZenDNN library is intended to be used in conjunction with the frameworks mentioned above and cannot be used independently.\n\nThe latest information on the ZenDNN release and installers is available on AMD.com portal (https://www.amd.com/en/developer/zendnn.html).\n\n# Supported OS and Compilers\nThis release of ZenDNN supports the following Operating Systems (OS) and compilers:\n## OS\n* Ubuntu® 22.04 LTS and later\n* Red Hat® Enterprise Linux® (RHEL) 9.2 and later\n* SLES15 SP5 and later\n* CentOS Stream 8.4\n* PyTorch v2.1 wheel files are supported on Anolis OS 8.8\n\n## Compilers\n* GCC 10.2 and later\n\nTheoretically, for wheel files any Linux based OS with GLIBC version later than 2.17 could be supported.\n\nFor C++ interface binaries, any Linux based OS with GLIBC version later than 2.31 could be supported.\n\n# Prerequisites\nThe following prerequisites must be met for this release of ZenDNN:\n* AOCL-BLAS v4.2 must be installed for optimal performance of the ZenDNN library.\n\n\n# AOCL-BLAS Library Installation\n\n**AOCL-BLAS** AOCL-BLAS is a high-performant implementation of the Basic Linear Algebra Subprograms (BLAS). The BLAS was designed to provide the essential kernels of matrix and vector computation and are the most commonly used computationally intensive operations in dense numerical linear algebra. This can be downloaded from AMD.com portal Developer Central (https://www.amd.com/en/developer/aocl/dense.html).\n\nNote: ZenDNN depends only on AOCL-BLAS and has no dependency on any other AOCL library.\n## General Convention\nThe following points must be considered while installing AOCL-BLAS:\n* Change to the preferred directory where AOCL-BLAS will be downloaded.\n* This parent folder is referred to as folder `\u003ccompdir\u003e` in the steps below.\n* Assume that the parent folder for user setup follows this convention: `/home/\u003cuser-id\u003e/my_work`.\n\n## AOCL-BLAS Library Setup\nComplete the following steps to setup the GCC compiled AOCL-BLAS library:\n1. Execute the command `cd \u003ccompdir\u003e`\n2. Download aocl-blis-linux-gcc-4.2.0.tar.gz.\n3. Execute the following commands:\n    ```bash\n    tar -xvf aocl-blis-linux-gcc-4.2.0.tar.gz\n    cd amd-blis\n\t```\nThis will set up the environment for AOCL-BLAS path:\n```bash\nexport ZENDNN_BLIS_PATH=$(pwd)\n```\nFor example:\n```bash\nexport ZENDNN_BLIS_PATH=/home/\u003cuser-id\u003e/my_work/amd-blis\n```\n\nThe bashrc file can be edited to setup ZENDNN_BLIS_PATH environment path.\nFor example, in the case of GCC compiled AOCL-BLAS:\n```bash\nexport ZENDNN_BLIS_PATH=/home/\u003cuser-id\u003e/my_work/amd-blis\n```\n\n# Composable Kernel Library Installation\n\n**Composable Kernel** aims to provide a programming model for writing performance critical kernels for machine learning workloads across multiple architectures including GPUs, CPUs, etc, through general purpose kernel languages, like HIP C++.  Composable Kernel can be downloaded from the AMD ROCm Software Platform Repository (https://github.com/ROCmSoftwarePlatform/composable_kernel).\n\n## Composable Kernel Library Setup\n\nComposable Kernel (CK) for CPU is currently only on the `cpu_avx2` branch of the Composable Kernel repository and is at the experimental stage of development.\n\n### Prerequisites\nCK is suitable for these compilers:\n1) hipclang: this is mainly used for compiling GPU hip kernels(require rocm environment), but also can be used for CPU. For a first trial use below compiler.\n2) gcc: at least gcc-9 is needed, you may need manually install a gcc-9 if ubuntu default is not gcc-9.\n\n### Download code\n```\ngit clone https://github.com/ROCmSoftwarePlatform/composable_kernel.git\ncd composable_kernel\ngit checkout origin/cpu_avx2 -b cpu_avx2\n```\n\n### Compile\n\nFrom the root directory of Composable Kernel (CK) build CK libraries:\n```\n# if use gcc\nsh script/cmake-avx2-gcc.sh\ncd build\nmake -j`nproc` example_cpu_conv2d_fwd example_cpu_conv2d_fwd_bias_relu_add\n```\n\n### Link from ZenDNN\n\nFrom the root directory of Composable Kernel (CK), this will set up the environment for including CK headers and linking to the CK libraries.\n```bash\n export ZENDNN_CK_PATH=$(pwd)\n```\n\nThe `Makefile` in this project contains a variable `DEPEND_ON_CK` which is set to `0` by default.  To enable CK use `DEPEND_ON_CK=1` when building the ZenDNN library.\n\nEither modify DEPEND_ON_CK in the Makefile or pass DEPEND_ON_CK as an argument to\nmake by editing scripts/zendnn_build.sh gcc.\n\nThe `LD_LIBRARY_PATH` variable needs to be updated in order to run code that depends on CK.\n```bash\nexport LD_LIBRARY_PATH=${ZENDNN_CK_PATH}/build/lib:${LD_LIBRARY_PATH}\n```\n\n# Runtime Dependencies\nZenDNN has the following runtime dependencies:\n* GNU C library (glibc.so)\n* GNU Standard C++ library (libstdc++.so)\n* Dynamic linking library (libdl.so)\n* POSIX Thread library (libpthread.so)\n* C Math Library (libm.so)\n* OpenMP (libomp.so)\n* Python v3.9-v3.12 for TensorFlow v2.16 (Plugin)\n* Python v3.8-v3.11 for PyTorch v2.1 (Plugin)\n* Python v3.8-v3.11 for ONNXRT v1.17.0\n\nSince ZenDNN is configured to use OpenMP, a C++ compiler with OpenMP 2.0 or later is required for runtime execution.\n\n# Build from Source\nClone ZenDNN git:\n```bash\ngit clone https://github.com/amd/ZenDNN.git\ncd ZenDNN\n```\n\n## GCC compiler\n**ZENDNN_BLIS_PATH** should be defined.\nexample:\n```bash\nexport ZENDNN_BLIS_PATH=/home/\u003cuser-id\u003e/my_work/amd-blis\nmake clean\nsource scripts/zendnn_build.sh gcc\n```\nWhen new terminal is opened, user need to set up environment variables:\n```bash\nsource scripts/zendnn_gcc_env_setup.sh\n```\nPlease note above scripts must be sourced only from ZenDNN Folder.\n\n## Validate the build\nAfter the library is built on Linux host, user can run unit tests using:\n```bash\nsource scripts/runApiTest.sh\n```\nCorresponding tests are located in the **tests/api_tests** directory. These unit tests don't produce any information/logs in the terminal. Library logs can be enabled with:\n```bash\nZENDNN_LOG_OPTS=ALL:2 source scripts/runApiTest.sh\n```\n\n# Logs\nLogging is disabled in the ZenDNN library by default. It can be enabled using the environment variable **ZENDNN_LOG_OPTS** before running any tests. Logging behavior can be specified by setting the environment variable **ZENDNN_LOG_OPTS** to a comma-delimited list of ACTOR:DBGLVL pairs.\n\nThe different ACTORS are as follows:\n| ACTORS  | Usage\n| :------ | :-----------\n| ALGO    | Logs all algorithms executed\n| CORE    | Logs all the core ZenDNN library operations\n| API     | Logs all the ZenDNN API calls\n| TEST    | Logs used in API tests, functionality tests and regression tests\n| PROF    | Logs the performance of operations in millisecond\n| FWK     | Logs all the framework (TensorFlow, ONNXRT, and PyTorch) specific calls\n\nFor example:\n* To turn on info logging, use **ZENDNN_LOG_OPTS=ALL:2**\n* To turn off all logging, use **ZENDNN_LOG_OPTS=ALL:-1**\n* To only log errors, use **ZENDNN_LOG_OPTS=ALL:0**\n* To only log info for ALGO, use **ZENDNN_LOG_OPTS=ALL:-1,ALGO:2**\n* To only log info for CORE, use **ZENDNN_LOG_OPTS=ALL:-1,CORE:2**\n* To only log info for API, use **ZENDNN_LOG_OPTS=ALL:-1,API:2**\n* To only log info for PROF (profile), use **ZENDNN_LOG_OPTS=ALL:-1,PROF:2**\n* To only log info for FWK, use **ZENDNN_LOG_OPTS=ALL:-1,FWK:2**\n\n\n\nThe Different Debug Levels (DBGLVL) are as follows:\n```bash\nenum LogLevel {\n  LOG_LEVEL_DISABLED = -1,\n  LOG_LEVEL_ERROR    =  0,\n  LOG_LEVEL_WARNING  =  1,\n  LOG_LEVEL_INFO     =  2,\n  LOG_LEVEL_VERBOSE0 =  3,\n  LOG_LEVEL_VERBOSE1 =  4,\n  LOG_LEVEL_VERBOSE2 =  5\n};\n```\n# License\nRefer to the \"[LICENSE](LICENSE)\" file for the full license text and copyright notice.\n\nThis distribution includes third party software governed by separate license terms.\n\nThis third party software, even if included with the distribution of the Advanced Micro Devices software, may be governed by separate license terms, including without limitation, third party license terms,  and open source software license terms. These separate license terms govern your use of the third party programs as set forth in the **THIRD-PARTY-PROGRAMS** file.\n\n# Technical Support\nPlease email zendnnsupport@amd.com for questions, issues, and feedback on ZenDNN.\n\nPlease submit your questions, feature requests, and bug reports on the\n[GitHub issues](https://github.com/amd/ZenDNN/issues) page.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famd%2FZenDNN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famd%2FZenDNN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famd%2FZenDNN/lists"}