awesome-gemm
📚 A curated list of awesome matrix-matrix multiplication (A * B = C) frameworks, libraries and software
https://github.com/yuninxia/awesome-gemm
Last synced: 18 days ago
JSON representation
Debugging and Profiling Tools 🔍
Language-Specific Libraries 🔤
Development Software: Debugging and Profiling
Language-Specific Libraries
-
nvprof
- line profiler for CUDA applications.
NVIDIA End User License Agreement -
Perf
- level metrics.
GPLv2 - gprofng-gui - 3.0.html)
- gprof
- FPChecker - point accuracy problems.
- HPCToolkit
Example Implementations
Other Resources
- YHs_Sample - Li/YHs_Sample/blob/master/LICENSE)
- GEMM
- GEMM Optimization with LIBXSMM - 3-Clause`](https://github.com/libxsmm/libxsmm/blob/main/LICENSE.md)
Example Implementations 💡
Blogs 🖋️
- NVIDIA_SGEMM_PRACTICE: Step-by-step optimization of CUDA SGEMM
- SGEMM_CUDA: Step-by-Step Optimization
- simple-gemm
- how-to-optimize-gemm (row-major matmul)
- chgemm: Int8 GEMM implementations
- CUTLASS-based Grouped GEMM: Efficient grouped GEMM operations - 2.0)
- CoralGemm: AMD high-performance GEMM implementations
- DeepBench - 2.0)
- TK-GEMM: a Triton FP8 GEMM kernel using SplitK parallelization
- CUDA-INT8-GEMM
- cuda-sgemm
- cute_gemm
- Cute-Learning
- CUTLASS GEMM - 3-Clause)
- Optimizing-SGEMM-on-NVIDIA-Turing-GPUs
- xGeMM: Accelerated General (FP32) Matrix Multiplication
- Toy HGEMM (Tensor Cores with MMA/WMMA)
Frameworks and Development Tools
Frameworks and Development Tools 🛠️
- BLIS: A modular framework for building high-performance BLAS-like libraries - 3-Clause)
- BLISlab: Educational framework for experimenting with BLIS-like GEMM algorithms
- Tensile: AMD ROCm JIT compiler for GPU kernels, specializing in GEMM and tensor contractions
- Tile Language: A concise DSL designed to streamline development of high-performance GPU/CPU kernels like GEMM
Fundamental Theories and Concepts 🧠
- Spatial-lang GEMM - High-level overview.
- General Matrix Multiply (Intel) - Intro from Intel.
- Strassen's Algorithm - Faster asymptotic complexity for large matrices.
- Winograd's Algorithm - Reduced multiplication count for improved performance.
General Optimization Techniques 🚀
- GEMM: From Pure C to SSE Optimized Micro Kernels - Detailed tutorial on going from naive to vectorized implementations.
- How To Optimize GEMM - Hands-on optimization guide.
Learning Resources
Blogs
- GEMM Caching
- The OpenBLAS Project and Matrix Multiplication Optimization
- OpenBLAS GEMM from Scratch
- The Proper Approach to CUDA for Beginners: How to Optimize GEMM
- ARMv7 4x4kernel Optimization Practice
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
Other Resources
University Courses & Tutorials
Learning Resources 📚
Blogs 🖋️
- Optimizing Matrix Multiplication
- Matrix Multiplication on CPU
- Optimizing Matrix Multiplication: Cache + OpenMP
- Tuning Matrix Multiplication (GEMM) for Intel GPUs
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: A Worklog
- Building a FAST Matrix Multiplication Algorithm
- Matrix-Matrix Product Experiments with BLAZE
- Matrix Multiplication Background Guide (NVIDIA)
- perf-book by Denis Bakhvalov
- CUDA Learn Notes
- CUDA GEMM Optimization
- Why GEMM is at the heart of deep learning
- Outperforming cuBLAS on H100: a Worklog
- Deep Dive on CUTLASS Ping-Pong GEMM Kernel
- CUTLASS Tutorial: Efficient GEMM kernel designs with Pipelining
- Developing CUDA Kernels for GEMM on NVIDIA Hopper Architecture using CUTLASS
- Fast Multidimensional Matrix Multiplication on CPU from Scratch
- A Case Study in CUDA Kernel Fusion: Implementing FlashAttention-2 on NVIDIA Hopper Architecture using the CUTLASS Library
- CUTLASS Tutorial: Fast Matrix-Multiplication with WGMMA on NVIDIA® Hopper™ GPUs
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Epilogue Fusion in CUTLASS with Epilogue Visitor Trees
- Mixed-input matrix multiplication performance optimizations
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- CUTLASS Tutorial: Persistent Kernels and Stream-K
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Optimizing Matrix Multiplication on RDNA3: 50 TFlops and 60% Faster Than rocBLAS
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Optimizing matrix multiplication (CPU): Discovering optimizations one at a time
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Mixed-Input GEMM Kernel Optimized for NVIDIA Hopper GPUs
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
Selected Papers 📝
- Anatomy of High-Performance Many-Threaded Matrix Multiplication (2014)
- Model-driven BLAS Performance on Loongson (2012)
- High-performance Implementation of the Level-3 BLAS (2008)
- BLIS: A Framework for Rapidly Instantiating BLAS Functionality (2015)
- Anatomy of High-Performance Matrix Multiplication (2008)
University Courses & Tutorials 🎓
- HLS Tutorial & Deep Learning Accelerator Lab1
- UCSB CS 240A: Applied Parallel Computing
- UC Berkeley: CS267 Parallel Computing
- ORNL: CUDA C++ Exercise: Basic Linear Algebra Kernels: GEMM Optimization Strategies
- Stanford: BLAS-level CPU Performance in 100 Lines of C
- Purdue: Optimizing Matrix Multiplication
- NJIT: Optimize Matrix Multiplication
- MIT OCW: 6.172 Performance Engineering
- Optimizing Matrix Multiplication using SIMD and Parallelization
- HPC Garage
- MIT: Optimizing Matrix Multiplication (6.172 Lecture Notes)
- CUDATutorial
- MIT: Optimizing Matrix Multiplication (6.172 Lecture Notes)
Libraries
CPU Libraries
- Xianyi Zhang
-
libFLAME
- performance dense linear algebra library.
BSD-3-Clause - OpenBLAS - 3-Clause`](https://github.com/xianyi/OpenBLAS/blob/develop/LICENSE)
GPU Libraries
- cutlass_fpA_intB_gemm - 2.0`](https://github.com/tlc-pack/cutlass_fpA_intB_gemm/blob/main/LICENSE)
- DGEMM on Int8 Tensor Core
- hipBLAS-common - common/blob/develop/LICENSE.md)
- OpenAI GEMM - gemm/blob/master/LICENSE)
-
ArrayFire
- purpose GPU library that simplifies GPU computing with high-level functions, including matrix operations.
BSD-3-Clause
Language-Specific Libraries
- Eigen
- MAGMA (Matrix Algebra on GPU and Multicore Architectures) - generation linear algebra libraries for heterogeneous computing.
- NumPy
- TensorFlow - source software library for machine learning.
- PyTorch - source software library for machine learning.
- ViennaCL - source linear algebra library for computations on many-core architectures (GPUs, MIC) and multi-core CPUs. The library is written in C++ and supports CUDA, OpenCL, and OpenMP (including switches at runtime).
- Blaze
- OpenBLAS
Libraries 🗂️
CPU Libraries 💻
- gemmlowp: Google's low-precision GEMM library - 2.0)
- blis_apple: BLIS optimized for Apple M1 - 3-Clause)
- Intel MKL: Highly optimized math routines for Intel CPUs
- FBGEMM: Meta's CPU GEMM for optimized server inference - 3-Clause)
- BLASFEO: Optimized for small- to medium-sized dense matrices - 2-Clause)
- LIBXSMM: Specializing in small/micro GEMM kernels - 3-Clause)
- libFLAME: High-performance dense linear algebra library - 3-Clause)
- OpenBLAS: Optimized BLAS implementation based on GotoBLAS2 - 3-Clause)
Cross-Platform Libraries 🌍
- LAPACK: Foundational linear algebra routines - 3-Clause)
- ARM Compute Library: Optimized for ARM platforms - 2.0/MIT)
- CUSP: C++ templates for sparse linear algebra - 2.0)
- CUV: C++/Python for CUDA-based vector/matrix ops
- oneDNN (MKL-DNN): Cross-platform deep learning primitives with optimized GEMM - 2.0)
- MAGMA: High-performance linear algebra on GPUs and multicore CPUs - 3-Clause)
- viennacl-dev: OpenCL-based linear algebra library
- Ginkgo: High-performance linear algebra on many-core systems - 3-Clause)
- oneDNN (MKL-DNN): Cross-platform deep learning primitives with optimized GEMM - 2.0)
GPU Libraries ⚡
- NVIDIA cuBLAS: Highly tuned BLAS for NVIDIA GPUs
- NVIDIA cuSPARSE: Sparse matrix computations on NVIDIA GPUs
- NVIDIA CUTLASS: Template library for CUDA GEMM kernels - 3-Clause)
- NVIDIA cuDNN: Deep learning primitives, including GEMM
- hipBLAS: BLAS for AMD GPU platforms (ROCm)
- hipBLASLt: Lightweight BLAS library on ROCm
- clBLAS: BLAS functions on OpenCL for portability - 2.0)
- CLBlast: Tuned OpenCL BLAS library - 2.0)