awesome-gemm
📚 A curated list of awesome matrix-matrix multiplication (A * B = C) frameworks, libraries and software
https://github.com/yuninxia/awesome-gemm
Last synced: 3 days ago
JSON representation
-
Learning Resources 📚
-
Blogs 🖋️
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Matrix Multiplication on CPU
- Building a FAST Matrix Multiplication Algorithm
- Matrix Multiplication Background Guide (NVIDIA)
- Outperforming cuBLAS on H100: a Worklog
- Deep Dive on CUTLASS Ping-Pong GEMM Kernel
- CUTLASS Tutorial: Efficient GEMM kernel designs with Pipelining
- CUDA GEMM Optimization
- CUDA Learn Notes
- CUTLASS Tutorial: Fast Matrix-Multiplication with WGMMA on NVIDIA® Hopper™ GPUs
- Epilogue Fusion in CUTLASS with Epilogue Visitor Trees
- Fast Multidimensional Matrix Multiplication on CPU from Scratch
- Matrix-Matrix Product Experiments with BLAZE
- Optimizing Matrix Multiplication
- Optimizing Matrix Multiplication: Cache + OpenMP
- perf-book by Denis Bakhvalov
- Tuning Matrix Multiplication (GEMM) for Intel GPUs
- Developing CUDA Kernels for GEMM on NVIDIA Hopper Architecture using CUTLASS
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Optimizing Matrix Multiplication on RDNA3: 50 TFlops and 60% Faster Than rocBLAS
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- A Case Study in CUDA Kernel Fusion: Implementing FlashAttention-2 on NVIDIA Hopper Architecture using the CUTLASS Library
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Mixed-input matrix multiplication performance optimizations
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: A Worklog
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Mixed-Input GEMM Kernel Optimized for NVIDIA Hopper GPUs
- Why GEMM is at the heart of deep learning
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Optimizing matrix multiplication (CPU): Discovering optimizations one at a time
- CUTLASS Tutorial: Persistent Kernels and Stream-K
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
-
University Courses & Tutorials 🎓
- HLS Tutorial & Deep Learning Accelerator Lab1
- MIT OCW: 6.172 Performance Engineering
- NJIT: Optimize Matrix Multiplication
- Optimizing Matrix Multiplication using SIMD and Parallelization
- ORNL: CUDA C++ Exercise: Basic Linear Algebra Kernels: GEMM Optimization Strategies
- Purdue: Optimizing Matrix Multiplication
- Stanford: BLAS-level CPU Performance in 100 Lines of C
- UC Berkeley: CS267 Parallel Computing
- UCSB CS 240A: Applied Parallel Computing
- MIT: Optimizing Matrix Multiplication (6.172 Lecture Notes)
- UT Austin: LAFF-On Programming for High Performance
- HPC Garage
- CUDATutorial
-
Selected Papers 📝
- BLIS: A Framework for Rapidly Instantiating BLAS Functionality (2015)
- Anatomy of High-Performance Many-Threaded Matrix Multiplication (2014)
- Model-driven BLAS Performance on Loongson (2012)
- High-performance Implementation of the Level-3 BLAS (2008)
- Anatomy of High-Performance Matrix Multiplication (2008)
-
-
Frameworks and Development Tools 🛠️
- BLIS: A modular framework for building high-performance BLAS-like libraries - 3-Clause)
- BLISlab: Educational framework for experimenting with BLIS-like GEMM algorithms
- Tensile: AMD ROCm JIT compiler for GPU kernels, specializing in GEMM and tensor contractions
- Tile Language: A concise DSL designed to streamline development of high-performance GPU/CPU kernels like GEMM
-
Libraries
-
- OpenBLAS
- Eigen
- MAGMA (Matrix Algebra on GPU and Multicore Architectures) - generation linear algebra libraries for heterogeneous computing.
- NumPy
- TensorFlow - source software library for machine learning.
- PyTorch - source software library for machine learning.
- ViennaCL - source linear algebra library for computations on many-core architectures (GPUs, MIC) and multi-core CPUs. The library is written in C++ and supports CUDA, OpenCL, and OpenMP (including switches at runtime).
- Blaze
-
GPU Libraries
- cutlass_fpA_intB_gemm - 2.0`](https://github.com/tlc-pack/cutlass_fpA_intB_gemm/blob/main/LICENSE)
- DGEMM on Int8 Tensor Core
- hipBLAS-common - common/blob/develop/LICENSE.md)
- OpenAI GEMM - gemm/blob/master/LICENSE)
- ArrayFire - purpose GPU library that simplifies GPU computing with high-level functions, including matrix operations. [`BSD-3-Clause`](https://github.com/arrayfire/arrayfire/blob/master/LICENSE)
-
CPU Libraries
- Xianyi Zhang
- libFLAME - performance dense linear algebra library. [`BSD-3-Clause`](https://github.com/flame/libflame/blob/master/LICENSE.txt)
- OpenBLAS - 3-Clause`](https://github.com/xianyi/OpenBLAS/blob/develop/LICENSE)
-
Language-Specific Libraries
-
-
Example Implementations
-
Other Resources
- YHs_Sample - Li/YHs_Sample/blob/master/LICENSE)
- GEMM
- GEMM Optimization with LIBXSMM - 3-Clause`](https://github.com/libxsmm/libxsmm/blob/main/LICENSE.md)
-
-
Libraries 🗂️
-
Language-Specific Libraries 🔤
- BLIS.jl - 3-Clause)
- PyTorch - 3-Clause)
- SciPy - 3-Clause)
- TensorFlow - 2.0) & [XLA](https://www.tensorflow.org/xla)
- Armadillo - 2.0/MIT)
- Eigen
- GemmKernels.jl - 3-Clause)
- Boost uBlas
- JAX - 2.0)
- NumPy - 3-Clause)
-
CPU Libraries 💻
- blis_apple: BLIS optimized for Apple M1 - 3-Clause)
- libFLAME: High-performance dense linear algebra library - 3-Clause)
- Intel MKL: Highly optimized math routines for Intel CPUs
- FBGEMM: Meta's CPU GEMM for optimized server inference - 3-Clause)
- gemmlowp: Google's low-precision GEMM library - 2.0)
- BLASFEO: Optimized for small- to medium-sized dense matrices - 2-Clause)
- LIBXSMM: Specializing in small/micro GEMM kernels - 3-Clause)
-
GPU Libraries ⚡
- NVIDIA cuBLAS: Highly tuned BLAS for NVIDIA GPUs
- NVIDIA cuSPARSE: Sparse matrix computations on NVIDIA GPUs
- NVIDIA CUTLASS: Template library for CUDA GEMM kernels - 3-Clause)
- TileFusion: Simplifying Kernel Fusion with Tile Processing
- clBLAS: BLAS functions on OpenCL for portability - 2.0)
- CLBlast: Tuned OpenCL BLAS library - 2.0)
- NVIDIA cuDNN: Deep learning primitives, including GEMM
- hipBLAS: BLAS for AMD GPU platforms (ROCm)
- hipBLASLt: Lightweight BLAS library on ROCm
- BitBLAS: Mixed-precision BLAS operations on GPUs
- BitBLAS-Benchmark
- TiledCUDA: Kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles
-
Cross-Platform Libraries 🌍
- CUSP: C++ templates for sparse linear algebra - 2.0)
- CUV: C++/Python for CUDA-based vector/matrix ops
- LAPACK: Foundational linear algebra routines - 3-Clause)
- ARM Compute Library: Optimized for ARM platforms - 2.0/MIT)
- MAGMA: High-performance linear algebra on GPUs and multicore CPUs - 3-Clause)
- oneDNN (MKL-DNN): Cross-platform deep learning primitives with optimized GEMM - 2.0)
- viennacl-dev: OpenCL-based linear algebra library
- Ginkgo: High-performance linear algebra on many-core systems - 3-Clause)
- oneDNN (MKL-DNN): Cross-platform deep learning primitives with optimized GEMM - 2.0)
-
-
Example Implementations 💡
-
Blogs 🖋️
- chgemm: Int8 GEMM implementations
- Toy HGEMM (Tensor Cores with MMA/WMMA)
- Cute-Learning
- CUTLASS GEMM - 3-Clause)
- NVIDIA_SGEMM_PRACTICE: Step-by-step optimization of CUDA SGEMM
- SGEMM_CUDA: Step-by-Step Optimization
- simple-gemm
- TK-GEMM: a Triton FP8 GEMM kernel using SplitK parallelization
- Optimizing-SGEMM-on-NVIDIA-Turing-GPUs
- xGeMM: Accelerated General (FP32) Matrix Multiplication
- CUTLASS-based Grouped GEMM: Efficient grouped GEMM operations - 2.0)
- CoralGemm: AMD high-performance GEMM implementations
- how-to-optimize-gemm (row-major matmul)
- DeepBench - 2.0)
- CUDA-INT8-GEMM
- cuda-sgemm
- cute_gemm
-
-
Fundamental Theories and Concepts 🧠
- Spatial-lang GEMM - High-level overview.
- General Matrix Multiply (Intel) - Intro from Intel.
- Strassen's Algorithm - Faster asymptotic complexity for large matrices.
- Winograd's Algorithm - Reduced multiplication count for improved performance.
-
Frameworks and Development Tools
-
Development Software: Debugging and Profiling
-
- gprof
- FPChecker - point accuracy problems.
- HPCToolkit
-
Language-Specific Libraries
- Perf - level metrics. [`GPLv2`](https://github.com/torvalds/linux/blob/master/COPYING)
- gprofng-gui - 3.0.html)
- nvprof - line profiler for CUDA applications. [`NVIDIA End User License Agreement`](https://docs.nvidia.com/cuda/eula/index.html)
-
-
Learning Resources
-
University Courses & Tutorials
-
Blogs
- The OpenBLAS Project and Matrix Multiplication Optimization
- OpenBLAS GEMM from Scratch
- The Proper Approach to CUDA for Beginners: How to Optimize GEMM
- ARMv7 4x4kernel Optimization Practice
- Distributed GEMM - A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems
- GEMM Caching
-
Other Resources
-
-
University Courses & Tutorials
-
Other Learning Resources
-
General Optimization Techniques 🚀
- GEMM: From Pure C to SSE Optimized Micro Kernels - Detailed tutorial on going from naive to vectorized implementations.
- How To Optimize GEMM - Hands-on optimization guide.
-
Debugging and Profiling Tools 🔍
Programming Languages
Categories
Learning Resources 📚
63
Libraries 🗂️
38
Debugging and Profiling Tools 🔍
18
Example Implementations 💡
17
Libraries
17
Learning Resources
9
Development Software: Debugging and Profiling
6
Frameworks and Development Tools 🛠️
4
Fundamental Theories and Concepts 🧠
4
Example Implementations
3
General Optimization Techniques 🚀
2
Frameworks and Development Tools
1
University Courses & Tutorials
1
Other Learning Resources
1
Sub Categories
Keywords
cuda
11
gemm
7
blas
6
matrix-multiplication
6
deep-learning
6
gpu
6
cpp
5
python
5
machine-learning
4
blis
4
opencl
4
c
3
hip
3
lapack
3
performance
3
oneapi
3
openmp
3
hpc
3
deep-neural-networks
3
linear-algebra
3
aarch64
3
neural-network
3
numpy
2
hgemm
2
code-optimization
2
armv7
2
armv8
2
linux
2
julia
2
scientific-computing
2
gemm-optimization
2
cuda-programming
2
blas-libraries
2
high-performance
2
high-performance-computing
2
linear-algebra-library
2
matrix-functions
2
xe-architecture
2
x86-64
2
x64
2
vnni
2
tbb
2
sycl
2
lapacke
2
gpu-computing
2
assembly
2
matrix-library
2
amx
2
avx512
2
bfloat16
2