https://github.com/tlc-pack/cutlass_fpA_intB_gemm

A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer
https://github.com/tlc-pack/cutlass_fpA_intB_gemm

Last synced: 1 day ago
JSON representation

A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer

• Host: GitHub
• URL: https://github.com/tlc-pack/cutlass_fpA_intB_gemm
• Owner: tlc-pack
• License: apache-2.0
• Created: 2023-06-06T22:26:34.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-02-28T06:15:34.000Z (8 months ago)
• Last Synced: 2024-08-02T14:05:07.564Z (3 months ago)
• Language: C++
• Size: 203 KB
• Stars: 79
• Watchers: 20
• Forks: 20
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-gemm - cutlass_fpA_intB_gemm

README

        
Extracted fp16 A and int8/4 B CUTLASS GEMM kernels from FasterTransformer for easier integration in third-party projects. See the original code below.

* https://github.com/NVIDIA/FasterTransformer/tree/main/src/fastertransformer/cutlass_extensions/include/cutlass_extensions

* https://github.com/NVIDIA/FasterTransformer/tree/main/src/fastertransformer/kernels/cutlass_kernels/fpA_intB_gemm

Build with

```

mkdir build && cd build

cmake ..

make

```