https://github.com/ifromeast/cuda_learning

learning how CUDA works
https://github.com/ifromeast/cuda_learning

Last synced: about 1 month ago
JSON representation

learning how CUDA works

• Host: GitHub
• URL: https://github.com/ifromeast/cuda_learning
• Owner: ifromeast
• Created: 2023-07-23T01:43:42.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-08-16T10:02:45.000Z (6 months ago)
• Last Synced: 2024-08-16T11:24:22.234Z (6 months ago)
• Language: Cuda
• Size: 3.78 MB
• Stars: 132
• Watchers: 4
• Forks: 14
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-cuda-triton-hpc - ifromeast/cuda_learning

README

        
# cuda_learning

learning how CUDA works

## project list:

- custom op [Done]

    - [CUDA 编程基础](https://zhuanlan.zhihu.com/p/645330027)

- memory & reduction [Done]

    - [GPU的内存体系及其优化指南](https://zhuanlan.zhihu.com/p/654027980)

- Gemm [Done]

    - [通用矩阵乘法：从入门到熟练](https://zhuanlan.zhihu.com/p/657632577)

- Transformer [Done]

    - 基础算子：

        - [LayerNorm 算子的 CUDA 实现与优化](https://zhuanlan.zhihu.com/p/694974164)

        - [SoftMax 算子的 CUDA 实现与优化](https://zhuanlan.zhihu.com/p/695307283)

        - [Cross Entropy 的 CUDA 实现](https://zhuanlan.zhihu.com/p/695594396)

        - [AdamW 优化器的 CUDA 实现](https://zhuanlan.zhihu.com/p/695611950)

        - [激活函数与残差连接的 CUDA 实现](https://zhuanlan.zhihu.com/p/695703671?)

        - [embedding 层与 LM head 层的 CUDA 实现](https://zhuanlan.zhihu.com/p/695785781)

    - 核心模块

        - [self-attention 的 CUDA 实现及优化 (上)](https://zhuanlan.zhihu.com/p/695898274)

        - [self-attention 的 CUDA 实现及优化 (下)](https://zhuanlan.zhihu.com/p/696197013)

    

- CUDA mode lectures [WIP]