{"id":51375858,"url":"https://github.com/mlbo/cuda_learning","last_synced_at":"2026-07-03T12:18:41.007Z","repository":{"id":343254694,"uuid":"1175157991","full_name":"mlbo/cuda_learning","owner":"mlbo","description":null,"archived":false,"fork":false,"pushed_at":"2026-03-09T15:56:10.000Z","size":30319,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-09T19:28:47.909Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlbo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-07T10:06:10.000Z","updated_at":"2026-03-09T15:56:15.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mlbo/cuda_learning","commit_stats":null,"previous_names":["mlbo/cuda_learning"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/mlbo/cuda_learning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlbo%2Fcuda_learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlbo%2Fcuda_learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlbo%2Fcuda_learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlbo%2Fcuda_learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlbo","download_url":"https://codeload.github.com/mlbo/cuda_learning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlbo%2Fcuda_learning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35085345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-03T12:18:38.720Z","updated_at":"2026-07-03T12:18:40.987Z","avatar_url":"https://github.com/mlbo.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CUDA 学习教程\n\n\u003e 从零开始学习 CUDA 并行编程，适合完全新手到高级优化\n\n## 项目简介\n\n本项目是一套完整的 CUDA 学习教程，包含：\n\n- **30 章教程文档**：循序渐进，从基础入门到高级优化\n- **30 个示例代码**：每章配套可运行代码，详细中文注释\n- **CMake 编译支持**：一键编译所有示例\n- **辅助脚本**：环境检查、编译、快速运行\n\n## 官方文档来源\n\n本教程内容参考并整合了以下 NVIDIA 官方文档：\n\n| 文档名称 | 版本 | 链接 | 本地路径 |\n|----------|------|------|----------|\n| **CUDA C++ Programming Guide** | 12.2.1 | [官方链接](https://docs.nvidia.com/cuda/archive/12.2.1/cuda-c-programming-guide/) | [本地文档](./lecture_slides/cuda_12_2_1_programming_guide/) |\n| CUDA Best Practices Guide | 12.2.1 | [官方链接](https://docs.nvidia.com/cuda/archive/12.2.1/cuda-c-best-practices-guide/) | - |\n| CUDA Runtime API | 12.2.1 | [官方链接](https://docs.nvidia.com/cuda/archive/12.2.1/cuda-runtime-api/) | - |\n\n\u003e **说明**：本地文档 `lecture_slides/cuda_12_2_1_programming_guide/` 包含完整的 CUDA 12.2.1 编程指南及配套图片，可供离线查阅。\n\n## 目录结构\n\n```\ncuda_learning/\n├── tutorials/                    # 教程文档 (30章)\n├── examples/                     # 示例代码 (与章节对应)\n│   ├── 01_gpu_basics/           # GPU基础\n│   ├── 02_cuda_intro/           # CUDA入门\n│   ├── ...\n│   └── 30_cuda_libraries/       # CUDA官方库\n├── lecture_slides/              # 课程讲义\n├── scripts/                      # 辅助脚本\n└── README.md\n```\n\n## 快速开始\n\n### 1. 环境要求\n\n- NVIDIA GPU（Compute Capability 5.3+）\n- CUDA Toolkit 11.0+\n- CMake 3.18+\n- Linux 操作系统\n\n### 2. 编译示例\n\n```bash\n# 使用脚本编译\n./scripts/compile_all.sh\n\n# 或手动编译\ncd examples\nmkdir build \u0026\u0026 cd build\ncmake .. -DCMAKE_CUDA_ARCHITECTURES=80  # 根据你的GPU选择架构\nmake -j\n```\n\n### 3. 运行示例\n\n```bash\n# 运行指定章节\n./scripts/run_example.sh 12    # 第12章：原子操作\n\n# 运行所有示例\n./scripts/run_example.sh --all\n```\n\n## 章节内容\n\n### 第一部分：基础入门 (1-7章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 01 | [什么是GPU并行计算](tutorials/01_什么是GPU并行计算.md) | 理解并行计算概念，CPU vs GPU 区别 |\n| 02 | [CUDA是什么](tutorials/02_CUDA是什么.md) | 了解 CUDA 架构，Host/Device 概念 |\n| 03 | [GPU硬件架构入门](tutorials/03_GPU硬件架构入门.md) | 认识 SM、CUDA Core、Warp |\n| 04 | [线程层级结构](tutorials/04_线程层级结构.md) | 掌握 Grid/Block/Thread 组织 |\n| 05 | [第一个CUDA程序](tutorials/05_第一个CUDA程序.md) | 编写并运行第一个程序 |\n| 06 | [内存管理基础](tutorials/06_内存管理基础.md) | 学会 cudaMalloc/cudaMemcpy |\n| 07 | [核函数深入](tutorials/07_核函数深入.md) | 理解 __global__、\u003c\u003c \u003c\u003e\u003e\u003e |\n\n### 第二部分：进阶优化 (8-11章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 08 | [性能分析入门](tutorials/08_性能分析入门.md) | 使用 nsys/ncu 分析性能 |\n| 09 | [内存访问优化](tutorials/09_内存访问优化.md) | 合并访问、向量化访存 |\n| 10 | [精度与性能](tutorials/10_精度与性能.md) | FP16 编程与优化 |\n| 11 | [Roofline模型](tutorials/11_Roofline模型.md) | 分析性能瓶颈 |\n\n### 第三部分：内存与同步机制 (12-16章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 12 | [原子操作与竞争条件](tutorials/12_原子操作与竞争条件.md) | 理解竞争条件，掌握原子操作 |\n| 13 | [共享内存深入](tutorials/13_共享内存深入.md) | 共享内存原理与优化 |\n| 14 | [规约算法优化](tutorials/14_规约算法优化.md) | 树状规约、Warp Shuffle |\n| 15 | [Bank Conflict优化](tutorials/15_Bank_Conflict优化.md) | Bank冲突检测与解决 |\n| 16 | [Cooperative Groups](tutorials/16_Cooperative_Groups.md) | 跨块同步与协作编程 |\n\n### 第四部分：核心算子实现 (17-20章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 17 | [GEMM优化入门](tutorials/17_GEMM优化入门.md) | 矩阵乘法Naive到分块 |\n| 18 | [GEMM分块优化](tutorials/18_GEMM分块优化.md) | 1D/2D Blocktiling、Warptiling |\n| 19 | [Tensor Core编程](tutorials/19_Tensor_Core编程.md) | WMMA API、混合精度GEMM |\n| 20 | [卷积算子实现](tutorials/20_卷积算子实现.md) | 直接卷积、im2col优化 |\n\n### 第五部分：系统级优化 (21-25章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 21 | [异步执行与延迟隐藏](tutorials/21_异步执行与延迟隐藏.md) | 双缓冲、软件流水线 |\n| 22 | [CUDA流与并发](tutorials/22_CUDA流与并发.md) | 多流、Event、并发内核 |\n| 23 | [数据传输优化](tutorials/23_数据传输优化.md) | Pinned Memory、Unified Memory |\n| 24 | [CUDA Graph](tutorials/24_CUDA_Graph.md) | 图捕获与执行优化 |\n| 25 | [多GPU编程](tutorials/25_多GPU编程.md) | P2P传输、NCCL、AllReduce |\n\n### 第六部分：工业级调优 (26-30章)\n\n| 章节 | 教程 | 学习目标 |\n|:----:|------|----------|\n| 26 | [低精度与量化](tutorials/26_低精度与量化.md) | FP16/BF16/INT8量化 |\n| 27 | [PTX与底层优化](tutorials/27_PTX与底层优化.md) | PTX汇编、内联PTX |\n| 28 | [微指令级调优](tutorials/28_微指令级调优.md) | 循环展开、编译器选项 |\n| 29 | [ILP与Warp Divergence](tutorials/29_ILP与Warp_Divergence.md) | 指令级并行、分支优化 |\n| 30 | [CUDA官方库实战](tutorials/30_CUDA官方库实战.md) | cuBLAS、cuDNN、CUB、CUTLASS |\n\n## 学习路线\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│  第一部分：基础入门 (1-7章)                                  │\n│  GPU概念 → CUDA入门 → 硬件架构 → 线程层级 →                 │\n│  第一个程序 → 内存管理 → 核函数深入                          │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              ▼\n┌─────────────────────────────────────────────────────────────┐\n│  第二部分：进阶优化 (8-11章)                                 │\n│  性能分析 → 内存访问优化 → 精度与性能 → Roofline模型         │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              ▼\n┌─────────────────────────────────────────────────────────────┐\n│  第三部分：内存与同步机制 (12-16章)                          │\n│  原子操作 → 共享内存 → 规约优化 → Bank Conflict → CG         │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              ▼\n┌─────────────────────────────────────────────────────────────┐\n│  第四部分：核心算子实现 (17-20章)                            │\n│  GEMM入门 → GEMM分块优化 → Tensor Core → 卷积算子            │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              ▼\n┌─────────────────────────────────────────────────────────────┐\n│  第五部分：系统级优化 (21-25章)                              │\n│  异步执行 → 多流 → 数据传输 → CUDA Graph → 多GPU             │\n└─────────────────────────────────────────────────────────────┘\n                              │\n                              ▼\n┌─────────────────────────────────────────────────────────────┐\n│  第六部分：工业级调优 (26-30章)                              │\n│  低精度量化 → PTX优化 → 微指令调优 → ILP/分支 → 官方库       │\n└─────────────────────────────────────────────────────────────┘\n```\n\n## GPU 架构参考\n\n| 架构 | GPU 示例 | Compute Capability | CMake 参数 |\n|------|----------|-------------------|------------|\n| Volta | V100 | 7.0 | `-DCMAKE_CUDA_ARCHITECTURES=70` |\n| Turing | RTX 2080, T4 | 7.5 | `-DCMAKE_CUDA_ARCHITECTURES=75` |\n| Ampere | A100 | 8.0 | `-DCMAKE_CUDA_ARCHITECTURES=80` |\n| Ampere | RTX 3090, A40 | 8.6 | `-DCMAKE_CUDA_ARCHITECTURES=86` |\n| Ada Lovelace | RTX 4090 | 8.9 | `-DCMAKE_CUDA_ARCHITECTURES=89` |\n| Hopper | H100 | 9.0 | `-DCMAKE_CUDA_ARCHITECTURES=90` |\n\n## 教程特点\n\n- **从零开始**：假设读者完全不了解 GPU 编程\n- **图解优先**：使用 Mermaid 图解释所有抽象概念\n- **代码说话**：每行代码都有详细中文注释\n- **即学即练**：每个概念都配可运行代码\n- **循序渐进**：每章只引入 2-3 个新概念\n- **结合官方文档**：每章标注对应的CUDA官方文档章节\n\n## 参考资料\n\n- [CUDA C++ Programming Guide](https://docs.nvidia.com/cuda/cuda-c-programming-guide/)\n- [CUDA Best Practices Guide](https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/)\n- [Nsight Compute Documentation](https://docs.nvidia.com/nsight-compute/)\n- [CUDA FP16 API](https://docs.nvidia.com/cuda/cuda-math-api/group__CUDA__MATH__INTRINSIC__HALF.html)\n\n## 课程来源\n\n本教程基于 InfiniTensor 大模型与人工智能系统训练营 2025 冬季课程内容整理。\n\n---\n\n*祝学习愉快！*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlbo%2Fcuda_learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlbo%2Fcuda_learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlbo%2Fcuda_learning/lists"}