awesome-gpu-engineering
GPU Engineering for AI Systems
https://github.com/goabiaryan/awesome-gpu-engineering
Last synced: 19 days ago
JSON representation
⭐ Acknowledgements
Learning Tools
🧠 Architecture and Low-Level Design
📘 Foundational Books
- Amazon - gpu-engineering/blob/main/notes/Abi's%20PMPP%20Notes.pdf)
- Amazon
- Web Version
💻 GPU Programming Frameworks
- CUDA
- cuBLAS
- ROCm
- OpenCL - platform parallel computing standard.
- SYCL / oneAPI
- Vulkan Compute - level GPU compute API.
- Metal Performance Shaders
- Mojo🔥 - Write like Python, run like C++.
🧾 License
Learning Tools
🧩 Optimization and Performance
- NVIDIA Nsight Systems - wide GPU profiler.
- Nsight Compute - level performance analysis.
- CUTLASS
- TensorRT - performance deep learning inference.
- OpenAI Triton - performance GPU kernels.
- Roofline Model
- Helion - A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.
📄 Research Papers and Articles
- Optimization techniques for GPU programming - Hijma, Pieter, et al.
- Efficient Multi-GPU Programming in Python: Reducing Synchronization and Access Overheads - Oden, Lena, and Klaus Nölp
- Evolving GPU Architecture
- Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision - Wei Gao et al
- Optimizing Machine Learning Models with CUDA: A Comprehensive Performance Analysis - Niteesh, L., and M. B. Ampareeshan
- Model Parallelism - LM](https://arxiv.org/pdf/1909.08053)*
- GPU Virtualization and Multi-Tenant Scheduling
- A Survey of Multi-Tenant Deep Learning Inference on GPU
- Efficient Performance-Aware GPU Sharing with Compatibility and Isolation through Kernel Space Interception
⚙️ Systems and Multi-GPU Engineering
- NCCL - GPU communication primitives.
- vLLM - Inference and serving engine for LLMs
- Hugging Face Accelerate - Simplify abstractions for distributed training
- SGLang
- TensorRT-LLM
- TGI by Hugging Face
- Horovod
- GPUDirect RDMA - copy GPU networking.
- Iris by AMD - open-source multi-GPU programming framework built for compiler-visible performance and optimized multi-GPU execution.
🧰 Tools and Utilities
Learning Tools
- LeetGPU
- GPU Glossary - A dictionary of terms related to programming GPUs
- Tensara
- Mojo🔥 GPU Puzzles
- Ingero - based GPU causal observability agent. Traces CUDA Runtime/Driver APIs and host kernel events to build causal chains explaining GPU latency. <2% overhead, production-safe.