https://github.com/srush/triton-puzzles

Puzzles for learning Triton
https://github.com/srush/triton-puzzles

machine-learning puzzle

Last synced: 30 days ago
JSON representation

Puzzles for learning Triton

• Host: GitHub
• URL: https://github.com/srush/triton-puzzles
• Owner: srush
• License: apache-2.0
• Created: 2024-03-19T00:40:03.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2024-09-25T18:04:28.000Z (about 2 months ago)
• Last Synced: 2024-09-27T03:04:39.519Z (about 2 months ago)
• Topics: machine-learning, puzzle
• Language: Jupyter Notebook
• Homepage:
• Size: 1010 KB
• Stars: 1,006
• Watchers: 10
• Forks: 64
• Open Issues: 6
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Triton Puzzles

w/ [Tejas Ramesh](https://tejas3070.github.io/) and [Keren Zhou](https://www.jokeren.tech/) based on [Triton-Viz](https://github.com/Deep-Learning-Profiling-Tools/triton-viz)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/srush/Triton-Puzzles/blob/main/Triton-Puzzles.ipynb)

Programming for accelerators such as GPUs is critical for modern AI systems.

This often means programming directly in proprietary low-level languages such as CUDA. [Triton](https://github.com/openai/triton/) is an alternative open-source language that allows you to code at a higher-level and compile to accelerators like GPU.

Coding for Triton is very similar to Numpy and PyTorch in both syntax and semantics. However, as a lower-level language there are a lot of details that you need to keep track of. In particular, one area that learners have trouble with is memory loading and storage which is critical for speed on low-level devices.

This set is puzzles is meant to teach you how to use Triton from first principles in an interactive fashion. You will start with trivial examples and build your way up to real algorithms like Flash Attention and Quantized neural networks. These puzzles **do not** need to run on GPU since they use a Triton interpreter.

Discord: https://discord.gg/gpumode #triton-puzzles

![image](https://github.com/srush/Triton-Puzzles/assets/35882/3e18a47d-1311-43d0-a025-ed1f593f919e)

If you are into this kind of thing, this is 7th in a series of these puzzles.

* https://github.com/srush/gpu-puzzles

* https://github.com/srush/tensor-puzzles

* https://github.com/srush/autodiff-puzzles

* https://github.com/srush/transformer-puzzles

* https://github.com/srush/GPTworld

* https://github.com/srush/LLM-Training-Puzzles