https://github.com/huggingface/kernels
Load compute kernels from the Hub
https://github.com/huggingface/kernels
Last synced: 3 months ago
JSON representation
Load compute kernels from the Hub
- Host: GitHub
- URL: https://github.com/huggingface/kernels
- Owner: huggingface
- License: apache-2.0
- Created: 2024-11-29T15:46:13.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2026-02-25T05:04:53.000Z (3 months ago)
- Last Synced: 2026-02-25T10:00:15.480Z (3 months ago)
- Language: Python
- Homepage:
- Size: 1.94 MB
- Stars: 445
- Watchers: 5
- Forks: 44
- Open Issues: 11
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# kernels
The Kernel Hub allows Python libraries and applications to load compute
kernels directly from the [Hub](https://hf.co/). To support this kind
of dynamic loading, Hub kernels differ from traditional Python kernel
packages in that they are made to be:
- Portable: a kernel can be loaded from paths outside `PYTHONPATH`.
- Unique: multiple versions of the same kernel can be loaded in the
same Python process.
- Compatible: kernels must support all recent versions of Python and
the different PyTorch build configurations (various CUDA versions
and C++ ABIs). Furthermore, older C library versions must be supported.
## Components
- You can load kernels from the Hub using the [`kernels`](kernels/) Python package.
- If you are a kernel author, you can build your kernels with [kernel-builder](builder/).
- Hugging Face maintains a set of kernels in [kernels-community](https://huggingface.co/kernels-community).
## 🚀 Quick Start
Install the `kernels` Python package with `pip` (requires `torch>=2.5` and CUDA):
```bash
pip install kernels
```
Here is how you would use the [activation](https://huggingface.co/kernels-community/activation) kernels from the Hugging Face Hub:
```python
import torch
from kernels import get_kernel
# Download optimized kernels from the Hugging Face hub
activation = get_kernel("kernels-community/activation", version=1)
# Random tensor
x = torch.randn((10, 10), dtype=torch.float16, device="cuda")
# Run the kernel
y = torch.empty_like(x)
activation.gelu_fast(y, x)
print(y)
```
You can [search for kernels](https://huggingface.co/models?other=kernels) on
the Hub.
## 📚 Documentation
Read the [documentation of kernels and kernel-builder](https://huggingface.co/docs/kernels/).