https://github.com/mariosieg/magnetron
(WIP) A small but powerful, homemade PyTorch from scratch.
https://github.com/mariosieg/magnetron
artificial-intelligence cpp cuda high-performance-computing machine-learning neuronal-network python pytorch research-project tensorflow tiny
Last synced: 17 days ago
JSON representation
(WIP) A small but powerful, homemade PyTorch from scratch.
- Host: GitHub
- URL: https://github.com/mariosieg/magnetron
- Owner: MarioSieg
- License: apache-2.0
- Created: 2025-01-02T21:25:56.000Z (4 months ago)
- Default Branch: master
- Last Pushed: 2025-04-08T09:25:35.000Z (17 days ago)
- Last Synced: 2025-04-08T12:02:24.505Z (17 days ago)
- Topics: artificial-intelligence, cpp, cuda, high-performance-computing, machine-learning, neuronal-network, python, pytorch, research-project, tensorflow, tiny
- Language: C++
- Homepage:
- Size: 25.4 MB
- Stars: 544
- Watchers: 4
- Forks: 24
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
magnetron
Minimalistic homemade PyTorch alternative, written in C99 and Python.
Explore the docs »
View Demo
|
Report Bug
|
Request Feature
Table of Contents
## News
- **[2025/01/14]** 🎉 CPU backend now uses multiple threads with dynamic scaling and thread pooling.
- **[2025/01/02]** 🎈 Magnetron released on GitHub.
## About
This project started as a learning experience and a way to understand the inner workings of PyTorch and other deep learning frameworks.
The goal is to create a minimalistic but still powerful deep learning framework that can be used for research and production.
The framework is written in C99 and Python and is designed to be easy to understand and modify.
### Work in Progress
* The project is still in its early stages and many features are missing.
* Developed by a single person in their free time.
* The project is not yet fully optimized for performance.
## Getting Started
To get a local copy up and running follow these simple steps.
Magnetron itself has **no** Python dependencies except for CFFI to call the C library from Python.
Some examples use matplotlib and numpy for plotting and data generation, but these are not required to use the framework.
### Prerequisites
* Linux, MacOS or Windows
* A C99 compiler (gcc, clang, msvc)
* Python 3.6 or higher
### Installation
*A pip installable package will be provided, as soon as all core features are implemented.*
1. Clone the repo
2. `cd magnetron/python` (VENV recommended).
3. `pip install -r requirements.txt` Install dependencies for examples.
4. `cd magnetron_framework && bash install_wheel_local.sh && cd ../` Install the Magnetron wheel locally, a pip installable package will be provided in the future.
5. `python examples/simple/xor.py` Run the XOR example.
## Usage
See the [Examples](python/examples) directory for examples on how to use the framework.
For usage in C and C++ see the [Unit Tests](test) directory in the root of the project.
## Features
* 6 Dimensional, linearized tensors
* Automatic Differentiation
* Multithreaded CPU Compute, SIMD optimized operators (SSE4, AVX2, AVX512, ARM NEON)
* Modern Python API (similar to PyTorch)
* Many operators with broadcasting support and in-place variants
* High level neural network building blocks
* Dynamic computation graph (eager evaluation)
* Modern PRNGs: Mersenne Twister and PCG
* Validation and friendly error messages
* Custom compressed tensor file formats
### Example
Code from the XOR example:
```python
def forward(self, x: Tensor) -> Tensor:
return (self.weight @ x + self.bias).sigmoid()
```
### Operators
Operation
Description
clone(x)
Creates a copy of the tensor
view(x)
Reshapes without changing data
transpose(x)
Swaps tensor dimensions
permute(x, d0, ...)
Reorders tensor dimensions
mean(x)
Mean across dimensions
min(x)
Minimum value of tensor
max(x)
Maximum value of tensor
sum(x)
Sum of elements
abs(x)
Element-wise absolute value
neg(x)
Element-wise negation
log(x)
Element-wise natural logarithm
sqr(x)
Element-wise square
sqrt(x)
Element-wise square root
sin(x)
Element-wise sine
cos(x)
Element-wise cosine
softmax(x)
Softmax along dimension
sigmoid(x)
Element-wise sigmoid
relu(x)
ReLU activation
gelu(x)
GELU activation
add(x, y)
Element-wise addition
sub(x, y)
Element-wise subtraction
mul(x, y)
Element-wise multiplication
div(x, y)
Element-wise division
matmul(A, B)
Matrix multiplication
## Roadmap
The goal is to implement training and inference for LLMs and other state of the art models, while providing a simple and small codebase that is easy to understand and modify.
- [ ] Compute on GPU (Cuda)
- [ ] Low-precision datatypes (f16, bf16, int8)
- [ ] Distributed Training and Inference
- [ ] CPU and GPU kernel JIT compilation
- [ ] Better examples with real world models (LLMs and state of the art models)
## Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
## License
Distributed under the Apache 2 License. See `LICENSE.txt` for more information.
## Similar Projects
* [GGML](https://github.com/ggerganov/ggml)
* [TINYGRAD](https://github.com/tinygrad/tinygrad)
* [MICROGRAD](https://github.com/karpathy/micrograd)
[contributors-shield]: https://img.shields.io/github/contributors/MarioSieg/magnetron.svg?style=for-the-badge
[contributors-url]: https://github.com/MarioSieg/magnetron/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/MarioSieg/magnetron.svg?style=for-the-badge
[forks-url]: https://github.com/MarioSieg/magnetron/network/members
[stars-shield]: https://img.shields.io/github/stars/MarioSieg/magnetron.svg?style=for-the-badge
[stars-url]: https://github.com/MarioSieg/magnetron/stargazers
[issues-shield]: https://img.shields.io/github/issues/MarioSieg/magnetron.svg?style=for-the-badge
[issues-url]: https://github.com/MarioSieg/magnetron/issues
[license-shield]: https://img.shields.io/github/license/MarioSieg/magnetron.svg?style=for-the-badge
[license-url]: https://github.com/MarioSieg/magnetron/blob/master/LICENSE.txt