Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/camille-004/feathernet
⏳ Lightweight deep learning library and compiler suite built for educational purposes.
https://github.com/camille-004/feathernet
compilers deep-learning deep-learning-compiler python
Last synced: 5 days ago
JSON representation
⏳ Lightweight deep learning library and compiler suite built for educational purposes.
- Host: GitHub
- URL: https://github.com/camille-004/feathernet
- Owner: camille-004
- Created: 2023-12-14T12:17:46.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-02T04:51:56.000Z (10 months ago)
- Last Synced: 2024-03-03T04:33:00.171Z (10 months ago)
- Topics: compilers, deep-learning, deep-learning-compiler, python
- Language: Python
- Homepage:
- Size: 463 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Feathernet
---
## Project Status[](#project-status)
---
## Introduction[](#introduction)
**Feathernet** is a lightweight deep learning and compiler suite. Aptly named for its light-as-a-feather footprint, Feathernet is specifically crafted for those keen on exploring the realms of deep learning and compiler technology. The `dl` package encompasses essential deep learning components, enabling users to effortlessly build, train, and evaluate basic neural network models. The `compiler` package offers tools for graph optimization and Intermediate Representations (IR), positioning itself as a potential resource for understanding and implementing advanced model optimization techniques.
---
## Table of Contents[](#table-of-contents)
- [Motivation](#motivation)
- [From Scratch in Python](#from-scratch-in-python)
- [Composition](#composition)
- [Running Tests](#running-tests)
- [Running Tests Locally](#running-tests-locally)
- [Running Tests in the Docker Container](#running-tests-in-the-docker-container)
---
## Motivation[](#motivation)
Deep learning compilers translate high-level neural network models into optimized, hardware-specific code, bridging the gap between theoretical model design and practical application. Examples of prominent deep learning compilers include [TensorRT](https://developer.nvidia.com/tensorrt) (inference optimization), [XLA (*Accelerated Linear Algebra*)](https://www.tensorflow.org/xla), and [Glow](https://ai.meta.com/tools/glow/).
The venture into deep learning compilers is driven by the need to:
- *Enhance Computational Efficiency*: Optimizing neural network models for specific hardware accelerates performance and reduces resource consumption.
- *Facilitate Cross-Platform Compatibility*: Compilers enable models to run seamlessly across different types of hardware.
- *Democratize AI Development*: By simplifying the process of model optimization, compilers make AI development more accessible to a broader range of developers and researchers.
Feathernet aims to provide an intuitive understanding of these concepts through its `compiler` package, offering tools for graph optimization and IRs.
Key resources for further reading on deep learning compilers:
- [AI Compilers Demystified](https://medium.com/geekculture/ai-compilers-ae28afbc4907)
- [A friendly introduction to machine learning compilers and optimizers](https://huyenchip.com/2021/09/07/a-friendly-introduction-to-machine-learning-compilers-and-optimizers.html)
[ ↑ Back to top ↑ ]
---
## From Scratch in Python[](#from-scratch-in-python)
Feathernet's journey of being developed from scratch in Python is driven by an ambition to unveil the synergy between DL libraries and compilers. This allows us to observe and understand how these two critical components of AI technology work in tandem.
Key aspects of this approach include:
- *Integrated Development*: Simultaneously crafting a deep learning library and a compiler reveals the intricate interplay between model creation and optimization. It provides a holistic view of how neural networks are translated into efficient, executable code.
- *Focus on CPU as Target*: While the deep learning field often focuses on GPU optimizations, Feathernet places its emphasis on CPU capabilities. This choice offers insides into how compilers optimize for different hardware architectures, particulraly the more universally accessible CPUs.
- *Practical Learning Experience*: This dual development approach grants practical insights into compiler development challenges, particularly in translating high-level neural network models into executable code.
> 💡 The code in the `dl` package of the Feathernet suite is intentionally less optimized. This is because the focus is more on the compiler aspect, emphasizing and understanding compiler technologies rather than optimizing the deep learning components themselves.
[ ↑ Back to top ↑ ]
---
## Composition[](#composition)
Feathernet is composed of two primary packages.
- [`dl`](https://github.com/camille-004/feathernet/tree/main/feathernet/dl): This package encompasses core deep learning components, including layers, optimizers, initializers, and losses. Additionally, each component in the `dl` package is equipped with serialization capabilities, enabling integration with the compiler's Intermediate Representation (IR).
- [`compiler`](https://github.com/camille-004/feathernet/tree/main/feathernet/compiler): This package offers tools and modules for graph optimization and Intermediate Representation (IR). It includes functionalities for layer fusion, pruning, and quantization.
[ ↑ Back to top ↑ ]
---
## Running Tests[](#run-tests)
To ensure the highest quality and reliability of the code, Feathernet includes a comprehensive suite of unit tests.
Reflecting its dual-package structure, the tests in Feathernet are divided into two main categories.
- Tests for the `dl` package cover deep learning components.
- Tests for the `compiler` package focus on DL compiler components like graph optimization and IR.
### Running Tests Locally
| **Prerequisite** | **Installation** |
| --- | ---|
| `make` | [GNU Make](https://www.gnu.org/software/make/) |
| `poetry` | [Poetry](https://python-poetry.org/docs/#installation) |