https://github.com/peremartra/rearchitecting-llms
Transform generic LLMs into specialized, fast and efficient models.
https://github.com/peremartra/rearchitecting-llms
Last synced: about 1 month ago
JSON representation
Transform generic LLMs into specialized, fast and efficient models.
- Host: GitHub
- URL: https://github.com/peremartra/rearchitecting-llms
- Owner: peremartra
- Created: 2025-08-19T11:11:42.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2025-08-28T16:35:31.000Z (about 1 month ago)
- Last Synced: 2025-08-28T16:38:25.231Z (about 1 month ago)
- Language: Jupyter Notebook
- Size: 101 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Rearchitecting-LLMs
Transform generic LLMs into specialized, fast and hyper-efficient models.
This repository contains a collection of Python notebooks and scripts focused on optimizing the architecture of Large Language Models.
The goal is to explore and practically demonstrate various structural modification techniques to improve the efficiency and fairness of the models.
## Content
The code explores, among other things, the following areas:
* Structured Pruning:
* Depth Pruning
* Width Pruning
* Optimization of Attention Mechanisms
* Knowledge Recovery post-optimization
* Analysis of Internal Activations for interpretability
* Fair Pruning: A methodology to mitigate biases directly in the model's architecture, combining efficiency and ethics.