{"id":25008768,"url":"https://github.com/bjornmelin/deep-learning-evolution","last_synced_at":"2026-05-09T07:06:16.376Z","repository":{"id":275625975,"uuid":"926669213","full_name":"BjornMelin/deep-learning-evolution","owner":"BjornMelin","description":"🧠 Deep-Learning Evolution: Unified collection of TensorFlow \u0026 PyTorch projects, featuring custom CUDA kernels, distributed training, memory‑efficient methods, and production‑ready pipelines. Showcases advanced GPU optimizations, from foundational models to cutting‑edge architectures. 🚀","archived":false,"fork":false,"pushed_at":"2025-02-03T17:10:51.000Z","size":8,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T01:26:27.442Z","etag":null,"topics":["ai-research","cuda","data-science","deep-learning","distributed-training","gan","gpu-acceleration","machine-learning","model-optimization","neural-networks","python","pytorch","tensorflow","training-pipeline","transformers"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BjornMelin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-03T16:59:12.000Z","updated_at":"2025-02-03T17:12:23.000Z","dependencies_parsed_at":"2025-02-03T18:25:50.136Z","dependency_job_id":"91356638-8e43-4e36-a8ef-57e86dbf05ee","html_url":"https://github.com/BjornMelin/deep-learning-evolution","commit_stats":null,"previous_names":["bjornmelin/deep-learning-evolution"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/BjornMelin/deep-learning-evolution","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BjornMelin%2Fdeep-learning-evolution","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BjornMelin%2Fdeep-learning-evolution/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BjornMelin%2Fdeep-learning-evolution/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BjornMelin%2Fdeep-learning-evolution/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BjornMelin","download_url":"https://codeload.github.com/BjornMelin/deep-learning-evolution/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BjornMelin%2Fdeep-learning-evolution/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32810385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"online","status_checked_at":"2026-05-09T02:00:06.633Z","response_time":123,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-research","cuda","data-science","deep-learning","distributed-training","gan","gpu-acceleration","machine-learning","model-optimization","neural-networks","python","pytorch","tensorflow","training-pipeline","transformers"],"created_at":"2025-02-05T03:28:41.398Z","updated_at":"2026-05-09T07:06:16.362Z","avatar_url":"https://github.com/BjornMelin.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Learning Evolution 🚀\n\n[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org/downloads/)\n[![TensorFlow](https://img.shields.io/badge/tensorflow-2.14%2B-orange.svg)](https://tensorflow.org/)\n[![PyTorch](https://img.shields.io/badge/pytorch-2.2%2B-red.svg)](https://pytorch.org/)\n[![CUDA](https://img.shields.io/badge/cuda-11.8%2B-green.svg)](https://developer.nvidia.com/cuda-toolkit)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n[![Contributions Welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](CONTRIBUTING.md)\n\n\u003e **Deep-Learning Evolution** combines both **TensorFlow** and **PyTorch** projects into a single, cohesive repository. From foundational neural networks to custom CUDA kernels and distributed training systems, you’ll find memory‑efficient training techniques, advanced GPU optimizations, and production‑ready deployments for real‑world AI applications.\n\n---\n\n## 📑 Table of Contents\n- [✨ Features](#-features)\n- [📁 Project Structure](#-project-structure)\n- [🔧 Prerequisites](#-prerequisites)\n- [📦 Installation](#-installation)\n- [🚀 Quick Start](#-quick-start)\n  - [TensorFlow Examples](#tensorflow-examples)\n  - [PyTorch Examples](#pytorch-examples)\n- [📚 Documentation](#-documentation)\n  - [Models](#models)\n  - [GPU Optimization](#gpu-optimization)\n  - [Benchmarks](#benchmarks)\n- [🤝 Contributing](#-contributing)\n- [📌 Versioning](#-versioning)\n- [✍️ Author](#-author)\n- [📝 Citation](#-citation)\n- [📄 License](#-license)\n- [🙏 Acknowledgments](#-acknowledgments)\n\n---\n\n## ✨ Features\n\n1. **Comprehensive Framework Coverage**  \n   - TensorFlow 2.14+ and PyTorch 2.2+ examples under one roof.\n\n2. **Custom Training Pipelines**  \n   - Efficient data loading, augmentation, and custom training loops.\n\n3. **Advanced GPU \u0026 CUDA Optimizations**  \n   - Memory‑efficient backprop, kernel fusion, and multi‑GPU data parallelism.\n\n4. **Distributed Training**  \n   - PyTorch’s `DistributedTrainer` and TensorFlow’s MirroredStrategy for large‑scale HPC setups.\n\n5. **Production‑Ready Models**  \n   - Demonstrations of how to package and deploy models with Docker, Streamlit, or other tools.\n\n6. **Comprehensive Documentation \u0026 Tests**  \n   - Detailed notebooks, examples, and unit tests in both frameworks.\n\n---\n\n## 📁 Project Structure\n\nDepending on your final layout, you might organize your repo as follows:\n\n```mermaid\ngraph TD\n    A[deep-learning-evolution] --\u003e B[tensorflow]\n    A --\u003e C[pytorch]\n    A --\u003e D[common]\n    A --\u003e E[benchmarks]\n    A --\u003e F[tests]\n    B --\u003e G[notebooks]\n    B --\u003e H[src]\n    B --\u003e I[apps]\n    C --\u003e J[examples]\n    C --\u003e K[src]\n    C --\u003e L[extensions]\n    D --\u003e M[utils]\n    D --\u003e N[datasets]\n    D --\u003e O[docs]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand possible directory structure\u003c/summary\u003e\n\n```plaintext\ndeep-learning-evolution/\n├── tensorflow/\n│   ├── notebooks/           # Jupyter notebooks for TF experiments\n│   ├── src/                 # TF source code (models, data, utils, cuda)\n│   ├── apps/                # Demo or production apps (e.g., Streamlit)\n│   └── ...\n├── pytorch/\n│   ├── examples/            # PyTorch example scripts\n│   ├── src/                 # PyTorch source (models, training utils)\n│   ├── extensions/          # Custom CUDA kernels/extensions\n│   └── ...\n├── common/\n│   ├── utils/               # Shared utility scripts\n│   ├── datasets/            # Shared data loaders or references\n│   └── docs/                # Additional documentation\n├── benchmarks/              # Performance comparisons across frameworks\n├── tests/                   # Unit tests for TF and PT code\n├── requirements.txt         # Dependencies\n└── README.md                # Main documentation\n```\n\u003c/details\u003e\n\n**Tip**: You can keep separate `requirements-tf.txt` and `requirements-pt.txt` if you prefer splitting dependencies.\n\n---\n\n## 🔧 Prerequisites\n\n- **Python** 3.8+\n- **CUDA** 11.8+\n- **TensorFlow** 2.14+  \n- **PyTorch** 2.2+  \n- **NVIDIA GPU** with compute capability 6.0+  \n- **(Optional)** C++ compiler for building custom CUDA extensions (e.g., PyTorch’s extension modules)\n\n---\n\n## 📦 Installation\n\n1. **Clone this repository:**\n\n   ```bash\n   git clone https://github.com/BjornMelin/deep-learning-evolution.git\n   cd deep-learning-evolution\n   ```\n\n2. **Set up a virtual environment:**\n\n   ```bash\n   python -m venv venv\n   source venv/bin/activate  # Linux/Mac\n   # or\n   .\\venv\\Scripts\\activate  # Windows\n   ```\n\n3. **Install dependencies:**\n\n   ```bash\n   # Single unified requirements:\n   pip install -r requirements.txt\n\n   # OR separate for each framework:\n   pip install -r requirements-tf.txt\n   pip install -r requirements-pt.txt\n   ```\n\n4. **(Optional) Build custom CUDA extensions (for PyTorch examples):**\n\n   ```bash\n   cd pytorch/extensions/cuda\n   python setup.py install\n   ```\n\n---\n\n## 🚀 Quick Start\n\n### TensorFlow Examples\n\n```python\n# TensorFlow: Basic CNN Example\nfrom tensorflow.src.models import BasicCNN\nfrom tensorflow.src.utils import DataLoader\n\ndata_loader = DataLoader('path/to/data')\nmodel = BasicCNN()\n\nmodel.train(data_loader, epochs=10)\nmodel.evaluate(...)\nmodel.save('saved_model')\n```\n\n### PyTorch Examples\n\n```python\n# PyTorch: Custom Transformer Example\nfrom pytorch.src import models, training\n\nmodel = models.TransformerWithCustomAttention(num_heads=8, d_model=512)\ntrainer = training.DistributedTrainer(model, gpu_list=[0,1])\ntrainer.train(dataset, epochs=10)\n```\n\n\u003e For more demos, see the [notebooks](./tensorflow/notebooks) in `tensorflow/` or the [examples](./pytorch/examples) folder in `pytorch/`.\n\n---\n\n## 📚 Documentation\n\n### Models\n\n| Model                 | Framework      | Description                                 | Performance        | Memory Efficiency |\n|-----------------------|----------------|---------------------------------------------|--------------------|-------------------|\n| `BasicCNN`            | TensorFlow     | Foundational CNN architecture               | 94% on MNIST       | High              |\n| `TransformerBlock`    | TensorFlow     | Custom transformer for NLP                  | BERT-comparable    | Medium            |\n| `AdvancedGAN`         | TensorFlow     | Generative adversarial network              | FID: 18.3          | High              |\n| `TransformerWithCustomAttention` | PyTorch | Advanced Transformer w/ fused attention  | 95% on GLUE        | High              |\n| `ViT`                 | PyTorch        | Vision transformer with optimized attention | 92% on ImageNet    | Medium            |\n| `CustomGAN`           | PyTorch        | Memory‑efficient adversarial network        | FID: 12.4          | High              |\n\n---\n\n### GPU Optimization\n\n- **Custom CUDA kernels** for specialized operations  \n- **Memory‑efficient training** (gradient checkpointing, fused ops)  \n- **Multi‑GPU data parallelism** (PyTorch DDP, TF MirroredStrategy)  \n- **Distributed training** for large‑scale HPC  \n- **Kernel fusion techniques** to minimize overhead\n\n---\n\n### Benchmarks\n\n| Model                 | Hardware | Framework   | Training Time | Memory Usage | Throughput        |\n|-----------------------|---------|------------|---------------|-------------|-------------------|\n| Custom BERT           | A100    | PyTorch    | 4.2 hrs       | 16GB        | 1250 samples/sec  |\n| ViT                   | V100    | PyTorch    | 6.8 hrs       | 24GB        | 850 samples/sec   |\n| ResNet-152            | 3090    | PyTorch    | 2.5 hrs       | 12GB        | 2100 samples/sec  |\n| BasicCNN (small)      | 4GB GPU | TensorFlow | 2.3 hrs       | ~2GB        | 900 samples/sec   |\n| TransformerBlock      | 8GB GPU | TensorFlow | 5.7 hrs       | ~6GB        | 600 samples/sec   |\n\n(Exact numbers are illustrative; refer to each framework’s benchmarks folder for detailed logs.)\n\n---\n\n## 🤝 Contributing\n\nWe welcome all contributions, from bug fixes to new model architectures and GPU kernels.\n\n- [Contributing Guidelines](CONTRIBUTING.md)  \n- [Code of Conduct](CODE_OF_CONDUCT.md)  \n- [Development Guide](DEVELOPMENT.md)\n\n1. **Fork** this repository  \n2. **Create** a new branch  \n3. **Commit** your changes  \n4. **Open** a Pull Request\n\n---\n\n## 📌 Versioning\n\nWe use [SemVer](http://semver.org/) for versioning. For available versions, check the [tags](https://github.com/BjornMelin/deep-learning-evolution/tags).\n\n---\n\n## ✍️ Author\n\n**Bjorn Melin**  \n- GitHub: [@BjornMelin](https://github.com/BjornMelin)  \n- LinkedIn: [Bjorn Melin](https://linkedin.com/in/bjorn-melin)\n\nFeel free to add additional contributors or maintainers here.\n\n---\n\n## 📝 Citation\n\nIf you find this project useful, please cite:\n\n```bibtex\n@misc{melin2024deeplearningevolution,\n  author = {Melin, Bjorn},\n  title = {Deep Learning Evolution: Unified TensorFlow \u0026 PyTorch Implementations},\n  year = {2024},\n  publisher = {GitHub},\n  url = {https://github.com/BjornMelin/deep-learning-evolution}\n}\n```\n\n---\n\n## 📄 License\n\nThis project is licensed under the **MIT License**. See the [LICENSE](LICENSE) file for details.\n\n---\n\n## 🙏 Acknowledgments\n\n- **TensorFlow** team for extensive documentation and tooling.  \n- **PyTorch** contributors for ongoing innovation.  \n- **NVIDIA** for the CUDA toolkit.  \n- **Open Source Community** for valuable feedback and inspiration.\n\n---\n\n**Made with 🧠 and ⚡ by Bjorn Melin**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbjornmelin%2Fdeep-learning-evolution","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbjornmelin%2Fdeep-learning-evolution","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbjornmelin%2Fdeep-learning-evolution/lists"}