{"id":24773214,"url":"https://github.com/burnycoder/ai-from-scratch","last_synced_at":"2025-07-12T16:06:26.520Z","repository":{"id":245608668,"uuid":"731419431","full_name":"BurnyCoder/ai-from-scratch","owner":"BurnyCoder","description":"Transformers, LLM, GPT-2, LSTM, Diffusion, U-Net, CNNs, MLP, Reinforcement learning, Bigram, Linear+Polynomial+Logistic Regression, GRPO, Selfplay, MCTS, PyTorch, Scikit-learn, Torchvision, NumPy, Matplotlib, Einops, Transformers, Plotly","archived":false,"fork":false,"pushed_at":"2025-04-16T16:33:15.000Z","size":2043,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-16T22:53:59.480Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BurnyCoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-14T03:27:21.000Z","updated_at":"2025-04-16T16:33:18.000Z","dependencies_parsed_at":"2024-06-23T01:51:31.062Z","dependency_job_id":"a11b7b26-c0f7-4dda-9150-ed23cf085b39","html_url":"https://github.com/BurnyCoder/ai-from-scratch","commit_stats":null,"previous_names":["burnycoder/pidi-gpt"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/BurnyCoder/ai-from-scratch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurnyCoder%2Fai-from-scratch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurnyCoder%2Fai-from-scratch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurnyCoder%2Fai-from-scratch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurnyCoder%2Fai-from-scratch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BurnyCoder","download_url":"https://codeload.github.com/BurnyCoder/ai-from-scratch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurnyCoder%2Fai-from-scratch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265017011,"owners_count":23698413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-29T04:39:56.300Z","updated_at":"2025-07-12T16:06:26.513Z","avatar_url":"https://github.com/BurnyCoder.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI From Scratch\n\nThis repository contains implementations of various AI and machine learning concepts, architectures, and exercises built from scratch. It serves as a learning resource for understanding the underlying principles of artificial intelligence and machine learning algorithms.\n\n## Repository Structure\n\n### Architectures\n\nThis directory contains implementations of different neural network architectures and machine learning models.\n\n- **convolutional_neural_network_reinforcement_learning_monte_carlo_tree_search_selfplay_alphazero_tictactoe.ipynb**: Implementation of a convolutional neural network with reinforcement learning using Monte Carlo Tree Search and self-play, similar to AlphaZero, applied to Tic-Tac-Toe.\n- **logistic_regression.ipynb**: Implementation of logistic regression algorithm.\n- **lstm.py**: Long Short-Term Memory neural network implementation.\n- **diffusion.py**: Diffusion model implementation.\n- **linear_regression.py**: Linear regression implementation.\n- **bigram.py**: Bigram language model implementation.\n- **multiple_linear_polynomical_sinus_etc_regression_and_gradient_descent.ipynb**: Implementation of various regression techniques including multiple linear, polynomial, and sine regression with gradient descent.\n- **reinforcement-learning-deep-q-learning**: Implementation of Deep Q-learning reinforcement learning and applied to Snake game.\n- **grpo_group_relative_policy_optimization.ipynb**: Implementation of Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm that enhances LLM's reasoning abilities by generating multiple responses to a given prompt, evaluating each using a reward function (solution correctness in math), and updating the model based on the relative performance of these responses within the group\n\n#### Transformer\n\n- **transformer.py**: Implementation of the transformer architecture.\n- **einops.py**: Implementation of einops operations for tensor manipulations.\n\n##### GPT-2\n\n- **train_gpttwo.py**: Training script for a GPT-2 style model.\n- **play.ipynb**: Interactive notebook for playing with the trained GPT-2 model.\n- **fineweb.py**: Fine-tuning utilities for web data.\n- **hellaswag.py**: Implementation for the HellaSwag benchmark.\n- **input.txt**: Training data for the model.\n\n### Physics-Inspired Neural Networks\n\n- **cooling/**: A collection of notebooks and code exploring physics-based applications of machine learning:\n  - **temp_pred.ipynb**: Neural network models and physics informed neural network model for predicting temperature dynamics in cooling systems, including implementation of L2 regularization techniques.\n  - **regularisation_ex.ipynb**: Demonstrates the application of regularization techniques in machine learning models to prevent overfitting, with visualizations comparing regularized vs. non-regularized polynomial regression.\n  - **network.py**: Basic neural network architecture for solving physics-based problems.\n  - **diff_equations.py**: Implementation of fundamental cooling law equations and gradient calculation for physics-based machine learning.\n\n### Exercises\n\n- **hyperplane_classifier_of_clothes.py**: Exercise implementation of a hyperplane classifier for clothing items.\n\n## Getting Started\n\nTo use this repository, clone it to your local machine and explore the different implementations. Each file is self-contained and includes the necessary code to understand and run the respective algorithm or model.\n\n```bash\ngit clone https://github.com/yourusername/ai-from-scratch.git\ncd ai-from-scratch\n```\n\n## Prerequisites\n\n- Python 3.x\n- NumPy\n- PyTorch (for some implementations)\n- Jupyter Notebook (for running .ipynb files)\n\n## License\n\nThis project is available for educational purposes.\n\n## Acknowledgments\n\n- Inspired by the desire to understand AI and machine learning concepts from first principles. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fburnycoder%2Fai-from-scratch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fburnycoder%2Fai-from-scratch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fburnycoder%2Fai-from-scratch/lists"}