{"id":23123448,"url":"https://github.com/mk2112/nn-zero-to-hero-notes","last_synced_at":"2025-04-05T07:01:49.604Z","repository":{"id":172016232,"uuid":"648722244","full_name":"MK2112/nn-zero-to-hero-notes","owner":"MK2112","description":"Jupyter Notebook notes on Andrej Karpathy's videos and the tutorial series, \"Neural Networks: Zero to Hero.\"","archived":false,"fork":false,"pushed_at":"2025-03-25T14:23:57.000Z","size":32088,"stargazers_count":156,"open_issues_count":0,"forks_count":22,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-29T06:05:39.474Z","etag":null,"topics":["deep-learning","gpt","neural-networks","nlp","nn-zero-to-hero","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MK2112.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-02T16:36:25.000Z","updated_at":"2025-03-28T12:52:29.000Z","dependencies_parsed_at":"2025-01-14T11:09:23.394Z","dependency_job_id":"e23da0b5-3364-4d14-b31c-9e2fd80b0047","html_url":"https://github.com/MK2112/nn-zero-to-hero-notes","commit_stats":null,"previous_names":["mk2112/nn-zero-to-hero-notes"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MK2112%2Fnn-zero-to-hero-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MK2112%2Fnn-zero-to-hero-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MK2112%2Fnn-zero-to-hero-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MK2112%2Fnn-zero-to-hero-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MK2112","download_url":"https://codeload.github.com/MK2112/nn-zero-to-hero-notes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247299828,"owners_count":20916190,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","gpt","neural-networks","nlp","nn-zero-to-hero","pytorch"],"created_at":"2024-12-17T07:34:23.610Z","updated_at":"2025-04-05T07:01:49.487Z","avatar_url":"https://github.com/MK2112.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Notes - Neural Networks: Zero to Hero\n\n[Andrej's GitHub Repository](https://github.com/karpathy/nn-zero-to-hero)\u003cbr\u003e\n[Andrej's YouTube](https://www.youtube.com/@AndrejKarpathy)\u003cbr\u003e\n[Eureka Labs Discord](https://discord.com/invite/3zy8kqD9Cp)\n\nJupyter Notebooks and Markdown files with detailed notes on Andrej Karpathy's tutorial series \"Neural Networks: Zero to Hero\", his talks and general audience videos. The notes aim to provide an additional overview and documentation of the concepts and techniques covered in the videos.\n\n## Zero to Hero Tutorial Series\n\n\u003ctable style=\"width: 100%;\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eVideo\u003c/th\u003e\n    \u003cth\u003eNotes\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=VMj-3S1tku0\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/VMj-3S1tku0/0.jpg\" alt=\"The spelled-out intro to neural networks and backpropagation: building micrograd\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N001 - Building Micrograd/N001 - Micrograd.ipynb\"\u003eN001 - Building Micrograd\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=PaCmpygFfXo\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/PaCmpygFfXo/0.jpg\" alt=\"The spelled-out intro to language modeling: building makemore\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N002 - Makemore 1/N002 - Makemore.ipynb\"\u003eN002 - Makemore 1\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=TCH_1BHY58I\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/TCH_1BHY58I/0.jpg\" alt=\"Building makemore Part 2: MLP\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N003 - Makemore 2 - MLP/N003 - Makemore_2.ipynb\"\u003eN003 - Makemore 2 - MLP\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=P6sfmUTpUmc\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/P6sfmUTpUmc/0.jpg\" alt=\"Building makemore Part 3: Activations \u0026 Gradients, BatchNorm\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N004 - Makemore 3 - Activations, BatchNorm/N004 - Makemore_3.ipynb\"\u003eN004 - Makemore 3 - Activations, BatchNorm\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=q8SA3rM6ckI\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/q8SA3rM6ckI/0.jpg\" alt=\"Building makemore Part 4: Becoming a Backprop Ninja\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N005 - Makemore 4 - Backprop Ninja/N005 - Makemore_4.ipynb\"\u003eN005 - Makemore 4 - Becoming a Backprop Ninja\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=t3YJ5hKiMQ0\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/t3YJ5hKiMQ0/0.jpg\" alt=\"Building makemore Part 5: Building a WaveNet\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N006 - Makemore 5 - WaveNet/N006 - Makemore_5.ipynb\"\u003eN006 - Makemore 5 - WaveNet\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=kCc8FmEb1nY\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/kCc8FmEb1nY/0.jpg\" alt=\"Let's build GPT: from scratch, in code, spelled out.\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N007 - GPT From Scratch/N007 - GPT.ipynb\"\u003eN007 - GPT From Scratch\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=zduSFxRajkE\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/zduSFxRajkE/0.jpg\" alt=\"Let's build the GPT Tokenizer\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N008 - GPT Tokenizer/N008 - Tokenization.ipynb\"\u003eN008 - GPT Tokenizer\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=l8pRSuU81PU\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/l8pRSuU81PU/0.jpg\" alt=\"Let's reproduce GPT-2 (124M)\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./N009 - Reproducing GPT-2/N009 - Reproducing_GPT-2.ipynb\"\u003eN009 - Reproducing GPT-2\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## General\n\n\u003ctable style=\"width: 100%;\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eVideo\u003c/th\u003e\n    \u003cth\u003eNotes\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=7xTGNNLPyMI\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/7xTGNNLPyMI/0.jpg\" alt=\"Deep Dive into LLMs like ChatGPT\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./G001 - Deep Dive into LLMs/G001 - Deep Dive into LLMs.md\"\u003eG001 - Deep Dive into LLMs like ChatGPT\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=EWvNQjAaOHw\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/EWvNQjAaOHw/0.jpg\" alt=\"How I use LLMs\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./G002 - How I Use LLMs/G002 - How I Use LLMs.md\"\u003eG002 - How I Use LLMs\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Talks\n\n\u003ctable style=\"width: 100%;\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eVideo\u003c/th\u003e\n    \u003cth\u003eNotes\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=bZQun8Y4L2A\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/bZQun8Y4L2A/0.jpg\" alt=\"State of GPT | BRK216HFS\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./T001 - State of GPT/T001 - State_of_GPT - Notes.md\"\u003eT001 - State of GPT\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003ca href=\"https://www.youtube.com/watch?v=zjkBMFhNj_g\" target=\"_blank\"\u003e\u003cimg src=\"https://img.youtube.com/vi/zjkBMFhNj_g/0.jpg\" alt=\"[1hr Talk] Intro to Large Language Models\" height=\"auto\" width=\"150\"\u003e\u003c/a\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003ca href=\"./T002 - Intro to LLMs - Director's Cut/T002 - Intro_to_LLMs - Director's_Cut - Notes.md\"\u003eT002 - Intro to LLMs - Director's Cut\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Getting Started\n\n1. Clone this repository to your local machine using the following command:\n\n```bash\ngit clone https://github.com/MK2112/nn-zero-to-hero-notes.git\n```\n\n2. Navigate to the repository, make sure to run `pip install -r requirements.txt`.\n3. Launch Jupyter Notebook by running the command:\n\n```bash\njupyter notebook\n```\n\n4. Within Jupyter, navigate to the repository and open the desired notebook.\n\n## Contributions\n\nContributions are welcome and encouraged. If you find any errors, have suggestions for improvements, or want to add to the materials, please submit a pull request.\n\n## Acknowledgments\n\nThank you Andrej Karpathy for creating and sharing the \"Neural Networks: Zero to Hero\" tutorial series. Find more of Andrej's work on his [GitHub](https://github.com/karpathy) and [𝕏/Twitter](https://x.com/karpathy).\n\n## License\n\nThis repository is licensed under the MIT License. Please attribute the original content to Andrej Karpathy and provide links to the tutorial series, the videos and this repo.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk2112%2Fnn-zero-to-hero-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmk2112%2Fnn-zero-to-hero-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk2112%2Fnn-zero-to-hero-notes/lists"}