{"id":18132755,"url":"https://github.com/madlittlemods/zig-ocr-neural-network","last_synced_at":"2025-04-06T16:18:41.890Z","repository":{"id":200918669,"uuid":"700679027","full_name":"MadLittleMods/zig-ocr-neural-network","owner":"MadLittleMods","description":"A from scratch neural network implementation in Zig, trained against the MNIST dataset to recognize handwritten digits.","archived":false,"fork":false,"pushed_at":"2023-12-13T15:38:24.000Z","size":116,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-12T22:18:09.543Z","etag":null,"topics":["deep-learning","machine-learning","mnist","mnist-handwriting-recognition","neural-network","ocr","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MadLittleMods.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-05T04:39:25.000Z","updated_at":"2024-10-18T12:54:55.000Z","dependencies_parsed_at":"2023-10-31T18:37:21.868Z","dependency_job_id":"8513f7e0-a58c-45a4-bf3c-aa0dc34c4e9f","html_url":"https://github.com/MadLittleMods/zig-ocr-neural-network","commit_stats":null,"previous_names":["madlittlemods/zig-ocr-neural-network"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLittleMods%2Fzig-ocr-neural-network","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLittleMods%2Fzig-ocr-neural-network/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLittleMods%2Fzig-ocr-neural-network/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MadLittleMods%2Fzig-ocr-neural-network/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MadLittleMods","download_url":"https://codeload.github.com/MadLittleMods/zig-ocr-neural-network/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247509238,"owners_count":20950232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-learning","mnist","mnist-handwriting-recognition","neural-network","ocr","zig"],"created_at":"2024-11-01T13:05:36.241Z","updated_at":"2025-04-06T16:18:41.870Z","avatar_url":"https://github.com/MadLittleMods.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Basic OCR example using a neural network against the MNIST dataset\n\n\u003e [!TIP]\n\u003e\n\u003e If you are looking for a neural network library that you can include in your project,\n\u003e I've created a library based on this project -\u003e\n\u003e https://github.com/MadLittleMods/zig-neural-networks\n\nA from scratch neural network implementation in Zig, trained against the MNIST dataset\nto recognize handwritten digits.\n\nA lot of the phrasing and concepts are taken from the resources linked in the\n[*developer notes*](./dev-notes.md). I'm just trying to piece together all of those\nresources into something that works and is understandable to me as I learn. Major kudos\nto Sebastian Lague, 3Blue1Brown, The Independent Code (Omar Aflak), and Samson Zhang for\ntheir excellent resources. And a special shoutout to Hans Musgrave\n([@hmusgrave](https://github.com/hmusgrave)) for the immense amount of help to get my\nhead around these concepts as I got stuck through this process.\n\n## Setup\n\nDownload and extract the MNIST dataset from http://yann.lecun.com/exdb/mnist/ to a\ndirectory called `data/` in the root of this project. Here is a copy-paste command\nyou can run:\n\n```sh\n# Make a data/ directory\nmkdir data/ \u0026\u0026\ncd data/ \u0026\u0026\n# Download the MNIST dataset\ncurl -O http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz \u0026\u0026\ncurl -O http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz \u0026\u0026\ncurl -O http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz \u0026\u0026\ncurl -O http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz \u0026\u0026\n# Unzip the files\ngunzip *.gz\n```\n\n\n## Building and running\n\nTested with Zig 0.11.0\n\nWith the OCR example, on my machine, I can complete 1 epoch of training in ~1 minute\nwhich gets to 94% accuracy and creeps to 97% after a few more epochs (60k training\nimages, 10k test images):\n\n```sh\n$ zig build run-mnist_ocr\ndebug: Created normalized data points. Training on 60000 data points, testing on 10000\ndebug: Here is what the first training data point looks like:\n┌──────────┐\n│ Label: 5 │\n┌────────────────────────────────────────────────────────┐\n│                                                        │\n│                                                        │\n│                                                        │\n│                                                        │\n│                                                        │\n│                        ░░░░░░░░▒▒▓▓▓▓░░▓▓████▒▒        │\n│                ░░░░▒▒▓▓▓▓████████████▓▓██████▒▒        │\n│              ░░████████████████████▒▒▒▒▒▒░░░░          │\n│              ░░██████████████▓▓████                    │\n│                ▒▒▓▓▒▒██████░░  ░░▓▓                    │\n│                  ░░░░▓▓██▒▒                            │\n│                      ▓▓██▓▓░░                          │\n│                      ░░▓▓██▒▒                          │\n│                        ░░████▓▓▒▒░░                    │\n│                          ▒▒██████▒▒░░                  │\n│                            ░░▓▓████▓▓░░                │\n│                              ░░▒▒████▓▓                │\n│                                  ██████▒▒              │\n│                            ░░▓▓▓▓██████░░              │\n│                        ░░▓▓██████████▓▓                │\n│                    ░░▒▒████████████▒▒                  │\n│                ░░▒▒████████████▒▒░░                    │\n│            ░░▓▓████████████▒▒░░                        │\n│        ░░▓▓████████████▓▓░░                            │\n│        ▓▓████████▓▓▓▓░░                                │\n│                                                        │\n│                                                        │\n│                                                        │\n└────────────────────────────────────────────────────────┘\ndebug: epoch 0   batch 0             3s -\u003e cost 331.64265899045563, accuracy with 100 test points 0.11\ndebug: epoch 0   batch 5             4s -\u003e cost 242.16033395427667, accuracy with 100 test points 0.56\ndebug: epoch 0   batch 10            5s -\u003e cost 155.62913461977217, accuracy with 100 test points 0.7\ndebug: epoch 0   batch 15            5s -\u003e cost 118.45908401769115, accuracy with 100 test points 0.75\n[...]\n```\n\n### Other examples\n\n#### Simple animal example \n\nThis is a small dataset that I made up to test the neural network. There are only 2\narbitrary features (x and y) where the labeled data points (fish and goat) occupy\ndistinct parts of the graph. Since there are only 2 input features (which means 2\ndimensions), we can easily graph the neural network's decision/classification boundary.\nIt's a good way to visualize the neural network and see how it evolves while training.\n\nThis example produces an image called `simple_xy_animal_graph.ppm` every 1,000 epochs\nshowing the decision/classification boundary.\n\n```sh\n$ zig build run-simple_xy_animal_sample\n```\n\n![](https://github.com/MadLittleMods/zig-ocr-neural-network/assets/558581/128ca52f-0f6f-42ae-8d7e-c557ad943706)\n\n\n#### Barebones XOR example\n\nThere is also a barebones XOR example which just trains a neural network to act like a\nXOR (\"exclusive or\") gate.\n\n```sh\n$ zig build run-xor\n```\n\n![](https://github.com/MadLittleMods/zig-ocr-neural-network/assets/558581/887e7323-41e7-4fda-aa58-5989dc437f97)\n\n\n## Testing\n\n```sh\n$ zig build test --summary all\n```\n\n## Dev notes\n\nSee the [*developer notes*](./dev-notes.md) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadlittlemods%2Fzig-ocr-neural-network","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmadlittlemods%2Fzig-ocr-neural-network","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadlittlemods%2Fzig-ocr-neural-network/lists"}