{"id":30855933,"url":"https://github.com/sermetpekin/microgradcpp","last_synced_at":"2025-09-07T11:11:59.112Z","repository":{"id":267083737,"uuid":"899651790","full_name":"SermetPekin/microgradCpp","owner":"SermetPekin","description":"MicrogradCPP: A tiny micrograd, inspired by Karpathy's original, but this time in C++. ","archived":false,"fork":false,"pushed_at":"2024-12-17T06:53:40.000Z","size":1728,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-01T08:31:23.575Z","etag":null,"topics":["autograd","machine-learning","micrograd","neural-network","neural-networks"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SermetPekin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-06T18:00:54.000Z","updated_at":"2025-07-31T19:38:20.000Z","dependencies_parsed_at":"2024-12-08T08:34:30.980Z","dependency_job_id":null,"html_url":"https://github.com/SermetPekin/microgradCpp","commit_stats":null,"previous_names":["sermetpekin/microgradcpp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SermetPekin/microgradCpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SermetPekin%2FmicrogradCpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SermetPekin%2FmicrogradCpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SermetPekin%2FmicrogradCpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SermetPekin%2FmicrogradCpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SermetPekin","download_url":"https://codeload.github.com/SermetPekin/microgradCpp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SermetPekin%2FmicrogradCpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274026718,"owners_count":25209740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","machine-learning","micrograd","neural-network","neural-networks"],"created_at":"2025-09-07T11:11:19.559Z","updated_at":"2025-09-07T11:11:59.095Z","avatar_url":"https://github.com/SermetPekin.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n![C++](https://img.shields.io/badge/C%2B%2B-17-blue.svg) ![License](https://img.shields.io/github/license/SermetPekin/microgradCpp)  ![MicrogradCpp](https://img.shields.io/badge/MicrogradCpp-C++-green)\n[![Run Google Tests](https://github.com/SermetPekin/microgradCpp/actions/workflows/package.yaml/badge.svg)](https://github.com/SermetPekin/microgradCpp/actions/workflows/package.yaml)\n\n# MicrogradCpp\n\n**MicrogradCPP** is a C++ library inspired by [Andrej Karpathy’s micrograd](https://github.com/karpathy/micrograd) that provides a small, educational automatic differentiation engine and a simple neural network library on top. \n\nIt’s geared toward understanding backpropagation, building small neural networks, and experimenting with gradient-based learning—entirely in C++.\n\nThis project aims to be easy to read and modify, making it a great learning tool for those who want to understand how automatic differentiation, feed-forward neural networks, and basic optimizers work at a low level.\n\n## Features\n\n- **Header-only**: Just include the headers, no extra linking needed.\n- **Autograd Engine**: A `Value` class that tracks data, gradients, and the computational graph, enabling `loss-\u003ebackward()` calls.\n- **Neural Network Components**:\n  - `Linear` layers for building multi-layer perceptrons (MLPs).\n  - `MLP` class that stacks `Linear` layers and applies non-linearities (like ReLU).\n- **Common Operations**: Basic arithmetic (+, -, *, /, unary negation), `log`, `exp`, `pow`, and more, all differentiable.\n- **Loss Functions**: A `Loss::cross_entropy` for classification tasks that integrates smoothly with `softmax` outputs.\n- **Optimizers**: A basic `SGD` optimizer to perform parameter updates.\n- **Simple Dataset Integration**: An `IrisLoader` utility to load the Iris dataset as an example.\n\n## Getting Started\n\n### Prerequisites\n- A C++17-compatible compiler (e.g. `g++` \u003e= 7.3, `clang` \u003e= 6.0).\n- CMake (optional) if you use the CMake build approach.\n- The Iris dataset CSV (`iris.csv`) located in `./data/` directory.\n\n\n\n```C++ \n    // main.cpp \n\n#include \"micrograd.hpp\"\nusing namespace microgradCpp;\n\nint main()\n{\n\n    DatasetType dataset = get_iris();\n    shuffle(dataset);\n    double TRAIN_SIZE{0.8};\n\n    // Define the model and hyperparameters\n\n    // Create Multi-Layer Perceptron (MLP)\n    // Input: 4 features, hidden layers: [10,10], output: 3 classes\n  \n    MLP model(4, {10, 10, 3});\n    double learning_rate = 0.01;\n    int epochs = 100;\n\n    // Train and evaluate the model\n    train_eval(dataset, TRAIN_SIZE, model, learning_rate, epochs);\n\n    return 0;\n}\n\n```\n\n### Building\n\n#### Without CMake (Direct g++ Command)\n\nSince it’s header-only, you just compile `main.cpp`:\n```bash\ng++ -std=c++17 -Iinclude -O2 -o main main.cpp\n```\nRun the executable:\n```bash\n./main\n```\nIf you placed iris.csv in data/, and main.cpp references ./data/iris.csv, ensure you run ./main from the microgradcpp directory.\n\n\n### 2. Building with Makefile\n\nThe project includes a `Makefile` to easily compile and run the code. To build and run the project, follow these steps:\n\n1. **Using `Makefile`:**\n\n   In your project directory, run:\n```bash\n   make run\n```\nThis will compile your code and automatically run the ./main executable.\n### 2. Building with CMake\n\n```bash\nmkdir build\ncd build\ncmake ..\nmake\n```\n## Usage\n\n### Defining a Model\nTo set up a model, just create an instance of the `MLP` class:\n```cpp\nMLP model(in_features, {hidden1, hidden2, ..., out_features});\n\n````\nThis defines a network with a given number of input features and any arrangement of hidden layers, ending with your desired output layer size.\nTraining\n\n    Set up an optimizer like SGD.\n    Load your dataset (e.g., Iris) and split it into training and test sets.\n    Run the model’s forward pass to get predictions, compute the loss, and then call loss-\u003ebackward() to perform backpropagation.\n    Finally, call optimizer.step() to update the model’s parameters.\n\nCustom Operations\n\nThe Value class supports a variety of arithmetic operations (+, -, *, /) and differentiable functions like log, exp, and pow. You can combine these to define custom computations that the model can learn from.\nExample with main.cpp\n\nThe provided main.cpp script walks you through the entire process. It:\n\n    Loads and preprocesses the Iris dataset.\n    Shuffles and splits the data into an 80-20 train-test ratio.\n    Defines a simple MLP and trains it with SGD.\n    Prints the training loss at each epoch and the test accuracy at regular intervals.\n\nYou can experiment directly by tweaking parameters in main.cpp—for example, changing the network architecture, the learning rate, or the number of epochs. Compile and run with:\n\n## Model Performance and Next Steps\n\nI’ve tested the model on the **Iris dataset**[^1] using an **80-20 train-test split**. The model achieved an average **accuracy of 93%** on the test set.\n\n\n[^1]: Fisher, R. (1936). Iris [Dataset]. UCI Machine Learning Repository. https://doi.org/10.24432/C56C76\n\n\n \n ## Ways to Improve the Model\n\nIf you want to push beyond the current 93% benchmark on the Iris dataset, consider these tweaks:\n\n- **Hyperparameters**: Adjust the learning rate, add more layers or neurons, or experiment with different numbers of epochs to see if you can squeeze out better performance.\n- **Regularization**: If the model’s overfitting, try adding dropout or using weight decay (L2 regularization) to help it generalize.\n- **Optimizers**: Sticking to SGD is fine, but you could also try Adam for potentially smoother, faster training.\n- **Cross-Validation and Scaling**: K-fold cross-validation can give you a more reliable performance estimate, and feature scaling may help if you’re applying this model to other, more complex datasets.\n\n### What’s Next?\n\nRight now, the model hits about 93% accuracy on Iris, which is a decent starting point. But this setup is flexible—feel free to run it on different datasets, tweak the architecture, or incorporate more advanced techniques as you get comfortable. Each adjustment helps you understand neural networks a bit more and how to tune them for better results.\n\nAnd credit where it’s due: Thanks to Andrej Karpathy for the original micrograd concept and the helpful video that inspired this C++ version.\n\n\n\n\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eexample output\u003c/summary\u003e\n\n  **Hidden content:**  \n\n\nexample output\n```plaintext\nLoaded 150 samples from ./data/iris.csv\nEpoch 1/100, Loss: 1.10912\nEpoch 1: Test Accuracy = 60%\nEpoch 2/100, Loss: 0.657723\nEpoch 3/100, Loss: 0.590328\nEpoch 4/100, Loss: 0.56049\nEpoch 5/100, Loss: 0.507944\nEpoch 6/100, Loss: 0.517438\nEpoch 7/100, Loss: 0.453947\nEpoch 8/100, Loss: 0.457171\nEpoch 9/100, Loss: 0.43995\nEpoch 10/100, Loss: 0.400679\nEpoch 11/100, Loss: 0.419943\nEpoch 11: Test Accuracy = 63.3333%\nEpoch 12/100, Loss: 0.387175\nEpoch 13/100, Loss: 0.46077\nEpoch 14/100, Loss: 0.353314\nEpoch 15/100, Loss: 0.372516\nEpoch 16/100, Loss: 0.416995\nEpoch 17/100, Loss: 0.372918\nEpoch 18/100, Loss: 0.389412\nEpoch 19/100, Loss: 0.339006\nEpoch 20/100, Loss: 0.376714\nEpoch 21/100, Loss: 0.360416\nEpoch 21: Test Accuracy = 80%\nEpoch 22/100, Loss: 0.333371\nEpoch 23/100, Loss: 0.282985\nEpoch 24/100, Loss: 0.323401\nEpoch 25/100, Loss: 0.333401\nEpoch 26/100, Loss: 0.317364\nEpoch 27/100, Loss: 0.252736\nEpoch 28/100, Loss: 0.278259\nEpoch 29/100, Loss: 0.335743\nEpoch 30/100, Loss: 0.256701\nEpoch 31/100, Loss: 0.385125\nEpoch 31: Test Accuracy = 86.6667%\nEpoch 32/100, Loss: 0.272716\nEpoch 33/100, Loss: 0.320718\nEpoch 34/100, Loss: 0.23242\nEpoch 35/100, Loss: 0.235965\nEpoch 36/100, Loss: 0.336758\nEpoch 37/100, Loss: 0.263255\nEpoch 38/100, Loss: 0.260777\nEpoch 39/100, Loss: 0.216942\nEpoch 40/100, Loss: 0.295679\nEpoch 41/100, Loss: 0.263334\nEpoch 41: Test Accuracy = 83.3333%\nEpoch 42/100, Loss: 0.206119\nEpoch 43/100, Loss: 0.239363\nEpoch 44/100, Loss: 0.217142\nEpoch 45/100, Loss: 0.219216\nEpoch 46/100, Loss: 0.231273\nEpoch 47/100, Loss: 0.236615\nEpoch 48/100, Loss: 0.174601\nEpoch 49/100, Loss: 0.1995\nEpoch 50/100, Loss: 0.248405\nEpoch 51/100, Loss: 0.182827\nEpoch 51: Test Accuracy = 83.3333%\nEpoch 52/100, Loss: 0.255753\nEpoch 53/100, Loss: 0.269484\nEpoch 54/100, Loss: 0.197941\nEpoch 55/100, Loss: 0.231213\nEpoch 56/100, Loss: 0.268628\nEpoch 57/100, Loss: 0.184304\nEpoch 58/100, Loss: 0.257628\nEpoch 59/100, Loss: 0.205261\nEpoch 60/100, Loss: 0.202221\nEpoch 61/100, Loss: 0.218491\nEpoch 61: Test Accuracy = 93.3333%\nEpoch 62/100, Loss: 0.272666\nEpoch 63/100, Loss: 0.206216\nEpoch 64/100, Loss: 0.242779\nEpoch 65/100, Loss: 0.199603\nEpoch 66/100, Loss: 0.159741\nEpoch 67/100, Loss: 0.165282\nEpoch 68/100, Loss: 0.184907\nEpoch 69/100, Loss: 0.144188\nEpoch 70/100, Loss: 0.249892\nEpoch 71/100, Loss: 0.158859\nEpoch 71: Test Accuracy = 93.3333%\nEpoch 72/100, Loss: 0.179014\nEpoch 73/100, Loss: 0.233659\nEpoch 74/100, Loss: 0.174708\nEpoch 75/100, Loss: 0.2855\nEpoch 76/100, Loss: 0.198777\nEpoch 77/100, Loss: 0.176887\nEpoch 78/100, Loss: 0.171887\nEpoch 79/100, Loss: 0.203625\nEpoch 80/100, Loss: 0.228634\nEpoch 81/100, Loss: 0.164969\nEpoch 81: Test Accuracy = 93.3333%\nEpoch 82/100, Loss: 0.213662\nEpoch 83/100, Loss: 0.167459\nEpoch 84/100, Loss: 0.198857\nEpoch 85/100, Loss: 0.227029\nEpoch 86/100, Loss: 0.229519\nEpoch 87/100, Loss: 0.270876\nEpoch 88/100, Loss: 0.178311\nEpoch 89/100, Loss: 0.165478\nEpoch 90/100, Loss: 0.187928\nEpoch 91/100, Loss: 0.146264\nEpoch 91: Test Accuracy = 93.3333%\nEpoch 92/100, Loss: 0.177392\nEpoch 93/100, Loss: 0.228931\nEpoch 94/100, Loss: 0.19446\nEpoch 95/100, Loss: 0.174602\nEpoch 96/100, Loss: 0.143752\nEpoch 97/100, Loss: 0.148301\nEpoch 98/100, Loss: 0.231661\nEpoch 99/100, Loss: 0.179442\nEpoch 100/100, Loss: 0.139438\nEpoch 100: Test Accuracy = 93.3333%\n\n```\n\u003c/details\u003e\n\n\n\n\n\n\n### Neural Network Architecture\n\nHere is a visual representation of the MLP architecture:\n```mermaid\ngraph TD\n    %% Styling for Neurons %%\n    classDef inputNeuron fill:#e6f7ff,stroke:#1d4e89,stroke-width:2px\n    classDef hiddenNeuron1 fill:#d9f7be,stroke:#389e0d,stroke-width:2px\n    classDef hiddenNeuron2 fill:#f6e6ff,stroke:#722ed1,stroke-width:2px\n    classDef outputNeuron fill:#fff1f0,stroke:#cf1322,stroke-width:2px\n\n    %% Input Layer %%\n    subgraph InputLayer [\"🎯 Input Layer\"]\n        InputNeuron1[\"Neuron 1\"]:::inputNeuron\n        InputNeuron2[\"Neuron 2\"]:::inputNeuron\n        InputNeuron3[\"Neuron 3\"]:::inputNeuron\n    end\n\n    %% Hidden Layer 1 %%\n    subgraph HiddenLayer1 [\"⚙️ Hidden Layer 1\"]\n        HiddenNeuron1_1[\"Neuron 1\"]:::hiddenNeuron1\n        HiddenNeuron1_2[\"Neuron 2\"]:::hiddenNeuron1\n        HiddenNeuron1_3[\"Neuron 3\"]:::hiddenNeuron1\n        HiddenNeuron1_4[\"Neuron 4\"]:::hiddenNeuron1\n    end\n\n    %% Hidden Layer 2 %%\n    subgraph HiddenLayer2 [\"⚙️ Hidden Layer 2\"]\n        HiddenNeuron2_1[\"Neuron 1\"]:::hiddenNeuron2\n        HiddenNeuron2_2[\"Neuron 2\"]:::hiddenNeuron2\n    end\n\n    %% Output Layer %%\n    subgraph OutputLayer [\"🔍 Output Layer\"]\n        OutputNeuron1[\"Neuron 1\"]:::outputNeuron\n        OutputNeuron2[\"Neuron 2\"]:::outputNeuron\n    end\n\n    %% Connections Between Layers %%\n    %% Input Layer to Hidden Layer 1 %%\n    InputNeuron1 --\u003e HiddenNeuron1_1\n    InputNeuron1 --\u003e HiddenNeuron1_2\n    InputNeuron1 --\u003e HiddenNeuron1_3\n    InputNeuron1 --\u003e HiddenNeuron1_4\n\n    InputNeuron2 --\u003e HiddenNeuron1_1\n    InputNeuron2 --\u003e HiddenNeuron1_2\n    InputNeuron2 --\u003e HiddenNeuron1_3\n    InputNeuron2 --\u003e HiddenNeuron1_4\n\n    InputNeuron3 --\u003e HiddenNeuron1_1\n    InputNeuron3 --\u003e HiddenNeuron1_2\n    InputNeuron3 --\u003e HiddenNeuron1_3\n    InputNeuron3 --\u003e HiddenNeuron1_4\n\n    %% Hidden Layer 1 to Hidden Layer 2 %%\n    HiddenNeuron1_1 --\u003e HiddenNeuron2_1\n    HiddenNeuron1_1 --\u003e HiddenNeuron2_2\n\n    HiddenNeuron1_2 --\u003e HiddenNeuron2_1\n    HiddenNeuron1_2 --\u003e HiddenNeuron2_2\n\n    HiddenNeuron1_3 --\u003e HiddenNeuron2_1\n    HiddenNeuron1_3 --\u003e HiddenNeuron2_2\n\n    HiddenNeuron1_4 --\u003e HiddenNeuron2_1\n    HiddenNeuron1_4 --\u003e HiddenNeuron2_2\n\n    %% Hidden Layer 2 to Output Layer %%\n    HiddenNeuron2_1 --\u003e OutputNeuron1\n    HiddenNeuron2_1 --\u003e OutputNeuron2\n\n    HiddenNeuron2_2 --\u003e OutputNeuron1\n    HiddenNeuron2_2 --\u003e OutputNeuron2\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsermetpekin%2Fmicrogradcpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsermetpekin%2Fmicrogradcpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsermetpekin%2Fmicrogradcpp/lists"}