{"id":21923883,"url":"https://github.com/amir-tav/primitive-nn","last_synced_at":"2026-05-15T20:02:07.540Z","repository":{"id":264500018,"uuid":"893474621","full_name":"Amir-Tav/primitive-NN","owner":"Amir-Tav","description":" How to build a simple neural network from scratch using Numpy and linear algebra without relying on high-level libraries like TensorFlow or Keras.","archived":false,"fork":false,"pushed_at":"2024-11-24T18:03:23.000Z","size":9134,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-20T06:35:59.099Z","etag":null,"topics":["backpropagation","gradientdescent","linear-algebra","machine-learning","mnist-dataset","neural-network"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Amir-Tav.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-24T14:44:49.000Z","updated_at":"2024-11-24T18:11:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"bb3ba54e-406c-4bff-a4fb-0ee6df681d15","html_url":"https://github.com/Amir-Tav/primitive-NN","commit_stats":null,"previous_names":["amir-tav/primitive-nn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Amir-Tav/primitive-NN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amir-Tav%2Fprimitive-NN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amir-Tav%2Fprimitive-NN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amir-Tav%2Fprimitive-NN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amir-Tav%2Fprimitive-NN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Amir-Tav","download_url":"https://codeload.github.com/Amir-Tav/primitive-NN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amir-Tav%2Fprimitive-NN/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33077925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-15T11:35:32.926Z","status":"ssl_error","status_checked_at":"2026-05-15T11:35:31.362Z","response_time":103,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backpropagation","gradientdescent","linear-algebra","machine-learning","mnist-dataset","neural-network"],"created_at":"2024-11-28T21:13:07.313Z","updated_at":"2026-05-15T20:02:07.522Z","avatar_url":"https://github.com/Amir-Tav.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# primitive Neural Network (NN From Scratch)\n\nWelcome to the **Neural Network From Scratch** project! In this project, we built a simple, yet powerful neural network from the ground up, without relying on libraries like TensorFlow or Keras. Instead, we used **Numpy** and **linear algebra** to understand the raw mechanics behind neural networks. Let's dive in! 🤖💡\n\n## Project Overview\nThe goal of this project was to implement a basic neural network with 3 layers:\n1. **Input Layer**: 784 nodes corresponding to the pixels in the 28x28 MNIST images.\n2. **Hidden Layer**: 10 nodes, which helps in learning complex patterns.\n3. **Output Layer**: 10 units, one for each digit (0-9) that the model is classifying.\n\n### Why Build a Neural Network From Scratch?\nBuilding a neural network from scratch is not only fun, but it also gives you a deeper understanding of the algorithms behind machine learning. Instead of relying on pre-built frameworks, we manually implement key components like forward propagation, backward propagation, and activation functions, including **ReLU** and **Softmax**.\n\n### The Dataset\nFor this project, we used the **MNIST** dataset, which consists of **28x28 grayscale images** of handwritten digits. Our task was to build a model that can classify these digits based on the pixel values. This makes it a **classification problem**.\n\n---\n\n## Key Concepts and Code Implementation ⚙️\n\nHere’s a brief overview of the important parts of the code:\n\n### 1. **Data Preprocessing** \nBefore training, the data is shuffled and split into training and development sets. The pixel values are also normalized to a range between 0 and 1.\n\n```python\n\ndata = np.array(data)\nm, n = data.shape\nnp.random.shuffle(data)  # Shuffle before splitting\n\n# Development set (1000 samples)\ndata_dev = data[0:1000].T\nY_dev = data_dev[0]\nX_dev = data_dev[1:n]\nX_dev = X_dev / 255.  # Normalize\n\n# Training set (remaining samples)\ndata_train = data[1000:m].T\nY_train = data_train[0]\nX_train = data_train[1:n]\nX_train = X_train / 255.  # Normalize\n```\n\n\n---\n\n### 2. **Network Initialization**\nWe initialize the weights and biases for the neural network using random values. This is where the magic starts! \n\n```python\n\ndef init_params():\n    W1 = np.random.rand(10, 784) - 0.5  # Weights for layer 1\n    b1 = np.random.rand(10, 1) - 0.5  # Bias for layer 1\n    W2 = np.random.rand(10, 10) - 0.5  # Weights for layer 2\n    b2 = np.random.rand(10, 1) - 0.5  # Bias for layer 2\n    return W1, b1, W2, b2\n\n```\n\n---\n\n### 3. **Forward Propagation**\nWe calculate activations at each layer to determine the output of the network. The ReLU activation function is applied to the hidden layer, and Softmax is used at the output layer to produce probabilities.\n\n```python \n\ndef forward_prop(W1, b1, W2, b2, X):\n    Z1 = W1.dot(X) + b1  # Weighted sum for hidden layer\n    A1 = ReLU(Z1)  # Apply ReLU activation\n    Z2 = W2.dot(A1) + b2  # Weighted sum for output layer\n    A2 = softmax(Z2)  # Apply Softmax to get probabilities\n    return Z1, A1, Z2, A2\n\n```\n\n---\n\n### 4. **Backward Propagation**\nWe compute the gradients for each weight and bias, helping the network adjust during training. This step allows the model to learn from its errors!\n\n```python\n\ndef backward_prop(Z1, A1, Z2, A2, W1, W2, X, Y):\n    one_hot_Y = one_hot(Y)  # One-hot encode the labels\n    dZ2 = A2 - one_hot_Y  # Error at output layer\n    dW2 = 1 / m * dZ2.dot(A1.T)  # Gradients for W2\n    db2 = 1 / m * np.sum(dZ2)  # Gradients for b2\n    dZ1 = W2.T.dot(dZ2) * ReLU_deriv(Z1)  # Error at hidden layer\n    dW1 = 1 / m * dZ1.dot(X.T)  # Gradients for W1\n    db1 = 1 / m * np.sum(dZ1)  # Gradients for b1\n    return dW1, db1, dW2, db2\n\n```\n\n---\n\n### 5. Training the Network**\nWe use gradient descent to minimize the loss and optimize the network's weights and biases over multiple iterations.\n\n```python  \n\ndef gradient_descent(X, Y, alpha, iterations):\n    W1, b1, W2, b2 = init_params()\n    for i in range(iterations):\n        Z1, A1, Z2, A2 = forward_prop(W1, b1, W2, b2, X)\n        dW1, db1, dW2, db2 = backward_prop(Z1, A1, Z2, A2, W1, W2, X, Y)\n        W1, b1, W2, b2 = update_params(W1, b1, W2, b2, dW1, db1, dW2, db2, alpha)\n        if i % 10 == 0:\n            print(\"Iteration:\", i)\n            predictions = get_predictions(A2)\n            print(get_accuracy(predictions, Y))\n    return W1, b1, W2, b2\n\n\n```\n\n---\n\n### 6. Results\nAfter training the model for 500 epochs with a learning rate of 0.1, we were able to achieve an average accuracy of 86%. Not bad for a simple neural network trained from scratch!\n\n* **Testing the Model:**\nWe tested the model by making predictions on a few images. The model successfully predicted 3 out of 4 images correctly, showing that it has learned useful patterns from the data.\n\n```python \n\ndef make_predictions(X, W1, b1, W2, b2):\n    _, _, _, A2 = forward_prop(W1, b1, W2, b2, X)\n    predictions = get_predictions(A2)\n    return predictions\n\ndef test_prediction(index, W1, b1, W2, b2):\n    current_image = X_train[:, index, None]\n    prediction = make_predictions(X_train[:, index, None], W1, b1, W2, b2)\n    label = Y_train[index]\n    print(\"Prediction:\", prediction)\n    print(\"Label:\", label)\n    \n    current_image = current_image.reshape((28, 28)) * 255\n    plt.gray()\n    plt.imshow(current_image, interpolation='nearest')\n    plt.show()\n\n\n```\n\n---\n\n### 7. Conclusion\nThis project has been an exciting journey of understanding how neural networks function at a fundamental level. We were able to create a basic neural network, train it on the MNIST dataset, and achieve an accuracy of 86%.\n\n**Future Work**\n* Fine-tune the model by adjusting the learning rate or adding dynamic learning rates.\n* Add more hidden layers to improve the model’s learning capacity.\n* Train the model for more epochs to achieve better performance.\n\n**Objective**\nThe main objective of this project was to gain a deeper understanding of neural networks, and this knowledge will be incredibly useful in more advanced machine learning tasks.\n\n**Credits** \nA big thank you to **Samson Zhang** for his tutorial that helped me understand how neural networks work from scratch. If you're interested, you can watch his full video [here](https://www.youtube.com/watch?v=w8yWXqWQYmU).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famir-tav%2Fprimitive-nn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famir-tav%2Fprimitive-nn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famir-tav%2Fprimitive-nn/lists"}