{"id":15640557,"url":"https://github.com/cosmic-cortex/neural-networks-from-scratch","last_synced_at":"2025-06-14T17:02:48.096Z","repository":{"id":61212359,"uuid":"174331088","full_name":"cosmic-cortex/neural-networks-from-scratch","owner":"cosmic-cortex","description":"An implementation of convolutional networks in NumPy!","archived":false,"fork":false,"pushed_at":"2021-04-19T13:58:33.000Z","size":50,"stargazers_count":81,"open_issues_count":0,"forks_count":16,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-14T17:02:20.144Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cosmic-cortex.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-07T11:24:41.000Z","updated_at":"2025-05-31T16:10:52.000Z","dependencies_parsed_at":"2022-10-12T20:27:24.246Z","dependency_job_id":null,"html_url":"https://github.com/cosmic-cortex/neural-networks-from-scratch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cosmic-cortex/neural-networks-from-scratch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-cortex%2Fneural-networks-from-scratch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-cortex%2Fneural-networks-from-scratch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-cortex%2Fneural-networks-from-scratch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-cortex%2Fneural-networks-from-scratch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cosmic-cortex","download_url":"https://codeload.github.com/cosmic-cortex/neural-networks-from-scratch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cosmic-cortex%2Fneural-networks-from-scratch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259851652,"owners_count":22921625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T11:37:22.071Z","updated_at":"2025-06-14T17:02:48.067Z","avatar_url":"https://github.com/cosmic-cortex.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# neural-networks-from-scratch\n\n# Contents\n- [Quickstart](#quickstart)\n  - [A simple example CNN](#CNN-example)\n  - [The `Net` object](#net)\n- [Layers](#layers)\n  - [`Linear`](#linear)\n  - [`Conv2D`](#conv2d)\n  - [`MaxPool2D`](#maxpool2d)\n  - [`BatchNorm2D`](#batchnorm2d)\n  - [`Flatten`](#flatten)\n- [Losses](#losses)\n  - [`CrossEntropyLoss`](#crossentropyloss)\n  - [`MeanSquareLoss`](#meansquareloss)\n- [Activations](#activations)\n  \n# Quickstart\u003ca name=\"quickstart\"\u003e\u003c/a\u003e\n\n## Installation\u003ca name=\"installation\"\u003e\u003c/a\u003e\nTo run the examples, creating a virtual environment is recommended.\n```bash\nvirtualenv neural-networks-from scratch\n```\nWhen a virtual environment is in place, all requirements can be installed with pip.\n```bash\nsource neural-networks-from-scratch/bin/activate\npip install -r requirements.txt\n```\n\n\n## A simple example CNN\u003ca name=\"CNN-example\"\u003e\u003c/a\u003e\nA simple convolutional network for image classification can be found in `CNN_custom_dataset.py`. To try it on your own dataset, you should prepare your images in the following format:\n```bash\nimages_folder\n   |-- class_01\n       |-- 001.png\n       |-- ...\n   |-- class_02\n       |-- 001.png\n       |-- ...\n   |-- ...\n```\nIts required argument is\n- `--dataset`: path to the dataset,  \n\nwhile the optional arguments are\n- `--epochs`: number of epochs,\n- `--batch_size`: size of the training batch,\n- `--lr`: learning rate.\n\n## The `Net` object\u003ca name=\"net\"\u003e\u003c/a\u003e\nTo define a neural network, the `nn.net.Net` object can be used. Its parameters are\n* `layers`: a list of layers from `nn.layers`, for example `[Linear(2, 4), ReLU(), Linear(4, 2)]`,\n* `loss`: a loss function from `nn.losses`, for example `CrossEntropyLoss` or `MeanSquareLoss`.\nIf you would like to train the model with data `X` and label `y`, you should\n1) perform the forward pass, during which local gradients are calculated,\n2) calculate the loss,\n3) perform the backward pass, where global gradients with respect to the variables and layer parameters are calculated,\n4) update the weights.\n\nIn code, this looks like the following:\n```python3\nout = net(X)\nloss = net.loss(out, y)\nnet.backward()\nnet.update_weights(lr)\n```\n\n# Layers\u003ca name=\"layers\"\u003e\u003c/a\u003e\nThe currently implemented layers can be found in `nn.layers`. Each layer is a callable object, where calling performs the forward pass and calculates local gradients. The most important methods are:\n- `.forward(X)`: performs the forward pass for X. Instead calling `forward` directly, the layer object should be called directly, which calculates and caches local gradients.\n- `.backward(dY)`: performs the backward pass, where `dY` is the gradient propagated backwards from the consequtive layer.\n- `.local_grad(X)`: calculates the local gradient of the input.\n\nThe input to the layers should always be a `numpy.ndarray` of shape `(n_batch, ...)`. For the 2D layers for images, the input should have shape `(n_batch, n_channels, n_height, n_width)`.\n\n## `Linear`\u003ca name=\"linear\"\u003e\u003c/a\u003e\nA simple fully connected layer. \nParameters:\n- `in_dim`: integer, dimensions of the input.\n- `out_dim`: integer, dimensions of the output.\n\nUsage:\n- input: `numpy.ndarray` of shape `(N, in_dim)`.\n- output: `numpy.ndarray` of shape `(N, out_dim)`.\n\n## `Conv2D`\u003ca name=\"conv2d\"\u003e\u003c/a\u003e\n2D convolutional layer. Parameters:\n- `in_channels`: integer, number of channels in the input image.\n- `out_channels`: integer, number of filters to be learned.\n- `kernel_size`: integer or tuple, the size of the filter to be learned. Defaults to 3.\n- `stride`: integer, stride of the convolution. Defaults to 1.\n- `padding`: integer, number of zeros to be added to each edge of the images. Defaults to 0.\n\nUsage:\n- input: `numpy.ndarray` of shape `(N, C_in, H_in, W_in)`.\n- output: `numpy.ndarray` of shape `(N, C_out, H_out, W_out)`.\n\n## `MaxPool2D`\u003ca name=\"maxpool2d\"\u003e\u003c/a\u003e\n2D max pooling layer. Parameters:\n- `kernel_size`: integer or tuple, size of the pooling window. Defaults to 2.\n\nUsage:\n- input: `numpy.ndarray` of shape `(N, C, H, W)`.\n- output: `numpy.ndarray` of shape `(N, C, H//KH, W//KW)` with kernel size `(KH, KW)`.\n\n## `BatchNorm2D`\u003ca name=\"batchnorm2d\"\u003e\u003c/a\u003e\n2D batch normalization layer. Parameters:\n- `n_channels`: integer, number of channels.\n- `epsilon`: epsilon parameter for BatchNorm, defaults to 1e-5.\n\nUsage:\n- input: `numpy.ndarray` of shape `(N, C, H, W)`.\n- output: `numpy.ndarray` of shape `(N, C, H, W)`.\n\n## `Flatten`\u003ca name=\"flatten\"\u003e\u003c/a\u003e\nA simple layer which flattens the outputs of a 2D layer for images.\n\nUsage:\n- input: `numpy.ndarray` of shape `(N, C, H, W)`.\n- output: `numpy.ndarray` of shape `(N, C*H*W)`.\n\n# Losses\u003ca name=\"losses\"\u003e\u003c/a\u003e\nThe implemented loss functions are located in `nn.losses`. As Layers, they are callable objects, with predictions and targets as input.\n\n## `CrossEntropyLoss`\u003ca name=\"crossentropyloss\"\u003e\u003c/a\u003e\nCross-entropy loss. Usage:\n- input: `numpy.ndarray` of shape `(N, D)` containing the class scores for each element in the batch.\n- output: `float`.\n\n## `MeanSquareLoss`\u003ca name=\"meansquareloss\"\u003e\u003c/a\u003e\nMean square loss. Usage:\n- input: `numpy.ndarray` of shape `(N, D)`.\n- output: `numpy.ndarray` of shape `(N, D)`.\n\n# Activations\u003ca name=\"activations\"\u003e\u003c/a\u003e\nThe activation layers for the network can be found in `nn.activations`. They are functions, applying the specified activation function elementwisely on a `numpy.ndarray`. Currently, the following activation functions are implemented:\n- ReLU\n- Leaky ReLU\n- Sigmoid\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmic-cortex%2Fneural-networks-from-scratch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcosmic-cortex%2Fneural-networks-from-scratch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcosmic-cortex%2Fneural-networks-from-scratch/lists"}