https://github.com/jfdev001/mlp-from-scratch

Implementation and extensive report on the multilayer perceptron to elucidate fundamental concepts in deep learning.
https://github.com/jfdev001/mlp-from-scratch

Last synced: 16 days ago
JSON representation

Implementation and extensive report on the multilayer perceptron to elucidate fundamental concepts in deep learning.

• Host: GitHub
• URL: https://github.com/jfdev001/mlp-from-scratch
• Owner: jfdev001
• Created: 2022-01-13T07:23:22.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2022-01-26T16:44:11.000Z (almost 3 years ago)
• Last Synced: 2024-10-28T12:07:50.833Z (2 months ago)
• Language: Python
• Homepage:
• Size: 10.2 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Multilayer Perceptron from Scratch

Here I implement the multilayer perceptron from "scratch" using `numpy`. See ` report.pdf` for details on the neural network and project results. The implementation is derived solely from the readings and sources in the _References_ section of the `report.pdf `. I compare my implementation with a reference `tensorflow` implementation and make use of N, K-fold cross validation with the construction of 99% confidence intervals to validate my models. The `report.pdf` also contains explanations on the foundational concepts of neural networks for personal elucidation.

# Reproducing Results

The `numpy`, `scikit-learn`, `scipy`, `matplotlib`, and `tensorflow` packages are needed for this repository. I recommend using a `conda` environment and running the following commands:

```

conda create --name mlp-from-scratch

conda activate mlp-from-scratch

conda install numpy scipy scikit-learn

conda install -c conda-forge tensorflow matplotlib

```

You can copy and paste the first or second line of `experiments.txt` into the terminal to reproduce my results.

The script `demo.py` was used to run experiments as well as debug the multilayer perceptron. The list of arguments to `demo.py` can be shown by running `python demo.py -h`, though I list the arguments below as well. Note that the `random-classification` task only supports binary classification for now.

```

$ python demo.py -h

usage: demo.py [-h] [--debug {True,False}] [--inspect-model-history {True,False}] [--random-seed RANDOM_SEED] [--verbose {True,False}] [--m-examples M_EXAMPLES]

               [--n-features N_FEATURES] [--t-targets T_TARGETS] [--c-categories C_CATEGORIES] [--num-layers NUM_LAYERS] [--num-hidden-units NUM_HIDDEN_UNITS]

               [--test-size TEST_SIZE] [--batch-size BATCH_SIZE] [--num-epochs NUM_EPOCHS] [--learning-rate LEARNING_RATE] [--confidence-level CONFIDENCE_LEVEL]

               [--n-kfold-iterations N_KFOLD_ITERATIONS] [--bar-chart-path BAR_CHART_PATH] [--bar-plot-title BAR_PLOT_TITLE] [--bar-width BAR_WIDTH]

               {breast-cancer-classification,diabetes-regression,random-regression,random-classification}

script for testing MLP from scratch.

positional arguments:

  {breast-cancer-classification,diabetes-regression,random-regression,random-classification}

                        specify supervised learning task.

optional arguments:

  -h, --help            show this help message and exit

  --debug {True,False}  bool to debug

  --inspect-model-history {True,False}

                        bool to print model history at the end of N, KFold CV

  --random-seed RANDOM_SEED

                        random seed for reproducibility. (default: 0)

  --verbose {True,False}

                        whether to print model fitting output. (default: False)

random-data-params:

  parameters for random dataset tasks

  --m-examples M_EXAMPLES

                        number of training examples. (default: 32)

  --n-features N_FEATURES

                        number of features. (default: 1)

  --t-targets T_TARGETS

                        number of targets for regression. (default: 1)

hyperparameters:

  hyperparametesr for MLP.

  --num-layers NUM_LAYERS

                        number of hidden layers. (default: 1)

  --num-hidden-units NUM_HIDDEN_UNITS

                        number of hidden units in hidden layers. (default: 2)

  --test-size TEST_SIZE

                        percent of data to devote to testing. (default: 0.2)

  --batch-size BATCH_SIZE

                        batch size for training. (default: 4)

  --num-epochs NUM_EPOCHS

                        number of epochs to fit model. (default: 2)

  --learning-rate LEARNING_RATE

                        learning rate for gradient descent. (default: 1e-2)

stats:

  parameters for report statistics.

  --confidence-level CONFIDENCE_LEVEL

                        confidence level for intervals (aka alpha). (default: 0.95)

  --n-kfold-iterations N_KFOLD_ITERATIONS

                        number of times to conduct k-fold cross validation. Also changes random state for shuffling n times. (default: 1)

figures:

  parameters for figures.

  --bar-chart-path BAR_CHART_PATH

                        path to save bar chart. (default: ./tex/figures/bar_chart.svg)

  --bar-plot-title BAR_PLOT_TITLE

                        title for bar plot. (default: N, K-Fold CV Model Performance Comparison)

  --bar-width BAR_WIDTH

                        width of bars in bar charts. (default: 0.25)

```