{"id":25023070,"url":"https://github.com/samuel-adekunle/autoencoders","last_synced_at":"2026-05-17T17:05:08.312Z","repository":{"id":101024392,"uuid":"261487234","full_name":"samuel-adekunle/Autoencoders","owner":"samuel-adekunle","description":"Autoencoders Tutorial","archived":false,"fork":false,"pushed_at":"2020-05-12T11:44:46.000Z","size":5159,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-05T14:40:30.255Z","etag":null,"topics":["autoencoders","python","tutorial"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samuel-adekunle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-05T13:58:32.000Z","updated_at":"2024-11-02T12:25:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"5780ad9a-beb6-42d2-a6c2-5b8185ae1f0c","html_url":"https://github.com/samuel-adekunle/Autoencoders","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuel-adekunle%2FAutoencoders","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuel-adekunle%2FAutoencoders/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuel-adekunle%2FAutoencoders/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuel-adekunle%2FAutoencoders/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samuel-adekunle","download_url":"https://codeload.github.com/samuel-adekunle/Autoencoders/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246320132,"owners_count":20758407,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoencoders","python","tutorial"],"created_at":"2025-02-05T14:36:04.555Z","updated_at":"2025-10-03T18:40:03.753Z","avatar_url":"https://github.com/samuel-adekunle.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"Written by [Samuel Adekunle](mailto:sja119@ic.ac.uk)\n\nFor [AI Core](http://www.theaicore.com)\n\n# Introduction to Autoencoders\n\n## Uses of Autoencoders\n\n### Image/Audio Denoising\n\nAutoencoders are very good at removing noise from images and generating a much clearer picture than the original. Later we will see how this can easily be implemented.\n\n![image](img/denoising_example.png)\n\n### Image Generation\n\nAn alternative to GANs are a variant of autoencoders known as [Variational Autoencoders](https://en.wikipedia.org/wiki/Autoencoder#Variational_autoencoder_(VAE)). There's a lot of complicated math involved but in summarhy, te input is an image, and the variational autoencoder learns it's distribution and can generate similar images.\n\n![faces generated with a vae](img/faces.png)\n\n*Faces generated with a Variational Autoencoder Model (source: [Wojciech Mormul on Github](https://github.com/WojciechMormul/vae))*\n\n### Image Inpainting and Photo Restoration\n\n![context encoders](img/inpainting.jpg)\n\n*Faces generated with a Variational Autoencoder Model (source: [Context Encoders: Feature Learning by Inpainting](https://people.eecs.berkeley.edu/~pathak/context_encoder/))*\n\n### Other Uses:\n - Anomaly Detection and Facial Recogniton\n - Feature Extraction and Data Compression\n - Language Translation\n\n\n## Autoencoder Basic Architecture\n\nAn [Autoencoder](https://en.wikipedia.org/wiki/Autoencoder) is a neural network architecture that learns efficient data encodings in an unsupervised manner. What this means is autoencoders learn to recognise the most important features of the data they are fed, and reject the less important ones (i.e. noise). In doing so, they can reduce the dimensionality of the number of features needed to represent the same data. It does this in two steps:\n\n - Data Encoding: The input data is forced through a bottleneck and transfomed into a feature space, which is typically much smaller than the input space. The encoder is trained so that this feature space represents the most important features in the input space that are needed to reconstruct the data. Note: If the feature space is not smaller than the input space, then the encoder might just learn the identity function.\n \n - Data Decoding: After the input data has been reduced to some feature space, the autoencoder tries to reconstruct the original data from the reduced feature space. This is why an autoencoder is often said to undergo **unsupervised training**. The original input data is what is compared against the output of the network and used to train it. Typically in training the autoencoder, the network tries to minimize a reconstruction loss, such as the Mean Squared Error between the input and the output.\n\n![image](img/transitions.png)\n\n*Mathematical Definition of an Autoencoder (source: [Wikipedia](https://en.wikipedia.org/wiki/Autoencoder))*\n\n# Feed-Forward Autoencoder\n\nThis basic architechture will take the input and try to reproduce it at the output.\n\n![feed_foward_autoencoder](img/encoder_decoder.png)\n\n*Basic Reconstruction Autoencoder Architecture (source: [Jeremy Jordan](https://www.jeremyjordan.me/autoencoders/))*\n\n\n```python\n# All requirements for this notebook\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torchvision\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n\nSEED = 5000\ntorch.manual_seed(SEED)\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n```\n\n\n```python\n# We will be using popular MNIST dataset\ntrain_data = torchvision.datasets.MNIST(root='MNIST-data',\n                                        transform=torchvision.transforms.ToTensor(),\n                                        train=True,\n                                        download=True\n                                        )\ntest_data = torchvision.datasets.MNIST(root='MNIST-data',\n                                       transform=torchvision.transforms.ToTensor(),\n                                       train=False\n                                       )\n```\n\n\n```python\nprint(f\"Shape of MNIST Training Dataset: {train_data.data.shape}\")\nprint(f\"Shape of MNIST Testing Dataset: {test_data.data.shape}\")\n```\n\n    Shape of MNIST Training Dataset: torch.Size([60000, 28, 28])\n    Shape of MNIST Testing Dataset: torch.Size([10000, 28, 28])\n\n\n\n```python\ndef show_image_helper(image):\n    image = image.view(28, 28)\n    plt.imshow(image.cpu().detach())\n    plt.show()\n    print(\"Max Element: \", rdm_img.max())\n    print(\"Min Element: \", rdm_img.min())\n    \ndef show_losses_helper(losses):\n    plt.plot(losses[1:])\n    plt.ylabel(\"Losses\")\n    plt.xlabel(\"Epochs\")\n    plt.title(\"Autoencoder Losses\")\n    plt.show()\n```\n\n\n```python\n# What are we working with and what will we be doing\nrdm_img = train_data.data[np.random.randint(\n    0, 100)] / 255.0  # get a random example\nshow_image_helper(rdm_img)\n```\n\n\n![png](img/output/output_9_0.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n\n\n\n```python\n# FURTHER SPLIT THE TRAINING INTO TRAINING AND VALIDATION\ntrain_data, val_data = torch.utils.data.random_split(train_data, [\n                                                     50000, 10000])\n\nBATCH_SIZE = 128\n\n# MAKE TRAINING DATALOADER\ntrain_loader = torch.utils.data.DataLoader(  # create a data loader\n    train_data,  # what dataset should it sample from?\n    shuffle=True,  # should it shuffle the examples?\n    batch_size=BATCH_SIZE  # how large should the batches that it samples be?\n)\n\n# MAKE VALIDATION DATALOADER\nval_loader = torch.utils.data.DataLoader(\n    val_data,\n    shuffle=True,\n    batch_size=BATCH_SIZE\n)\n\n# MAKE TEST DATALOADER\ntest_loader = torch.utils.data.DataLoader(\n    test_data,\n    shuffle=True,\n    batch_size=BATCH_SIZE\n)\n```\n\n\n```python\nclass AutoEncoder(nn.Module):\n    def __init__(self, input_size, hidden_size, code_size):\n        super().__init__()\n\n        self.encoder = nn.Sequential(\n            nn.Linear(input_size, hidden_size),\n            nn.ReLU(),\n            nn.Linear(hidden_size, code_size),\n            nn.ReLU()\n        )\n\n        self.decoder = nn.Sequential(\n            nn.Linear(code_size, hidden_size),\n            nn.ReLU(),\n            nn.Linear(hidden_size, input_size),\n            nn.Sigmoid()\n        )\n\n    def forward(self, x):\n        return self.decoder(self.encoder(x))\n```\n\n\n```python\ndef train(model, num_epochs=10, learning_rate=0.01):\n    global EPOCHS\n    model.train()\n    losses = []\n    optimiser = torch.optim.Adam(model.parameters(), lr=learning_rate)\n    criterion = nn.BCELoss()\n#     criterion = nn.MSELoss()\n\n    for epoch in range(num_epochs):\n        EPOCHS += 1\n        total_loss = 0\n        num_batches = 0\n        for org_img, _ in train_loader:\n            optimiser.zero_grad()\n\n            org_img = org_img.double().view(-1, 784).to(device) / 255.0\n            gen_img = model(org_img).double()\n\n            loss = criterion(gen_img, org_img)\n            total_loss += loss\n            num_batches += 1\n\n            loss.backward()  # backpropagate\n            optimiser.step()\n        \n        average_loss = total_loss / num_batches\n        losses.append(average_loss)\n        print(f\"Epoch {EPOCHS}:\\tScore: {1/average_loss}\")\n    \n    return losses\n```\n\n\n```python\nEPOCHS = 0\nINPUT_SIZE = 28*28\nHIDDEN_SIZE = 128\nCODE_SIZE = 32\nLEARNING_RATE = 0.01\n\nautoencoder = AutoEncoder(\n    INPUT_SIZE, HIDDEN_SIZE, CODE_SIZE).double().to(device)\n```\n\n\n```python\nnum_epochs = 25\nlosses = train(autoencoder, num_epochs, LEARNING_RATE)\nshow_losses_helper(losses)\n```\n\n    Epoch 1:\tScore: 65.9746830403687\n    Epoch 2:\tScore: 253.71439743573538\n    Epoch 3:\tScore: 254.1112540841705\n    Epoch 4:\tScore: 254.6927016062727\n    Epoch 5:\tScore: 255.29870898289508\n    Epoch 6:\tScore: 256.4334503161277\n    Epoch 7:\tScore: 257.90301234822346\n    Epoch 8:\tScore: 259.5075457397731\n    Epoch 9:\tScore: 260.4631195092867\n    Epoch 10:\tScore: 261.1034743810059\n    Epoch 11:\tScore: 261.6928707329267\n    Epoch 12:\tScore: 262.2515987754876\n    Epoch 13:\tScore: 262.90792133414806\n    Epoch 14:\tScore: 264.5439896168884\n    Epoch 15:\tScore: 266.52383513231877\n    Epoch 16:\tScore: 267.958849893919\n    Epoch 17:\tScore: 269.1381944234346\n    Epoch 18:\tScore: 270.09315124139346\n    Epoch 19:\tScore: 270.8515580173019\n    Epoch 20:\tScore: 271.4857478976898\n    Epoch 21:\tScore: 272.0478296076016\n    Epoch 22:\tScore: 272.7034256223991\n    Epoch 23:\tScore: 273.1984607430912\n    Epoch 24:\tScore: 273.837528485409\n    Epoch 25:\tScore: 274.55321836051\n\n\n\n![png](img/output/output_14_1.png)\n\n\n\n```python\ndef validate(model):\n    model.eval()\n    criterion = torch.nn.BCELoss()\n#     criterion = torch.nn.MSELoss()\n    total_loss = 0\n    num_batches = 0\n    for val_img, _ in val_loader:\n        val_img = val_img.double().view(-1, 784).to(device) / 255.0\n        gen_img = model(val_img).double()\n        loss = criterion(gen_img, val_img)\n        total_loss += loss\n        num_batches += 1\n    average_loss = total_loss / num_batches\n    return 1/average_loss.item()\n```\n\n\n```python\nscore = validate(autoencoder)\nprint(\"Score: \", score)\n```\n\n    Score:  273.6599220477289\n\n\n\n```python\ndef test(model):\n    model.eval()\n    criterion = torch.nn.BCELoss()\n#   criterion = torch.nn.MSELoss()\n    total_loss = 0\n    num_batches = 0\n    stored_images = []\n    for test_img, _ in test_loader:\n        test_img = test_img.double().view(-1, 784).to(device) / 255.0\n        gen_img = model(test_img)\n        loss = criterion(gen_img.double(), test_img).item()\n        total_loss += loss\n        num_batches += 1\n        if np.random.random() \u003e 0.90:\n            stored_images.append(\n                (test_img[0].clone().detach(), gen_img[0].clone().detach()))\n\n    score = average_loss = total_loss / num_batches\n    print(f\"Score: {1/score}\\n\")\n\n    for original, generated in stored_images:\n        print(\"Original: \")\n        show_image_helper(original)\n        print(\"Generated: \")\n        show_image_helper(generated)\n```\n\n\n```python\ntest(autoencoder)\n```\n\n    Score: 271.5048760406239\n    \n    Original: \n\n\n\n![png](img/output/output_18_1.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_3.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_5.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_7.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_9.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_11.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_13.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_15.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_17.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_19.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_21.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_23.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_25.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_27.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_29.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_31.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_33.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_35.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_37.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_39.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_41.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_43.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_45.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_47.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_49.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_51.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_53.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_55.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_57.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_59.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_61.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_63.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_65.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_67.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_18_69.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_18_71.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n\n\n## Comparing MSE to BCE\n\nGenerally, when dealing with Autoencoders or similar problems, we train using a loss like MSE which would compare the generated image and the original one, pixel by pixel in order to calculate the error. \n\nThis is fine most of the time, but would not have been optimal in our case. Our images have values varying only between 0 and 1 and most of them are zero anyways, so this means the mean square error will always be very low, which will not allow our model to train effectively.\n\n![mean_square_error_loss](img/mse_losses.png)\n\nThe alternative we used was the Binary Cross Entropy Error. Typically this is used for categorical problems, but in our case we are trying to distinguish between a high (1.0) and a low(0.0) so the cross entropy loss can still be used. Because our numbers are between 0 and 1 we use a binary cross entropy.\n\n![binary_cross_entropy_loss](img/bce.png)\n\n# Application - Denoising an Image\n\nThis adds some noise to the input before passing it in to the autoencoder network but uses the original image as the ground truth, effectively training the autoencoder network to reject the noise and learn the data encodings that represent the data beneath the noise. The only difference is in the training loop\n\n![denoising_autoencoder_architecture](img/denoising.png)\n\n*Denoising Autoencoder Architecture (source: [Jeremy Jordan](https://www.jeremyjordan.me/autoencoders/))*\n\n\n\n```python\ndef add_noise(clean_image, noise_factor=0.0):\n    random_noise = torch.randn_like(clean_image)\n    random_noise /= random_noise.max() # between -1 and 1\n    noisy_image = clean_image + (noise_factor * random_noise)\n    return noisy_image\n```\n\n\n```python\ndef train_noise(model, num_epochs=10, learning_rate=0.01, noise_factor=0.0):\n    global EPOCHS\n    model.train()\n    losses = []\n    optimiser = torch.optim.Adam(model.parameters(), lr=learning_rate)\n    criterion = nn.BCELoss()\n#     criterion = nn.MSELoss()\n\n    for _ in range(num_epochs):\n        EPOCHS += 1\n        total_loss = 0\n        num_batches = 0\n        for org_img, _ in train_loader:\n            optimiser.zero_grad()\n            org_img = org_img.double().view(-1, 784).to(device) / 255.0\n            noisy_img = add_noise(org_img, noise_factor)\n            gen_img = model(noisy_img).double()\n\n            loss = criterion(gen_img, org_img)\n            total_loss += loss\n            num_batches += 1\n\n            loss.backward()  # backpropagate\n            optimiser.step()\n\n        average_loss = total_loss / num_batches\n        losses.append(average_loss)\n        print(f\"Epoch {EPOCHS}:\\tScore: {1/average_loss}\")\n    return losses\n```\n\n\n```python\nEPOCHS = 0\nINPUT_SIZE = 28*28\nHIDDEN_SIZE = 128\nCODE_SIZE = 32\nLEARNING_RATE = 0.01\nNOISE_FACTOR = 0.001\n\ndenoise_autoencoder = AutoEncoder(\n    INPUT_SIZE, HIDDEN_SIZE, CODE_SIZE).double().to(device)\n```\n\n\n```python\nnum_epochs = 25\nlosses = train_noise(denoise_autoencoder, num_epochs, LEARNING_RATE, NOISE_FACTOR)\nshow_losses_helper(losses)\n```\n\n    Epoch 1:\tScore: 59.75679771690447\n    Epoch 2:\tScore: 251.18074917481994\n    Epoch 3:\tScore: 252.6921020280419\n    Epoch 4:\tScore: 254.09790694141552\n    Epoch 5:\tScore: 254.819997697417\n    Epoch 6:\tScore: 255.6102181550768\n    Epoch 7:\tScore: 256.63636913198377\n    Epoch 8:\tScore: 257.94384904717685\n    Epoch 9:\tScore: 258.76483061365417\n    Epoch 10:\tScore: 259.70921886758015\n    Epoch 11:\tScore: 260.5467640070574\n    Epoch 12:\tScore: 260.9838936349969\n    Epoch 13:\tScore: 261.37337065872185\n    Epoch 14:\tScore: 262.5729687955811\n    Epoch 15:\tScore: 264.4646967983865\n    Epoch 16:\tScore: 265.9826024554563\n    Epoch 17:\tScore: 266.93235612723765\n    Epoch 18:\tScore: 267.65796666980407\n    Epoch 19:\tScore: 268.40720731143205\n    Epoch 20:\tScore: 269.01269006078854\n    Epoch 21:\tScore: 269.60631692091505\n    Epoch 22:\tScore: 270.46841955209504\n    Epoch 23:\tScore: 271.47174606097565\n    Epoch 24:\tScore: 272.0981936757506\n    Epoch 25:\tScore: 272.5683784998365\n\n\n\n![png](img/output/output_25_1.png)\n\n\n\n```python\ndef validate_noise(model, noise_factor=NOISE_FACTOR):\n    model.eval()\n    criterion = torch.nn.BCELoss()\n#     criterion = torch.nn.MSELoss()\n    total_loss = 0\n    num_batches = 0\n    for val_img, _ in val_loader:\n        val_img = val_img.double().view(-1, 784).to(device) / 255.0\n        gen_img = model(add_noise(val_img, noise_factor)).double()\n\n        loss = criterion(gen_img, val_img)\n        total_loss += loss\n        num_batches += 1\n    average_loss = total_loss / num_batches\n    return 1/average_loss.item()\n```\n\n\n```python\nscore = validate_noise(denoise_autoencoder)\nprint(\"Score: \", score)\n```\n\n    Score:  272.01511380963933\n\n\n\n```python\ndef test_noise(model, noise_factor=NOISE_FACTOR):\n    model.eval()\n    criterion = torch.nn.BCELoss()\n#   criterion = torch.nn.MSELoss()\n    total_loss = 0\n    num_batches = 0\n    stored_images = []\n    for test_img, _ in test_loader:\n        test_img = test_img.double().view(-1, 784).to(device) / 255.0\n        noisy_img = add_noise(test_img, noise_factor)\n        gen_img = model(noisy_img).double()\n        \n        loss = criterion(gen_img, test_img)\n        total_loss += loss\n        num_batches += 1\n        if np.random.random() \u003e 0.90:\n            stored_images.append((test_img[0].clone().detach(\n            ), noisy_img[0].clone().detach(), gen_img[0].clone().detach()))\n\n    score = average_loss = total_loss / num_batches\n    print(f\"Score: {1/score}\\n\")\n\n    for original, noisy, generated in stored_images:\n        print(\"Original: \")\n        show_image_helper(original)\n        print(\"Noisy: \")\n        show_image_helper(noisy)\n        print(\"Generated: \")\n        show_image_helper(generated)\n```\n\n\n```python\ntest_noise(denoise_autoencoder)\n```\n\n    Score: 269.54182307638195\n    \n    Original: \n\n\n\n![png](img/output/output_29_1.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_3.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_5.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_7.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_9.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_11.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_13.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_15.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_17.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_19.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_21.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_23.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_25.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_27.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_29.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_31.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_33.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_35.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_37.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_39.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_41.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_43.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_45.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_47.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_49.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_51.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_53.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_55.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_57.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_59.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_61.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_63.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_65.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_67.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_69.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_71.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_73.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_75.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_77.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_79.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_81.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_83.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_85.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_87.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_89.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Original: \n\n\n\n![png](img/output/output_29_91.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Noisy: \n\n\n\n![png](img/output/output_29_93.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n    Generated: \n\n\n\n![png](img/output/output_29_95.png)\n\n\n    Max Element:  tensor(1.)\n    Min Element:  tensor(0.)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamuel-adekunle%2Fautoencoders","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamuel-adekunle%2Fautoencoders","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamuel-adekunle%2Fautoencoders/lists"}