{"id":16537576,"url":"https://github.com/phreakyphoenix/face-generation","last_synced_at":"2026-06-05T08:31:50.746Z","repository":{"id":104660833,"uuid":"267762981","full_name":"phreakyphoenix/Face-Generation","owner":"phreakyphoenix","description":"DCGAN architecture to generate faces from CelebA dataset, made with :heart: in PyTorch. Do :star2: the repo if you find it useful.","archived":false,"fork":false,"pushed_at":"2020-05-29T05:26:04.000Z","size":26142,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-03T21:09:01.029Z","etag":null,"topics":["create-faces","dcgan","dcgan-pytorch","deep-learning-nanodegree","dlnd","dlnd-face-generation","face-generation","gan","generative-adversarial-networks","machine-learning","pytorch","pytorch-gan","udacity","udacity-deep-learning","udacity-nanodegree","udacity-project"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phreakyphoenix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-29T04:14:29.000Z","updated_at":"2020-05-29T13:02:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"e6e08fdd-df18-4cae-b6ff-3636ef1b40c2","html_url":"https://github.com/phreakyphoenix/Face-Generation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/phreakyphoenix/Face-Generation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreakyphoenix%2FFace-Generation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreakyphoenix%2FFace-Generation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreakyphoenix%2FFace-Generation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreakyphoenix%2FFace-Generation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phreakyphoenix","download_url":"https://codeload.github.com/phreakyphoenix/Face-Generation/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phreakyphoenix%2FFace-Generation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33937661,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["create-faces","dcgan","dcgan-pytorch","deep-learning-nanodegree","dlnd","dlnd-face-generation","face-generation","gan","generative-adversarial-networks","machine-learning","pytorch","pytorch-gan","udacity","udacity-deep-learning","udacity-nanodegree","udacity-project"],"created_at":"2024-10-11T18:42:51.353Z","updated_at":"2026-06-05T08:31:50.692Z","avatar_url":"https://github.com/phreakyphoenix.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Face Generation\n### Project 4 in Udacity Deep Learning Nanodegree\n\nIn this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate *new* images of faces that look as realistic as possible! \n\nTo check out the Project, use any of these links:\n1. [Markdown](https://github.com/phreakyphoenix/Face-generation/tree/master/README.md) \n2. [IPYNB](https://github.com/phreakyphoenix/Face-generation/blob/master/dlnd_face_generation.ipynb)\n3. [HTML](https://github.com/phreakyphoenix/Face-generation/blob/master/dlnd_face_generation.html)  \n\n\u003e**g.pt and d.pt are the trained generator and discriminator files**\n\nThe project will be broken down into a series of tasks from **loading in data to defining and training adversarial networks**. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.\n\n### Get the Data\n\nYou'll be using the [CelebFaces Attributes Dataset (CelebA)](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) to train your adversarial networks.\n\nThis dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.\n\n### Pre-processed Data\n\nSince the project's main focus is on building the GANs, we've done *some* of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.\n\n\u003cimg src='assets/processed_face_data.png' width=60% /\u003e\n\n\u003e If you are working locally, you can download this data [by clicking here](https://s3.amazonaws.com/video.udacity-data.com/topher/2018/November/5be7eb6f_processed-celeba-small/processed-celeba-small.zip)\n\nThis is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data `processed_celeba_small/`\n\n\n```python\n# can comment out after executing\n# !unzip processed_celeba_small.zip\n```\n\n\n```python\ndata_dir = 'processed_celeba_small/'\n\nimport pickle as pkl\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport problem_unittests as tests\n#import helper\n\n%matplotlib inline\n```\n\n## Visualize the CelebA Data\n\nThe [CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with [3 color channels (RGB)](https://en.wikipedia.org/wiki/Channel_(digital_image)#RGB_Images) each.\n\n### Pre-process and Load the Data\n\nSince the project's main focus is on building the GANs, we've done *some* of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This *pre-processed* dataset is a smaller subset of the very large CelebA data.\n\n\u003e There are a few other steps that you'll need to **transform** this data and create a **DataLoader**.\n\n#### Exercise: Complete the following `get_dataloader` function, such that it satisfies these requirements:\n\n* Your images should be square, Tensor images of size `image_size x image_size` in the x and y dimension.\n* Your function should return a DataLoader that shuffles and batches these Tensor images.\n\n#### ImageFolder\n\nTo create a dataset given a directory of images, it's recommended that you use PyTorch's [ImageFolder](https://pytorch.org/docs/stable/torchvision/datasets.html#imagefolder) wrapper, with a root directory `processed_celeba_small/` and data transformation passed in.\n\n\n```python\n# necessary imports\nimport torch\nfrom torchvision import datasets\nfrom torchvision import transforms\n```\n\n\n```python\ndef get_dataloader(batch_size, image_size, data_dir='processed_celeba_small/'):\n    \"\"\"\n    Batch the neural network data using DataLoader\n    :param batch_size: The size of each batch; the number of images in a batch\n    :param img_size: The square size of the image data (x, y)\n    :param data_dir: Directory where image data is located\n    :return: DataLoader with batched data\n    \"\"\"\n    \n    # TODO: Implement function and return a dataloader\n    transform = transforms.Compose([transforms.Resize(image_size),\n                                   transforms.ToTensor()]) \n    image_dataset = datasets.ImageFolder(data_dir, transform)\n    print (image_dataset)\n    return torch.utils.data.DataLoader(image_dataset, batch_size = batch_size, shuffle=True)\n```\n\n## Create a DataLoader\n\n#### Exercise: Create a DataLoader `celeba_train_loader` with appropriate hyperparameters.\n\nCall the above function and create a dataloader to view images. \n* You can decide on any reasonable `batch_size` parameter\n* Your `image_size` **must be** `32`. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!\n\n\n```python\n# Define function hyperparameters\nbatch_size = 128\nimg_size = 32\n\n# Call your function and get a dataloader\nceleba_train_loader = get_dataloader(batch_size, img_size)\n```\n\n    Dataset ImageFolder\n        Number of datapoints: 89931\n        Root Location: processed_celeba_small/\n        Transforms (if any): Compose(\n                                 Resize(size=32, interpolation=PIL.Image.BILINEAR)\n                                 ToTensor()\n                             )\n        Target Transforms (if any): None\n\n\nNext, you can view some images! You should seen square images of somewhat-centered faces.\n\nNote: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested `imshow` code is below, but it may not be perfect.\n\n\n```python\n# helper display function\ndef imshow(img):\n    npimg = img.numpy()\n    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n\n# obtain one batch of training images\ndataiter = iter(celeba_train_loader)\nimages, _ = dataiter.next() # _ for no labels\n\n# plot the images in the batch, along with the corresponding labels\nfig = plt.figure(figsize=(20, 4))\nplot_size=20\nfor idx in np.arange(plot_size):\n    ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])\n    imshow(images[idx])\n```\n\n\n![png](assets/output_9_0.png)\n\n\n#### Exercise: Pre-process your image data and scale it to a pixel range of -1 to 1\n\nYou need to do a bit of pre-processing; you know that the output of a `tanh` activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)\n\n\n```python\n# TODO: Complete the scale function\ndef scale(x, feature_range=(-1, 1)):\n    ''' Scale takes in an image x and returns that image, scaled\n       with a feature_range of pixel values from -1 to 1. \n       This function assumes that the input x is already scaled from 0-1.'''\n    # assume x is scaled to (0, 1)\n    # scale to feature_range and return scaled x   \n#     min_val, max_val = feature_range\n#     return x * (max_val - min_val) + min_val\n    return x*2-1         #it's always called with -1,1 so for speed\n\n```\n\n\n```python\n# check scaled range\n# should be close to -1 to 1\nimg = images[0]\nscaled_img = scale(img)\n\nprint('Min: ', scaled_img.min())\nprint('Max: ', scaled_img.max())\n```\n\n    Min:  tensor(-0.9059)\n    Max:  tensor(0.8588)\n\n\n---\n# Define the Model\n\nA GAN is comprised of two adversarial networks, a discriminator and a generator.\n\n## Discriminator\n\nYour first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with **normalization**. You are also allowed to create any helper functions that may be useful.\n\n#### Exercise: Complete the Discriminator class\n* The inputs to the discriminator are 32x32x3 tensor images\n* The output should be a single value that will indicate whether a given image is real or fake\n\n\n\n```python\nimport torch.nn as nn\nimport torch.nn.functional as F\n```\n\n\n```python\n# helper conv function\ndef conv(in_channels, out_channels, kernel_size, stride=2, padding=1, batch_norm=True):\n    \"\"\"Creates a convolutional layer, with optional batch normalization.\n    \"\"\"\n    layers = []\n    conv_layer = nn.Conv2d(in_channels=in_channels, out_channels=out_channels, \n                           kernel_size=kernel_size, stride=stride, padding=padding, bias=False)\n    \n    layers.append(conv_layer)\n\n    if batch_norm:\n        layers.append(nn.BatchNorm2d(out_channels))\n    return nn.Sequential(*layers)\n```\n\n\n```python\nclass Discriminator(nn.Module):\n\n    def __init__(self, conv_dim):\n        \"\"\"\n        Initialize the Discriminator Module\n        :param conv_dim: The depth of the first convolutional layer\n        \"\"\"\n        super(Discriminator, self).__init__()\n        self.conv_dim = conv_dim\n        self.conv1 = conv(3, conv_dim, 4, batch_norm=False) # (16, 16, conv_dim)\n        self.conv2 = conv(conv_dim, conv_dim*2, 4) # (8, 8, conv_dim*2)\n        self.conv3 = conv(conv_dim*2, conv_dim*4, 4) # (4, 4, conv_dim*4)\n        self.conv4 = conv(conv_dim*4, conv_dim*8, 4) # (2, 2, conv_dim*8) \n        \n        self.classifier = nn.Linear(conv_dim*8*2*2, 1)\n\n        \n\n    def forward(self, x):\n        \"\"\"\n        Forward propagation of the neural network\n        :param x: The input to the neural network     \n        :return: Discriminator logits; the output of the neural network\n        \"\"\"\n        # define feedforward behavior\n        out = F.selu(self.conv1(x), 0.2)\n        out = F.selu(self.conv2(out), 0.2)\n        out = F.selu(self.conv3(out), 0.2)\n        out = F.selu(self.conv4(out), 0.2)\n        \n        out = out.view(-1, self.conv_dim*8*2*2)\n        out = self.classifier(out)\n        return out\n\n\ntests.test_discriminator(Discriminator)\n```\n\n    Tests Passed\n\n\n## Generator\n\nThe generator should upsample an input and generate a *new* image of the same size as our training data `32x32x3`. This should be mostly transpose convolutional layers with normalization applied to the outputs.\n\n#### Exercise: Complete the Generator class\n* The inputs to the generator are vectors of some length `z_size`\n* The output should be a image of shape `32x32x3`\n\n\n```python\ndef deconv(in_channels, out_channels, kernel_size, stride=2, padding=1, batch_norm=True):\n    layers = []\n    layers.append(nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride, padding, bias=False))\n    if batch_norm:\n        layers.append(nn.BatchNorm2d(out_channels))\n    return nn.Sequential(*layers)\n```\n\n\n```python\nclass Generator(nn.Module):\n    \n    def __init__(self, z_size, conv_dim):\n        \"\"\"\n        Initialize the Generator Module\n        :param z_size: The length of the input latent vector, z\n        :param conv_dim: The depth of the inputs to the *last* transpose convolutional layer\n        \"\"\"\n        super(Generator, self).__init__()\n        self.conv_dim = conv_dim\n\n        self.fc = nn.Linear(z_size, conv_dim*8*2*2)\n        \n        self.t_conv1 = deconv(conv_dim*8, conv_dim*4, 4)\n        self.t_conv2 = deconv(conv_dim*4, conv_dim*2, 4)\n        self.t_conv3 = deconv(conv_dim*2, conv_dim, 4)\n        self.t_conv4 = deconv(conv_dim, 3, 4, batch_norm=False)\n        \n\n    def forward(self, x):\n        \"\"\"\n        Forward propagation of the neural network\n        :param x: The input to the neural network     \n        :return: A 32x32x3 Tensor image as output\n        \"\"\"\n        # define feedforward behavior\n        out = self.fc(x)\n        out = out.view(-1, self.conv_dim*8, 2, 2) # (batch_size, depth, 4, 4)\n        \n        out = F.relu(self.t_conv1(out))\n        out = F.relu(self.t_conv2(out))\n        out = F.relu(self.t_conv3(out))\n        \n        # last layer: tanh activation instead of relu\n        out = self.t_conv4(out)\n        out = F.tanh(out)\n        \n        return out\n\n\ntests.test_generator(Generator)\n```\n\n    Tests Passed\n\n\n## Initialize the weights of your networks\n\nTo help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the [original DCGAN paper](https://arxiv.org/pdf/1511.06434.pdf), they say:\n\u003e All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.\n\nSo, your next task will be to define a weight initialization function that does just this!\n\nYou can refer back to the lesson on weight initialization or even consult existing model code, such as that from [the `networks.py` file in CycleGAN Github repository](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/models/networks.py) to help you complete this function.\n\n#### Exercise: Complete the weight initialization function\n\n* This should initialize only **convolutional** and **linear** layers\n* Initialize the weights to a normal distribution, centered around 0, with a standard deviation of 0.02.\n* The bias terms, if they exist, may be left alone or set to 0.\n\n\n```python\nfrom torch.nn import init\n\ndef weights_init_normal(m):\n    \"\"\"\n    Applies initial weights to certain layers in a model: convolutional and linear\n    The weights are taken from a normal distribution \n    with mean = 0, std dev = 0.02.\n    :param m: A module or layer in a network    \n    \"\"\"\n    # classname will be something like:\n    # `Conv`, `BatchNorm2d`, `Linear`, etc.\n    classname = m.__class__.__name__\n    isConvolution = classname.find('Conv') != -1\n    isLinear = classname.find('Linear') != -1\n    if (hasattr(m, 'weight') and isConvolution or isLinear):\n        init.normal_(m.weight.data, 0.0, 0.02)\n```\n\n## Build complete network\n\nDefine your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.\n\n\n```python\ndef build_network(d_conv_dim, g_conv_dim, z_size):\n    # define discriminator and generator\n    D = Discriminator(d_conv_dim)\n    G = Generator(z_size=z_size, conv_dim=g_conv_dim)\n\n    # initialize model weights\n    D.apply(weights_init_normal)\n    G.apply(weights_init_normal)\n\n    print(D)\n    print()\n    print(G)\n    \n    return D, G\n\n```\n\n#### Exercise: Define model hyperparameters\n\n\n```python\n# Define model hyperparams\nd_conv_dim = 64\ng_conv_dim = 64\nz_size = 100\n\nD, G = build_network(d_conv_dim, g_conv_dim, z_size)\n```\n\n    Discriminator(\n      (conv1): Sequential(\n        (0): Conv2d(3, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n      )\n      (conv2): Sequential(\n        (0): Conv2d(64, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (conv3): Sequential(\n        (0): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (conv4): Sequential(\n        (0): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (classifier): Linear(in_features=2048, out_features=1, bias=True)\n    )\n    \n    Generator(\n      (fc): Linear(in_features=100, out_features=2048, bias=True)\n      (t_conv1): Sequential(\n        (0): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (t_conv2): Sequential(\n        (0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (t_conv3): Sequential(\n        (0): ConvTranspose2d(128, 64, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n        (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n      )\n      (t_conv4): Sequential(\n        (0): ConvTranspose2d(64, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1), bias=False)\n      )\n    )\n\n\n### Training on GPU\n\nCheck if you can train on GPU. Here, we'll set this as a boolean variable `train_on_gpu`. Later, you'll be responsible for making sure that \n\u003e* Models,\n* Model inputs, and\n* Loss function arguments\n\nAre moved to GPU, where appropriate.\n\n\n```python\n\nimport torch\n\n# Check for a GPU\ntrain_on_gpu = torch.cuda.is_available()\nif not train_on_gpu:\n    print('No GPU found. Please use a GPU to train your neural network.')\nelse:\n    print('Training on GPU!')\n```\n\n    Training on GPU!\n\n\n---\n## Discriminator and Generator Losses\n\nNow we need to calculate the losses for both types of adversarial networks.\n\n### Discriminator Losses\n\n\u003e * For the discriminator, the total loss is the sum of the losses for real and fake images, `d_loss = d_real_loss + d_fake_loss`. \n* Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.\n\n\n### Generator Loss\n\nThe generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to *think* its generated images are *real*.\n\n#### Exercise: Complete real and fake loss functions\n\n**You may choose to use either cross entropy or a least squares error loss to complete the following `real_loss` and `fake_loss` functions.**\n\n\n```python\ndef real_loss(D_out):\n    '''\n    Calculates how close discriminator outputs are to being real.\n    param, D_out: discriminator logits\n    return: real loss\n    '''\n    labels = torch.ones(D_out.size(0))      #D_out[0] is batch size\n    if train_on_gpu:\n        labels = labels.cuda()\n    \n    criterion = nn.BCEWithLogitsLoss()\n    return criterion(D_out.squeeze(), labels)\n\ndef fake_loss(D_out):\n    '''\n    Calculates how close discriminator outputs are to being fake.\n    param, D_out: discriminator logits\n    return: fake loss\n    '''\n    labels = torch.zeros(D_out.size(0))     #D_out[0] is batch size\n    if train_on_gpu:\n        labels = labels.cuda()\n    \n    criterion = nn.BCEWithLogitsLoss()\n    return criterion(D_out.squeeze(), labels)\n```\n\n## Optimizers\n\n#### Exercise: Define optimizers for your Discriminator (D) and Generator (G)\n\nDefine optimizers for your models with appropriate hyperparameters.\n\n\n```python\nimport torch.optim as optim\n\nlr = 0.0002\nbeta1=0.5\nbeta2=0.999 # default value\n\n# Create optimizers for the discriminator D and generator G\nd_optimizer = optim.Adam(D.parameters(), lr, [beta1, beta2])\ng_optimizer = optim.Adam(G.parameters(), lr, [beta1, beta2])\n```\n\n---\n## Training\n\nTraining will involve alternating between training the discriminator and the generator. You'll use your functions `real_loss` and `fake_loss` to help you calculate the discriminator losses.\n\n* You should train the discriminator by alternating on real and fake images\n* Then the generator, which tries to trick the discriminator and should have an opposing loss function\n\n\n#### Saving Samples\n\nYou've been given some code to print out some loss statistics and save some generated \"fake\" samples.\n\n#### Exercise: Complete the training function\n\nKeep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.\n\n\n```python\nfrom workspace_utils import active_session\n```\n\n\n```python\ndef train(D, G, n_epochs, print_every=50):\n    '''Trains adversarial networks for some number of epochs\n       param, D: the discriminator network\n       param, G: the generator network\n       param, n_epochs: number of epochs to train for\n       param, print_every: when to print and record the models' losses\n       return: D and G losses'''\n    \n    # move models to GPU\n    if train_on_gpu:\n        D.cuda()\n        G.cuda()\n\n    # keep track of loss and generated, \"fake\" samples\n    samples = []\n    losses = []\n\n    # Get some fixed data for sampling. These are images that are held\n    # constant throughout training, and allow us to inspect the model's performance\n    sample_size=16\n    fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))\n    fixed_z = torch.from_numpy(fixed_z).float()\n    # move z to GPU if available\n    if train_on_gpu:\n        fixed_z = fixed_z.cuda()\n\n    # epoch training loop\n    for epoch in range(n_epochs):\n\n        # batch training loop\n        for batch_i, (real_images, _) in enumerate(celeba_train_loader):\n\n            batch_size = real_images.size(0)\n            real_images = scale(real_images)\n\n            # ===============================================\n            #         YOUR CODE HERE: TRAIN THE NETWORKS\n            # ===============================================\n            \n            # 1. Train the discriminator on real and fake images\n            d_optimizer.zero_grad()\n            if train_on_gpu:\n                real_images = real_images.cuda()\n                \n            D_real = D(real_images)\n            d_real_loss = real_loss(D_real)\n            \n            z = np.random.uniform(-1, 1, size = (batch_size, z_size))\n            z = torch.from_numpy(z).float()\n            if train_on_gpu:\n                z = z.cuda()\n            \n            fake_images = G(z)\n            D_fake = D(fake_images)\n            d_fake_loss = fake_loss(D_fake)\n            \n            d_loss = d_real_loss + d_fake_loss\n            d_loss.backward()\n            d_optimizer.step()\n\n            # 2. Train the generator with an adversarial loss\n            g_optimizer.zero_grad()\n            \n            z = np.random.uniform(-1, 1, size = (batch_size, z_size))\n            z = torch.from_numpy(z).float()\n            if train_on_gpu:\n                z = z.cuda()\n            \n            fake_images = G(z)\n            D_fake = D(fake_images)\n            g_loss = real_loss(D_fake)\n            \n            g_loss.backward()\n            g_optimizer.step()\n            \n            \n            # ===============================================\n            #              END OF YOUR CODE\n            # ===============================================\n\n            # Print some loss stats\n            if batch_i % print_every == 0:\n                # append discriminator loss and generator loss\n                losses.append((d_loss.item(), g_loss.item()))\n                # print discriminator and generator loss\n                print('Epoch [{:5d}/{:5d}] | batch: {:5d} | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(\n                        epoch+1, n_epochs, batch_i+1, d_loss.item(), g_loss.item()))\n\n\n        ## AFTER EACH EPOCH##    \n        torch.save(G.state_dict(), 'g'+str(epoch+1)+'.pt')\n        torch.save(D.state_dict(), 'd'+str(epoch+1)+'.pt')\n\n        # this code assumes your generator is named G, feel free to change the name\n        # generate and save sample, fake images\n        G.eval() # for generating samples\n        samples_z = G(fixed_z)\n        samples.append(samples_z)\n        G.train() # back to training mode\n        print('---------------Epoch [{:5d}/{:5d}]---------------'.format(epoch+1, n_epochs))\n    # Save training generator samples\n    with open('train_samples.pkl', 'wb') as f:\n        pkl.dump(samples, f)\n    \n    # finally return losses\n    return losses\n```\n\nSet your number of training epochs and train your GAN!\n\n\n```python\n# G.load_state_dict(torch.load('g.pt'))\n# D.load_state_dict(torch.load('d.pt'))\n```\n\n\n```python\n# set number of epochs \nn_epochs = 20\n\n\n# call training function\nwith active_session():\n    losses = train(D, G, n_epochs=n_epochs)\n```\n\n    Epoch [    1/   20] | batch:     1 | d_loss: 0.8111 | g_loss: 1.2661\n    Epoch [    1/   20] | batch:    51 | d_loss: 0.9952 | g_loss: 2.3659\n    Epoch [    1/   20] | batch:   101 | d_loss: 0.9237 | g_loss: 2.7058\n    Epoch [    1/   20] | batch:   151 | d_loss: 0.9340 | g_loss: 1.3858\n    Epoch [    1/   20] | batch:   201 | d_loss: 0.9927 | g_loss: 1.6645\n    Epoch [    1/   20] | batch:   251 | d_loss: 0.8794 | g_loss: 1.3428\n    Epoch [    1/   20] | batch:   301 | d_loss: 1.0240 | g_loss: 1.8747\n    Epoch [    1/   20] | batch:   351 | d_loss: 0.8578 | g_loss: 2.3406\n    Epoch [    1/   20] | batch:   401 | d_loss: 1.0471 | g_loss: 1.4166\n    Epoch [    1/   20] | batch:   451 | d_loss: 0.8953 | g_loss: 2.2885\n    Epoch [    1/   20] | batch:   501 | d_loss: 0.8606 | g_loss: 1.5577\n    Epoch [    1/   20] | batch:   551 | d_loss: 0.8905 | g_loss: 1.9201\n    Epoch [    1/   20] | batch:   601 | d_loss: 0.9602 | g_loss: 2.0931\n    Epoch [    1/   20] | batch:   651 | d_loss: 0.9449 | g_loss: 2.2206\n    Epoch [    1/   20] | batch:   701 | d_loss: 1.0528 | g_loss: 1.5514\n    ---------------Epoch [    1/   20]---------------\n    Epoch [    2/   20] | batch:     1 | d_loss: 0.8771 | g_loss: 1.3734\n    Epoch [    2/   20] | batch:    51 | d_loss: 1.0019 | g_loss: 1.2820\n    Epoch [    2/   20] | batch:   101 | d_loss: 0.9900 | g_loss: 1.6812\n    Epoch [    2/   20] | batch:   151 | d_loss: 1.0659 | g_loss: 2.1863\n    Epoch [    2/   20] | batch:   201 | d_loss: 0.9917 | g_loss: 1.6778\n    Epoch [    2/   20] | batch:   251 | d_loss: 0.8652 | g_loss: 1.5668\n    Epoch [    2/   20] | batch:   301 | d_loss: 0.7992 | g_loss: 2.3568\n    Epoch [    2/   20] | batch:   351 | d_loss: 1.0523 | g_loss: 1.2216\n    Epoch [    2/   20] | batch:   401 | d_loss: 1.0096 | g_loss: 2.4254\n    Epoch [    2/   20] | batch:   451 | d_loss: 1.1680 | g_loss: 1.1488\n    Epoch [    2/   20] | batch:   501 | d_loss: 0.9187 | g_loss: 1.2972\n    Epoch [    2/   20] | batch:   551 | d_loss: 0.9080 | g_loss: 1.9679\n    Epoch [    2/   20] | batch:   601 | d_loss: 1.0048 | g_loss: 2.4279\n    Epoch [    2/   20] | batch:   651 | d_loss: 1.2464 | g_loss: 1.1955\n    Epoch [    2/   20] | batch:   701 | d_loss: 1.0181 | g_loss: 2.1282\n    ---------------Epoch [    2/   20]---------------\n    Epoch [    3/   20] | batch:     1 | d_loss: 0.9845 | g_loss: 3.0135\n    Epoch [    3/   20] | batch:    51 | d_loss: 0.8247 | g_loss: 2.0062\n    Epoch [    3/   20] | batch:   101 | d_loss: 0.9052 | g_loss: 2.0246\n    Epoch [    3/   20] | batch:   151 | d_loss: 0.8806 | g_loss: 1.8005\n    Epoch [    3/   20] | batch:   201 | d_loss: 1.1608 | g_loss: 1.1466\n    Epoch [    3/   20] | batch:   251 | d_loss: 1.0030 | g_loss: 1.7211\n    Epoch [    3/   20] | batch:   301 | d_loss: 1.2835 | g_loss: 1.8969\n    Epoch [    3/   20] | batch:   351 | d_loss: 0.8584 | g_loss: 1.6190\n    Epoch [    3/   20] | batch:   401 | d_loss: 0.7295 | g_loss: 1.6601\n    Epoch [    3/   20] | batch:   451 | d_loss: 1.0188 | g_loss: 1.8397\n    Epoch [    3/   20] | batch:   501 | d_loss: 1.2436 | g_loss: 2.5559\n    Epoch [    3/   20] | batch:   551 | d_loss: 1.0364 | g_loss: 2.0659\n    Epoch [    3/   20] | batch:   601 | d_loss: 1.0645 | g_loss: 1.2833\n    Epoch [    3/   20] | batch:   651 | d_loss: 0.8505 | g_loss: 2.2852\n    Epoch [    3/   20] | batch:   701 | d_loss: 0.9450 | g_loss: 1.7768\n    ---------------Epoch [    3/   20]---------------\n    Epoch [    4/   20] | batch:     1 | d_loss: 0.9740 | g_loss: 1.1168\n    Epoch [    4/   20] | batch:    51 | d_loss: 0.9093 | g_loss: 1.2597\n    Epoch [    4/   20] | batch:   101 | d_loss: 1.0745 | g_loss: 0.9085\n    Epoch [    4/   20] | batch:   151 | d_loss: 0.7689 | g_loss: 1.8919\n    Epoch [    4/   20] | batch:   201 | d_loss: 1.2617 | g_loss: 3.2608\n    Epoch [    4/   20] | batch:   251 | d_loss: 1.0825 | g_loss: 1.3975\n    Epoch [    4/   20] | batch:   301 | d_loss: 1.0497 | g_loss: 0.9826\n    Epoch [    4/   20] | batch:   351 | d_loss: 0.8368 | g_loss: 1.8344\n    Epoch [    4/   20] | batch:   401 | d_loss: 0.8927 | g_loss: 1.4510\n    Epoch [    4/   20] | batch:   451 | d_loss: 0.8737 | g_loss: 2.1484\n    Epoch [    4/   20] | batch:   501 | d_loss: 0.8620 | g_loss: 1.7640\n    Epoch [    4/   20] | batch:   551 | d_loss: 1.0488 | g_loss: 1.1829\n    Epoch [    4/   20] | batch:   601 | d_loss: 0.9488 | g_loss: 1.2433\n    Epoch [    4/   20] | batch:   651 | d_loss: 0.8552 | g_loss: 2.1256\n    Epoch [    4/   20] | batch:   701 | d_loss: 1.2086 | g_loss: 1.2018\n    ---------------Epoch [    4/   20]---------------\n    Epoch [    5/   20] | batch:     1 | d_loss: 0.6779 | g_loss: 2.0733\n    Epoch [    5/   20] | batch:    51 | d_loss: 0.9240 | g_loss: 0.9773\n    Epoch [    5/   20] | batch:   101 | d_loss: 0.8876 | g_loss: 1.7137\n    Epoch [    5/   20] | batch:   151 | d_loss: 1.0030 | g_loss: 2.4516\n    Epoch [    5/   20] | batch:   201 | d_loss: 0.9148 | g_loss: 1.8731\n    Epoch [    5/   20] | batch:   251 | d_loss: 0.9607 | g_loss: 1.9129\n    Epoch [    5/   20] | batch:   301 | d_loss: 0.9252 | g_loss: 1.8296\n    Epoch [    5/   20] | batch:   351 | d_loss: 0.7578 | g_loss: 2.0952\n    Epoch [    5/   20] | batch:   401 | d_loss: 0.7759 | g_loss: 1.5163\n    Epoch [    5/   20] | batch:   451 | d_loss: 0.8367 | g_loss: 1.9044\n    Epoch [    5/   20] | batch:   501 | d_loss: 0.8096 | g_loss: 1.8013\n    Epoch [    5/   20] | batch:   551 | d_loss: 0.7723 | g_loss: 1.8913\n    Epoch [    5/   20] | batch:   601 | d_loss: 0.7605 | g_loss: 2.4772\n    Epoch [    5/   20] | batch:   651 | d_loss: 1.3507 | g_loss: 1.2231\n    Epoch [    5/   20] | batch:   701 | d_loss: 0.7732 | g_loss: 2.0357\n    ---------------Epoch [    5/   20]---------------\n    Epoch [    6/   20] | batch:     1 | d_loss: 0.8181 | g_loss: 1.4688\n    Epoch [    6/   20] | batch:    51 | d_loss: 1.3336 | g_loss: 1.3665\n    Epoch [    6/   20] | batch:   101 | d_loss: 0.8859 | g_loss: 1.9091\n    Epoch [    6/   20] | batch:   151 | d_loss: 0.8380 | g_loss: 2.1336\n    Epoch [    6/   20] | batch:   201 | d_loss: 0.8269 | g_loss: 3.3221\n    Epoch [    6/   20] | batch:   251 | d_loss: 0.9188 | g_loss: 1.9334\n    Epoch [    6/   20] | batch:   301 | d_loss: 0.9195 | g_loss: 1.7430\n    Epoch [    6/   20] | batch:   351 | d_loss: 1.2432 | g_loss: 3.4875\n    Epoch [    6/   20] | batch:   401 | d_loss: 0.7508 | g_loss: 1.7305\n    Epoch [    6/   20] | batch:   451 | d_loss: 0.9282 | g_loss: 1.9829\n    Epoch [    6/   20] | batch:   501 | d_loss: 0.8910 | g_loss: 2.0563\n    Epoch [    6/   20] | batch:   551 | d_loss: 0.7907 | g_loss: 1.6141\n    Epoch [    6/   20] | batch:   601 | d_loss: 0.8585 | g_loss: 2.6222\n    Epoch [    6/   20] | batch:   651 | d_loss: 0.7723 | g_loss: 1.0951\n    Epoch [    6/   20] | batch:   701 | d_loss: 0.8480 | g_loss: 2.0792\n    ---------------Epoch [    6/   20]---------------\n    Epoch [    7/   20] | batch:     1 | d_loss: 0.8413 | g_loss: 1.3066\n    Epoch [    7/   20] | batch:    51 | d_loss: 0.7940 | g_loss: 2.6432\n    Epoch [    7/   20] | batch:   101 | d_loss: 0.8089 | g_loss: 1.8077\n    Epoch [    7/   20] | batch:   151 | d_loss: 0.9595 | g_loss: 1.6319\n    Epoch [    7/   20] | batch:   201 | d_loss: 1.0729 | g_loss: 0.9551\n    Epoch [    7/   20] | batch:   251 | d_loss: 0.9064 | g_loss: 2.4957\n    Epoch [    7/   20] | batch:   301 | d_loss: 0.7105 | g_loss: 2.1476\n    Epoch [    7/   20] | batch:   351 | d_loss: 1.0778 | g_loss: 2.9749\n    Epoch [    7/   20] | batch:   401 | d_loss: 0.9246 | g_loss: 1.3092\n    Epoch [    7/   20] | batch:   451 | d_loss: 1.0882 | g_loss: 0.7023\n    Epoch [    7/   20] | batch:   501 | d_loss: 1.0619 | g_loss: 1.6870\n    Epoch [    7/   20] | batch:   551 | d_loss: 0.8053 | g_loss: 2.0815\n    Epoch [    7/   20] | batch:   601 | d_loss: 0.7979 | g_loss: 2.2491\n    Epoch [    7/   20] | batch:   651 | d_loss: 0.8395 | g_loss: 1.9564\n    Epoch [    7/   20] | batch:   701 | d_loss: 1.0466 | g_loss: 1.3503\n    ---------------Epoch [    7/   20]---------------\n    Epoch [    8/   20] | batch:     1 | d_loss: 0.6671 | g_loss: 1.4386\n    Epoch [    8/   20] | batch:    51 | d_loss: 0.9737 | g_loss: 1.9353\n    Epoch [    8/   20] | batch:   101 | d_loss: 0.7338 | g_loss: 1.6522\n    Epoch [    8/   20] | batch:   151 | d_loss: 0.8512 | g_loss: 2.1538\n    Epoch [    8/   20] | batch:   201 | d_loss: 0.7490 | g_loss: 2.2567\n    Epoch [    8/   20] | batch:   251 | d_loss: 0.8791 | g_loss: 2.4892\n    Epoch [    8/   20] | batch:   301 | d_loss: 0.7260 | g_loss: 1.8245\n    Epoch [    8/   20] | batch:   351 | d_loss: 0.8386 | g_loss: 2.1277\n    Epoch [    8/   20] | batch:   401 | d_loss: 0.9722 | g_loss: 2.4713\n    Epoch [    8/   20] | batch:   451 | d_loss: 0.6980 | g_loss: 2.2593\n    Epoch [    8/   20] | batch:   501 | d_loss: 1.0766 | g_loss: 3.4284\n    Epoch [    8/   20] | batch:   551 | d_loss: 0.9441 | g_loss: 2.4267\n    Epoch [    8/   20] | batch:   601 | d_loss: 0.9615 | g_loss: 1.1553\n    Epoch [    8/   20] | batch:   651 | d_loss: 0.9098 | g_loss: 2.4200\n    Epoch [    8/   20] | batch:   701 | d_loss: 0.8843 | g_loss: 1.3202\n    ---------------Epoch [    8/   20]---------------\n    Epoch [    9/   20] | batch:     1 | d_loss: 0.7078 | g_loss: 1.8200\n    Epoch [    9/   20] | batch:    51 | d_loss: 1.0578 | g_loss: 1.2410\n    Epoch [    9/   20] | batch:   101 | d_loss: 0.8762 | g_loss: 1.8029\n    Epoch [    9/   20] | batch:   151 | d_loss: 0.8846 | g_loss: 1.5970\n    Epoch [    9/   20] | batch:   201 | d_loss: 0.9776 | g_loss: 3.1247\n    Epoch [    9/   20] | batch:   251 | d_loss: 0.7748 | g_loss: 1.2802\n    Epoch [    9/   20] | batch:   301 | d_loss: 0.7308 | g_loss: 2.0371\n    Epoch [    9/   20] | batch:   351 | d_loss: 0.9546 | g_loss: 1.3827\n    Epoch [    9/   20] | batch:   401 | d_loss: 0.9979 | g_loss: 2.7320\n    Epoch [    9/   20] | batch:   451 | d_loss: 0.8290 | g_loss: 1.6430\n    Epoch [    9/   20] | batch:   501 | d_loss: 0.8173 | g_loss: 1.2279\n    Epoch [    9/   20] | batch:   551 | d_loss: 0.7015 | g_loss: 1.6330\n    Epoch [    9/   20] | batch:   601 | d_loss: 1.2070 | g_loss: 2.3610\n    Epoch [    9/   20] | batch:   651 | d_loss: 1.2740 | g_loss: 0.5588\n    Epoch [    9/   20] | batch:   701 | d_loss: 0.8654 | g_loss: 3.2853\n    ---------------Epoch [    9/   20]---------------\n    Epoch [   10/   20] | batch:     1 | d_loss: 1.0506 | g_loss: 1.6972\n    Epoch [   10/   20] | batch:    51 | d_loss: 0.8623 | g_loss: 1.2557\n    Epoch [   10/   20] | batch:   101 | d_loss: 0.7753 | g_loss: 1.7687\n    Epoch [   10/   20] | batch:   151 | d_loss: 0.6656 | g_loss: 1.5877\n    Epoch [   10/   20] | batch:   201 | d_loss: 0.9033 | g_loss: 1.9987\n    Epoch [   10/   20] | batch:   251 | d_loss: 0.8301 | g_loss: 1.5547\n    Epoch [   10/   20] | batch:   301 | d_loss: 0.9235 | g_loss: 2.9890\n    Epoch [   10/   20] | batch:   351 | d_loss: 0.8235 | g_loss: 1.8149\n    Epoch [   10/   20] | batch:   401 | d_loss: 0.6793 | g_loss: 2.3275\n    Epoch [   10/   20] | batch:   451 | d_loss: 0.7816 | g_loss: 3.2800\n    Epoch [   10/   20] | batch:   501 | d_loss: 0.7944 | g_loss: 1.7540\n    Epoch [   10/   20] | batch:   551 | d_loss: 0.9119 | g_loss: 1.2870\n    Epoch [   10/   20] | batch:   601 | d_loss: 1.1382 | g_loss: 2.2872\n    Epoch [   10/   20] | batch:   651 | d_loss: 0.8099 | g_loss: 1.2067\n    Epoch [   10/   20] | batch:   701 | d_loss: 0.6867 | g_loss: 1.9326\n    ---------------Epoch [   10/   20]---------------\n    Epoch [   11/   20] | batch:     1 | d_loss: 0.8320 | g_loss: 1.1754\n    Epoch [   11/   20] | batch:    51 | d_loss: 0.6943 | g_loss: 1.6403\n    Epoch [   11/   20] | batch:   101 | d_loss: 0.9558 | g_loss: 3.0082\n    Epoch [   11/   20] | batch:   151 | d_loss: 0.8052 | g_loss: 2.7568\n    Epoch [   11/   20] | batch:   201 | d_loss: 0.7848 | g_loss: 1.9918\n    Epoch [   11/   20] | batch:   251 | d_loss: 0.8150 | g_loss: 1.4829\n    Epoch [   11/   20] | batch:   301 | d_loss: 0.7332 | g_loss: 2.1810\n    Epoch [   11/   20] | batch:   351 | d_loss: 0.8433 | g_loss: 1.8746\n    Epoch [   11/   20] | batch:   401 | d_loss: 0.6758 | g_loss: 1.3094\n    Epoch [   11/   20] | batch:   451 | d_loss: 0.8459 | g_loss: 1.6590\n    Epoch [   11/   20] | batch:   501 | d_loss: 0.8478 | g_loss: 1.3339\n    Epoch [   11/   20] | batch:   551 | d_loss: 0.8105 | g_loss: 2.5290\n    Epoch [   11/   20] | batch:   601 | d_loss: 0.8847 | g_loss: 2.9163\n    Epoch [   11/   20] | batch:   651 | d_loss: 0.9158 | g_loss: 3.0656\n    Epoch [   11/   20] | batch:   701 | d_loss: 0.7829 | g_loss: 2.7488\n    ---------------Epoch [   11/   20]---------------\n    Epoch [   12/   20] | batch:     1 | d_loss: 0.7352 | g_loss: 1.9948\n    Epoch [   12/   20] | batch:    51 | d_loss: 0.9262 | g_loss: 1.6865\n    Epoch [   12/   20] | batch:   101 | d_loss: 0.8878 | g_loss: 2.6026\n    Epoch [   12/   20] | batch:   151 | d_loss: 0.7256 | g_loss: 1.7047\n    Epoch [   12/   20] | batch:   201 | d_loss: 0.8756 | g_loss: 1.5276\n    Epoch [   12/   20] | batch:   251 | d_loss: 1.1679 | g_loss: 0.8444\n    Epoch [   12/   20] | batch:   301 | d_loss: 0.6936 | g_loss: 2.6643\n    Epoch [   12/   20] | batch:   351 | d_loss: 0.8501 | g_loss: 2.4405\n    Epoch [   12/   20] | batch:   401 | d_loss: 0.7047 | g_loss: 2.2575\n    Epoch [   12/   20] | batch:   451 | d_loss: 0.6888 | g_loss: 2.6093\n    Epoch [   12/   20] | batch:   501 | d_loss: 0.7325 | g_loss: 1.3854\n    Epoch [   12/   20] | batch:   551 | d_loss: 1.0565 | g_loss: 3.9395\n    Epoch [   12/   20] | batch:   601 | d_loss: 0.7159 | g_loss: 3.6967\n    Epoch [   12/   20] | batch:   651 | d_loss: 0.7333 | g_loss: 1.4379\n    Epoch [   12/   20] | batch:   701 | d_loss: 0.7963 | g_loss: 1.4610\n    ---------------Epoch [   12/   20]---------------\n    Epoch [   13/   20] | batch:     1 | d_loss: 0.6865 | g_loss: 2.8950\n    Epoch [   13/   20] | batch:    51 | d_loss: 0.9207 | g_loss: 1.5573\n    Epoch [   13/   20] | batch:   101 | d_loss: 1.2883 | g_loss: 1.4500\n    Epoch [   13/   20] | batch:   151 | d_loss: 0.8684 | g_loss: 2.6025\n    Epoch [   13/   20] | batch:   201 | d_loss: 0.7481 | g_loss: 1.6527\n    Epoch [   13/   20] | batch:   251 | d_loss: 0.8783 | g_loss: 1.3452\n    Epoch [   13/   20] | batch:   301 | d_loss: 0.7375 | g_loss: 2.0968\n    Epoch [   13/   20] | batch:   351 | d_loss: 0.6422 | g_loss: 2.5757\n    Epoch [   13/   20] | batch:   401 | d_loss: 0.7280 | g_loss: 1.3058\n    Epoch [   13/   20] | batch:   451 | d_loss: 0.7422 | g_loss: 1.4346\n    Epoch [   13/   20] | batch:   501 | d_loss: 0.9727 | g_loss: 3.5498\n    Epoch [   13/   20] | batch:   551 | d_loss: 0.6053 | g_loss: 2.1011\n    Epoch [   13/   20] | batch:   601 | d_loss: 1.5844 | g_loss: 0.5962\n    Epoch [   13/   20] | batch:   651 | d_loss: 0.5554 | g_loss: 2.5573\n    Epoch [   13/   20] | batch:   701 | d_loss: 0.6035 | g_loss: 2.5171\n    ---------------Epoch [   13/   20]---------------\n    Epoch [   14/   20] | batch:     1 | d_loss: 0.7339 | g_loss: 3.6615\n    Epoch [   14/   20] | batch:    51 | d_loss: 0.8485 | g_loss: 2.3583\n    Epoch [   14/   20] | batch:   101 | d_loss: 0.6495 | g_loss: 1.8807\n    Epoch [   14/   20] | batch:   151 | d_loss: 0.5651 | g_loss: 1.4821\n    Epoch [   14/   20] | batch:   201 | d_loss: 0.7518 | g_loss: 2.2164\n    Epoch [   14/   20] | batch:   251 | d_loss: 0.7359 | g_loss: 1.7076\n    Epoch [   14/   20] | batch:   301 | d_loss: 1.4818 | g_loss: 3.6353\n    Epoch [   14/   20] | batch:   351 | d_loss: 0.8072 | g_loss: 3.1063\n    Epoch [   14/   20] | batch:   401 | d_loss: 0.6912 | g_loss: 1.5440\n    Epoch [   14/   20] | batch:   451 | d_loss: 0.7894 | g_loss: 1.2850\n    Epoch [   14/   20] | batch:   501 | d_loss: 0.8157 | g_loss: 3.7143\n    Epoch [   14/   20] | batch:   551 | d_loss: 0.8888 | g_loss: 4.6283\n    Epoch [   14/   20] | batch:   601 | d_loss: 0.7662 | g_loss: 1.5938\n    Epoch [   14/   20] | batch:   651 | d_loss: 0.8177 | g_loss: 1.5437\n    Epoch [   14/   20] | batch:   701 | d_loss: 1.0911 | g_loss: 1.2381\n    ---------------Epoch [   14/   20]---------------\n    Epoch [   15/   20] | batch:     1 | d_loss: 0.6332 | g_loss: 2.4432\n    Epoch [   15/   20] | batch:    51 | d_loss: 0.5178 | g_loss: 2.3152\n    Epoch [   15/   20] | batch:   101 | d_loss: 0.5655 | g_loss: 2.4855\n    Epoch [   15/   20] | batch:   151 | d_loss: 1.4360 | g_loss: 4.9096\n    Epoch [   15/   20] | batch:   201 | d_loss: 0.6065 | g_loss: 1.9198\n    Epoch [   15/   20] | batch:   251 | d_loss: 0.6617 | g_loss: 2.7955\n    Epoch [   15/   20] | batch:   301 | d_loss: 0.5642 | g_loss: 2.5870\n    Epoch [   15/   20] | batch:   351 | d_loss: 0.7548 | g_loss: 1.6740\n    Epoch [   15/   20] | batch:   401 | d_loss: 0.7114 | g_loss: 1.2314\n    Epoch [   15/   20] | batch:   451 | d_loss: 0.7264 | g_loss: 2.8277\n    Epoch [   15/   20] | batch:   501 | d_loss: 0.8249 | g_loss: 1.9137\n    Epoch [   15/   20] | batch:   551 | d_loss: 0.6711 | g_loss: 1.9008\n    Epoch [   15/   20] | batch:   601 | d_loss: 0.7163 | g_loss: 1.7807\n    Epoch [   15/   20] | batch:   651 | d_loss: 0.7018 | g_loss: 1.8669\n    Epoch [   15/   20] | batch:   701 | d_loss: 1.2824 | g_loss: 0.8433\n    ---------------Epoch [   15/   20]---------------\n    Epoch [   16/   20] | batch:     1 | d_loss: 0.7795 | g_loss: 1.9912\n    Epoch [   16/   20] | batch:    51 | d_loss: 0.7561 | g_loss: 2.7653\n    Epoch [   16/   20] | batch:   101 | d_loss: 0.9359 | g_loss: 1.5608\n    Epoch [   16/   20] | batch:   151 | d_loss: 0.5931 | g_loss: 2.3988\n    Epoch [   16/   20] | batch:   201 | d_loss: 0.8050 | g_loss: 1.1277\n    Epoch [   16/   20] | batch:   251 | d_loss: 0.4762 | g_loss: 2.5125\n    Epoch [   16/   20] | batch:   301 | d_loss: 0.6604 | g_loss: 2.3707\n    Epoch [   16/   20] | batch:   351 | d_loss: 1.1260 | g_loss: 1.0176\n    Epoch [   16/   20] | batch:   401 | d_loss: 0.5498 | g_loss: 2.5035\n    Epoch [   16/   20] | batch:   451 | d_loss: 0.6702 | g_loss: 2.3546\n    Epoch [   16/   20] | batch:   501 | d_loss: 0.5987 | g_loss: 2.7054\n    Epoch [   16/   20] | batch:   551 | d_loss: 0.6563 | g_loss: 1.9455\n    Epoch [   16/   20] | batch:   601 | d_loss: 0.4369 | g_loss: 2.3982\n    Epoch [   16/   20] | batch:   651 | d_loss: 0.6192 | g_loss: 1.6578\n    Epoch [   16/   20] | batch:   701 | d_loss: 0.8847 | g_loss: 3.6035\n    ---------------Epoch [   16/   20]---------------\n    Epoch [   17/   20] | batch:     1 | d_loss: 0.8319 | g_loss: 1.7466\n    Epoch [   17/   20] | batch:    51 | d_loss: 0.8202 | g_loss: 2.0495\n    Epoch [   17/   20] | batch:   101 | d_loss: 0.9077 | g_loss: 2.2762\n    Epoch [   17/   20] | batch:   151 | d_loss: 0.9110 | g_loss: 1.5152\n    Epoch [   17/   20] | batch:   201 | d_loss: 0.6959 | g_loss: 3.1359\n    Epoch [   17/   20] | batch:   251 | d_loss: 0.6086 | g_loss: 3.1093\n    Epoch [   17/   20] | batch:   301 | d_loss: 0.9669 | g_loss: 3.7573\n    Epoch [   17/   20] | batch:   351 | d_loss: 0.6061 | g_loss: 2.2916\n    Epoch [   17/   20] | batch:   401 | d_loss: 1.3833 | g_loss: 0.8206\n    Epoch [   17/   20] | batch:   451 | d_loss: 0.7337 | g_loss: 1.2652\n    Epoch [   17/   20] | batch:   501 | d_loss: 0.5545 | g_loss: 2.2966\n    Epoch [   17/   20] | batch:   551 | d_loss: 0.4312 | g_loss: 2.7339\n    Epoch [   17/   20] | batch:   601 | d_loss: 1.2815 | g_loss: 4.5698\n    Epoch [   17/   20] | batch:   651 | d_loss: 0.5700 | g_loss: 2.5264\n    Epoch [   17/   20] | batch:   701 | d_loss: 0.6704 | g_loss: 3.2834\n    ---------------Epoch [   17/   20]---------------\n    Epoch [   18/   20] | batch:     1 | d_loss: 0.4970 | g_loss: 2.2410\n    Epoch [   18/   20] | batch:    51 | d_loss: 0.6295 | g_loss: 4.3854\n    Epoch [   18/   20] | batch:   101 | d_loss: 0.6976 | g_loss: 4.3022\n    Epoch [   18/   20] | batch:   151 | d_loss: 1.0637 | g_loss: 2.3074\n    Epoch [   18/   20] | batch:   201 | d_loss: 0.6216 | g_loss: 1.5955\n    Epoch [   18/   20] | batch:   251 | d_loss: 0.6220 | g_loss: 2.0259\n    Epoch [   18/   20] | batch:   301 | d_loss: 0.4838 | g_loss: 2.7805\n    Epoch [   18/   20] | batch:   351 | d_loss: 0.7369 | g_loss: 3.7731\n    Epoch [   18/   20] | batch:   401 | d_loss: 0.6132 | g_loss: 1.4443\n    Epoch [   18/   20] | batch:   451 | d_loss: 0.5376 | g_loss: 3.1501\n    Epoch [   18/   20] | batch:   501 | d_loss: 0.4992 | g_loss: 1.8216\n    Epoch [   18/   20] | batch:   551 | d_loss: 0.9730 | g_loss: 3.3499\n    Epoch [   18/   20] | batch:   601 | d_loss: 0.4792 | g_loss: 2.8586\n    Epoch [   18/   20] | batch:   651 | d_loss: 0.6218 | g_loss: 2.6634\n    Epoch [   18/   20] | batch:   701 | d_loss: 0.6697 | g_loss: 2.0219\n    ---------------Epoch [   18/   20]---------------\n    Epoch [   19/   20] | batch:     1 | d_loss: 0.5197 | g_loss: 1.8495\n    Epoch [   19/   20] | batch:    51 | d_loss: 0.5086 | g_loss: 1.6420\n    Epoch [   19/   20] | batch:   101 | d_loss: 0.6986 | g_loss: 1.1413\n    Epoch [   19/   20] | batch:   151 | d_loss: 0.6020 | g_loss: 1.8165\n    Epoch [   19/   20] | batch:   201 | d_loss: 0.9451 | g_loss: 1.4040\n    Epoch [   19/   20] | batch:   251 | d_loss: 0.4777 | g_loss: 3.1983\n    Epoch [   19/   20] | batch:   301 | d_loss: 0.9183 | g_loss: 4.0350\n    Epoch [   19/   20] | batch:   351 | d_loss: 0.4961 | g_loss: 2.3933\n    Epoch [   19/   20] | batch:   401 | d_loss: 0.8045 | g_loss: 1.3214\n    Epoch [   19/   20] | batch:   451 | d_loss: 0.5569 | g_loss: 2.1318\n    Epoch [   19/   20] | batch:   501 | d_loss: 0.5449 | g_loss: 2.9328\n    Epoch [   19/   20] | batch:   551 | d_loss: 0.5860 | g_loss: 1.8094\n    Epoch [   19/   20] | batch:   601 | d_loss: 0.5079 | g_loss: 1.9517\n    Epoch [   19/   20] | batch:   651 | d_loss: 0.5289 | g_loss: 2.2257\n    Epoch [   19/   20] | batch:   701 | d_loss: 0.5177 | g_loss: 1.4699\n    ---------------Epoch [   19/   20]---------------\n    Epoch [   20/   20] | batch:     1 | d_loss: 0.8772 | g_loss: 3.8187\n    Epoch [   20/   20] | batch:    51 | d_loss: 1.0291 | g_loss: 4.1244\n    Epoch [   20/   20] | batch:   101 | d_loss: 0.4135 | g_loss: 2.1966\n    Epoch [   20/   20] | batch:   151 | d_loss: 0.4319 | g_loss: 2.7543\n    Epoch [   20/   20] | batch:   201 | d_loss: 0.5398 | g_loss: 2.8416\n    Epoch [   20/   20] | batch:   251 | d_loss: 0.4549 | g_loss: 2.5017\n    Epoch [   20/   20] | batch:   301 | d_loss: 0.6065 | g_loss: 2.1791\n    Epoch [   20/   20] | batch:   351 | d_loss: 0.3795 | g_loss: 3.5890\n    Epoch [   20/   20] | batch:   401 | d_loss: 0.6528 | g_loss: 1.6973\n    Epoch [   20/   20] | batch:   451 | d_loss: 0.5359 | g_loss: 1.7315\n    Epoch [   20/   20] | batch:   501 | d_loss: 0.3617 | g_loss: 3.3402\n    Epoch [   20/   20] | batch:   551 | d_loss: 0.5715 | g_loss: 2.3122\n    Epoch [   20/   20] | batch:   601 | d_loss: 0.5038 | g_loss: 2.7294\n    Epoch [   20/   20] | batch:   651 | d_loss: 0.4265 | g_loss: 2.0100\n    Epoch [   20/   20] | batch:   701 | d_loss: 0.5678 | g_loss: 1.9734\n    ---------------Epoch [   20/   20]---------------\n\n\n## Training loss\n\nPlot the training losses for the generator and discriminator, recorded after each epoch.\n\n\n```python\nfig, ax = plt.subplots()\nlosses = np.array(losses)\nplt.plot(losses.T[0], label='Discriminator', alpha=0.5)\nplt.plot(losses.T[1], label='Generator', alpha=0.5)\nplt.title(\"Training Losses\")\nplt.legend()\n```\n\n\n\n\n    \u003cmatplotlib.legend.Legend at 0x7f13f4acc128\u003e\n\n\n\n\n![png](assets/output_40_1.png)\n\n\n## Generator samples from training\n\nView samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.\n\n\n```python\n# helper function for viewing a list of passed in sample images\ndef view_samples(epoch, samples):\n    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)\n    for ax, img in zip(axes.flatten(), samples[epoch]):\n        img = img.detach().cpu().numpy()\n        img = np.transpose(img, (1, 2, 0))\n        img = ((img + 1)*255 / (2)).astype(np.uint8)\n        ax.xaxis.set_visible(False)\n        ax.yaxis.set_visible(False)\n        im = ax.imshow(img.reshape((32,32,3)))\n```\n\n\n```python\n# Load samples from generator, taken while training\nwith open('train_samples.pkl', 'rb') as f:\n    samples = pkl.load(f)\n```\n\n\n```python\nview_samples(-1, samples)\n```\n\n\n![png](assets/output_44_0.png)\n\n\n### Question: What do you notice about your generated samples and how might you improve this model?\nWhen you answer this question, consider the following factors:\n* The dataset is biased; it is made of \"celebrity\" faces that are mostly white\n* Model size; larger models have the opportunity to learn more features in a data feature space\n* Optimization strategy; optimizers and number of epochs affect your final result\n\n\n**Answer:** \nThe training data does not have the complete face. Features like chins are not visible. As a results, the generated images miss chins. Data should preferably have the complete face.\n\nObtaining higher resolution images. I would like to study NVIDIA's face generation paper where they trained HD faces and can even merge two faces to crate a new one.\n\nA larger model size with more hidden dimensions would work better but take more training time\n\nTraining for longer will be ueful as the discriminator loss is steadily decreasing anf the generator loss curve is jagged indicating, it is imagining new features well, many of which are incorrect, but that's a step in the right direction.\n\n### Submitting This Project\nWhen submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as \"dlnd_face_generation.ipynb\" and save it as a HTML file under \"File\" -\u003e \"Download as\". Include the \"problem_unittests.py\" files in your submission.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphreakyphoenix%2Fface-generation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphreakyphoenix%2Fface-generation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphreakyphoenix%2Fface-generation/lists"}