{"id":15158280,"url":"https://github.com/fanisgl/cnn_machine_learning","last_synced_at":"2026-02-24T07:15:55.492Z","repository":{"id":255027820,"uuid":"848317651","full_name":"FanisGl/CNN_Machine_Learning","owner":"FanisGl","description":"A Machine Learning exercise that trains Convolutional Neural Network (CNN) using the tensorflow 2 and Keras libraries to predict images from the CIFAR-10 dataset. ","archived":false,"fork":false,"pushed_at":"2024-08-29T14:56:01.000Z","size":348,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-13T18:21:31.423Z","etag":null,"topics":["data-science","keras","machine-learning","neural-networks","numpy","pandas","pooling-layers","relu","softmax","tensorflow2"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FanisGl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-27T14:41:01.000Z","updated_at":"2024-08-29T14:56:04.000Z","dependencies_parsed_at":"2024-09-26T20:43:37.255Z","dependency_job_id":"4e628fbc-7543-41eb-86b8-44328dc43bba","html_url":"https://github.com/FanisGl/CNN_Machine_Learning","commit_stats":{"total_commits":5,"total_committers":1,"mean_commits":5.0,"dds":0.0,"last_synced_commit":"927bc07e14cc7feb1d81adead684842cb776809b"},"previous_names":["fanisgl/cnn_machine_learning"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FanisGl%2FCNN_Machine_Learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FanisGl%2FCNN_Machine_Learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FanisGl%2FCNN_Machine_Learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FanisGl%2FCNN_Machine_Learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FanisGl","download_url":"https://codeload.github.com/FanisGl/CNN_Machine_Learning/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247684480,"owners_count":20979072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","keras","machine-learning","neural-networks","numpy","pandas","pooling-layers","relu","softmax","tensorflow2"],"created_at":"2024-09-26T20:42:13.318Z","updated_at":"2025-10-29T12:46:55.645Z","avatar_url":"https://github.com/FanisGl.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Convolutional Neural Networks (CNNs) - Coloured Image Data\r\n\r\n\u003e Tags: [Data Science], [Machine Learning]\r\n\u003e\r\n\u003e Technical Skills: [Python], [Numpy], [Pandas], [Tensorflow 2], [Keras], [API]\r\n\u003e\r\n\u003e Theoretical Frameworks: [Neural Networks], [Convolutions], [Pooling Layers]\r\n\r\n\u003e [!NOTE]\r\n\u003e\r\n\u003eThis exercise comes directly from Jose Portilla and his team's fantastic course in Udemy of [Complete Tensorflow 2 and Keras Deep Learning Bootcamp](https://www.udemy.com/course/complete-tensorflow-2-and-keras-deep-learning-bootcamp/).\r\n\u003e\r\n\u003e This was part of the [Workearly](https://www.workearly.gr) bootcamp.\r\n\r\nLoading the CIFAR-10 Dataset through Keras API, which consists 32 by 32 coloured images of 10 different objects, and then creating a Machine Learning (CNN) model using Tensorflow 2 that attempts to predict a given image.\r\n\r\n## 1. Importing the dataset \u0026 model\r\n\r\n\r\n```python\r\nimport pandas as pd\r\nimport numpy as np\r\nimport matplotlib.pyplot as plt\r\n%matplotlib inline\r\n```\r\n\r\n\r\n```python\r\nfrom tensorflow.keras.datasets import cifar10\r\n```\r\n\r\n\r\n```python\r\n(x_train,y_train),(x_test,y_test) = cifar10.load_data()\r\n```\r\n\r\n### Checking the properties of the dataset and the image data\r\n\r\n\r\n```python\r\nx_train.shape\r\n```\r\n\r\n\r\n\r\n\r\n    (50000, 32, 32, 3)\r\n\r\n\r\n\r\nThis tells us that there are 50000 pictures, 32x32 pixels, with 3 colour channels (RGB).\r\n\r\n\r\n```python\r\nx_train[0]\r\n```\r\n\r\n\r\n\r\n\r\n    array([[[ 59,  62,  63],\r\n            [ 43,  46,  45],\r\n            [ 50,  48,  43],\r\n            ...,\r\n            [158, 132, 108],\r\n            [152, 125, 102],\r\n            [148, 124, 103]],\r\n    \r\n           [[ 16,  20,  20],\r\n            [  0,   0,   0],\r\n            [ 18,   8,   0],\r\n            ...,\r\n            [123,  88,  55],\r\n            [119,  83,  50],\r\n            [122,  87,  57]],\r\n    \r\n           [[ 25,  24,  21],\r\n            [ 16,   7,   0],\r\n            [ 49,  27,   8],\r\n            ...,\r\n            [118,  84,  50],\r\n            [120,  84,  50],\r\n            [109,  73,  42]],\r\n    \r\n           ...,\r\n    \r\n           [[208, 170,  96],\r\n            [201, 153,  34],\r\n            [198, 161,  26],\r\n            ...,\r\n            [160, 133,  70],\r\n            [ 56,  31,   7],\r\n            [ 53,  34,  20]],\r\n    \r\n           [[180, 139,  96],\r\n            [173, 123,  42],\r\n            [186, 144,  30],\r\n            ...,\r\n            [184, 148,  94],\r\n            [ 97,  62,  34],\r\n            [ 83,  53,  34]],\r\n    \r\n           [[177, 144, 116],\r\n            [168, 129,  94],\r\n            [179, 142,  87],\r\n            ...,\r\n            [216, 184, 140],\r\n            [151, 118,  84],\r\n            [123,  92,  72]]], dtype=uint8)\r\n\r\n\r\n\r\n\r\n```python\r\nx_train[0].shape\r\n```\r\n\r\n\r\n\r\n\r\n    (32, 32, 3)\r\n\r\n\r\n\r\n[0] is a picture indexed 0, which we can see when we \".shape\" by having 32x32 pixels and 3 colour channels. If we are interested about the consistency of the pixels of the image data, it can be observed on its raw form under the result of \"x_train[0].\" The arrays are combinations of values of colours that make up what we say 32 by 32.\r\n\r\nTo be more practical, let's show that picture indexed [0] below.\r\n\r\n\r\n```python\r\nplt.imshow(x_train[0])\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cmatplotlib.image.AxesImage at 0x1fa22d32e50\u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_12_1.png)\r\n    \r\n\r\n\r\nIt's a blurred frog! :)\r\n\r\nThis can be done with any given number between [0-49999]. \r\n\r\n\r\n```python\r\nplt.imshow(x_train[49999])\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cmatplotlib.image.AxesImage at 0x1fa262117f0\u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_14_1.png)\r\n    \r\n\r\n\r\n\r\n```python\r\nplt.imshow(x_train[3567])\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cmatplotlib.image.AxesImage at 0x1fa2659f340\u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_15_1.png)\r\n    \r\n\r\n\r\nAnd so on!\r\n\r\n## Data Preprocessing\r\n\r\n\r\n```python\r\nx_train[0].max()\r\n```\r\n\r\n\r\n\r\n\r\n    255\r\n\r\n\r\n\r\nThis tells us that indeed, the maximum value that a picture can take for any given variable/value within an array is 255, and therefore has the range of [0-255].\r\n\r\nWith that logic we proceed to scale the model on all it's dimensions by 255.\r\n\r\n\r\n```python\r\nx_train = x_train/255\r\n```\r\n\r\n\r\n```python\r\nx_test = x_test/255\r\n```\r\n\r\n\r\n```python\r\nx_test.shape\r\n```\r\n\r\n\r\n\r\n\r\n    (10000, 32, 32, 3)\r\n\r\n\r\n\r\n\r\n```python\r\ny_train\r\n```\r\n\r\n\r\n\r\n\r\n    array([[6],\r\n           [9],\r\n           [9],\r\n           ...,\r\n           [9],\r\n           [1],\r\n           [1]], dtype=uint8)\r\n\r\n\r\n\r\nThe array labels work as continuous values in the network, and if left as such then the CNN would try to predict in a numerical way, which would not be very helpful as it would name a predicted image as '[6]' instead of 'Frog' or 'Car'.\r\n\r\nSo we need to tackle this as a multi-class classification problem, and one-hot encode it. Technically, converting the labels of the array into categorical data.\r\n\r\n\r\n```python\r\nfrom tensorflow.keras.utils import to_categorical\r\n```\r\n\r\n\r\n```python\r\ny_cat_train = to_categorical(y_train,10)\r\n```\r\n\r\n\r\n```python\r\ny_cat_test = to_categorical(y_test,10)\r\n```\r\n\r\nMaking sure that 10 classes are specified within the mapping of the categorical encoding.\r\n\r\nTo clarify the logic above, within the CIFAR-10 dataset, we know for a fact (by simply [googling](https://www.google.com/search?q=cifar10+label+number+6\u0026client=opera\u0026hs=6y0\u0026sca_esv=acb05f42373aaad6\u0026sca_upv=1\u0026ei=M9fNZvmqDcCrxc8Pz4qTyQw\u0026ved=0ahUKEwi5kMWHppWIAxXAVfEDHU_FJMkQ4dUDCA8\u0026uact=5\u0026oq=cifar10+label+number+6\u0026gs_lp=Egxnd3Mtd2l6LXNlcnAiFmNpZmFyMTAgbGFiZWwgbnVtYmVyIDYyBRAhGKABMgUQIRigAUiEIFC6BFj3HnAGeAGQAQCYAWmgAc8IqgEEMTQuMbgBA8gBAPgBAZgCFaAC9gjCAgoQABiwAxjWBBhHwgIGEAAYFhgewgILEAAYgAQYhgMYigXCAggQABiABBiiBMICCxAAGIAEGJECGIoFwgIFEAAYgATCAgcQIRigARgKmAMAiAYBkAYIkgcEMjAuMaAHsCg\u0026sclient=gws-wiz-serp) it) that frog's label is mapped to 6, and that can be seen when we evaluate the array for image indexed [0]\r\n\r\n\r\n```python\r\ny_train[0]\r\n```\r\n\r\n\r\n\r\n\r\n    array([6], dtype=uint8)\r\n\r\n\r\n\r\n\r\n```python\r\nplt.imshow(x_train[0])\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cmatplotlib.image.AxesImage at 0x1fa2719fee0\u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_31_1.png)\r\n    \r\n\r\n\r\n## Building the ML (CNN) model\r\n\r\n\r\n```python\r\nfrom tensorflow.keras.models import Sequential\r\nfrom tensorflow.keras.layers import Dense, Conv2D, MaxPool2D, Flatten\r\n```\r\n\r\n\r\n```python\r\nmodel = Sequential()\r\n\r\n# Convolutional Layer (1)\r\nmodel.add(Conv2D(filters=32, kernel_size=(4,4), input_shape=(32,32,3), activation='relu',))\r\n\r\n# Pooling Layer (1)\r\nmodel.add(MaxPool2D(pool_size=(2,2)))\r\n\r\n# Convolutional Layer (2)\r\nmodel.add(Conv2D(filters=32, kernel_size=(4,4), input_shape=(32,32,3), activation='relu',))\r\n\r\n# Pooling Layer (2)\r\nmodel.add(MaxPool2D(pool_size=(2,2)))\r\n\r\nmodel.add(Flatten())\r\n\r\nmodel.add(Dense(256, activation='relu'))\r\n\r\nmodel.add(Dense(10, activation='softmax'))\r\n\r\nmodel.compile(loss='categorical_crossentropy',optimizer='adam',\r\n              metrics=['accuracy'])\r\n```\r\n\r\n    B:\\Anaconda\\envs\\mytfenv\\lib\\site-packages\\keras\\src\\layers\\convolutional\\base_conv.py:107: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\r\n      super().__init__(activity_regularizer=activity_regularizer, **kwargs)\r\n    \r\n\r\nThe raison d'etre of the two convolutional layers and the two pooling layers, is due to the complexity of the images. The Data load that the model is called to process, is 32*32*3 = 3072.\r\n\r\n'relu' refers to *rectified linear unit*, an activation function that helps incorporate nonlinearity in neural networks, and additionally helps tackle the vanishing gradient problem.\r\n\r\n'softmax' because we are dealing with a multiclass problem\r\n\r\nAlso, we will ignore the UserWarning I was just given, because I embrace chaos.\r\n\r\n\r\n```python\r\nmodel.summary()\r\n```\r\n\r\n\r\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"font-weight: bold\"\u003eModel: \"sequential\"\u003c/span\u003e\r\n\u003c/pre\u003e\r\n\r\n\r\n\r\n\r\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\r\n┃\u003cspan style=\"font-weight: bold\"\u003e Layer (type)                         \u003c/span\u003e┃\u003cspan style=\"font-weight: bold\"\u003e Output Shape                \u003c/span\u003e┃\u003cspan style=\"font-weight: bold\"\u003e         Param # \u003c/span\u003e┃\r\n┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\r\n│ conv2d (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eConv2D\u003c/span\u003e)                      │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e29\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e29\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e32\u003c/span\u003e)          │           \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e1,568\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ max_pooling2d (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eMaxPooling2D\u003c/span\u003e)         │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e14\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e14\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e32\u003c/span\u003e)          │               \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e0\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ conv2d_1 (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eConv2D\u003c/span\u003e)                    │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e11\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e11\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e32\u003c/span\u003e)          │          \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e16,416\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ max_pooling2d_1 (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eMaxPooling2D\u003c/span\u003e)       │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e5\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e5\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e32\u003c/span\u003e)            │               \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e0\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ flatten (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eFlatten\u003c/span\u003e)                    │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e800\u003c/span\u003e)                 │               \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e0\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ dense (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eDense\u003c/span\u003e)                        │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e256\u003c/span\u003e)                 │         \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e205,056\u003c/span\u003e │\r\n├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\r\n│ dense_1 (\u003cspan style=\"color: #0087ff; text-decoration-color: #0087ff\"\u003eDense\u003c/span\u003e)                      │ (\u003cspan style=\"color: #00d7ff; text-decoration-color: #00d7ff\"\u003eNone\u003c/span\u003e, \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e10\u003c/span\u003e)                  │           \u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e2,570\u003c/span\u003e │\r\n└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘\r\n\u003c/pre\u003e\r\n\r\n\r\n\r\n\r\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"font-weight: bold\"\u003e Total params: \u003c/span\u003e\u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e225,610\u003c/span\u003e (881.29 KB)\r\n\u003c/pre\u003e\r\n\r\n\r\n\r\n\r\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"font-weight: bold\"\u003e Trainable params: \u003c/span\u003e\u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e225,610\u003c/span\u003e (881.29 KB)\r\n\u003c/pre\u003e\r\n\r\n\r\n\r\n\r\n\u003cpre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"\u003e\u003cspan style=\"font-weight: bold\"\u003e Non-trainable params: \u003c/span\u003e\u003cspan style=\"color: #00af00; text-decoration-color: #00af00\"\u003e0\u003c/span\u003e (0.00 B)\r\n\u003c/pre\u003e\r\n\r\n\r\n\r\n### Creating an early stop\r\n\r\n\r\n```python\r\nfrom tensorflow.keras.callbacks import EarlyStopping\r\n```\r\n\r\n\r\n```python\r\nearly_stop = EarlyStopping(monitor='val_loss',patience=2)\r\n```\r\n\r\n### Fitting the train data\r\n\r\n\r\n```python\r\nmodel.fit(x_train,y_cat_train,epochs=15,\r\n         validation_data=(x_test,y_cat_test),callbacks=[early_stop])\r\n```\r\n\r\n    Epoch 1/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m12s\u001b[0m 7ms/step - accuracy: 0.3761 - loss: 1.7052 - val_accuracy: 0.5249 - val_loss: 1.3182\r\n    Epoch 2/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m10s\u001b[0m 7ms/step - accuracy: 0.5628 - loss: 1.2429 - val_accuracy: 0.5923 - val_loss: 1.1439\r\n    Epoch 3/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m10s\u001b[0m 7ms/step - accuracy: 0.6254 - loss: 1.0659 - val_accuracy: 0.6133 - val_loss: 1.0917\r\n    Epoch 4/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m12s\u001b[0m 8ms/step - accuracy: 0.6695 - loss: 0.9525 - val_accuracy: 0.6474 - val_loss: 1.0103\r\n    Epoch 5/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 7ms/step - accuracy: 0.6951 - loss: 0.8712 - val_accuracy: 0.6529 - val_loss: 1.0017\r\n    Epoch 6/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 7ms/step - accuracy: 0.7263 - loss: 0.7795 - val_accuracy: 0.6616 - val_loss: 0.9860\r\n    Epoch 7/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m9s\u001b[0m 6ms/step - accuracy: 0.7580 - loss: 0.6857 - val_accuracy: 0.6680 - val_loss: 0.9821\r\n    Epoch 8/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m9s\u001b[0m 6ms/step - accuracy: 0.7782 - loss: 0.6341 - val_accuracy: 0.6735 - val_loss: 1.0034\r\n    Epoch 9/15\r\n    \u001b[1m1563/1563\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m9s\u001b[0m 6ms/step - accuracy: 0.8031 - loss: 0.5684 - val_accuracy: 0.6684 - val_loss: 1.0446\r\n    \r\n\r\n\r\n\r\n\r\n    \u003ckeras.src.callbacks.history.History at 0x1fa272d5cd0\u003e\r\n\r\n\r\n\r\nDefault accuracy should be 10% based on the dataset, but as epochs go by, we observe the model becomes better and better at accurately predicting.\r\n\r\nThe early_stop has finished on 9 epochs, and has returned an accuracy of 80.31% as of the 9th epoch.\r\n\r\n## Evaluating the model\r\n\r\n\r\n```python\r\nmetrics = pd.DataFrame(model.history.history)\r\n```\r\n\r\n\r\n```python\r\nmetrics.columns\r\n```\r\n\r\n\r\n\r\n\r\n    Index(['accuracy', 'loss', 'val_accuracy', 'val_loss'], dtype='object')\r\n\r\n\r\n\r\n\r\n```python\r\nmetrics[['accuracy', 'val_accuracy']].plot()\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cAxes: \u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_46_1.png)\r\n    \r\n\r\n\r\n\r\n```python\r\nmetrics[['loss', 'val_loss']].plot()\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cAxes: \u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_47_1.png)\r\n    \r\n\r\n\r\nIt seems validation loss has started going up since the 6th epoch, so the early stop was correct to stop on the 8th epoch. The trade off for the added accuracy, was desirable.\r\n\r\n\r\n```python\r\nmodel.evaluate(x_test,y_cat_test,verbose=0)\r\n```\r\n\r\n\r\n\r\n\r\n    [1.0445741415023804, 0.66839998960495]\r\n\r\n\r\n\r\n\r\n```python\r\nfrom sklearn.metrics import classification_report,confusion_matrix\r\n```\r\n\r\n\r\n```python\r\npredicted_classes = np.argmax(predictions, axis=1)\r\n```\r\n\r\n\r\n```python\r\nprint(classification_report(y_test, predicted_classes))\r\n```\r\n\r\n                  precision    recall  f1-score   support\r\n    \r\n               0       0.73      0.70      0.72      1000\r\n               1       0.77      0.80      0.78      1000\r\n               2       0.59      0.56      0.57      1000\r\n               3       0.46      0.49      0.47      1000\r\n               4       0.67      0.56      0.61      1000\r\n               5       0.53      0.57      0.55      1000\r\n               6       0.75      0.74      0.74      1000\r\n               7       0.74      0.69      0.72      1000\r\n               8       0.77      0.79      0.78      1000\r\n               9       0.69      0.79      0.74      1000\r\n    \r\n        accuracy                           0.67     10000\r\n       macro avg       0.67      0.67      0.67     10000\r\n    weighted avg       0.67      0.67      0.67     10000\r\n    \r\n    \r\n\r\nThe f1-score for accuracy is 67%. Since this data set has 10 classes, then as mentioned earlier, the default prediction should be at 10%, meaning that if you took a random guess, you would have 10% chance to be correct about the category of the image. Therefore, 67% is a good overall performance in accuracy.\r\n\r\nInterestingly so, the model we just created, tends to not perform well for the label: 3, which as seen in [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html) it's cats. Funnily enough, we observe the same with label: 5 which is dogs, with f1-scores being very closely aligned (0.47 for label: 3 and 0.55 for label: 5). This could be explained by how limited the quality of the pixels is on 32x32 on depicting a dog or a cat and the computer being able to tell the difference.\r\n\r\nLet's test some predictions.\r\n\r\n\r\n```python\r\nconfusion_matrix(y_test,predicted_classes)\r\n```\r\n\r\n\r\n\r\n\r\n    array([[704,  31,  63,  19,  21,   7,  12,   9,  77,  57],\r\n           [ 28, 795,  11,   7,   2,   4,  11,   3,  31, 108],\r\n           [ 63,  12, 560,  88,  59,  85,  62,  30,  22,  19],\r\n           [ 17,  19,  71, 487,  48, 217,  53,  41,  20,  27],\r\n           [ 18,   6,  92,  88, 558,  62,  67,  74,  22,  13],\r\n           [ 18,   8,  55, 196,  37, 566,  29,  58,  16,  17],\r\n           [  6,  14,  47,  87,  27,  33, 742,  10,  17,  17],\r\n           [ 21,  13,  29,  53,  64,  74,   8, 694,   5,  39],\r\n           [ 63,  44,  15,  13,   6,   8,   3,   7, 791,  50],\r\n           [ 26,  92,   9,  20,   6,  14,   8,   9,  29, 787]], dtype=int64)\r\n\r\n\r\n\r\n\r\n```python\r\nimport seaborn as sns\r\n\r\nplt.figure(figsize=(10,6))\r\nsns.heatmap(confusion_matrix(y_test,predicted_classes),annot=True)\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cAxes: \u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_55_1.png)\r\n    \r\n\r\n\r\nBased on the colouring here, we see indeed that pairs (3,5) and (5,3) which is (cats,dogs) and (dogs,cats) are indeed troublesome as they are more closely correlated by the computer.\r\n\r\nTo clarify. the lighter the colour of a block (towards 700), it means that the machine predicts that label to be the matching label. For example, all predictions made for pictures labeled 4, the machine likely classified it as 4 indeed, and that's why we see a strong light colour in (4,4) but in any other combination with 4, it is dark (below 100), as it finds no correlation to the other pair.\r\n\r\nWhen we see labels 3 and 5, we see that not only do they get a higher hue in their respective classes (3,3), (5,5) but also that they are a scoring a bit more than 200 in combinations (3,5) and (5,3) meaning the machine is indeed confusing the two categories when it tries to predict.\r\n\r\n\r\n```python\r\npred_image = x_test[0]\r\n```\r\n\r\n\r\n```python\r\nplt.imshow(pred_image)\r\n```\r\n\r\n\r\n\r\n\r\n    \u003cmatplotlib.image.AxesImage at 0x1fa4a24bd90\u003e\r\n\r\n\r\n\r\n\r\n    \r\n![png](README_files/README_58_1.png)\r\n    \r\n\r\n\r\n\r\n```python\r\ny_test[0]\r\n```\r\n\r\n\r\n\r\n\r\n    array([3], dtype=uint8)\r\n\r\n\r\n\r\n\r\n```python\r\nnp.argmax(model.predict(pred_image.reshape(1,32,32,3)), axis=-1)\r\n```\r\n\r\n    \u001b[1m1/1\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 15ms/step\r\n    \r\n\r\n\r\n\r\n\r\n    array([3], dtype=int64)\r\n\r\n\r\n\r\nAnd there we go! \r\n\r\nWe took a picture in our test model indexed [0] which is a bit *dubious* as we can see under the image show line, and using the prediction model results to \"array[3]\" which is the label number for cat, and indeed the picture we see is that of a cat!\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffanisgl%2Fcnn_machine_learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffanisgl%2Fcnn_machine_learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffanisgl%2Fcnn_machine_learning/lists"}