{"id":19068992,"url":"https://github.com/kmohamedalie/dtm-machine_learning_final_project","last_synced_at":"2026-05-15T01:05:56.802Z","repository":{"id":109207331,"uuid":"603708502","full_name":"Kmohamedalie/DTM-Machine_Learning_final_Project","owner":"Kmohamedalie","description":"Machine learning final project ","archived":false,"fork":false,"pushed_at":"2023-08-20T11:45:09.000Z","size":2035,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-02T15:32:06.864Z","etag":null,"topics":["cifar-10","classification","cnn","computer-vision","convolutional-neural-networks","deep-learning","deep-neural-networks","hog-features","image-classification","keras-tensorflow","machine-learning","svm"],"latest_commit_sha":null,"homepage":"https://kmohamedalie.github.io/DTM-Machine_Learning_final_Project/","language":"CSS","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kmohamedalie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-19T10:49:52.000Z","updated_at":"2024-04-01T16:07:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"f28af9fb-d008-4fd4-8138-34877064f345","html_url":"https://github.com/Kmohamedalie/DTM-Machine_Learning_final_Project","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kmohamedalie%2FDTM-Machine_Learning_final_Project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kmohamedalie%2FDTM-Machine_Learning_final_Project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kmohamedalie%2FDTM-Machine_Learning_final_Project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kmohamedalie%2FDTM-Machine_Learning_final_Project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kmohamedalie","download_url":"https://codeload.github.com/Kmohamedalie/DTM-Machine_Learning_final_Project/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240122476,"owners_count":19751140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar-10","classification","cnn","computer-vision","convolutional-neural-networks","deep-learning","deep-neural-networks","hog-features","image-classification","keras-tensorflow","machine-learning","svm"],"created_at":"2024-11-09T01:12:56.709Z","updated_at":"2026-05-15T01:05:56.754Z","avatar_url":"https://github.com/Kmohamedalie.png","language":"CSS","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Machine Learning vs Deep Learning on Image Recognition\n\u003cp style=\"text-align: center;\"\u003eEkin Yilmaz, Leonardo Ghiani, Mohamed Alie Kamara \u003c/p\u003e \u003cbr/\u003e\n\u003cp style=\"text-align: center;\"\u003eInstructor: \u003ca href=\"https://www.unibo.it/sitoweb/guido.borghi\"\u003eGuido Borghi\u003c/a\u003e \u003c/p\u003e\n\u003cp style=\"text-align: center;\"\u003e\u003ca href=\"mailto:guido.borghi@unibo.it\"\u003eEmail: guido.borghi@unibo.it\u003c/a\u003e\u003c/p\u003e\n\n### Practical Demo Websites\n\u003ca href=\"https://cs231n.github.io/convolutional-networks/\"\u003eConvolution Demo\u003c/a\u003e\u003cbr/\u003e\n\u003ca href=\"https://playground.tensorflow.org/#activation=relu\u0026batchSize=10\u0026dataset=circle\u0026regDataset=reg-plane\u0026learningRate=0.03\u0026regularizationRate=0\u0026noise=0\u0026networkShape=4,2\u0026seed=0.25169\u0026showTestData=false\u0026discretize=false\u0026percTrainData=50\u0026x=true\u0026y=true\u0026xTimesY=false\u0026xSquared=false\u0026ySquared=false\u0026cosX=false\u0026sinX=false\u0026cosY=false\u0026sinY=false\u0026collectStats=false\u0026problem=classification\u0026initZero=false\u0026hideText=false\"\u003eTensorflow Neural Network playground\u003c/a\u003e\u003cbr/\u003e\n \u003ca href=\"https://adamharley.com/nn_vis/cnn/2d.html\"\u003e2D fully-connected network visualization\u003c/a\u003e\u003cbr/\u003e\n \u003ca href=\"https://deeplizard.com/resource/pavq7noze3\"\u003eMax Pooling Operation in Neural Networks\u003c/a\u003e\n \n\n### Keywords\nComputer vision; machine learning; deep learning; feature descriptor; \u003ca href=\"https://github.com/topics/hog-features-extraction\"\u003ehog\u003c/a\u003e; \u003ca href=\"https://www.cs.toronto.edu/~kriz/cifar.html\"\u003ecifar-10\u003c/a\u003e; \u003ca href=\"https://en.wikipedia.org/wiki/MNIST_database\"\u003eMnist\u003c/a\u003e; classification; \u003ca href=\"https://scikit-image.org/\"\u003escikit-image\u003c/a\u003e; \u003ca href=\"https://tqdm.github.io/\"\u003etqdm\u003c/a\u003e; \u003ca href=\"https://opencv.org/\"\u003ecv2\u003c/a\u003e; \u003ca href=\"https://www.ibm.com/topics/convolutional-neural-networks\"\u003econvolutional neural networks\u003c/a\u003e; \u003ca href=\"https://it.wikipedia.org/wiki/Macchine_a_vettori_di_supporto\"\u003eSVM\u003c/a\u003e.\n\n### 1. Introduction\n   Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos, and other visual inputs - and take actions or make recommendations based on that information. Computer vision enables machines to see, observe and understand, while AI enables computers to think [1].\n    In this paper we will do a comparative analysis of image recognition between machine learning using feature descriptions and support vector machine \u003ca href=\"https://scikit-learn.org/stable/modules/svm.html\"\u003e(svm)\u003c/a\u003e Classifier vs deep learning using convolutional neural networks \u003ca href=\"https://insightsimaging.springeropen.com/articles/10.1007/s13244-018-0639-9#:~:text=CNN%20is%20a%20type%20of,%2D%20to%20high%2Dlevel%20patterns\"\u003e(CNN)\u003c/a\u003e, on CIFAR-10 dataset. The comparison will be based on the following aspects: data, accuracy, training time, hardware, features, interpretability.\n\n\u003cdiv align=\"center\"\u003e\n\n![image](https://user-images.githubusercontent.com/63104472/219057648-0096a8cb-2048-4d14-9d9c-a79ed146da70.png)\n\n\u003c/div\u003e\n\nFigure 1: Machine learning vs deep learning    source: \u003ca href=\"https://www.projectpro.io/article/deep-learning-vs-machine-learning-whats-the-difference/414#toc-5\"\u003eprojectpro\u003c/a\u003e\n\n\na. CIFAR-10 Description\n    The CIFAR-10 dataset (Canadian Institute for Advanced Research, 10 classes) is a subset of \nthe Tiny Images dataset and consists of 60000 32x32 color images. The images are labelled with\none of 10 mutually exclusive classes: airplane, automobile (but not truck or pickup truck), bird,\ncat, deer, dog, frog, horse, ship, and truck (but not pickup truck). There are 6000 images per class\nwith 5000 training and 1000 testing images per class [2]. \n    The \u003ca href=\"https://www.cs.toronto.edu/~kriz/cifar.html\"\u003edata\u003c/a\u003e provided by the university of Toronto is already pre-processed and stored in batch files [3] however we will use raw images rather than already pre-processed in our analysis which we downloaded from \u003ca href=\"https://www.kaggle.com/datasets/oxcdcd/cifar10\"\u003eKaggle [4]\u003c/a\u003e.\n\n\u003cdiv align=\"center\"\u003e\n\n![image](https://user-images.githubusercontent.com/63104472/219051600-89dd5ffa-ef60-4705-8b28-0c614ae90826.png)\n\n\u003c/div\u003e\n\nFigure 2: Cifar-10 images                         source: \u003ca href=\"https://www.cs.toronto.edu/~kriz/cifar.html\"\u003eCIFAR-10\u003c/a\u003e\n\n\n### 2. Related Work\na. \u003ca href=\"https://iopscience.iop.org/article/10.1088/1742-6596/1314/1/012148\"\u003eA Comparison of Traditional Machine Learning and Deep Learning in Image Recognition\u003c/a\u003e by \nYunfei Lai. In Lai’s paper he made a comparative analysis between old traditional machine learning technique using \u003ca href=\"https://it.wikipedia.org/wiki/Vladimir_Vapnik\"\u003e \"Vapnik’s\u003c/a\u003e svm (RBF (Radial Basis Function)) ‘kernel trick’ on \u003ca href=\"https://it.wikipedia.org/wiki/Yann_LeCun\"\u003eYann LeCun’s\u003c/a\u003e famous \u003ca href=\"https://www.tensorflow.org/datasets/catalog/mnist?hl=it\"\u003eMnist dataset\u003c/a\u003e, which is a large dataset black and white images (one channel) of handwritten digits that is commonly employed as training and testing set in the field of machine learning, and deep learning consisting of three-layer convolution neural shown in figure 3. \n\n\u003cdiv align=\"center\"\u003e\n\n![image](https://user-images.githubusercontent.com/63104472/219052056-b895ebd7-08da-4139-8e9c-aa6429ae5658.png)\n\n\u003c/div\u003e\n\nFigure 3: Yunfei Lai CNN network                                 source:\u003ca href=\"https://iopscience.iop.org/article/10.1088/1742-6596/1314/1/012148\"\u003eiopscience\u003c/a\u003e\n\nThe overall result achieved in the testing set using SVM is 93.92% total accuracy and 98.85% using CNN, clearly demonstrating that deep learning is more suitable for modelling of image data because of its ability to extract features from two-dimensional image data automatically, unlike traditional machine learning methods which need subjective feature extraction to convert binary vectors into one-dimensional vectors.    \n    However, Yunfei also take into consideration that more training time and resources are need for deep learning, but better prediction accuracy and generalization can be achieved [5]. \nFigure 4 shows detailed summary of Yunfei Lai’s experiment. \n\n\n\u003cdiv align=\"center\"\u003e\n\n ![image](https://user-images.githubusercontent.com/63104472/219052674-d50e8ae7-7e56-4644-9667-98c9331aedbb.png)\n\n\u003c/div\u003e\n\n  Figure 4: Yunfei’s SVM and CNNs comparison            source:\u003ca href=\"https://iopscience.iop.org/article/10.1088/1742-6596/1314/1/012148\"\u003eiopscience\u003c/a\u003e\n\n\n### 3. Proposed Method\na. SVM and feature descriptors\n(i) The dataset (CIFAR-10) used in our analysis is divided into training and testing, however we further split the training set into trainset and validation set with a ratio of 70% to 30% respectively.\n \n(ii). Feature descriptors: is a method of feature extraction for transforming raw data into numerical features that can be processed while preserving the information in the original data set. It can be accomplished manually or automatically using deep networks. In our analysis we will extract features manually using the following python libraries cv2 and scikit-image for image processing, tqdm that allows you to output a smart progress bar that shows elapsed and estimated time remaining, Histogram of Oriented Gradients normally refers to ‘hog’. (HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image [6]. \n\n(iii). SVM (Support Vector Machines) algorithm: Support vector machines (SVMs) are a set of\nsupervised learning methods used for classification, regression, and outlier detection [7]. It was proposed by Vladimir N. Vapnik and Alexey Ya. Chervonenkis in 1963 but was only able to deal with linear classification problems, because of this it was later improved to a method called ‘kernel trick’, which was able to solve non – linear classification problems. \n\n\u003cdiv align=\"center\"\u003e\n \n ![image](https://user-images.githubusercontent.com/63104472/219052930-e6badf27-8a17-44c9-b711-ac367ad171d0.png)\n \n\u003c/div\u003e\n\nFigure 5: The intuition of SVM Algorithm\n\n\nThere are so many different kernel tricks and for our analysis we will use RBF (Radial Basis Function) because of its similarity to k-nearest neighbor algorithm.\n \n\u003cdiv align=\"center\"\u003e\n \n![image](https://user-images.githubusercontent.com/63104472/219053121-b99260a2-2f9b-4481-b358-7ef32d0f9495.png)\n\n\u003c/div\u003e\nFigure 6: The svm classifier\n\n\nThe RBF equation: \n\u003cdiv align=\"center\"\u003e\n \n![image](https://user-images.githubusercontent.com/63104472/219053358-058eb1f5-9add-4d5e-9b84-af244c235d35.png)\n\n\u003c/div\u003e\n\nModel Architecture: SVM (RBF) + HOG + 128\nWhere feature type: HOG, image size: 128, model: SVM\n![image](https://user-images.githubusercontent.com/63104472/219053483-489c9798-e18f-4d88-89e8-714f156b382a.png)\n\nFigure 7: The svm classifier\n\nb. \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003eConvolutional Neural Networks\u003c/a\u003e  \nFirst introduced by Kunihiko FuKushima in the 1979 called “Neocognitron”, a basic image recognition neural network [8]. Kunihiko’s Neocognitron laid the foundation of research around convolutional neural networks by Yann LeCun in 1980 [9]. Since then, CNNs (Convolutional Neural Networks) have been widely adopted for image recognition and object detection. Some known CNN architectures are the classic \u003ca href=\"https://en.wikipedia.org/wiki/LeNet\"\u003eLeNet-5\u003c/a\u003e, \u003ca href=\"https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf\"\u003eAlexNet\u003c/a\u003e, \u003ca href=\"https://arxiv.org/pdf/1409.1556.pdf\"\u003eVGGNet\u003c/a\u003e, \u003ca href=\"https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43022.pdf\"\u003eGoogleNet\u003c/a\u003e, \u003ca href=\"https://arxiv.org/pdf/1512.03385v1.pdf\"\u003eResNet\u003c/a\u003e, \u003ca href=\"https://towardsdatascience.com/zfnet-an-explanation-of-paper-with-code-f1bd6752121d\"\u003eZFNet\u003c/a\u003e [10]. In this paper we used a simple deep network which is explained and demonstrated below.\n\nModel Architecture:\nThe proposed solution has four \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003econvolutional\u003c/a\u003e  and \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003e pooling layers\u003c/a\u003e, one flattened, two full connected layers and an output layer. The first convolution layer has a filter of 32, kernel size of 3, three channel image size of 64x64, and a relu activation function. The second layer is \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003eMax pooling \u003c/a\u003e with pool size and strides set to 2. One flattened layer, two fully connected layers with \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003erelu activation function\u003c/a\u003e, a final output layer with 10 neurons for each one of the 10 classes and a \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003eSoftMax activation function\u003c/a\u003e for multiclass, which is suitable and better than \u003ca href=\"https://en.wikipedia.org/wiki/Convolutional_neural_network\"\u003esigmoid\u003c/a\u003e.\n\n\u003cdiv align=\"center\"\u003e\n\n ![image](https://user-images.githubusercontent.com/63104472/219053733-92dea681-3bd4-4590-9821-1427825179aa.png)\n\n \u003c/div\u003e\n\nFigure 8: Model architecture                    source: project notebook\n\n                           \n### 4. Results\nThe SVM model had a validation accuracy of 57.01% on the training set and 57.2% on the test set, unlike CNN which had 78.12% on the training set and 75.48% on the test set, clearly showing why svm are not the best for 2-dimensional data. The CNN performance was incredibly good with image recognition. Computational time for machine learning (SVM) was 4.7hrs unlike deep learning (CNN) which was 1.26hrs given the same hardware resources (Google Colab TPU with high RAM of 35.2GB), breaking the myth that it takes more time to train and develop CNN models. A summary of our analysis is shown below in Figure 8. \n\n\n![image](https://user-images.githubusercontent.com/63104472/219054223-fffa2e72-997e-4790-bcce-9d144b32d82b.png)\n\nFigure 9: SVM and CNN comparison\n\n\n### 5. Conclusions\nIt is true that computer vision is in all aspects of our daily lives from the smart phones we use to our fun social media sites, health, military, law enforcement authorities and business enterprises. This field is making a huge improvement. In our paper we have proven that CNNs are better than SVM in image classification, recognition, and object detection tasks. \n   Also, another groundbreaking discovery is that with more computational resources like GPUs (Graphics Processing Unit) and TPUs convolutional neural networks perform better and take less time to train and deploy than SVM. \n    However, we should mention that SVM is still widely used in machine learning and one of the most powerful and most effective algorithms in high dimensional spaces [11].\n\n### 6. Acknowledgement\nThanks to Professor Guido Borghi for his guidance and provision of initial code templates on which this paper was built on.\n\n### Group organization\n- GitHub repository: L. Ghiani , E. Yilmaz, M. A. Kamara\n- Markdown \u0026 Web template: L. Ghiani , E. Yilmaz\n- Jupyter Notebook : Mohamed Alie Kamara\n\n\n \n\n### References\n[1] IBM, “What is computer vision”.\n\n[2] Paperswithcode, “Cifar-10”,  https://paperswithcode.com/dataset/cifar-10\n[3] A. Krizhevsky https://www.ibm.com/topics/convolutional-neural-networks sky and V. Nair and \n     G. Hinton, “The CIFAR-10 dataset”, s, Department of Computer Science, University of Toronto, \n     https://www.cs.toronto.edu/~kriz/cifar.html\n\n[4] Kaggle, “Cifar-10 raw image data”, https://www.kaggle.com/datasets/oxcdcd/cifar10 \n\n [ 5] Yunfei Lai, “A comparison of Traditional Machine Learning and Deep learning in Image \n      Recognition”, https://iopscience.iop.org/article/10.1088/1742-6596/1314/1/012148 \n\n[6] Wikipedia, “Histogram of Oriented Gradients”\n\n[7] Scikit-learn, “Support vector machines”,  https://scikit-learn.org/stable/modules/svm.html \n\n[8] Wikipedia, “Neocognitron”, https://en.wikipedia.org/wiki/Neocognitron\n\n[9] IBM, “What are convolutional neural networks?”, \n      https://www.ibm.com/topics/convolutional-neural-networks\n\n[10] IBM, “Types of convolutional neural networks”,\n      https://www.ibm.com/topics/convolutional-neural-networks\n\n[11] Scikit-learn, “Support Vector Machine”\n       \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkmohamedalie%2Fdtm-machine_learning_final_project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkmohamedalie%2Fdtm-machine_learning_final_project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkmohamedalie%2Fdtm-machine_learning_final_project/lists"}