{"id":16450902,"url":"https://github.com/tushar-ml/darkneuron","last_synced_at":"2025-03-23T08:32:33.202Z","repository":{"id":56659236,"uuid":"267849401","full_name":"Tushar-ml/DarkNeuron","owner":"Tushar-ml","description":"This Library will deal with the implementation of Automatic Deep Learning which can reduce the time and Complexity for non-technical users and Specialists in this field to train their own networks with significant accuracies and can perform Image Classification, Object Detection and hybridization of Models.","archived":false,"fork":false,"pushed_at":"2022-12-22T05:54:18.000Z","size":144137,"stargazers_count":5,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-18T19:36:09.058Z","etag":null,"topics":["automation-framework","classification-model","keras","keras-tensorflow","object-detection","python","yolov4"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tushar-ml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":["Tushar-ml"]}},"created_at":"2020-05-29T12:13:06.000Z","updated_at":"2022-10-11T12:11:46.000Z","dependencies_parsed_at":"2022-08-15T22:50:16.285Z","dependency_job_id":null,"html_url":"https://github.com/Tushar-ml/DarkNeuron","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tushar-ml%2FDarkNeuron","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tushar-ml%2FDarkNeuron/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tushar-ml%2FDarkNeuron/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tushar-ml%2FDarkNeuron/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tushar-ml","download_url":"https://codeload.github.com/Tushar-ml/DarkNeuron/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245078067,"owners_count":20557274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation-framework","classification-model","keras","keras-tensorflow","object-detection","python","yolov4"],"created_at":"2024-10-11T10:06:12.311Z","updated_at":"2025-03-23T08:32:31.917Z","avatar_url":"https://github.com/Tushar-ml.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/Tushar-ml"],"categories":[],"sub_categories":[],"readme":"# Welcome to DarkNeuron !\r\n\r\n**Dark Neuron**  will deal with implementation of Automatic Deep Learning  which can reduce the time and Complexity for non-technical users to train their own netwroks without Comprimising Accuracies for Classification of Images and Object Detection, most demanding tecniques for **Autonomous Systems** and **Medical Fields**.\r\n\r\n\u003e \" By augmenting human performance, **AI** has the potential to markedly improve productivity, efficiency, workflow, accuracy and speed, both for physicians and for patients … What I’m most excited about is using the future to bring back the past: to restore the care in healthcare. \" -  Eric Topol \r\n\r\n## CONTENTS :\r\n\r\n\t\r\n\r\n \r\n\r\n - \u003ca  href='#classification'\u003e Classification of Images \u003c/a\u003e\r\n\t  \r\n\t - \u003ca href='#class_init'\u003eInitialization\u003c/a\u003e\r\n\t - \u003ca href='#class_prepare'\u003ePreparation of Data\u003c/a\u003e\r\n\t -  \u003ca href='#class_model'\u003eModel Creation\u003c/a\u003e\r\n\t -  \u003ca href='#class_train'\u003eTraining\u003c/a\u003e\r\n\t -  \u003ca href='#class_prediction'\u003ePrediction\u003c/a\u003e\r\n\t -  \u003ca href='#class_visual'\u003eVisualization of Prediction and Metrics\u003c/a\u003e\r\n\t\r\n -  \u003ca href='#object_detection'\u003eObject Detection (YOLOv4)\u003c/a\u003e\r\n\t -  \u003ca href='#obj_init'\u003eInitialization\u003c/a\u003e\r\n\t -  \u003ca href='#obj_prepare'\u003ePreparation of Data\u003c/a\u003e\r\n\t -  \u003ca href='#obj_train'\u003eModel Training\u003c/a\u003e\r\n\t - \u003ca href='#obj_detection'\u003eDetection \u003c/a\u003e\r\n -  \u003ca href='#release'\u003e Further Release \u003c/a\u003e\r\n -  \u003ca href='#license'\u003e License \u003c/a\u003e\r\n\r\n# Installation \r\n\t\t\r\n\r\n    pip install DarkNeurons\r\n\r\n\r\n## DarkNeuron Target Audience:\r\n**DarkNeuron** is an Open Source Library . Target Audience  are:\r\n\r\n - All Experienced Data Scientists who wants to increase productivity with reduction in complexities and time.\r\n -  Professionals in Autonomous and Healthcare Industries to implement high accuracy models for the Production use.\r\n - Students of Data Science.\r\n\r\n\u003ch2 id='classification'\u003eClassification of Images\u003c/h2\u003e\r\n\r\n**DarkNeuron Classification** has feature of implementing Pretrained Models on ImageNet Data. Users can directly  train pretrained models or can retrain their own models.  Models provided are:\r\n\t\t\t\t\r\n\r\n - InceptionV3\r\n - Xception\r\n - ResNet50\r\n - VGG16\r\n - VGG19\r\nFurther will be added on upcoming releases.\r\n\r\n\u003ch3 id='class_init'\u003eInitialization of Classification Model\u003c/h3\u003e\r\n\r\nInitialization of Classification Model of DarkNeuron requires **working_directory**\r\nas a main argument. It should have models and Raw_Data in it.\r\nIt can be Initialized as below:\r\n\r\n    from DarkNeurons import Classify_Images\r\n    classify = Classify_Images(working_directory = \"Working Directory\")\r\n   \r\n\u003ch3 id='class_prepare'\u003ePreparation of Data\u003c/h3\u003e\r\nPreparation of Data for Classification takes place in terms of whether the user wants to \u003cb\u003eTrain the Model or Predict from the Model\u003c/b\u003e and the \u003cb\u003eMethod of Importing Images\u003c/b\u003e:\r\n\r\n - **Directory** : To Import the whole folder with Images distributed in respective class folders.\r\n - **DataFrame** : Having a Dataframe containing Image filenames and corresponding Labels.\r\n - **Point** : To provide input as an array like :  X_train , Y_train .\r\n - **Image** : To provide Single Image as an Input (suggested for Prediction Phase)\r\n\u003cb\u003eLet's See each of them with their necessary arguments\u003c/b\u003e\r\n\r\n\u003ch4\u003e\u003cu\u003e Method: Directory\u003c/u\u003e\u003c/h4\u003e\r\n\u003cb\u003e Code Syntax:\u003c/b\u003e (Continue from above....)\r\n\r\n    train,val,labels = classify.Preprocess_the_Image(method = 'directory', train =True,\r\n\t\t\t\tnum_classes = 2, batch_size = 32, #Default\r\n\t\t\t\ttarget_image_size = (224,224,3) #Default\r\n\t\t\t\tmodel_name = 'InceptionV3',\r\n\t\t\t\tuser_model = None, #Default,\r\n\t\t\t\ttraining_image_directory = 'Directory_Path\r\n\t\t\t\tvalidation_image_directory = None,\r\n\t\t\t\t)\r\n\r\nLet's See each argument and their default values:\r\n\r\n - **train**: **False** (for Prediction) and **True** (Training)\r\n - **num_classes** : No of classes in user data     (Default: 2)\r\n - **batch_size** : Batch Size of Training Data      (Default:32)\r\n - **target_image_size** :  Image Input Size used for Creation and Preprocessing of the Input\r\n - **model_name**: Name of Pretrained Model , **not required when provide user_model name** (Default: None)(**Same for all Method**)\r\n - **user_model**: user pretrained model , input taken as an model means Load the model using `classify.load_model(user_model_name)(**Same for all Method**)\r\n - **training_image_directory**: Full Path of Training Image Directory (Must for Training )\t\t(Default: None)\r\n - **validation_image_directory**: Full Path of Validation Image Directory (Default=None)\r\n - **test_image_directory**: Test Image Directory path, **only used when train = False**\r\n \r\n\u003ch4\u003e\u003cu\u003eMethod: DataFrame\u003c/u\u003e\u003c/h4\u003e\r\n\u003cb\u003eCode Syntax:\u003c/b\u003e\r\n\r\n    train,val,labels = classify.Preprocess_the_Image(method = 'dataframe', train = True,\r\n\t\t\t\tnum_classes = 2,batch_size = 32,\r\n\t\t\t\tdataframe = df ,\r\n\t\t\t\tx_col_name = 'filename',\r\n\t\t\t\ty_col_name = 'label',\r\n\t\t\t\timage_directory = None,\r\n\t\t\t\tsplit = 0.1 )\r\n\r\nLet's Understand the above arguments:\r\n\r\n - **dataframe**: Loaded Dataframe variable ( refer Pandas for defining DataFrame)\r\n - **x_col_name**: name of Image file names containing column name\r\n - **y_col_name**: name of Labels containing column name\r\n - **image_directory**: **Only required if x_col_name has relative path for images**\r\n - **split**: Spliting of data automatically for validation and Training puroses\r\n \r\n \u003ch4\u003e\u003cu\u003eMethod: Point\u003c/u\u003e\u003c/h4\u003e\r\n \u003cb\u003eCode Example....\u003c/b\u003e\r\n \r\n\r\n    train,val,labels = classify.Preprocess_the_Image(method = 'point',train = True,\r\n\t\t\t\tx_train = x_train,y_train = y_train,\r\n\t\t\t\tx_test = x_test,y_test = y_test)\r\n\r\nLet's Understand each argument:\r\n\r\n - **x_train**:  Input X variable for Training\r\n - **y_train**: Target Y variable for Training\r\n - **x_test**: Input X variable for Testing and Validation\r\n - **y_test**: Target Y variable for Testing and Validation\r\n\r\n\u003ch4\u003e\u003cu\u003eMethod: Image\u003c/u\u003e\u003c/h4\u003e\r\n\u003cb\u003eCode Syntax:\u003c/b\u003e\r\n\r\n    test = classify.Preprocess_the_Image(method='image',train = False,\r\n\t\t\t\timage_path = 'Path of the Image',\r\n\t\t\t\tgrayscale=False\r\n\t\t\t\t)\r\n\r\n - **image_path**: Path of the Image to predicted\r\n - **grayscale**: To load the image with grayscale feature\r\n\r\n\u003ch3 id='class_model'\u003e Model Creation\u003c/h3\u003e\r\nThis Feature takes no argument , but \u003cb\u003enecessary when user provide model_name\u003c/b\u003e  .\r\n\u003cbr\u003e\r\nIt will create the full structure of the model  based on the data provided in Prepare the Data function call.\r\n\r\n    model = classify.Create_the_Model()\r\n\r\nThat's it. Model will be created and Generated.\r\n**If you have PreDownloaded weights, then must sure the following:**\r\n\r\n - Put the model in working_directory\r\n - If Training is **False**: (Names of the model to be save)\r\n\t - InceptionV3 : 'inceptionv3_model.h5'\r\n\t - ResNet50 : 'resnet50_model.h5'\r\n\t - VGG16: 'vgg16_model.h5'\r\n\t - VGG19: 'vgg19_model.h5'\r\n\t - Xception: 'xception_model.h5'\r\n  - If Training is **True**: (Names of the model to be save)\r\n\t - InceptionV3 : 'inceptionv3_notop_model.h5'\r\n\t - ResNet50 : 'resnet50_notop_model.h5'\r\n\t - VGG16: 'vgg16_notop_model.h5'\r\n\t - VGG19: 'vgg19_notop_model.h5'\r\n\t - Xception: 'xception_notop_model.h5'\r\n\r\n**Otherwise, it will automatically Download the weights.**\r\n\r\n\u003ch3 id='class_train'\u003e Model Training\u003c/h3\u003e\r\n\r\nThis Feature will be used for Model Training purposes .\r\n\u003cb\u003e Code Syntax:\u003c/b\u003e\r\n\t\r\n\r\n    model = classify.Train_the_Model(model = model,\r\n\t\t\t\t    rebuild = False,\r\n\t\t\t\t    train_data_object = train,\r\n\t\t\t\t    validation_data_object = train,\r\n\t\t\t\t    epochs = 10,\r\n\t\t\t\t    optimizers = 'adam',\r\n\t\t\t\t    loss = 'binary_crossentropy',\r\n\t\t\t\t    fine_tuning = False,\r\n\t\t\t\t    layers = 20,\r\n\t\t\t\t    metrics = ['accuracy'],\r\n\t\t\t\t    validation_steps = 80,\r\n\t\t\t\t    steps_per_epoch = 50,\r\n\t\t\t\t    callbacks = None\r\n\t\t\t\t    )\r\n\r\n - **model**: model created from previous step.\r\n - **rebuild**: **only**, when model_name provided, set to **True**\r\n - **train_data_object**: generators get from Prepare the Data function\r\n - **epochs**: No of Steps for training\r\n - **optimizer**: Suitable Optimizer for model\r\n - **loss**: Loss function for model\r\n - **fine_tuning**: To do Fine_tuning or not \r\n - **layers**: **Only required when Fine Tuning set to True**, number of layers from bottom to be trained or to train all layers provide **'all'** argument\r\n - **metrics**: To be provided as **list**.\r\n - **callbacks**: To be provided as **list** by the user for early_stopping or Checkpoint.\r\n\r\n\u003ch3 id='class_prediction'\u003e Prediction \u003c/h3\u003e\r\nThis Feature will be used for Prediction from the model on the Test Dataset.\r\n\u003cbr\u003e\u003cb\u003eTo do this step, First Prepare the Data with train argument set to False and obtain test object from it.\u003c/b\u003e\r\n\u003cbr\u003e\u003cbr\u003e\u003cb\u003eCode Syntax:\u003c/b\u003e\r\n\r\n    classify.Predict_from_the_Model(labels = labels,\r\n\t\t\t\t\tmodel = model,\r\n\t\t\t\t\timg = None,\r\n\t\t\t\t\tgenerator = None,\r\n\t\t\t\t\ttop = 5\r\n\t\t\t\t\t)\r\n\r\n - **labels**: Labels provided as List or provided from generated labels in Prepare the data function during training.  (See Above)\r\n - **model**: Model generated by Training or due to loading user own model.\r\n - **img**: only if method: image\r\n - **generator**: Test Data Object generated from Prepare the Data Function call.\r\n - **top**: Top k predictions for image.\r\n\r\n\u003ch3 id='class_visual'\u003eVisualization of Predictions and Metrics\u003c/h3\u003e\r\n\r\n\u003ch4\u003e Metrics Visualization\u003c/h4\u003e\r\n\r\n    classify.Visualize_the_Metrics()\r\n\r\n\u003ch4\u003e Prediction Visualization\u003c/h4\u003e\r\n\r\n    classify.Visualize_the_Predictions(number = 20)\r\n\r\n - **number**: No of Images or Predictions to Visualize\r\n\r\n\u003ch3\u003e Here Comes the Ending to Classification Part\u003c/h3\u003e\r\n\u003ch3\u003e Let's move on to Object Detection Part\u003c/h3\u003e\r\n\r\n\u003ch2 id = 'object_detection'\u003eObject Detection (YOLOv4)\u003c/h2\u003e\r\n\r\n\r\n\u003ch3 id='obj_init'\u003eInitialization of Object Detection Model\u003c/h3\u003e\r\nThis Function will take working directory as an argument where the training data is present and weights be present . If no weights are there then it will be downloaded.\u003cbr\u003e\r\nIf you have predefined yolov4 weights : Named it as --\u003e 'yolov4.weights'\r\nIf you have predefined yolov4 model: Named it as --\u003e 'yolov4.h5'\r\n\r\n    from DarkNeuron import YOLOv4\r\n    yolo = YOLOv4( working_directory , output_directory)\r\n\r\n\u003ch3 id='obj_prepare'\u003e Preparation of Data\u003c/h3\u003e\r\n\r\nFor this Function, All Images and corressponding labels should be in working_Directory within no sub folder.( For Simplicity, Train directory = Working directory).\r\nThis Function take file in three formats and converted them into YOLO Format Automatically:\r\n\r\n - csv\r\n - xml\r\n - text files\r\n \r\n \u003cb\u003eCode Syntax:\u003c/b\u003e\r\n \r\n\r\n    yolo.Prepare_the_Data(file_type,file_path,\r\n\t\t\t    dataframe_name = None,\r\n\t\t\t    class_file_name = None\r\n\t\t\t    )\r\n\r\n - **file_type**: This contain file_type: whether csv, xml, or text_files\r\n - **file_path**: This contain the path to data directory\r\n - **dataframe_name**: This should be given as name of csv file in working_directory\r\n - **class_file_name**: provide name of the  class list as text file in working directory\r\n\r\n\u003ch3 id='obj_train'\u003e Model Training\u003c/h3\u003e\r\nThis Function will be used to Train the model on user custom data set.\r\n\u003cbr\u003e\r\nThere are two process involved :\r\n\r\n - Process_1 : Simple Training\r\n - Process_2 : After Process_1, Fine tuning (Highly Recommended)\r\n\r\n\u003cb\u003eCode Syntax:\u003c/b\u003e\r\n\r\n    yolo.Train_the_Yolo(model_name = 'yolov4.h5',\r\n\t\t\t    input_shape = (608,608) #Multiple of 32 required\r\n\t\t\t    score = 0.5,\r\n\t\t\t    iou = 0.5,\r\n\t\t\t    epochs1 = 50, #For Process 1\r\n\t\t\t    epochs2 = 51, #For Process 2\r\n\t\t\t    batch_size1 = 32,\r\n\t\t\t    batch_size2 = 4,\r\n\t\t\t    validation_split = 0.1,\r\n\t\t\t    process1 = True,\r\n\t\t\t    process2 = True\r\n\t\t\t   )\r\n\r\n - **model_name**: If user have predefine model, can provide the name.\r\n - **input_shape**: Input Shape for the model .\r\n - **score**: Score Threshold.\r\n - **iou**: Intersection Over Union thresholf over training (must change for better accuracy)\r\n - **epochs1, epochs2**: Epochs for Different Processes described above.\r\n - **batch_size1, batch_size2**: Batch Size for Differn Purposes\r\n - **process1, process2**: Process to be Done (Default: True)\r\n\r\n \u003ch3 id='obj_detection'\u003e Detection \u003c/h3\u003e\r\nThis Function will be used to detect objects from video and Images.\r\nThis Function has following features:\u003cbr\u003e\r\n\r\n - **Web Cam Detection** --\u003e It will Detect using webcams and can also be used by Mobile Phone Cameras ( see IPWebCam )\r\n - **Choose Class** --\u003e You can choose your own prediction classes , means which object to predict which to not. For Example, on COCO dataset , it has 80 labels, then you should pass person to the function, it will detect only person, leave everthing else as it is.\r\n \r\n \r\n \u003cb\u003e Code Synatax:\u003c/b\u003e\r\n \r\n\r\n    yolo.Detect(test_folder_name = 'test',\r\n\t\t    model_name = None,\r\n\t\t    cam = False,\r\n\t\t    videopath = 0,\r\n\t\t    classes = [],\r\n\t\t    score = 0.5,\r\n\t\t    tracking = False\r\n\t\t    )\r\n\r\n - **test_folder_name**: Test folder name in working directory ( images and video both, it will detect automatically and take actions according to it)\r\n - **model_name**: Model name saved in working_directory by Training, otherwise it will take yolov4.h5 by default.\r\n - **cam**: To enable Web Cam Detection\r\n - **videopath**: Path to the video to detect\r\n - **classes**: Selelctive choosing of Classes for Detections (**Provide as List**)\r\n - **score**:  Threshold of Score for Prediction\r\n - **tracking**: DeepSort Tracking to be enable or not\r\n\r\n\u003ch2 id='release'\u003e Further Release\u003c/h2\u003e\r\n\r\n - [ ] Improvement in Tracking \r\n - [ ] Artificial Neural Networks user friendly implementation\r\n - [ ] Visualization of Neural Networks\r\n \r\n \u003ch2 id='license'\u003eLicense\u003c/h2\u003e\r\n MIT  License\r\n\r\n**Copyright  (c)  2020  DarkNeuron  Tushar-ml**\r\n\r\nPermission  is  hereby  granted,  free  of  charge,  to  any  person  obtaining  a  copy\r\n\r\nof  this  software  and  associated  documentation  files  (the  \"Software\"),  to  deal\r\n\r\nin  the  Software  without  restriction,  including  without  limitation  the  rights\r\n\r\nto  use,  copy,  modify,  merge,  publish,  distribute,  sublicense,  and/or  sell\r\n\r\ncopies  of  the  Software,  and  to  permit  persons  to  whom  the  Software  is\r\n\r\nfurnished  to  do  so,  subject  to  the  following  conditions:\r\n\r\nThe  above  copyright  notice  and  this  permission  notice  shall  be  included  in  all\r\n\r\ncopies  or  substantial  portions  of  the  Software.\r\n\r\nTHE  SOFTWARE  IS  PROVIDED  \"AS  IS\",  WITHOUT  WARRANTY  OF  ANY  KIND,  EXPRESS  OR\r\n\r\nIMPLIED,  INCLUDING  BUT  NOT  LIMITED  TO  THE  WARRANTIES  OF  MERCHANTABILITY,\r\n\r\nFITNESS  FOR  A  PARTICULAR  PURPOSE  AND  NONINFRINGEMENT.  IN  NO  EVENT  SHALL  THE\r\n\r\nAUTHORS  OR  COPYRIGHT  HOLDERS  BE  LIABLE  FOR  ANY  CLAIM,  DAMAGES  OR  OTHER\r\n\r\nLIABILITY,  WHETHER  IN  AN  ACTION  OF  CONTRACT,  TORT  OR  OTHERWISE,  ARISING  FROM,\r\n\r\nOUT  OF  OR  IN  CONNECTION  WITH  THE  SOFTWARE  OR  THE  USE  OR  OTHER  DEALINGS  IN  THE SOFTWARE.\r\n\r\n \r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftushar-ml%2Fdarkneuron","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftushar-ml%2Fdarkneuron","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftushar-ml%2Fdarkneuron/lists"}