{"id":18299039,"url":"https://github.com/fabprezja/deep-fast-vision","last_synced_at":"2025-04-05T13:34:19.221Z","repository":{"id":204115120,"uuid":"633096585","full_name":"fabprezja/deep-fast-vision","owner":"fabprezja","description":"A Python library for rapid prototyping of deep transfer learning vision models. ","archived":false,"fork":false,"pushed_at":"2023-11-15T19:58:25.000Z","size":2938,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-09-17T05:18:51.206Z","etag":null,"topics":["auto-ml","computer-vision","deep-learning","machine-learning","python","tensorflow-keras"],"latest_commit_sha":null,"homepage":"https://fabprezja.github.io/deep-fast-vision/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabprezja.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.bib","codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-26T19:17:17.000Z","updated_at":"2023-10-25T12:09:36.000Z","dependencies_parsed_at":"2023-11-15T20:48:12.590Z","dependency_job_id":null,"html_url":"https://github.com/fabprezja/deep-fast-vision","commit_stats":null,"previous_names":["fabprezja/deep-fast-vision"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabprezja%2Fdeep-fast-vision","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabprezja%2Fdeep-fast-vision/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabprezja%2Fdeep-fast-vision/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabprezja%2Fdeep-fast-vision/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabprezja","download_url":"https://codeload.github.com/fabprezja/deep-fast-vision/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223192623,"owners_count":17103564,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auto-ml","computer-vision","deep-learning","machine-learning","python","tensorflow-keras"],"created_at":"2024-11-05T15:07:46.476Z","updated_at":"2024-11-05T15:07:47.042Z","avatar_url":"https://github.com/fabprezja.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Fast Vision: Accelerated Deep Transfer Learning Vision Prototyping and Beyond\n\u003cimg src=\"https://user-images.githubusercontent.com/87379098/234222583-5f1fcbf6-368d-471d-8f64-fa25b6ccf925.png\" alt=\"logo\" width=\"60%\"\u003e\n\nDeep Fast Vision is a versatile Python library for rapid prototyping of deep transfer learning vision models. It caters to users of various levels, offering different levels of abstraction from high-level configurations for beginners to mid and low-level customization for professional data scientists and developers. Built around Keras and TensorFlow, this library also includes handy utilities.\n\nCompute mode depends on Tensorflow configuration: GPU or CPU (GPU is recommended).\n\n## (a few) Key Features\n\n1. Auto loss/target type determination\n2. Auto generator setup\n3. Auto output layer setup\n4. Auto pre-training of new dense layers before unfreezing transfer architecture (in parts or as a whole)\n5. Auto augmentation setup (from templates and/or Custom)\n6. Auto best weights saving and loading\n7. Auto class weights calculation\n8. Auto validation curves plot (with minimúm loss \u0026 maximum metric epoch highlight)\n9. Auto confusion matrices for test/external data\n10. Easy dense layer configuration\n11. Easy regularization set up and mixing (Dropout, L2, L1, Early Stop, etc.)\n12. Access to all Keras optimizers \u0026 callback support\n\nComprehensive documentation for Deep Fast Vision is available both in the docs folder and at the [**documentation page**](https://fabprezja.github.io/deep-fast-vision/).\n\n\n## Install using pip:\n\nYou can install `deepfastvision` using pip with the following command:\n\n```shell\npip install deepfastvision\n```\nYou can install `deepfastvision` with the older `tensorflow-gpu` using the following command:\n\n```shell\npip install deepfastvision[gpu]\n```\n## How to cite:\n\nIf you find the work usefull in your project, please cite:\n\n```bibtex\n@misc{prezja2023deep,\n      title={Deep Fast Vision: A Python Library for Accelerated Deep Transfer Learning Vision Prototyping}, \n      author={Fabi Prezja},\n      year={2023},\n      eprint={2311.06169},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\u003e **Note:** When referencing, please consider additional attributions to Tensorflow and Keras, as the library is built around them.\n\n# Usage Examples by Abstraction Level (and Capabilitity details)\nBellow are examples by level of abstraction while detailing automation and configuration capabilities.\n\n## High Level of Abstraction\n```python\nimport wandb\nfrom wandb.keras import WandbCallback\nfrom deepfastvision.core import DeepTransferClassification\n\n# Initialize wandb\nwandb.init(project='your_project_name', entity='your_username')\n\n# Create a DeepTransferClassification object\nexperiment = DeepTransferClassification(paths={'train_val_data': 'path_to_train_val_data',\n                                               'test_data_folder': 'path_to_test_data'},\n                                        saving={'save_weights_folder':'path_to_save_weights'},\n                                        model= {'transfer_arch': 'VGG16',\n                                                'dense_layers': [144,89,55],\n                                                'unfreeze_block': ['block5']},\n                                        training={'epochs': 15,\n                                                  'learning_rate': 0.0001,\n                                                  'metrics': ['accuracy'],\n                                                  'callback': [WanDBCallback()]})\n\nmodel, results = experiment.run()\n```\n\nThe above code will return:\n\n-   **Trained model**: The final model after training.\n-   **Results dictionary**: A results dictionary which contains evaluation and training results, as well as model and training configuration.\n-   **Validation curves**: Automatically configured based on the target type and provided labels \u0026 metrics.\n-   **Confusion matrix**: Automatically configured based on the target type and provided labels \u0026 metrics.\n\n**User provided:**\n1. Data paths: The library identifies and loads the data from the provided paths for training, validation, and testing.\n2. Transfer learning architecture (VGG16): The library fetches the VGG16 pre-trained model and uses it as the base for the new neural network.\n3. Dense layer configuration [144, 89, 55]: The library creates dense layers with 144, 89, and 55 neurons, respectively, and adds them to the neural network.\n4. Any callback: Insert the user-defined callback (WanDB in this case).\n\n**After processing the user-provided information, the library automatically performs the following tasks to create, train, and evaluate the neural network model (customizable in lower levels of abstraction):**\n\n1. Determine Loss function\n2. Identify Train-Val and Test folders in the provided path\n3. Establish output layer size and activation functions\n4. Calculate and apply class weights (adjustable)\n5. Insert dropout between the dense layers (adjustable).\n6. Generate appropriate data generators for train, val, and test data\n7. Resize all images to 224 x 224 which i.e, the transfer model specification (adjustable).\n8. Retrieve transfer architecture's preprocessing function for augmentation settings\n9. Prepare data augmentation for training data generator (adjustable)\n10. Pre-train dense layers with a frozen transfer model, then unfreeze and train as specified (to mitigate destructive effects on unfrozen transfer architecture)  (adjustable)\n11. Monitor and load optimal weights based on validation results before testing (adjustable)\n12. Conduct test after loading the best weights (adjustable)\n13. Create validation curves highlighting the best metric value and validation epoch (adjustable)\n14. Produce confusion matrices for test set(s) (adjustable)\n15. Provide model architecture summary and trainable layers summary (adjustable)\n\n## Random Run Example\n![image](https://user-images.githubusercontent.com/87379098/233811708-dea3958d-6439-424f-bdac-9af669da769c.png)\n\n## Medium Level of Abstraction\n```python\nfrom deepfastvision.core import DeepTransferClassification\n\nexperiment = DeepTransferClassification(paths={'train_val_data': 'path_to_train_val_data',\n                                               'test_data_folder': 'path_to_test_data'},\n                                        saving={'save_weights_folder':'path_to_save_weights'},\n                                        model= {'transfer_arch': 'VGG19',\n                                                'dense_layers': [ 377, 233, 144, 89 ],\n                                                'dense_activations': 'relu' ,\n                                                'regularization': 'Dropout+L1',\n                                                'dropout_rate': 0.35 ,\n                                                'unfreeze_block': ['block5']},\n                                        training={'epochs': 15,\n                                                  'batch_size': 32 ,\n                                                  'optimizer_name': 'Adam' ,\n                                                  'add_optimizer_params': {'clipnorm': 0.8} ,\n                                                  'augmentation': 'advanced' ,\n                                                  'learning_rate': 0.0001,\n                                                  'metrics': [ 'accuracy' , 'recall', 'precision']})\n\nmodel, results = experiment.run()\n```\nIn the medium level of abstraction example, the user provides more specific configurations for the model and training process in addition to the detailed automations, as compared to the high level of abstraction example. Here are the main differences:\n\n1. **Transfer learning architecture**: This example selects VGG19 instead of VGG16.\n1. **Dense layer configuration**: This example defines another dense layer configuration, specifying 377, 233, 144, and 89 neurons in respective layers.\n2. **Dense activation function**: This example provides a specific activation function, 'ReLU', for the dense layers.\n3. **Regularization**: This example chooses 'Dropout + L1 as for dense layer regularization.\n4. **Dropout rate**: This example sets a custom dropout rate of 0.35.\n5. **Training parameters**: This example provides more specific training parameters, including batch size (32), optimizer name ('Adam'), and additional optimizer parameters such as gradient cliping ({'clipnorm': 0.8}).\n6. **Data augmentation**: This example selects an advanced data augmentation strategy (instead of the default 'basic').\n7. **Metrics**: This example specifies additional metrics for evaluation, including recall and precision, along with accuracy. The first metric in the list is used as the metric sub-plot in validation curves.\n\nThese changes give more control over the model architecture and training process, allowing for more tailored experiments and better performance tuning. All previous detailed automations still apply.\n\n## Low Level of Abstraction \u0026 Default Configuration\n```python\nfrom deepfastvision.core import DeepTransferClassification\n\nexperiment = DeepTransferClassification(paths={\n    'train_val_data': 'path_to_train_val_data',\n    'test_data_folder': 'path_to_test_data',\n    'external_test_data_folder': 'path_to_external_test_data',\n},\nmodel={\n    'image_size': (224, 224),\n    'transfer_arch': 'VGG19',\n    'pre_trained': 'imagenet',\n    'before_dense': 'Flatten',\n    'dense_layers': [610, 377, 233, 144, 89, 55],\n    'dense_activations': 'elu',\n    'initializer': 'he_normal',\n    'batch_norm': True,\n    'regularization': 'Dropout+L2',\n    'l2_strength': 0.001,\n    'dropout_rate': 0.35,\n    'unfreeze_block': ['block1', 'block2', 'block5'],\n    'freeze_up_to': 'flatten',\n},\ntraining={\n    'epochs': 9,\n    'batch_size': 32,\n    'learning_rate': 2e-5,\n    'optimizer_name': 'Adam',\n    'add_optimizer_params': {'clipnorm': 0.8},\n    'class_weights': True,\n    'metrics': ['accuracy', 'recall', 'precision'],\n    'augmentation': 'custom',\n    'custom_augmentation':[user_function]\n    'callback': [WandbCallback(), learning_rate_schedule],\n    'early_stop': 0.20,\n    'warm_pretrain_dense': True,\n    'warm_pretrain_epochs': 9,\n},\nevaluation={\n    'auto_mode': True,\n},\nsaving={\n    'save_weights': True,\n    'save_weights_folder': 'path_to_save_weights',\n    'save_best_weights': 'val_loss',\n},\nmisc={\n    'show_summary': True,\n    'plot_curves': True,\n    'show_min_max_plot': True,\n    'plot_conf': True,\n})\n\nmodel, results = experiment.run()\n```\n\nIn this low-level of abstraction example, the user has greater control over the model and training process, providing a more detailed configuration. This example includes new user changes and options, as well as the default values in the configuration. Compared to the previous example, the user can additionally specify:\n\nChanges compared to the Medium level of abstraction:\n1. **Paths:** The user added an 'external_test_data_folder' for additional test data.\n2. **Batch normalization** was enabled.\n3. **More Dense Layers** provided (610, 377, 233, 144, 89, 55)\n4. **Regularization** was set to 'Dropout+L2', with L2 strength specified as 0.001.\n5. The user specified which **blocks to unfreeze** ('block1', 'block2', 'block5') and froze layers up to 'flatten'.\n6. **Augmentation** was set to 'custom' and provided by user.\n7. **User-provided callbacks** were added: WandbCallback and learning_rate_schedule.\n8. **Early stopping** tolerance was enabled with a threshold of 0.20 (as a ratio of total epochs).\n9. **Epochs** were increased to 25.\n10. **Warm pretraining** for dense layers was enabled with 9 epochs.\n\nActiavated by default:\n1. **Weight initializer** is set to 'he_normal'.\n2. **Class weights** are enabled.\n3. **Image size** is explicitly set to (224, 224).\n4. **Pre-trained weights** are specified to be from 'imagenet'.\n5. **Layer before dense layers** is set to 'Flatten'.\n6. **Dense layer activations** are set to 'elu'.\n7. **Learning rate** is set to 2e-5.\n8. **Evaluation configuration**: 'auto_mode' is True. (automatic evaluation of best weights)\n9. **Saving configuration:** Saving best weights based on 'val_loss'.\n10. **Miscellaneous settings:** Showing model summary, plotting curves, showing min-max plot, and plotting confusion matrix.\n\nAs in the previous examples all automations are applied.\n\n### Loading Model Checkpoint and/or Inference Only\nWhen evaluate_mode is True, training cannot occur. Similarly, when a preloaded weights path is given, the model initializes with the preloaded weights and allows further training or inference.\n\n```python\nsaving={\n    'evaluate_mode': True,\n    'auto_mode': True,\n    'preloaded_weights_path': 'path_to_preloaded_weights',\n}\n```\n\n# Available Class Methods\n\n**1. Model Prediction**\n\n```python\npredictions = experiment.model_predict('folder_path')\n```\nThe model_predict method uses the trained model to predict all images in a given folder. The method returns image, path, predicted label, confidence, and variance for each image in the folder. It can be sorted by variance (across labels) for identifying confusing instances or by metric (e.g., accuracy).\n\n**2. Export Results and Model**\n```python\nexperiment.export_all(results, base_path='folder_path_to_results', export_model=True, additive=True)\n```\nThe export_all method exports all results, best weights, and the trained model into a folder. With additive=True, the user may iterate the experiment and obtain results in new randomly named folders.\n\n**3. Extract Features**\n```python\nX_train, y_train, X_val, y_val, X_test, y_test, X_test_external, y_test_external = experiment.model_feature_extract(layer_index=None, layer_name='block5')\n```\nThe model_feature_extract method can be used to extract features from any layer in the model while respecting the used train, val, test(s) indices.\n\n# Extra Utilities\n\n**2. Data Splitter**\nA class to split any data into the required partition format (train, val, test(s)). The splits are stratified.\n```python\nfrom deepfastvision.data_management.data_helpers import DatasetSplitter\n\n# Define the paths to the original dataset and the destination directory for the split datasets\ndata_dir = 'path/to/original/dataset'\ndestination_dir = 'path/to/destination/directory'\n\n# Instantiate the DatasetSplitter class with the desired train, validation, and test set ratios\nsplitter = DatasetSplitter(data_dir, destination_dir, train_ratio=0.7,\n                           val_ratio=0.10, test_ratio=0.10, test_ratio_2=0.10, seed=42)\n\n# Split the dataset into train, validation, and test sets\nsplitter.run()\n```\n**2. Data Sub Sampler (miniaturize)**\nA class to sub-sample (miniaturize) any dataset give a ratio:\n```python\nfrom deepfastvision.data_management.data_helpers import DataSubSampler\n\n# Define the paths to the original dataset and the destination directory for the subsampled dataset\ndata_dir = 'path/to/original/dataset'\nsubsampled_destination_dir = 'path/to/subsampled/dataset'\n\n# Instantiate the DataSubSampler class with the desired fraction of files to sample\nsubsampler = DataSubSampler(data_dir, subsampled_destination_dir, fraction=0.5, seed=42)\n\n# Create a smaller dataset by randomly sampling a fraction (in this case, 50%) of files from the original dataset\nsubsampler.create_miniature_dataset()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabprezja%2Fdeep-fast-vision","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabprezja%2Fdeep-fast-vision","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabprezja%2Fdeep-fast-vision/lists"}