Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/amishra15/enhancing-motion-blurred-images-in-normal-and-low-light-condition-for-mobile-photography

DATA: 606 | Capstone Project
https://github.com/amishra15/enhancing-motion-blurred-images-in-normal-and-low-light-condition-for-mobile-photography

autoencoder-neural-network cnn-architecture deep-learning exploratory-data-analysis gan image-processing unet-keras

Last synced: 2 months ago
JSON representation

DATA: 606 | Capstone Project

Awesome Lists containing this project

README 

        
# Dual-Dataset Deblurring Project



## Overview

This project leverages deep learning to deblur images under both normal and low-light conditions using a dual-dataset approach. Our goal is to enhance mobile photography by improving image clarity impacted by various blurs, focusing on Gaussian blur due to computational constraints. 


**This Research demands very high computational power**



## Datasets



### HDR+ Dataset

- **Description**: High-quality images aimed at enhancing mobile photography under diverse lighting conditions, stored in raw DNG format with metadata.

- **License**: Creative Commons (CC-BY-SA)

- **Samples Used**: 200



### ExDark Dataset

- **Description**: Images captured in very low-light conditions, designed to enhance low-light image processing for object detection and recognition.

- **License**: Available for academic research.

- **Samples Used**: 300



## Methodology



### Models Employed

- **On HDR+ Dataset (200 Samples)**:

  - Convolutional Neural Networks (CNN)

  - AutoEncoders

  - U-Net

- **On ExDark Dataset (300 Samples)**:

  - Generative Adversarial Networks (GANs)

  - CNN

  - AutoEncoders

  - U-Net

- **On Combined Dataset (500 samples)**:

  - Advanced CNNs for optimal deblurring.



### Workflow

1. **Data Loading**: Import and prepare HDR+ and ExDark datasets.

2. **Exploratory Data Analysis (EDA)**: Analyze the datasets to understand the effects of Gaussian blur.

3. **Model Training**:

   - Independently on HDR+ and ExDark datasets.

   - On the merged dataset for enhanced results.

4. **Deblurring**: Evaluate the deblurring performance using SSIM, PSNR values, Validation/Loss Graph and Image Clarity.



## Results

The implementation of complex CNNs on the merged dataset significantly enhanced image clarity, demonstrating superior deblurring performance. Below is an example of the deblurring results:



![Deblurred Image Example](https://i.imgur.com/CVMoQig.png)



- Training History



![Training History](https://i.imgur.com/nBBNxNu.png)



Average SSIM: 0.4122613634069845

Average PSNR: 13.250429498226481



## Conclusion



This project has successfully demonstrated the potential of using deep learning techniques to deblur images under normal and low-light conditions. By employing a dual-dataset approach with the HDR+ and ExDark datasets, we have developed models that significantly enhance the quality of blurred images. The most effective results were achieved using a complex CNN architecture, which provided the clearest deblurred images among the tested models.



Throughout this research, we focused on Gaussian blur due to computational constraints, yet the outcomes suggest promising avenues for further development. Future enhancements, such as expanding the types of blur processed, increasing the sample size, integrating additional datasets, and implementing real-time deblurring, are expected to broaden the project's impact and applicability.



The integration of more advanced deep learning techniques and automated model selection could further refine the deblurring process, tailoring solutions to the specific characteristics of each image. This would not only improve the model's performance but also its practical utility in real-world applications, such as in mobile photography and real-time video processing.



In conclusion, the Dual-Dataset Deblurring Project serves as a foundational step towards more sophisticated image processing capabilities, setting the stage for innovative future work that could transform how we enhance and utilize digital images in various lighting conditions.



## Technologies Used

- Python

- TensorFlow/Keras

- OpenCV

- Google Colab Pro+



## Datasets

- HDR+ Burst Photography Dataset: Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F.,

Chen, J., & Levoy, M. (2016). Burst photography for high dynamic range and low-light imaging on mobile

cameras. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2016), 35(6).

http://hdrplusdata.org/dataset.html

- ExDark Dataset Loh, Y. P., Chan, C. S., Por, L. Y., & Ling, C. Y. (2018). Getting to Know Low-light Images with

The Exclusively Dark Dataset. arXiv preprint arXiv:1805.11227.

https://paperswithcode.com/dataset/exdark



## Usage

To use this project effectively, follow these detailed steps:



1. **Obtain the Datasets**:

   - Download the HDR+ and ExDark datasets from their respective sources.

   - Due to licensing, the full datasets cannot be directly included in this repository.



2. **Setup Google Drive**:

   - Upload the downloaded datasets to your Google Drive in a specific folder, e.g., `datasets/your-project-name`.



3. **Google Colab Setup**:

   - Download the provided Jupyter notebook from the `notebooks/` directory of this repository.

   - Upload the notebook to your Google Drive, preferably in the same project folder where the datasets are stored.



4. **Open with Google Colab**:

   - Navigate to Google Drive and open the uploaded notebook with Google Colab.

   - Ensure that the notebook is configured to use the correct path to the datasets in your Google Drive.



5. **Consider Upgrading to Google Colab Pro+**:

   - Since this project requires significant computational power to process and deblur the images effectively, consider subscribing to Google Colab Pro+. This will provide you with enhanced access to faster GPUs and longer runtime, which are crucial for training deep learning models efficiently.



6. **Run the Notebook**:

   - Follow the instructions within the notebook to train the models and perform image deblurring. Make sure all paths in the code are correctly set to where your data is stored in Google Drive.



## Limitations and Restrictions



### Computational Requirements

- **High Computational Power**: This project demands significant computational resources, which may not be accessible to all users. High-performance computing environments are required for effective model training and evaluation.



### Data Handling

- **Dataset Size and Diversity**: The project currently utilizes only 500 samples from two datasets, potentially limiting the models' generalizability and robustness. Larger and more diverse datasets are needed for comprehensive results.

- **Raw Data Format**: The HDR+ dataset is in raw DNG format, requiring specific tools for processing, which may limit its usability.



### Licensing

- **Dataset Licenses**: Specific licenses govern the use of the datasets:

  - **HDR+ Dataset**: Licensed under Creative Commons (CC-BY-SA), requiring attribution and sharing under similar terms, possibly limiting commercial use.

  - **ExDark Dataset**: Restricted to academic research, limiting its use in commercial or non-academic environments.



### Model Complexity

- **Advanced Model Architectures**: The project uses complex models, such as Advanced CNNs, which may be challenging for users without deep neural network expertise.



### Specificity to Blur Types

- **Focus on Gaussian Blur**: Models are currently optimized for Gaussian blur, potentially underperforming with other types of blurs like motion or defocus blur.



### Real-Time Processing

- **Lack of Real-Time Capability**: The project does not currently include real-time deblurring capabilities for mobile devices or cameras.



### Future Enhancements

- **Dependence on Future Improvements**: The project's success and utility depend on future developments, including more datasets, expanded blur types, and real-time processing capabilities.



## Future Directions



Following the success of our current model in deblurring images using a dual-dataset approach and a focus on Gaussian blur, we outline several promising avenues for further research and development:



### Exploring Various Types of Blurs

- **Objective**: Extend the model's capability to handle various types of blurs beyond Gaussian. This includes motion blur, defocus blur, and others which frequently occur in real-world scenarios.

- **Rationale**: Diversifying the types of blurs addressed can enhance the model's robustness and applicability across different photographic conditions and challenges.



### Integrating More Datasets

- **Objective**: Incorporate additional datasets to train the models, particularly those capturing a broader range of environments and conditions.

- **Rationale**: Adding more datasets can improve the model's generalizability and performance across a wider array of lighting conditions and camera settings.



### Real-Time Deblurring

- **Objective**: Develop a real-time deblurring system that can be implemented in mobile devices or in-camera software.

- **Rationale**: Providing real-time processing capabilities would significantly enhance user experience by delivering high-quality images instantaneously during the capture process.



### Increasing Sample Size

- **Objective**: Significantly increase the number of images used in training and testing the models.

- **Rationale**: Expanding the sample size will help in enhancing the statistical significance of the results and improve the robustness of the models across varied scenarios. This approach is particularly critical given that only 500 samples from combined datasets were used initially.



### Automated Model Selection

- **Objective**: Create a system that can automatically select the optimal model based on the specific characteristics of the input image.

- **Rationale**: Automating model selection can maximize performance by tailoring the processing to the specific needs of each image, enhancing efficiency and effectiveness in diverse scenarios.





Implementing these directions will require addressing computational limitations, possibly through optimized algorithms or more powerful computing resources. The success in these areas could significantly push the boundaries of what's currently possible in image processing technology.



## Contributing

Interested in contributing? Great! Please open an issue to discuss proposed changes or submit a pull request.



## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details.



```bibtex

@misc{Enhancing Motion Blurred Images in normal and low light conditions,

  author = {Abhinav Mishra, Anilkumar Reddy, Ramesh Chandra},

  title = {Deblurring Motion Blurred Images in Normal and Low Light: Deep Learning-based Deblurring Using Dual Datasets},

  year = {2024},

  publisher = {GitHub},

  journal = {https://github.com/amishra15/enhancing-motion-blurred-images-in-normal-and-low-light-condition-for-mobile-photography},

  howpublished = {\\https://github.com/amishra15/enhancing-motion-blurred-images-in-normal-and-low-light-condition-for-mobile-photography}}

}