{"id":28158029,"url":"https://github.com/anshkgoyal/image-enhancer","last_synced_at":"2025-05-15T09:17:30.207Z","repository":{"id":286331496,"uuid":"961087315","full_name":"AnshKGoyal/Image-Enhancer","owner":"AnshKGoyal","description":"Hybrid Image Enhancement platform combining FastAPI and Streamlit for customizable multi-stage pipelines using deep learning. Supports flexible deblurring and dehazing workflows, user authentication, and output history. Easily extendable architecture with pre-trained model weights for advanced image restoration.","archived":false,"fork":false,"pushed_at":"2025-04-05T18:43:25.000Z","size":19007,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T19:28:54.557Z","etag":null,"topics":["deblur","dehazing","fastapi-sqlalchemy","hybrid-architecture","streamlit-webapp","unet-pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnshKGoyal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-04-05T18:14:18.000Z","updated_at":"2025-04-05T18:50:43.000Z","dependencies_parsed_at":"2025-04-05T19:39:08.577Z","dependency_job_id":null,"html_url":"https://github.com/AnshKGoyal/Image-Enhancer","commit_stats":null,"previous_names":["anshkgoyal/image-enhancer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnshKGoyal%2FImage-Enhancer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnshKGoyal%2FImage-Enhancer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnshKGoyal%2FImage-Enhancer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnshKGoyal%2FImage-Enhancer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnshKGoyal","download_url":"https://codeload.github.com/AnshKGoyal/Image-Enhancer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254310484,"owners_count":22049470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deblur","dehazing","fastapi-sqlalchemy","hybrid-architecture","streamlit-webapp","unet-pytorch"],"created_at":"2025-05-15T09:15:57.532Z","updated_at":"2025-05-15T09:17:30.197Z","avatar_url":"https://github.com/AnshKGoyal.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hybrid Image Enhancement API and Frontend\n\nThis project provides a FastAPI backend and Streamlit frontend for **hybrid image enhancement** using deep learning models. It enables users to design **custom multi-stage pipelines** combining **deblurring** and **dehazing** in any sequence, for advanced and flexible image restoration workflows.\n\n---\n\n## Features\n\n- **Configurable Hybrid Pipelines:**  \n  Design custom image enhancement workflows by **adding multiple deblurring and dehazing stages sequentially**. This hybrid approach allows tailored, multi-step processing beyond single-model pipelines.\n\n- **Deep Learning Models:**  \n  - **Deblurring:** Custom-trained **NAFNet** model  \n  - **Dehazing:** Custom-trained **U-Net** with mit_b3 encoder  \n  Both models are integrated into the hybrid pipeline.\n\n- **User Authentication:**  \n  Secure login and registration system, with personalized output history.\n\n- **Output History:**  \n  View processed images, including intermediate and final outputs, linked to user accounts.\n\n- **Interactive Streamlit Frontend:**  \n  User-friendly interface for uploading images, configuring hybrid pipelines, running processing, and visualizing results.\n\n- **Scalable Architecture:**  \n  Modular design allows easy integration of new models and processing stages.\n\n---\n\n## Architecture\n\n### Backend (FastAPI)\n\n- **API Endpoints:**  \n  Provides RESTful endpoints for:\n  - User authentication (`/login`, `/users/`, `/user/detail/`)\n  - Hybrid pipeline execution (`/predict_pipeline/`)\n  - Model output management (`/model_outputs/global/`, `/model_outputs/user/`)\n\n- **Business Logic:**  \n  - **User Management:** Registration, login, token generation, and validation  \n  - **Image Uploading:** Saves images, computes hashes to avoid duplicates, associates images with users  \n  - **Hybrid Pipeline Execution:**  \n    - Accepts a user-defined sequence of stages (deblur/dehaze)  \n    - Runs each model sequentially, saving intermediate outputs  \n    - Stores all outputs in the database, linked to the original image and user  \n  - **Model Management:** Loads NAFNet and U-Net models at startup for inference  \n  - **Database:** SQLAlchemy ORM with SQLite/MySQL, storing users, images, pipeline outputs, and relationships  \n  - **Security:** Token-based authentication with per-user access control\n\n### Frontend (Streamlit)\n\n- **Pipeline Configuration:**  \n  Users can **add multiple processing stages sequentially** via an \"Add Stage\" button. For each stage, they select either **deblur** or **dehaze**. The pipeline executes stages **in the order added**.\n\n- **User Interface:**  \n  - Register and login  \n  - Upload images  \n  - Configure hybrid pipelines  \n  - Run processing and view outputs for each stage and final result  \n  - Browse global and user-specific output histories with filtering options\n\n- **API Client:**  \n  Communicates with the FastAPI backend for authentication, pipeline execution, and data retrieval.\n\n---\n\n## Demo Video\n\nWatch a demonstration of the application in action:  \n\n\nhttps://github.com/user-attachments/assets/a55d0d8d-de94-45aa-8475-0440dacdf8bf\n\n\n\n---\n\n## Model Training\n\n### Deblurring Model (NAFNet)\n\n- **Dataset Preparation:**  \n  - Loads a curated subset of the [A Curated List of Image Deblurring Datasets](https://www.kaggle.com/datasets/jishnuparayilshibu/a-curated-list-of-image-deblurring-datasets)  \n  - Excludes datasets like 'CelebA'  \n  - Maps paired blurry and sharp images  \n  - Creates train/test DataFrames  \n\n- **DataLoader Setup:**  \n  - Custom PyTorch `Dataset` class loads blurry and sharp pairs  \n  - Resizes images to 384x384, normalizes to [0,1]  \n  - DataLoader with batch size 16, shuffling, prefetching, pin_memory, persistent_workers  \n  - Batch visualization to verify data\n\n- **Model \u0026 Loss:**  \n  - NAFNet architecture with projected_in_channels=24, shallow-to-deep layers  \n  - Composite loss: weighted sum of  \n    - **PSNR Loss** (weight 0.7)  \n    - **Perceptual Loss** (VGG16 feature-based, weight 0.3)  \n    - No contribution from MSE or L1 losses (weights 0.0 and 0)\n\n- **Training:**  \n  - PyTorch Lightning framework  \n  - Mixed precision (`precision='16-mixed'`)  \n  - AdamW optimizer  \n  - Progress bar callback  \n\n- **Evaluation:**  \n  - Runs inference on validation samples  \n  - Visualizes blurry input, model output, and ground truth side by side  \n  - Saves final model weights\n\n---\n\n### Dehazing Model (U-Net with mit_b3 Encoder)\n\n- **Dataset Preparation:**  \n  - Uses [RESIDE-OUT](https://www.kaggle.com/datasets/anshkgoyal/reside-out) dataset  \n  - Pairs hazy and ground truth images, verifies filenames  \n  - Saves paired paths as CSVs  \n  - Loads CSVs into DataFrames  \n  - Visualizes samples for verification\n\n- **DataLoader Setup:**  \n  - Custom PyTorch `Dataset` class loads hazy and GT pairs  \n  - Resizes images to 384x384, normalizes to [0,1]  \n  - DataLoader with batch size 16, shuffling, prefetching, pin_memory, persistent_workers  \n  - Batch visualization to verify data\n\n- **Model \u0026 Loss:**  \n  - U-Net with mit_b3 encoder, pretrained on ImageNet, activation sigmoid  \n  - Composite loss: weighted sum of  \n    - **Perceptual Loss** (VGG16 feature-based, weight 0.4)  \n    - **MSE Loss** (weight 0.2)  \n    - **L1 Loss** (weight 0.4)\n\n- **Training:**  \n  - PyTorch Lightning framework  \n  - Mixed precision  \n  - AdamW optimizer  \n  - Progress bar callback  \n\n- **Evaluation:**  \n  - Runs inference on validation samples  \n  - Visualizes hazy input, model output, and ground truth side by side  \n  - Saves final model weights\n\n[kaggle notebook link](https://www.kaggle.com/code/anshkgoyal/dehazing)\n---\n\n---\n\n## Project Structure\n\n- `main.py`: FastAPI backend application with API endpoints and hybrid pipeline logic  \n- `frontend/app.py`: Streamlit frontend application with pipeline configuration and visualization  \n- `models.py`: SQLAlchemy models for users, images, and model outputs  \n- `schemas.py`: Pydantic schemas for API validation  \n- `utils.py`: Utilities for password hashing, image hashing, inference, and database helpers  \n- `helpers.py`: File saving, image preprocessing, and postprocessing  \n- `deeplearning_models.py`: PyTorch Lightning model classes for deblurring and dehazing  \n- `notebooks/`:  \n  - `deblurring.ipynb`: Training pipeline for NAFNet  \n  - `dehazing.ipynb`: Training pipeline for U-Net  \n- `models/`:  \n  - `deblurring/model_state_19_new.pth`  \n  - `dehazing/model_state_26_jan_final_epoch_plus25.pth`  \n- `output_images/`: Saved outputs from pipeline stages  \n- `uploaded_images/`: User-uploaded images  \n- `requirements.txt`: Python dependencies\n\n---\n\n## Setup and Installation\n\n### 0. Download Pre-trained Model Weights\n\n- Download the pre-trained model weights from Google Drive:  \n  [**Model Weights Download Link**](https://drive.google.com/drive/folders/1poqEPc01gX4ZgnO5plK3JLnyujLeJFnx?usp=sharing)\n\n- After downloading, extract and place the files into the `models/` directory as follows:\n\n```\nmodels/\n├── deblurring/\n│   └── model_state_19_new.pth\n├── dehazing/\n│   └── model_state_26_jan_final_epoch_plus25.pth\n```\n\n- **Ensure the folder structure matches above** so the application can load the weights correctly.\n\n- **Optional:** You can customize the model weights paths in `main.py` by editing:\n\n```python\nMODEL_WEIGHTS_PATH = r\"models\\\\deblurring\\\\model_state_19_new.pth\"\nMODEL_WEIGHTS_PATH_DEHAZING = r\"models\\\\dehazing\\\\model_state_26_jan_final_epoch_plus25.pth\"\n```\n\n---\n\n### 1. Clone the Repository\nDownload or clone this repository, then open the project directory in your preferred editor (e.g., VS Code).\n\n---\n\n### 2. Install MySQL Server\n- **Download and install MySQL Community Server:**  \n  https://dev.mysql.com/downloads/mysql/\n- **Start the MySQL server** and ensure it is running.\n\n---\n\n### 3. Create the Database\nOpen a terminal or MySQL client and run:\n\n```sql\nCREATE DATABASE photo_enhancer;\n```\n\n- By default, the app connects with username `root` and password `root` on `localhost:3306`.\n- **To customize credentials or host**, edit the connection string in `db.py`:\n\n```python\nSQLALCHEMY_DATABASE_URL = \"mysql+pymysql://\u003cusername\u003e:\u003cpassword\u003e@\u003chost\u003e:\u003cport\u003e/\u003cdatabase_name\u003e\"\n```\n\n---\n\n### 4. (Optional) Set up a Virtual Environment\nIt is recommended to use a virtual environment to manage dependencies:\n\n```bash\npython -m venv venv\n# On Linux/macOS:\nsource venv/bin/activate\n# On Windows:\nvenv\\Scripts\\activate\n```\n\n---\n\n### 5. Install Python Dependencies\nInstall all required packages:\n\n```bash\npip install -r requirements.txt\n```\n\n---\n\n### 6. Initialize the Database Schema\nNo manual migration is needed.  \n**The database tables will be created automatically** when you first run the backend server.\n\n---\n\n### 7. Run the Backend API Server\nStart the FastAPI backend with:\n\n```bash\nuvicorn main:app --reload\n```\n\nThe API will be available at `http://127.0.0.1:8000`.\n\n---\n\n### 8. Run the Streamlit Frontend\nIn a new terminal (with the virtual environment activated), run:\n\n```bash\nstreamlit run frontend/app.py\n```\n\nThe frontend will open in your browser, typically at `http://localhost:8501`.\n\n\n---\n\n## Usage\n\n1. **Register/Login** via the frontend.\n2. **Upload an image**.\n3. **Configure your hybrid pipeline** by **adding multiple stages sequentially** and selecting the model for each.\n4. **Run the pipeline** and view outputs for each stage and the final result.\n5. **Explore output history** (global and user-specific) with filtering options.\n\n---\n\n## Datasets\n\n- **Deblurring:**  \n  [A Curated List of Image Deblurring Datasets](https://www.kaggle.com/datasets/jishnuparayilshibu/a-curated-list-of-image-deblurring-datasets) (filtered subset)\n\n- **Dehazing:**  \n  [RESIDE-OUT](https://www.kaggle.com/datasets/anshkgoyal/reside-out)\n\n---\n\n## Licensing\n\n- **Code:** [Apache 2.0 License](https://github.com/AnshKGoyal/Image-Enhancer/blob/main/LICENSE)  \n- **Models:** [Apache 2.0 License](https://github.com/AnshKGoyal/Image-Enhancer/blob/main/LICENSE)\n\n---\n\n## Contributing\n\nContributions are welcome! Please fork the repository and submit pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanshkgoyal%2Fimage-enhancer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanshkgoyal%2Fimage-enhancer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanshkgoyal%2Fimage-enhancer/lists"}