{"id":20585855,"url":"https://github.com/harshit433/image-captioning-cantilever","last_synced_at":"2026-04-25T03:11:31.561Z","repository":{"id":250414038,"uuid":"834409572","full_name":"harshit433/Image-Captioning-Cantilever","owner":"harshit433","description":"This project is a Flask web application that allows users to upload images and generate captions for them using a custom AI model. The model utilizes EfficientNet for the Convolutional Neural Network (CNN) component, a custom Long Short-Term Memory (LSTM) network, and a multihead attention layer. The model has an accuracy of 80%.","archived":false,"fork":false,"pushed_at":"2024-12-18T22:34:55.000Z","size":3025,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-16T23:12:46.920Z","etag":null,"topics":["artificial-intelligence","attention-mechanism","cnn-keras","deep-learning","efficientnet","flask-application","html-css","keras-tensorflow","lstm-neural-networks","python","webapp"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/harshit433.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-27T07:20:15.000Z","updated_at":"2024-12-18T22:35:13.000Z","dependencies_parsed_at":"2024-12-18T23:23:33.130Z","dependency_job_id":"e75edf1b-066d-40c5-98cb-4cdd7562404c","html_url":"https://github.com/harshit433/Image-Captioning-Cantilever","commit_stats":null,"previous_names":["harshit433/image-captioning-cantilever-"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harshit433%2FImage-Captioning-Cantilever","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harshit433%2FImage-Captioning-Cantilever/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harshit433%2FImage-Captioning-Cantilever/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harshit433%2FImage-Captioning-Cantilever/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/harshit433","download_url":"https://codeload.github.com/harshit433/Image-Captioning-Cantilever/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242213559,"owners_count":20090694,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","attention-mechanism","cnn-keras","deep-learning","efficientnet","flask-application","html-css","keras-tensorflow","lstm-neural-networks","python","webapp"],"created_at":"2024-11-16T07:09:33.054Z","updated_at":"2026-04-25T03:11:31.521Z","avatar_url":"https://github.com/harshit433.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Image Captioning Project\n\nThis project is a Flask web application that allows users to upload images and generate captions for them using a custom AI model. The model utilizes EfficientNet for the Convolutional Neural Network (CNN) component, a custom Long Short-Term Memory (LSTM) network, and a multihead attention layer. The model has an accuracy of 80%.\n\n## ScreenShots\n\n#### HomePage\n![img_cap1](https://github.com/user-attachments/assets/b7411d67-2267-454b-90ec-5fc4a4e38eb8)\n\n#### Upload Image\n![img_cap2](https://github.com/user-attachments/assets/df9f81ec-5216-4c07-afbd-fa6da4a027fa)\n\n#### Image Uploaded\n![img_cap3](https://github.com/user-attachments/assets/9da11943-a789-4ac3-ad60-c7751419718f)\n\n#### Image captioned\n![img_cap4](https://github.com/user-attachments/assets/5a6f959d-6785-4b67-9900-3e8a2e763567)\n\n## Directory Structure\n```markdown\nImage Captioning Project\n│\n├── app.py\n├── prediction.py\n├── templates\n│   └── index.html\n├── static\n│   ├── css\n│   ├── images\n│   └── media\n├── Models\n│   ├── model.h5\n│   └── tokenizer.json\n├── test images\n└── model_training.pynb\n```\n\n- **app.py**: Main Flask application file.\n- **prediction.py**: Contains the logic for image captioning using the AI model.\n- **templates/index.html**: HTML template for the main page.\n- **static/css**: Directory for CSS files.\n- **static/images**: Directory for image files.\n- **static/media**: Directory for media files.\n- **Models**: Directory containing the pre-trained model and tokenizer.\n- **test images**: Directory containing test images.\n- **model_training.pynb**: Jupyter notebook containing the code for training the AI model.\n\n## Setup and Installation\n\n### Prerequisites\n\n- Python 3.8 or higher\n- Pip (Python package installer)\n- Jupyter Notebook (for running `model_training.pynb`)\n\n### Installation\n\n1. **Clone the repository:**\n   ```bash\n   git clone https://github.com/harshit433/Image-Captioning-Cantilever-.git\n   cd Image Captioning Project\n   ```\n\n2. **Create and activate a virtual environment:**\n   ```bash\n   python -m venv venv\n   source venv/bin/activate  # On Windows, use `venv\\Scripts\\activate`\n   ```\n\n3. **Install the required packages:**\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n### Create `requirements.txt`\n\nIf `requirements.txt` is not already present, you can generate it with the following command after installing the necessary packages:\n\n```bash\npip freeze \u003e requirements.txt\n```\n\n### Running the Application\n\n1. **Start the Flask application:**\n   ```bash\n   python app.py\n   ```\n\n2. **Open your browser and go to:**\n   ```\n   http://127.0.0.1:5000/\n   ```\n\n### Training the Model\n\nTo train the model, open the `model_training.pynb` file in Jupyter Notebook and run the cells. This notebook contains the code for training the AI model using EfficientNet for the CNN component, a custom LSTM network, and a multihead attention layer.\n\n### Usage\n\n1. **Upload an Image:** Click on the \"Choose File\" button to select an image from your computer.\n2. **Generate Caption:** After selecting the image, click on the \"Upload\" button to generate a caption for the image.\n3. **View Result:** The generated caption and the uploaded image will be displayed on the same page.\n\n## Project Details\n\n### app.py\n\nThis is the main Flask application file which handles the web server, routes, and the logic for handling image uploads and generating captions.\n\n### prediction.py\n\nThis file contains the core logic for generating captions using a custom AI model. The model utilizes EfficientNet for the CNN component, a custom LSTM network, and a multihead attention layer. The model and tokenizer are loaded from the `Models` directory.\n\n### model_training.pynb\n\nThis Jupyter notebook contains the code for training the AI model. It includes data preprocessing, model architecture, training loop, and evaluation metrics.\n\n### templates/index.html\n\nThis HTML file serves as the front-end for the application where users can upload images and view the generated captions.\n\n### static\n\n- **css**: This directory is intended for any CSS files needed for styling the web pages.\n- **images**: This directory can be used to store images used in the project.\n- **media**: This directory is used to store the user uploaded images.\n\n### Models\n\nThis directory contains the pre-trained model (`model.h5`) and the tokenizer (`tokenizer.json`) used for generating captions.\n\n### test images\n\nThis directory can be used to store images for testing the application.\n\n## Model Details\n\n- **CNN Component**: Utilizes EfficientNet for feature extraction from images.\n- **LSTM Network**: A custom LSTM network is used for sequence generation.\n- **Multihead Attention Layer**: Enhances the model's ability to focus on different parts of the image when generating captions.\n- **Accuracy**: The model has an accuracy of 42%.\n\n## Contributing\n\nFeel free to fork this repository and make your changes. Pull requests are welcome.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharshit433%2Fimage-captioning-cantilever","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fharshit433%2Fimage-captioning-cantilever","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharshit433%2Fimage-captioning-cantilever/lists"}