{"id":21383242,"url":"https://github.com/sreeeswaran/image-captioning-transformer","last_synced_at":"2025-03-16T11:24:02.461Z","repository":{"id":246573121,"uuid":"821519198","full_name":"SreeEswaran/Image-Captioning-Transformer","owner":"SreeEswaran","description":"This project demonstrates an image captioning model using a Transformer architecture. The model takes an image as input and generates a descriptive caption. We use the COCO dataset for training and evaluation.","archived":false,"fork":false,"pushed_at":"2024-11-16T16:20:07.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-22T23:25:25.490Z","etag":null,"topics":["coco","coco-dataset","image-caption-generator","image-captioning","model","neural-networks","transformer","transformer-models","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SreeEswaran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-28T18:17:02.000Z","updated_at":"2024-11-16T16:20:10.000Z","dependencies_parsed_at":"2024-11-16T17:30:26.504Z","dependency_job_id":null,"html_url":"https://github.com/SreeEswaran/Image-Captioning-Transformer","commit_stats":null,"previous_names":["sreeeswaran/image-captioning-transformer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SreeEswaran%2FImage-Captioning-Transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SreeEswaran%2FImage-Captioning-Transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SreeEswaran%2FImage-Captioning-Transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SreeEswaran%2FImage-Captioning-Transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SreeEswaran","download_url":"https://codeload.github.com/SreeEswaran/Image-Captioning-Transformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243859473,"owners_count":20359537,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coco","coco-dataset","image-caption-generator","image-captioning","model","neural-networks","transformer","transformer-models","transformers"],"created_at":"2024-11-22T11:21:03.543Z","updated_at":"2025-03-16T11:24:02.438Z","avatar_url":"https://github.com/SreeEswaran.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Image-Captioning-Transformer\n\nThis project demonstrates an image captioning model using a Transformer architecture. The model takes an image as input and generates a descriptive caption. We use the COCO dataset for training and evaluation.\n\n## Project Overview\n\nImage captioning is a challenging task that involves understanding the content of an image and describing it in natural language. In this project, we build an image captioning model using a Transformer-based architecture. The model is trained on the COCO dataset and can generate captions for new images.\n\n## Dataset\n\nWe use the COCO (Common Objects in Context) dataset for training and evaluating the model. The dataset contains images of complex scenes with various objects, along with annotations describing the objects and their relationships.\n\n### Download Dataset\n\nDownload the COCO dataset from [here](http://cocodataset.org/#download).\n\n### Preprocessing\n\nThe dataset needs to be preprocessed before feeding it into the model. The preprocessing steps include resizing images, tokenizing captions, and creating data loaders for training and evaluation.\n\n## Model Architecture\n\nThe image captioning model consists of two main components:\n1. **Encoder**: A convolutional neural network (CNN) that extracts features from the input image.\n2. **Decoder**: A Transformer model that generates captions based on the extracted image features.\n\nThe encoder is a pre-trained CNN (e.g., ResNet-50), and the decoder is a Transformer with self-attention mechanisms.\n\n## Installation\n\n1. Clone the repository:\n    ```bash\n    git clone https://github.com/SreeEswaran/Image-Captioning-Transformer.git\n    cd Image-Captioning-Transformer\n    ```\n\n2. Install the required dependencies:\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n3. Download and preprocess the COCO dataset as described above.\n\n## Usage\n\n### Training\n\nTo train the model, run:\n```bash\npython train_model.py --config configs/train_config.yaml\n```\n\u003c!--python train_model.py --config configs/train_config.yaml--\u003e\n\n\nTo genearte the images, run:\n```bash\npython infer.py --image_path path/to/your/image.jpg --model_path path/to/saved/model.pth\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsreeeswaran%2Fimage-captioning-transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsreeeswaran%2Fimage-captioning-transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsreeeswaran%2Fimage-captioning-transformer/lists"}