{"id":14977134,"url":"https://github.com/imsanko/image_caption_generator_with_transformers","last_synced_at":"2025-10-27T23:31:48.614Z","repository":{"id":236347892,"uuid":"792424330","full_name":"imSanko/Image_Caption_Generator_With_Transformers","owner":"imSanko","description":"This repository contains code for generating captions for images using a Transformer-based model. The model used is the `VisionEncoderDecoderModel` from the Hugging Face Transformers library, specifically the `nlpconnect/vit-gpt2-image-captioning` model.","archived":true,"fork":false,"pushed_at":"2024-09-02T16:30:06.000Z","size":239,"stargazers_count":9,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-11T14:42:58.185Z","etag":null,"topics":["collaboration","ipython-notebook","jypyternotebook","llm","pretrained-language-model","pthon3","transfromers"],"latest_commit_sha":null,"homepage":"https://imsanko.github.io/Image_Caption_Generator_With_Transformers/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/imSanko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["iMsanko"],"patreon":"user?u=12720438","custom":["http://bit.ly/iMsanko"]}},"created_at":"2024-04-26T16:33:45.000Z","updated_at":"2024-10-07T15:21:14.000Z","dependencies_parsed_at":"2024-04-26T17:47:20.001Z","dependency_job_id":"6a9085ba-37a3-4338-9de9-46399f2fade3","html_url":"https://github.com/imSanko/Image_Caption_Generator_With_Transformers","commit_stats":{"total_commits":38,"total_committers":3,"mean_commits":"12.666666666666666","dds":0.1842105263157895,"last_synced_commit":"4291ea3f962cb5a3a02a1b2de51a8d43697a5be6"},"previous_names":["imsanko/image_caption_generator_with_transformers"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imSanko%2FImage_Caption_Generator_With_Transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imSanko%2FImage_Caption_Generator_With_Transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imSanko%2FImage_Caption_Generator_With_Transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/imSanko%2FImage_Caption_Generator_With_Transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/imSanko","download_url":"https://codeload.github.com/imSanko/Image_Caption_Generator_With_Transformers/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219860467,"owners_count":16556017,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collaboration","ipython-notebook","jypyternotebook","llm","pretrained-language-model","pthon3","transfromers"],"created_at":"2024-09-24T13:55:10.589Z","updated_at":"2025-10-27T23:31:48.264Z","avatar_url":"https://github.com/imSanko.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/iMsanko","https://patreon.com/user?u=12720438","http://bit.ly/iMsanko"],"categories":[],"sub_categories":[],"readme":"---\n\n# Image Caption Generator With Transformers\n\nThis repository contains code for generating captions for images using a Transformer-based model. The model used is the `VisionEncoderDecoderModel` from the Hugging Face Transformers library, specifically the `nlpconnect/vit-gpt2-image-captioning` model.\n\n## Installation\n\nTo run this code, you'll need to install the following packages:\n\n- [Transformers](https://huggingface.co/transformers/)\n- [PyTorch](https://pytorch.org/)\n- [Pillow (Python Imaging Library)](https://python-pillow.org/)\n\nCertainly! Here's an updated installation section with more detailed instructions:\n\n---\n\n## Installation\n\n### 1. Clone the Repository\n\nFirst, clone this repository to your local machine using Git:\n\n```bash\ngit clone https://github.com/your-username/Image_Caption_Generator_With_Transformers.git\n```\n\nReplace `your-username` with your GitHub username.\n\n### 2. Install Required Packages\n\nNavigate to the cloned repository and install the required Python packages using pip:\n\n```bash\ncd Image_Caption_Generator_With_Transformers\npip install -r requirements.txt\n```\n\n### 3. Download Pre-trained Model and Tokenizer\n\nDownload the pre-trained `nlpconnect/vit-gpt2-image-captioning` model and tokenizer from the Hugging Face model hub using the `transformers` library:\n\n```bash\npython download_model.py\n```\n\nThis will download the necessary model files and save them to the `models` directory.\n\n### 4. Verify Installation\n\nTo verify that the installation was successful, you can run the provided example usage code:\n\n```bash\npython example_usage.py\n```\n\nThis will generate captions for a sample image (`sample.jpg`) and print the captions to the console.\n\n---\n\n## Usage\n\n1. Import the required libraries and load the pre-trained model and tokenizer:\n\n```python\nfrom transformers import VisionEncoderDecoderModel, ViTFeatureExtractor, AutoTokenizer\nimport torch\nfrom PIL import Image\n```\n\n2. Call the `predict_step` function with a list of image paths to generate captions:\n\n```python\ncaptions = predict_step(['sample2.jpg'])\nprint(captions)\n```\n\nThis will output the generated captions for the given image(s).\n\nThe provided code includes an example usage:\n\n```python\npredict_step(['sample2.jpg'])\n```\n\n---\n\n## Deployment\n\n### 1. Prepare Your Streamlit App\n\nMake sure your Streamlit app (`app.py`) is ready for deployment. Ensure that it includes all necessary dependencies and functionality.\n\n### 2. Create a Requirements File\n\nCreate a `requirements.txt` file in your project directory listing all the dependencies needed by your Streamlit app. You can generate this file using `pip freeze \u003e requirements.txt` if you're using a virtual environment.\n\n### 3. Set Up a GitHub Repository\n\nIf you haven't already, set up a GitHub repository for your Streamlit app. Push your `app.py` and `requirements.txt` files to this repository.\n\n### 4. Deploy on Streamlit Sharing\n\n1. Go to [Streamlit Sharing](https://share.streamlit.io/) and sign in with your GitHub account.\n2. Click on \"New app\" and select your GitHub repository.\n3. Configure the settings for your app (e.g., branch, path to `app.py`).\n4. Click on \"Deploy\" to deploy your Streamlit app.\n\n### 5. Monitor Deployment\n\nStreamlit will start building and deploying your app. You can monitor the deployment process in the Streamlit Sharing dashboard.\n\n### 6. View Your App\n\nOnce deployed, you can access your Streamlit app using the provided URL. Share this URL with others to showcase your app.\n\n### 7. Update Your App\n\nIf you make changes to your app, simply push the changes to your GitHub repository. Streamlit Sharing will automatically redeploy your app with the new changes.\n\n### 8. Manage Your App\n\nYou can manage your deployed app in the Streamlit Sharing dashboard. From here, you can view logs, change settings, and monitor usage.\n\n---\n\n### Docker Deployment\n\n```Dockerfile\n# Use the official Python image as the base image\nFROM python:3.9\n\n# Set the working directory in the container\nWORKDIR /app\n\n# Copy the requirements file\nCOPY requirements.txt .\n\n# Install the required packages\nRUN pip install --no-cache-dir -r requirements.txt\n\n# Copy the rest of the application code\nCOPY . .\n\n# Expose the port for the Streamlit app (default is 8501)\nEXPOSE 8501\n\n# Run the Streamlit app\nCMD [\"streamlit\", \"run\", \"app.py\"]\n```\n\nHere's what the different parts of the Dockerfile do:\n\n1. `FROM python:3.12`: This line specifies the base image for your Docker container. In this case, we're using the official Python 3.12 image.\n\n2. `WORKDIR /app`: This sets the working directory inside the container to `/app`.\n\n3. `COPY requirements.txt .`: This copies the `requirements.txt` file from your local machine to the container's working directory.\n\n4. `RUN pip install --no-cache-dir -r requirements.txt`: This line installs the Python packages listed in the `requirements.txt` file.\n\n5. `COPY . .`: This copies the entire contents of your local project directory (including the Streamlit app code) to the container's working directory.\n\n6. `EXPOSE 8501`: This exposes port 8501 in the container, which is the default port that Streamlit runs on.\n\n7. `CMD [\"streamlit\", \"run\", \"app.py\"]`: This is the command that will be executed when the container starts. It runs the `streamlit run app.py` command to start the Streamlit app.\n\nTo build the Docker image, navigate to the directory containing the Dockerfile and run the following command:\n\n```\ndocker build -t image-caption-generator .\n```\n\nThis will build a Docker image with the tag `image-caption-generator`.\n\nTo run the container and start the Streamlit app, use the following command:\n\n```\ndocker run -p 8501:8501 image-caption-generator\n```\n\nThis command maps the container's port 8501 to the host's port 8501, so you can access the Streamlit app in your web browser at `http://localhost:8501`.\n\nMake sure to replace `app.py` with the name of your Streamlit app file if it's different.\n\nWith this Dockerfile, you can easily build and run your Streamlit image caption generator app in a Docker container, ensuring a consistent and isolated environment for your application.\n\n---\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimsanko%2Fimage_caption_generator_with_transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimsanko%2Fimage_caption_generator_with_transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimsanko%2Fimage_caption_generator_with_transformers/lists"}