{"id":15628099,"url":"https://github.com/yashk2810/image-captioning","last_synced_at":"2025-07-21T20:33:41.405Z","repository":{"id":77964812,"uuid":"92924815","full_name":"yashk2810/Image-Captioning","owner":"yashk2810","description":"Image Captioning using InceptionV3 and beam search","archived":false,"fork":false,"pushed_at":"2020-08-26T09:37:23.000Z","size":78260,"stargazers_count":327,"open_issues_count":4,"forks_count":122,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-31T09:08:18.395Z","etag":null,"topics":["beam-search","cnn","image-captioning","keras","lstm","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yashk2810.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-05-31T08:44:20.000Z","updated_at":"2025-03-21T16:06:33.000Z","dependencies_parsed_at":"2023-02-24T15:45:42.356Z","dependency_job_id":null,"html_url":"https://github.com/yashk2810/Image-Captioning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yashk2810%2FImage-Captioning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yashk2810%2FImage-Captioning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yashk2810%2FImage-Captioning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yashk2810%2FImage-Captioning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yashk2810","download_url":"https://codeload.github.com/yashk2810/Image-Captioning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247640462,"owners_count":20971557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beam-search","cnn","image-captioning","keras","lstm","tensorflow"],"created_at":"2024-10-03T10:20:53.064Z","updated_at":"2025-04-07T11:07:16.057Z","avatar_url":"https://github.com/yashk2810.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Image-Captioning using InceptionV3 and Beam Search\n\nUsing Flickr8k dataset since the size is 1GB. MS-COCO is 14GB!\n\nUsed \u003ca href=\"https://keras.io/\"\u003eKeras\u003c/a\u003e with \u003ca href=\"https://www.tensorflow.org/\"\u003eTensorflow\u003c/a\u003e backend for the code. **InceptionV3** is used for extracting the features.\n\nI am using Beam search with **k=3, 5, 7** and an Argmax search for predicting the captions of the images.\n\nThe loss value of **1.5987** has been achieved which gives good results. You can check out some examples below. The rest of the examples are in the jupyter notebook. You can run the Jupyter Notebook and try out your own examples. *unique.p* is a pickle file which contains all the unique words in the vocabulary. \n\nEverything is implemented in the Jupyter notebook which will hopefully make it easier to understand the code.\n\nI have also written a blog post describing my experience while implementing this project. You can find it \u003ca href=\"https://yashk2810.github.io/Image-Captioning-using-InceptionV3-and-Beam-Search/\"\u003ehere\u003c/a\u003e.\n\nYou can download the weights \u003ca href='https://github.com/yashk2810/Image-Captioning/raw/master/weights/time_inceptionV3_2.8876_loss.h5'\u003ehere\u003c/a\u003e.\n\n# Examples\n\n![\"first2\"](https://raw.githubusercontent.com/yashk2810/Image-Captioning/master/images/first2.jpg \"first2\")\n![\"second2\"](https://raw.githubusercontent.com/yashk2810/Image-Captioning/master/images/second2.jpg \"second2\")\n![\"third\"](https://raw.githubusercontent.com/yashk2810/Image-Captioning/master/images/third.jpg \"third\")\n![\"last1\"](https://raw.githubusercontent.com/yashk2810/Image-Captioning/master/images/last1.jpg \"last1\")\n\n# Dependencies\n\n* Keras 1.2.2\n* Tensorflow 0.12.1\n* tqdm\n* numpy\n* pandas\n* matplotlib\n* pickle\n* PIL\n* glob\n\n# References\n\n[1] M. Hodosh, P. Young and J. Hockenmaier (2013) \"Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics\", Journal of Artificial Intelligence Research, Volume 47, pages 853-899 \u003ca href=\"http://www.jair.org/papers/paper3994.html\"\u003ehttp://www.jair.org/papers/paper3994.html\u003c/a\u003e \n\n[2] Oriol Vinyals, Alexander Toshev, Samy Bengio, Dumitru Erhan \u003ca href=\"https://arxiv.org/abs/1411.4555\"\u003eShow and Tell: A Neural Image Caption Generator\u003c/a\u003e\n\n[3] CS231n Winter 2016 Lesson 10 Recurrent Neural Networks, Image Captioning and LSTM \u003ca href=\"https://youtu.be/cO0a0QYmFm8?t=32m25s\"\u003ehttps://youtu.be/cO0a0QYmFm8?t=32m25s\u003c/a\u003e \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyashk2810%2Fimage-captioning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyashk2810%2Fimage-captioning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyashk2810%2Fimage-captioning/lists"}