{"id":20023278,"url":"https://github.com/cyanogenoid/pytorch-vqa","last_synced_at":"2025-04-07T06:07:49.994Z","repository":{"id":23751939,"uuid":"98831127","full_name":"Cyanogenoid/pytorch-vqa","owner":"Cyanogenoid","description":"Strong baseline for visual question answering","archived":false,"fork":false,"pushed_at":"2023-03-13T19:42:05.000Z","size":22,"stargazers_count":239,"open_issues_count":5,"forks_count":99,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-31T04:07:35.478Z","etag":null,"topics":["baseline","pytorch","visual-question-answering","vqa"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cyanogenoid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-30T23:41:38.000Z","updated_at":"2025-02-17T14:12:06.000Z","dependencies_parsed_at":"2024-11-13T08:45:53.006Z","dependency_job_id":"015563de-806c-4229-ab0d-242326f669f3","html_url":"https://github.com/Cyanogenoid/pytorch-vqa","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cyanogenoid%2Fpytorch-vqa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cyanogenoid%2Fpytorch-vqa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cyanogenoid%2Fpytorch-vqa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cyanogenoid%2Fpytorch-vqa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cyanogenoid","download_url":"https://codeload.github.com/Cyanogenoid/pytorch-vqa/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601448,"owners_count":20964864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["baseline","pytorch","visual-question-answering","vqa"],"created_at":"2024-11-13T08:45:43.462Z","updated_at":"2025-04-07T06:07:49.978Z","avatar_url":"https://github.com/Cyanogenoid.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Strong baseline for visual question answering\n\nThis is a re-implementation of Vahid Kazemi and Ali Elqursh's paper [Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering][0] in [PyTorch][1].\n\nThe paper shows that with a relatively simple model, using only common building blocks in Deep Learning, you can get better accuracies than the majority of previously published work on the popular [VQA v1][2] dataset.\n\nThis repository is intended to provide a straightforward implementation of the paper for other researchers to build on.\nThe results closely match the reported results, as the majority of details should be exactly the same as the paper. (Thanks to the authors for answering my questions about some details!)\nThis implementation seems to consistently converge to about 0.1% better results –\nthere are two main implementation differences:\n\n- Instead of setting a limit on the maximum number of words per question and cutting off all words beyond this limit, this code uses per-example dynamic unrolling of the language model.\n- [An issue with the official evaluation code](https://github.com/Cyanogenoid/pytorch-vqa/issues/5) makes some questions unanswerable. This code does not normalize machine-given answers, which avoids this problem. As the vast majority of questions are not affected by this issue, it's very unlikely that this will have any significant impact on accuracy.\n\nA fully trained model (convergence shown below) is [available for download][5].\n\n![Graph of convergence of implementation versus paper results](http://i.imgur.com/moWYEm8.png)\n\nNote that the model in [my other VQA repo](https://github.com/Cyanogenoid/vqa-counting) performs better than the model implemented here.\n\n\n## Running the model\n\n- Clone this repository with:\n```\ngit clone https://github.com/Cyanogenoid/pytorch-vqa --recursive\n```\n- Set the paths to your downloaded [questions, answers, and MS COCO images][4] in `config.py`.\n  - `qa_path` should contain the files `OpenEnded_mscoco_train2014_questions.json`, `OpenEnded_mscoco_val2014_questions.json`, `mscoco_train2014_annotations.json`, `mscoco_val2014_annotations.json`.\n  - `train_path`, `val_path`, `test_path` should contain the train, validation, and test `.jpg` images respectively.\n- Pre-process images (93 GiB of free disk space required for f16 accuracy) with [ResNet152 weights ported from Caffe][3] and vocabularies for questions and answers with:\n```\npython preprocess-images.py\npython preprocess-vocab.py\n```\n- Train the model in `model.py` with:\n```\npython train.py\n```\nThis will alternate between one epoch of training on the train split and one epoch of validation on the validation split while printing the current training progress to stdout and saving logs in the `logs` directory.\nThe logs contain the name of the model, training statistics, contents of `config.py`,  model weights, evaluation information (per-question answer and accuracy), and question and answer vocabularies.\n- During training (which takes a while), plot the training progress with:\n```\npython view-log.py \u003cpath to .pth log\u003e\n```\n\n\n## Python 3 dependencies (tested on Python 3.6.2)\n\n- torch\n- torchvision\n- h5py\n- tqdm\n\n\n\n[0]: https://arxiv.org/abs/1704.03162\n[1]: https://github.com/pytorch/pytorch\n[2]: http://visualqa.org/\n[3]: https://github.com/ruotianluo/pytorch-resnet\n[4]: http://visualqa.org/vqa_v1_download.html\n[5]: https://github.com/Cyanogenoid/pytorch-vqa/releases\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyanogenoid%2Fpytorch-vqa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyanogenoid%2Fpytorch-vqa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyanogenoid%2Fpytorch-vqa/lists"}