{"id":19204297,"url":"https://github.com/wangzheallen/stl-vqa","last_synced_at":"2026-03-08T07:31:44.340Z","repository":{"id":80125466,"uuid":"112789401","full_name":"wangzheallen/STL-VQA","owner":"wangzheallen","description":"The good practice in the VQA system such as pos-tag attention, structed triplet learning and triplet attention is very general and can be inserted into almost any visual and language task","archived":false,"fork":false,"pushed_at":"2018-01-23T22:31:49.000Z","size":1274,"stargazers_count":19,"open_issues_count":1,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-09-23T21:32:44.393Z","etag":null,"topics":["deep-learning","practice","tensorflow","vision-and-language","vqa"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wangzheallen.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-12-01T21:38:02.000Z","updated_at":"2025-01-21T03:56:47.000Z","dependencies_parsed_at":"2023-03-02T21:45:31.681Z","dependency_job_id":null,"html_url":"https://github.com/wangzheallen/STL-VQA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/wangzheallen/STL-VQA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangzheallen%2FSTL-VQA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangzheallen%2FSTL-VQA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangzheallen%2FSTL-VQA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangzheallen%2FSTL-VQA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wangzheallen","download_url":"https://codeload.github.com/wangzheallen/STL-VQA/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangzheallen%2FSTL-VQA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30248864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-08T05:41:50.788Z","status":"ssl_error","status_checked_at":"2026-03-08T05:41:39.075Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","practice","tensorflow","vision-and-language","vqa"],"created_at":"2024-11-09T13:07:07.597Z","updated_at":"2026-03-08T07:31:44.324Z","avatar_url":"https://github.com/wangzheallen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Structured Triplet Learning with Pos-tag Guided Attention for Visual Question Answering\r\nThis is the code for \"Structured Triplet Learning with Pos-tag Guided Attention for Visual Question Answering, WACV 2018 (Zhe Wang, Xiaoyi Liu, Liangjian Chen, Limin Wang, Yu Qiao, Xiaohui Xie, Charless Fowlkes)\", The good practice in the VQA system such as pos-tag attention, structured triplet learning and triplet attention is very general and can be inserted into almost any visual and language task.\r\n\r\nIf you find the code useful, please cite the paper: \r\n##### Structured Triplet Learning with Pos-tag Guided Attention for Visual Question Answering WACV 2018 (Zhe Wang, Xiaoyi Liu, Liangjian Chen, Limin Wang, Yu Qiao, Xiaohui Xie, Charless Fowlkes)\r\nIf you have feedback for the code, please contact:\r\n##### buptwangzhe2012 at gmail dot com\r\n\r\n# Performance\r\n\r\nBelow is the step by step effectiveness verification of our method, note to speed up the verification, we use the 7by7 feature instead of 14by14 feature\r\n\r\n| Method | V7W | VQA validation |\r\n| ------ | ------ | ------ |\r\n| Our Baseline | 65.6 | 58.3 |\r\n| +POS tag guided attention (POS-Att) | 66.3 | 58.7 |\r\n| +Convolutional N-Gram (Conv N-Gram) |  66.2 | 59.3 |\r\n| +POS-Att +Conv N-Gram | 66.6 | 59.5 |\r\n| +POS-Att +Conv N-Gram +Triplet attention-Q | 66.8 | 60.1 |\r\n| +POS-Att +Conv N-Gram +Triplet attention-A | 67.0 | 60.1 |\r\n| +POS-Att +Conv N-Gram +Triplet attention-Q+A | 67.3 | 60.2 |\r\n| +POS-Att +Conv N-Gram +Triplet attention-Q+A + structured Learning Triplets | 67.5 | 60.3 |\r\n\r\nOur full model performance\r\n\r\n| Method | V7W Telling | VQA Test Standard | VQA Test Dev | VQA Test Dev Y/N | VQA Test Dev Num | VQA Test Dev Other |\r\n| ------ | ------ | ------ | ------ | ------ | ------ | ------ |\r\n| Ours | 68.2 | 69.6 | 69.7 | 81.9 | 44.3 | 64.7 |\r\n\r\n\r\n# Pre-requisite\r\n\r\ntensorflow, torch, pandas, h5py, ipdb, cv2, pdb, spacy, sklearn, matplotlib, PIL, nltk\r\n\r\n# Quick Demo\r\nDownload the V7W telling feature shared on https://drive.google.com/open?id=1Hofquxw22j8soyjE0vuZqxcNuvJd-e9V\r\nAnd run \"CUDA_VISIBLE_DEVICES=0 python v7w.py\"\r\n\r\n# Data pre-processing\r\n\r\nDownload Visual7W: http://web.stanford.edu/~yukez/visual7w/\r\nAnd glove: http://nlp.stanford.edu/data/wordvecs/glove.6B.zip  from https://github.com/stanfordnlp/GloVe\r\nDownload: https://d2j0dndfm35trm.cloudfront.net/resnet-200.t7\r\n\r\npython data_preprocessing_7w.py --data_set telling\r\n\r\npython prepro_7w.py\r\n\r\nth prepro_img_residule.lua\r\n\r\n# Visualization\r\n**Architecture**: \r\n\r\n![Architecture](https://github.com/wangzheallen/STL-VQA/blob/master/architecture.png )\r\n\r\n\r\n**Good Practice**: \r\n\r\npython comparisons_wacv.py\r\n\r\n![goodpractice](https://github.com/wangzheallen/STL-VQA/blob/master/goodpractice.png )\r\n**Good Samples**: \r\n\r\npython draw_heat_new.py\r\n\r\n![good samples](https://github.com/wangzheallen/STL-VQA/blob/master/goodsample.png )\r\n**Bad Samples**: \r\n![bad samples](https://github.com/wangzheallen/STL-VQA/blob/master/badsample.png )\r\n\r\n\r\n# License\r\n\r\nMIT \r\n\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwangzheallen%2Fstl-vqa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwangzheallen%2Fstl-vqa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwangzheallen%2Fstl-vqa/lists"}