{"id":13784194,"url":"https://github.com/ap229997/Conditional-Batch-Norm","last_synced_at":"2025-05-11T19:32:33.218Z","repository":{"id":158764861,"uuid":"119690498","full_name":"ap229997/Conditional-Batch-Norm","owner":"ap229997","description":"Pytorch implementation of NIPS 2017 paper \"Modulating early visual processing by language\"","archived":false,"fork":false,"pushed_at":"2019-02-23T09:07:51.000Z","size":41,"stargazers_count":61,"open_issues_count":0,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-08-03T19:08:39.266Z","etag":null,"topics":["cbn","modulated-resnet","pytorch","vqa"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ap229997.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-01-31T13:26:08.000Z","updated_at":"2024-05-10T15:23:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"44314241-66d5-4f71-8d49-d0e589933d79","html_url":"https://github.com/ap229997/Conditional-Batch-Norm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ap229997%2FConditional-Batch-Norm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ap229997%2FConditional-Batch-Norm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ap229997%2FConditional-Batch-Norm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ap229997%2FConditional-Batch-Norm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ap229997","download_url":"https://codeload.github.com/ap229997/Conditional-Batch-Norm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225086567,"owners_count":17418750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cbn","modulated-resnet","pytorch","vqa"],"created_at":"2024-08-03T19:00:36.900Z","updated_at":"2024-11-17T20:31:37.129Z","avatar_url":"https://github.com/ap229997.png","language":"Python","funding_links":[],"categories":["2017"],"sub_categories":[],"readme":"## Conditional Batch Normalization\nPytorch implementation of NIPS 2017 paper \"Modulating early visual processing by language\" \n[[Link]](https://papers.nips.cc/paper/7237-modulating-early-visual-processing-by-language.pdf) \u003c/br\u003e\n\n### Introduction\nThe authors present a novel approach to incorporate language information into extracting visual features by conditioning the Batch Normalization parameters on the language. They apply Conditional Batch Normalization (CBN) to a pre-trained ResNet and show that this significantly improves performance on visual question answering tasks. \u003c/br\u003e\n\n### Setup\nThis repository is compatible with python 2. \u003c/br\u003e\n- Follow instructions outlined on [PyTorch Homepage](https://pytorch.org/) for installing PyTorch (Python2). \n- The python packages required are ``` nltk ``` ``` tqdm ``` which can be installed using pip. \u003c/br\u003e\n\n### Data\nTo download the VQA dataset please use the script 'scripts/vqa_download.sh': \u003c/br\u003e\n```\nscripts/vqa_download.sh `pwd`/data\n```\n\n### Process Data\nDetailed instructions for processing data are provided by [GuessWhatGame/vqa](https://github.com/GuessWhatGame/vqa#introduction). \u003c/br\u003e\n\n#### Create dictionary\nTo create the VQA dictionary, use the script preprocess_data/create_dico.py. \u003c/br\u003e\n```\npython preprocess_data/create_dictionary.py --data_dir data --year 2014 --dict_file dict.json\n```\n\n#### Create GLOVE dictionary\nTo create the GLOVE dictionary, download the original glove file and run the script preprocess_data/create_gloves.py. \u003c/br\u003e\n```\nwget http://nlp.stanford.edu/data/glove.42B.300d.zip -P data/\nunzip data/glove.42B.300d.zip -d data/\npython preprocess_data/create_gloves.py --data_dir data --glove_in data/glove.42B.300d.txt --glove_out data/glove_dict.pkl --year 2014\n```\n\n### Train Model\nTo train the network, set the required parameters in ``` config.json ``` and run the script main.py.\n```\npython main.py --gpu gpu_id --data_dir data --img_dir images --config config.json --exp_dir exp --year 2014\n```\n\n### Citation\nIf you find this code useful, please consider citing the original work by authors:\n```\n@inproceedings{de2017modulating,\nauthor = {Harm de Vries and Florian Strub and J\\'er\\'emie Mary and Hugo Larochelle and Olivier Pietquin and Aaron C. Courville},\ntitle = {Modulating early visual processing by language},\nbooktitle = {Advances in Neural Information Processing Systems 30},\nyear = {2017}\nurl = {https://arxiv.org/abs/1707.00683}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fap229997%2FConditional-Batch-Norm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fap229997%2FConditional-Batch-Norm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fap229997%2FConditional-Batch-Norm/lists"}