{"id":16368906,"url":"https://github.com/apsdehal/nli-batch-optimizations","last_synced_at":"2025-04-15T00:31:38.517Z","repository":{"id":92133136,"uuid":"108935867","full_name":"apsdehal/nli-batch-optimizations","owner":"apsdehal","description":"Better convergence speeds for NLI task","archived":false,"fork":false,"pushed_at":"2017-12-19T05:23:51.000Z","size":132,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-28T12:51:12.039Z","etag":null,"topics":["batch-augmentations","natural-language-inference","natural-language-understanding"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apsdehal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-31T02:44:58.000Z","updated_at":"2021-07-16T15:06:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"23129502-96c2-4ab0-84e3-f3d42616c4f6","html_url":"https://github.com/apsdehal/nli-batch-optimizations","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apsdehal%2Fnli-batch-optimizations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apsdehal%2Fnli-batch-optimizations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apsdehal%2Fnli-batch-optimizations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apsdehal%2Fnli-batch-optimizations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apsdehal","download_url":"https://codeload.github.com/apsdehal/nli-batch-optimizations/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248984305,"owners_count":21193720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch-augmentations","natural-language-inference","natural-language-understanding"],"created_at":"2024-10-11T02:54:09.934Z","updated_at":"2025-04-15T00:31:38.504Z","avatar_url":"https://github.com/apsdehal.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NLI Batch Optimization\n\nBatch optimizations for NLI. Project done for DS-GA 1011. \n\nTLDR: A method for accelerating convergence speeds in NLI task\n\n\n## Long Description\n\nThis project presents a method to organize mini-batches for training models on Natural Language Inference (NLI) task in order to accelerate convergence speeds and in some cases better accuracy. Techniques such as batch normalization and neural data filter which augment and adapt the varying mini-batches often result in accelerating training time. Inspired by these techniques we propose a method for creating mini-batches for NLI task. We test our proposed method on different models published for solving NLI. We fine tune these models and compare convergence rate and validation accuracy achieved to gather quantitative data to support our claim. Experiments show that using our method can accelerate the convergence speed of NLI models as is seen in case ESIM, Parikh et. al., Chen et. al. and others by 2-5 times. We also present fine-tuned results of several models that were trained on SNLI for MultiNLI.\n\n## Models tested\n\n- Parikh et. al. 2016 (Decomposable Attentional Model for NLI) [Paper Link](https://arxiv.org/abs/1606.01933) [Folder](https://github.com/apsdehal/nli-batch-optimizations/tree/master/parikh-et-al-and-bilstm-max-pool)\n- ESIM (Chen et. al. 2016) [Paper Link](https://arxiv.org/abs/1609.06038) [Folder](https://github.com/apsdehal/nli-batch-optimizations/tree/master/esim-chen-et-al-2016)\n- Chen et. al. 2017a [Paper Link](https://arxiv.org/abs/1708.01353) [Folder](https://github.com/apsdehal/nli-batch-optimizations/tree/master/chen-et-al-2017)\n- BiLSTM with Max Pooling [Folder](https://github.com/apsdehal/nli-batch-optimizations/tree/master/parikh-et-al-and-bilstm-max-pool)\n\n\n## Installation\n\n- In this project, run `conda env create`\n- `source activate nlu-project`\n\nThis will install all of the dependencies needed to run the project\n\nAlternatively, you can use `pip install -r requirements.txt` to install all of the requirements.\n\n## Running and Training\n\nCheck respective model's folder's README to run and train that model.\n\n## Pretrained Models\n\nCheck each model's respective folder to find link to pretrained models. \n\n## Results\n\nFollowing table shows the results obtained:\n\n![Results](https://i.imgur.com/etEUOvZ.png)\n\nGradients values for hidden state matrix of BiLSTM:\n\n\u003cimg src=\"https://i.imgur.com/45E70Mh.png\" width=\"40%\"/\u003e\n\n## LICENSE\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapsdehal%2Fnli-batch-optimizations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapsdehal%2Fnli-batch-optimizations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapsdehal%2Fnli-batch-optimizations/lists"}