{"id":18081213,"url":"https://github.com/thundergolfer/reasoning-about-entailment-tensorflow","last_synced_at":"2025-07-06T22:32:59.259Z","repository":{"id":39740480,"uuid":"85446944","full_name":"thundergolfer/reasoning-about-entailment-tensorflow","owner":"thundergolfer","description":":school: Tensorflow implementation of \"Reasoning About Entailment with Neural Attention\"","archived":false,"fork":false,"pushed_at":"2023-03-24T23:55:11.000Z","size":51,"stargazers_count":10,"open_issues_count":4,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-04T12:55:25.647Z","etag":null,"topics":["deepmind","implementation","machine-learning","natural-language-inference","natural-language-processing","tensorflow"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thundergolfer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-19T02:58:31.000Z","updated_at":"2022-03-02T09:59:59.000Z","dependencies_parsed_at":"2022-09-09T14:40:29.629Z","dependency_job_id":null,"html_url":"https://github.com/thundergolfer/reasoning-about-entailment-tensorflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thundergolfer%2Freasoning-about-entailment-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thundergolfer%2Freasoning-about-entailment-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thundergolfer%2Freasoning-about-entailment-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thundergolfer%2Freasoning-about-entailment-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thundergolfer","download_url":"https://codeload.github.com/thundergolfer/reasoning-about-entailment-tensorflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248586257,"owners_count":21128995,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deepmind","implementation","machine-learning","natural-language-inference","natural-language-processing","tensorflow"],"created_at":"2024-10-31T13:13:32.110Z","updated_at":"2025-04-12T14:52:05.838Z","avatar_url":"https://github.com/thundergolfer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Implementation: *Reasoning About Entailment With Neural Attention* [![Build Status](https://travis-ci.com/thundergolfer/reasoning-about-entailment-tensorflow.svg?token=yHGWQ42iK2BPk1FjaUMc\u0026branch=master)](https://travis-ci.com/thundergolfer/reasoning-about-entailment-tensorflow) [![Code Climate](https://codeclimate.com/repos/58fb8909e6e27a027b001f9f/badges/901756218f7b5cb74762/gpa.svg)](https://codeclimate.com/repos/58fb8909e6e27a027b001f9f/feed) [![Test Coverage](https://codeclimate.com/repos/58fb8909e6e27a027b001f9f/badges/901756218f7b5cb74762/coverage.svg)](https://codeclimate.com/repos/58fb8909e6e27a027b001f9f/coverage)\n\n----\n\nTensorflow implementation of [*Reasoning about Entailment with Neural Attention*](https://arxiv.org/abs/1509.06664), a paper which addresses the problem of [Natural Language Inference](https://nlp.stanford.edu/~manning/talks/SIGIR2016-Deep-Learning-NLI.pdf) with an end-to-end Neural Network architecture. The paper was a collaboration from [Deepmind](https://deepmind.com/), [Oxford University](https://www.cs.ox.ac.uk/activities/machinelearning/), and [University College London](http://mr.cs.ucl.ac.uk/).\n\n----\n\n## Abstract\n\n\u003e While most approaches to automatically recognizing entailment relations have\nused classifiers employing hand engineered features derived from complex natural\nlanguage processing pipelines, in practice their performance has been only\nslightly better than bag-of-word pair classifiers using only lexical similarity. The\nonly attempt so far to build an end-to-end differentiable neural network for entailment\nfailed to outperform such a simple similarity classifier. In this paper, we\npropose a neural model that reads two sentences to determine entailment using\nlong short-term memory units. We extend this model with a word-by-word neural\nattention mechanism that encourages reasoning over entailments of pairs of words\nand phrases. Furthermore, we present a qualitative analysis of attention weights\nproduced by this model, demonstrating such reasoning capabilities. On a large\nentailment dataset this model outperforms the previous best neural model and a\nclassifier with engineered features by a substantial margin. It is the first generic\nend-to-end differentiable system that achieves state-of-the-art accuracy on a textual\nentailment dataset.\n\n## Requirements\n\n* `python 2.7` (probably works on Python 3 but untested)\n* `tensorflow 1.0.0`\n* `gensim` (for loading word embeddings)\n* `numpy` and `pandas`\n\n## Installation\n\n#### Prerequisites\n\nFirst clone this repository with `git clone git@github.com:thundergolfer/reasoning-about-entailment-tensorflow.git` **OR** you can also [fork](https://github.com/thundergolfer/reasoning-about-entailment-tensorflow#fork-destination-box) the repository.\n\nOnce cloned, install [Miniconda](https://conda.io/miniconda.html) with `./install_miniconda.sh`. That should succeed outputting `Install complete \u003cINSTALL LOCATION\u003e` in the terminal.\n\nNow the Miniconda has been installed you can install all packages into a nice `conda` virtual environment with `./install.sh`. *[Currently this doesn't support Windows, sorry]*\n\nIf everything in the install script went OK, you can enter the created virtual environment with `source ./run_in_environment.sh`.\n\n#### Data \u0026 Datasets\n\n* Download the SNLI Dataset ([info](https://nlp.stanford.edu/projects/snli/)): `./scripts/get_snli_dataset.sh`\n* Download the pre-trained Word2Vec model ([info](https://en.wikipedia.org/wiki/Word2vec)): `./scripts/get_word2vec.sh`\n\n##### Other Datasets\n\nThere are two other datasets that I thought would be interesting to use. First is [*Sentences Involving Compositional Knowledge*](http://clic.cimec.unitn.it/composes/sick.html), a precursor to the *SNLI* dataset.\n\nSecond is the [*Multi-Genre NLI Corpus*](https://www.nyu.edu/projects/bowman/multinli/) (*MultiNLI*), which is a successor to the *SNLI* dataset and in some ways superior to it. This new corpus can be combined with the *SNLI* for training. It is also of the same format to the *SNLI*, so the same loading and processing code can be used.\n\n* To get *SICK* run `scripts/get_sick_dataset.sh`\n* To get *MultiNLI* run `scripts/get_multiNLI_dateset.sh`\n\n## Usage\n\nCurrently training is done by `cd implementation/ \u0026\u0026 python train.py`\n\n## Results\n\n\u003e Note: These results do not acheive the accuracy results of the paper. Investigating improvements.\n\n![Model accuracy](http://i.imgur.com/wSsEA4r.png)\n![Model loss](http://i.imgur.com/Y3OhW8a.png)\n![Model performance summary](http://i.imgur.com/dKyefaj.png)\n\n## Training Details\n\n`coming soon`\n\n----------\n\n### References\n\n\n1. **Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Phil Blunsom**, *[Reasoning about Entailment with Neural Attention](https://arxiv.org/abs/1509.06664)*, 2015.\n\n2. **Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean**, *[Distributed Representations of Words and Phrases and their Compositionality](https://arxiv.org/abs/1310.4546)*, 2013.\n\n3. **Google**, *[Large-Scale Machine Learning on Heterogeneous Systems](http://tensorflow.org/)*, 2015.\n\n4. **Samuel R. Bowman, Gabor Angeli, Christopher Potts, Christopher D. Manning, The Stanford Natural Language Processing Group**, *[A large annotated corpus for learning natural language inference](http://nlp.stanford.edu/projects/snli/)*,  2015.\n\n### Credit\n\nThe work done in [borelien/entailment-neural-attention-lstm-tf](https://github.com/borelien/entailment-neural-attention-lstm-tf) and [shyamupa/snli-entailment](https://github.com/shyamupa/snli-entailment) was helpful to me in this project. I hope you find that my work has extended and expanded theirs in interesting and useful ways.\n\n### License\n\nThis project is available under the [MIT License](https://choosealicense.com/licenses/mit/).\n\n### Author\n\nJonathon Belotti ([thundergolfer](https://github.com/thundergolfer)) - [@jonobelotti_IO](https://twitter.com/jonobelotti_IO) - [jonathonbelotti.com](http://jonathonbelotti.com)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthundergolfer%2Freasoning-about-entailment-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthundergolfer%2Freasoning-about-entailment-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthundergolfer%2Freasoning-about-entailment-tensorflow/lists"}