{"id":14959025,"url":"https://github.com/siddk/entity-network","last_synced_at":"2026-02-23T23:16:10.556Z","repository":{"id":68633384,"uuid":"82630461","full_name":"siddk/entity-network","owner":"siddk","description":"Tensorflow implementation of \"Tracking the World State with Recurrent Entity Networks\" [https://arxiv.org/abs/1612.03969] by Henaff, Weston, Szlam, Bordes, and LeCun.","archived":false,"fork":false,"pushed_at":"2017-03-08T18:02:45.000Z","size":19957,"stargazers_count":56,"open_issues_count":4,"forks_count":16,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-07T02:08:56.606Z","etag":null,"topics":["embeddings","key-vectors","recurrent-entity-networks","recurrent-neural-networks","tensorflow","tensorflow-models"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/siddk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-02-21T03:12:55.000Z","updated_at":"2025-01-17T12:58:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"8ed5a5bf-41fe-46c2-a3c0-b870daeb4f79","html_url":"https://github.com/siddk/entity-network","commit_stats":{"total_commits":32,"total_committers":2,"mean_commits":16.0,"dds":0.03125,"last_synced_commit":"b18fbb7c749e98192a77132af9e537e399ac24c7"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siddk%2Fentity-network","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siddk%2Fentity-network/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siddk%2Fentity-network/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siddk%2Fentity-network/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/siddk","download_url":"https://codeload.github.com/siddk/entity-network/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252045995,"owners_count":21685932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embeddings","key-vectors","recurrent-entity-networks","recurrent-neural-networks","tensorflow","tensorflow-models"],"created_at":"2024-09-24T13:18:42.903Z","updated_at":"2026-02-23T23:16:05.523Z","avatar_url":"https://github.com/siddk.png","language":"Python","readme":"# Recurrent Entity Networks\nTensorflow/TFLearn Implementation of [\"Tracking the World State with Recurrent Entity Networks\"](https://arxiv.org/abs/1612.03969) by Henaff et. al.\n\n### Punchline ###\nBy building a set of disparate memory cells, each responsible for different concepts, entities, or other content, Recurrent Entity Networks (EntNets) are able to efficiently and robustly maintain a “world-state” - one that can be updated easily and effectively with the influx of new information. \n\nFurthermore, one can either let EntNet cell keys vary, or specifically seed them with specific embeddings, thereby forcing the model to track a given set of entities/objects/locations, allowing for the easy interpretation of the underlying decision-making process.\n\n### Results ###\nImplementation results are as follows (graphs of training/validation loss will be added later). Some of the tasks \nare fairly computationally intensive, so it might take a while to get benchmark results.\n\nNote that for efficiency, training stopped after validation accuracy passed a threshold of 95%. This is different than\nthe method used in the paper, which runs tasks for 200 epochs, and reports the best model across 10 different runs. The number of runs, epochs to converge, and final train/validation/test accuracies (best on validation over different runs) for this repository relative to the paper results are as follows:\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/f27575f13e0a5dd47d5090418b7121b4.svg?invert_in_darkmode\" align=middle width=1016.8059pt height=432.5607pt/\u003e\u003c/p\u003e\n\nNote that the italics above indicate examples of *overfitting*. Note that the notes rows consist of single runs\nof the model - this is probably why multiple runs are necessary. If this continues to happen, I'll look into ways\nto better regularize the network (via dropout, for example).\n\nThe **bold** above denotes failure to convergence. I'm not sure why this is happening, but I'll note that Jim\nFleming reports the same sort of issue in [his implementation](https://github.com/jimfleming/recurrent-entity-networks).\n\nAdditionally, plots of the training/validation loss and accuracies through training can be found in eval/qa_id, where\nid is the id of the task at hand. As an example, here is the plot for the graph of Task 1 - Single Supporting Fact's \ntraining:\n\n![alt text](https://github.com/siddk/entity-network/blob/master/eval/qa_1/run_1.png \"Task 1 - Single Supporting Fact\")\n\n### Components ###\n\nEntity Networks consist of three separate components:\n\n1) An Input Encoder, that takes the input sequence at a given time step, and encodes it into a fixed-size vector representation \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/1f1c28e0a1b1708c6889fb006c886784.svg?invert_in_darkmode\" align=middle width=12.623985pt height=14.10255pt/\u003e\n\n2) The Dynamic Memory (the core of the model), that keeps a disparate set of memory cells, each with a different vector key \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/40cca55dbe7b8452cf1ede03d21fe3ed.svg?invert_in_darkmode\" align=middle width=17.806305pt height=14.10255pt/\u003e (the location), and a hidden state memory \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/6d22be1359e204374e6f0b45e318d561.svg?invert_in_darkmode\" align=middle width=15.517425pt height=22.74591pt/\u003e (the content)\n\n3) The Output Module, that takes the hidden states, and applies a series of transformations to generate the output \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/deceeaf6940a8c7a5a02373728002b0f.svg?invert_in_darkmode\" align=middle width=8.61696pt height=14.10255pt/\u003e.\n\nA breakdown of the components are as follows:\n\n**Input Encoder**: Takes the input from the environment (i.e. a sentence from a story), and maps it to a fixed size state vector \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/1f1c28e0a1b1708c6889fb006c886784.svg?invert_in_darkmode\" align=middle width=12.623985pt height=14.10255pt/\u003e.\n\nThis repository (like the paper) utilizes a learned multiplicative mask, where each embedding of the sentence is multiplied element-wise with a mask vector \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/9b6dbadab1b122f6d297345e9d3b8dd7.svg?invert_in_darkmode\" align=middle width=12.65154pt height=22.74591pt/\u003e and then summed together. \n\nAlternatively, one could just as easily imagine an LSTM or CNN encoder to generate this initial input.\n\n**Dynamic Memory**: Core of the model, consists of a series of key vectors \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/5ccebbf530ff52e71bfb606d574fdaca.svg?invert_in_darkmode\" align=middle width=98.039205pt height=14.10255pt/\u003e and memory (hidden state) vectors \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/68db9e670b455c9eef5d6b82287b3676.svg?invert_in_darkmode\" align=middle width=91.17273pt height=22.74591pt/\u003e.\n\nThe keys and state vectors function similarly to how the program keys and program embeddings function in the NPI/NTM - the keys represent location, while the memories are content.\nOnly the content (memories) get updated at inference time, with the influx of new information. \n\nFurthermore, one can seed and fix the key vectors such that they reflect certain words/entities =\u003e the paper does this by fixing key vectors to certain word embeddings, and using a simple BoW state encoding.\nThis repository currently only supports random key vector seeds.\n\nThe Dynamic Memory updates given an input \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/1f1c28e0a1b1708c6889fb006c886784.svg?invert_in_darkmode\" align=middle width=12.623985pt height=14.10255pt/\u003e are as follows - this is very similar to the GRU update equations:\n\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/e30634013819f430680ff7d9d2d67190.svg?invert_in_darkmode\" align=middle width=195.352245pt height=27.59823pt/\u003e \n    - Gating function, determines how much memory j should be affected by the given input.\n\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/070eb4a8ad370755d533e0f8c6dea9aa.svg?invert_in_darkmode\" align=middle width=259.068645pt height=30.55107pt/\u003e \n    - New state update - U, V, W are model parameters that are shared across all memory cells .\n    - Model can be simplified by constraining U, V, W to be zero, or identity.\n\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/1f3fc749aea58a01cb3dfe4942924983.svg?invert_in_darkmode\" align=middle width=116.74773pt height=30.55107pt/\u003e\n    - Gated update, elementwise product of g with $\\tilde{h}$.\n    - Dictates how much the given memory should be updated.\n\n**Output Module**: Model interface, takes in the memories and a query vector q, and transforms them into the required output.\n\nFunctions like a 1-hop Memory Network (Sukhbaatar, Weston), building a weighting mechanism over each input, then combines and feeds them through some intermediate layers. \n\nThe actual updates are as follows:\n\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/95f55f07bc9f2f920afdda389426a490.svg?invert_in_darkmode\" align=middle width=141.104535pt height=27.59823pt/\u003e\n    - Normalizes states based on cosine similarity.\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/341317311489f34a57af823138e4fd8a.svg?invert_in_darkmode\" align=middle width=88.94622pt height=24.65793pt/\u003e\n    - Weighted sum of hidden states\n+ \u003cimg src=\"https://rawgit.com/siddk/entity-network/master/eval/svgs/0ac5cd1ce89de7dbc1e4329a272457d6.svg?invert_in_darkmode\" align=middle width=192.040695pt height=24.56553pt/\u003e \n    - R, H are trainable model parameters.\n    - As long as you can build some sort of loss using y, then the entirety of the model is trainable via Backpropagation-Through-Time (BPTT).\n\n### Repository Structure ###\nDirectory is structured in the following way:\n\n+ model/ - Model definition code, including the definition of the Dynamic Memory Cell.\n\n+ preprocessor/ - Preprocessing code to load and vectorize the bAbI Tasks.\n\n+ tasks/ - Raw bAbI Task files.\n\n+ run.py - Core script for training and evaluating the Recurrent Entity Network. \n\n### References ###\nBig shout-out to Jim Fleming for his initial Tensorflow Implementation - his Dynamic Memory Cell Implementation \nspecifically made things a lot easier.\n\nReference: [Jim Fleming's EntNet Memory Cell](https://github.com/jimfleming/recurrent-entity-networks/blob/master/entity_networks/dynamic_memory_cell.py)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiddk%2Fentity-network","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsiddk%2Fentity-network","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiddk%2Fentity-network/lists"}