{"id":18673752,"url":"https://github.com/carpedm20/memn2n-tensorflow","last_synced_at":"2025-04-04T08:07:02.880Z","repository":{"id":66039959,"uuid":"47553188","full_name":"carpedm20/MemN2N-tensorflow","owner":"carpedm20","description":"\"End-To-End Memory Networks\" in Tensorflow","archived":false,"fork":false,"pushed_at":"2017-03-14T01:33:15.000Z","size":1970,"stargazers_count":828,"open_issues_count":10,"forks_count":250,"subscribers_count":47,"default_branch":"master","last_synced_at":"2025-03-28T07:06:22.464Z","etag":null,"topics":["memory-network","nlp","tensorflow"],"latest_commit_sha":null,"homepage":"http://arxiv.org/abs/1503.08895v4","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/carpedm20.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-12-07T13:18:50.000Z","updated_at":"2025-03-01T06:14:30.000Z","dependencies_parsed_at":"2023-06-18T15:16:47.522Z","dependency_job_id":null,"html_url":"https://github.com/carpedm20/MemN2N-tensorflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FMemN2N-tensorflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FMemN2N-tensorflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FMemN2N-tensorflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carpedm20%2FMemN2N-tensorflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/carpedm20","download_url":"https://codeload.github.com/carpedm20/MemN2N-tensorflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247142063,"owners_count":20890652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["memory-network","nlp","tensorflow"],"created_at":"2024-11-07T09:16:28.906Z","updated_at":"2025-04-04T08:07:02.862Z","avatar_url":"https://github.com/carpedm20.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"End-To-End Memory Networks in Tensorflow\n========================================\n\nTensorflow implementation of [End-To-End Memory Networks](http://arxiv.org/abs/1503.08895v4) for language modeling (see Section 5). The original torch code from Facebook can be found [here](https://github.com/facebook/MemNN/tree/master/MemN2N-lang-model).\n\n![alt tag](http://i.imgur.com/nv89JLc.png)\n\n\nPrerequisites\n-------------\n\nThis code requires [Tensorflow](https://www.tensorflow.org/). There is a set of sample Penn Tree Bank (PTB) corpus in `data` directory, which is a popular benchmark for measuring quality of these models. But you can use your own text data set which should be formated like [this](data/).\n\n\nWhen you use docker image tensorflw/tensorflow:latest-gpu, you need to python package future.\n\n    $ pip install future\n    \nIf you want to use `--show True` option, you need to install python package `progress`.\n\n    $ pip install progress\n    \nUsage\n-----\n\nTo train a model with 6 hops and memory size of 100, run the following command:\n\n    $ python main.py --nhop 6 --mem_size 100\n\nTo see all training options, run:\n\n    $ python main.py --help\n\nwhich will print:\n\n    usage: main.py [-h] [--edim EDIM] [--lindim LINDIM] [--nhop NHOP]\n                  [--mem_size MEM_SIZE] [--batch_size BATCH_SIZE]\n                  [--nepoch NEPOCH] [--init_lr INIT_LR] [--init_hid INIT_HID]\n                  [--init_std INIT_STD] [--max_grad_norm MAX_GRAD_NORM]\n                  [--data_dir DATA_DIR] [--data_name DATA_NAME] [--show SHOW]\n                  [--noshow]\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      --edim EDIM           internal state dimension [150]\n      --lindim LINDIM       linear part of the state [75]\n      --nhop NHOP           number of hops [6]\n      --mem_size MEM_SIZE   memory size [100]\n      --batch_size BATCH_SIZE\n                            batch size to use during training [128]\n      --nepoch NEPOCH       number of epoch to use during training [100]\n      --init_lr INIT_LR     initial learning rate [0.01]\n      --init_hid INIT_HID   initial internal state value [0.1]\n      --init_std INIT_STD   weight initialization std [0.05]\n      --max_grad_norm MAX_GRAD_NORM\n                            clip gradients to this norm [50]\n      --checkpoint_dir CHECKPOINT_DIR\n                            checkpoint directory [checkpoints]\n      --data_dir DATA_DIR   data directory [data]\n      --data_name DATA_NAME\n                            data set name [ptb]\n      --is_test IS_TEST     True for testing, False for Training [False]\n      --nois_test\n      --show SHOW           print progress [False]\n      --noshow\n\n(Optional) If you want to see a progress bar, install `progress` with `pip`:\n\n    $ pip install progress\n    $ python main.py --nhop 6 --mem_size 100 --show True\n\nAfter training is finished, you can test and validate with:\n\n    $ python main.py --is_test True --show True\n\nThe training output looks like:\n\n    $ python main.py --nhop 6 --mem_size 100 --show True\n    Read 929589 words from data/ptb.train.txt\n    Read 73760 words from data/ptb.valid.txt\n    Read 82430 words from data/ptb.test.txt\n    {'batch_size': 128,\n    'data_dir': 'data',\n    'data_name': 'ptb',\n    'edim': 150,\n    'init_hid': 0.1,\n    'init_lr': 0.01,\n    'init_std': 0.05,\n    'lindim': 75,\n    'max_grad_norm': 50,\n    'mem_size': 100,\n    'nepoch': 100,\n    'nhop': 6,\n    'nwords': 10000,\n    'show': True}\n    I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 12\n    I tensorflow/core/common_runtime/direct_session.cc:45] Direct session inter op parallelism threads: 12\n    Training |################################| 100.0% | ETA: 0s\n    Testing |################################| 100.0% | ETA: 0s\n    {'perplexity': 507.3536108810464, 'epoch': 0, 'valid_perplexity': 285.19489755719286, 'learning_rate': 0.01}\n    Training |################################| 100.0% | ETA: 0s\n    Testing |################################| 100.0% | ETA: 0s\n    {'perplexity': 218.49577035468886, 'epoch': 1, 'valid_perplexity': 231.73457031084268, 'learning_rate': 0.01}\n    Training |################################| 100.0% | ETA: 0s\n    Testing |################################| 100.0% | ETA: 0s\n    {'perplexity': 163.5527845871247, 'epoch': 2, 'valid_perplexity': 175.38771414841014, 'learning_rate': 0.01}\n    Training |################################| 100.0% | ETA: 0s\n    Testing |################################| 100.0% | ETA: 0s\n    {'perplexity': 136.1443535538306, 'epoch': 3, 'valid_perplexity': 161.62522958776597, 'learning_rate': 0.01}\n    Training |################################| 100.0% | ETA: 0s\n    Testing |################################| 100.0% | ETA: 0s\n    {'perplexity': 119.15373237680929, 'epoch': 4, 'valid_perplexity': 149.00768378137946, 'learning_rate': 0.01}\n    Training |##############                  | 44.0% | ETA: 378s\n\n\nPerformance\n-----------\n\nThe perplexity on the test sets of Penn Treebank corpora.\n\n| # of hidden | # of hops | memory size | MemN2N (Sukhbaatar 2015) |  This repo. |\n|:-----------:|:---------:|:-----------:|:------------------------:|:-----------:|\n|     150     |     3     |     100     |            122           |     129     |\n|     150     |     6     |     150     |            114           | in progress |\n\n\nAuthor\n------\n\nTaehoon Kim / [@carpedm20](http://carpedm20.github.io/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarpedm20%2Fmemn2n-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarpedm20%2Fmemn2n-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarpedm20%2Fmemn2n-tensorflow/lists"}