{"id":20447344,"url":"https://github.com/snowkylin/ntm","last_synced_at":"2025-08-20T22:31:06.386Z","repository":{"id":49562274,"uuid":"96532341","full_name":"snowkylin/ntm","owner":"snowkylin","description":"TensorFlow implementation of Neural Turing Machines (NTM), with its application on one-shot learning (MANN)","archived":false,"fork":false,"pushed_at":"2020-03-19T10:46:14.000Z","size":1526,"stargazers_count":182,"open_issues_count":12,"forks_count":65,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-12-10T02:50:48.738Z","etag":null,"topics":["mann","neural-turing-machines","ntm","one-shot-learning","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snowkylin.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-07T11:29:28.000Z","updated_at":"2024-12-04T08:22:03.000Z","dependencies_parsed_at":"2022-09-06T15:10:54.911Z","dependency_job_id":null,"html_url":"https://github.com/snowkylin/ntm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowkylin%2Fntm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowkylin%2Fntm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowkylin%2Fntm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowkylin%2Fntm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snowkylin","download_url":"https://codeload.github.com/snowkylin/ntm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230462906,"owners_count":18229864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mann","neural-turing-machines","ntm","one-shot-learning","tensorflow"],"created_at":"2024-11-15T10:25:55.930Z","updated_at":"2024-12-19T16:10:48.574Z","avatar_url":"https://github.com/snowkylin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NTM and MANN in TensorFlow\n\nTensorFlow implementation of Neural Turing Machines (NTM), as well as its application on one-shot learning (MANN). \n\nThe models are ready to use -- they are encapsulated into classes `NTMCell` and `MANNCell`, and the usage is similar to `LSTMCell` in TensorFlow, so you can apply these models easily in other programs. The sample code is also provided.\n\nYou can see [my slide](https://docs.google.com/presentation/d/1FqU7q-vWN9uV7sMRt9It9F_el9nIdqzBfMPm91hJ4B0/edit?usp=sharing) for more details about NTM and MANN.\n\n## Prerequisites\n\n* Python 3.5\n* TensorFlow 1.2.0\n* NumPy\n* Pillow (For MANN, prepoccessing of Omniglot dataset)\n\n## Implementation of NTM\n\n### Paper\n\nGraves, Alex, Greg Wayne, and Ivo Danihelka. \"[Neural turing machines.](https://arxiv.org/abs/1410.5401)\" _arXiv preprint arXiv:1410.5401_ (2014).\n\n### Usage\n\n#### Class NTMCell()\n\nThe usage of class `NTMCell` in `ntm/ntm_cell.py` is similar to `tf.contrib.rnn.BasicLSTMCell` in TensorFlow. The basic pseudocode is as follows:\n\n```python\nimport ntm.ntm_cell as ntm_cell\ncell = ntm_cell.NTMCell(\n    rnn_size=200,           # Size of hidden states of controller \n    memory_size=128,        # Number of memory locations (N)\n    memory_vector_dim=20,   # The vector size at each location (M)\n    read_head_num=1,        # # of read head\n    write_head_num=1,       # # of write head\n    addressing_mode='content_and_location', # Address Mechanisms, 'content_and_location' or 'content'\n    reuse=False,            # Whether to reuse the variable in the model (if the length of sequence is not fixed, you might need to build more than one model using the same variable, and this will be useful)\n)\nstate = cell.zero_state(batch_size, tf.float32)\noutput_list = []\nfor t in range(seq_length):\n    output, state = cell(input[i], state)\n    output_list.append(output)\n```\n\n#### Train and Test\n\nTo train the model, run:\n\n```\npython copy_task.py\n```\nYou can specify training options including parameters to the model via flags, such as `--model` (default is NTM), `--batch_size` and so on. See code for more detail.\n\nTo test the model, run:\n\n```\npython copy_task.py --mode test\n```\n\nYou can specify testing options via flags such as `--test_seq_length`.\n\n### Result (Copy task)\n\n![](images/copy_task_head.png) | ![](images/copy_task_loss.png)\n---|---\nVector of weighting (left: read vector; right: write vector; shift range: 1) | Training loss\n\n## One-shot Learning with NTM (MANN)\n\n### Paper\n\nSantoro, Adam, et al. \"[One-shot learning with memory-augmented neural networks.](https://arxiv.org/abs/1605.06065)\" _arXiv preprint arXiv:1605.06065_ (2016).\n\n### Usage\n\n#### Class MANNCell()\n\nThe usage of class `MANNCell` in `ntm/mann_cell.py` is similar to `tf.contrib.rnn.BasicLSTMCell` in TensorFlow. The basic pseudocode is as follows:\n\n```python\nimport ntm.mann_cell as mann_cell\ncell = mann_cell.MANNCell(\n    rnn_size=200,           # Size of hidden states of controller \n    memory_size=128,        # Number of memory locations (N)\n    memory_vector_dim=40,   # The vector size at each location (M)\n    head_num=1,             # # of read \u0026 write head (in MANN, #(read head) = #(write head))\n    gamma=0.95              # Usage decay of the write weights (in eq 20)\n    k_strategy='separate'   # In the original MANN paper, query key vector 'k' are used in both reading (eq 17) and writing (eq 23). You can set k_strategy='summary' if you want this way. However, in the NTM paper they are esparated. If you set k_strategy='separate', the controller will generate a new add vector 'a' to replace the query vector 'k' in eq 23.\n)\nstate = cell.zero_state(batch_size, tf.float32)\noutput_list = []\nfor t in range(seq_length):\n    output, state = cell(input[i], state)\n    output_list.append(output)\n```\n\nThere is another implementation of `MANNCell` translated from [tristandeleu's Theano version of MANN](https://github.com/tristandeleu/ntm-one-shot). You can find it in `ntm/mann_cell_2.py` and the usage is just the same. The performance is not fully tested but it seems to work fine too.\n\n#### Train and Test\n\nTo train the model, first you need to prepare the Omniglot dataset. Download [images_background.zip](https://github.com/brendenlake/omniglot/blob/master/python/images_background.zip) (964 classes) and [images_evaluation.zip](https://github.com/brendenlake/omniglot/blob/master/python/images_evaluation.zip) (679 classes), then combine them in a new `data` folder so you have 1623 classes. Your `data` folder may looks like:\n\n    /data\n        /Alphabet_of_the_Magi\n            /character01\n                0709_01.png\n                ...\n                0709_20.png\n            ...\n            /character20\n        ...\n        /ULOG\n\nThen, run:\n\n```\npython one_shot_learning.py\n```\n\nYou can specify training options including parameters to the model via flags, such as `--model` (default is MANN), `--batch_size` and so on. See code for more detail.\n\nTo test the model, run:\n\n```\npython one_shot_learning.py --mode test\n```\n\nYou can specify testing options via flags such as `--test_batch_num` (default: 100), `--n_train_classes` (default: 1200) and `--n_test_classes` (default: 423).\n\n### Result\n\nOmniglot Classification:\n\n![](images/LSTM_one_hot.png) | ![](images/MANN_one_hot.png)\n---|---\nLSTM, five random classes/episode, one-hot vector labels | MANN, five random classes/episode, one-hot vector labels\n![](images/LSTM_five_hot.png) | ![](images/MANN_five_hot.png)\nLSTM, fifteen random classes/episode, five-character string labels | MANN, fifteen random classes/episode, five-character string labels\n\nTest-set classification accuracies for LSTM and MANN trained on the Omniglot dataset, using one-hot encodings of labels and five classes presented per episode:\n\nModel | 1st | 2nd | 3rd | 4th | 5th | 6th | 7th | 8th | 9th | 10th | loss\n---|---|---|---|---|---|---|---|---|---|---|---\nLSTM | 0.2333 | 0.5897 | 0.6581 | 0.681 | 0.7077 | 0.7156 | 0.7141 | 0.7305 | 0.7281 | 0.7233 | 42.6427\nMANN | 0.3558 | 0.8881 | 0.9497 | 0.9651 | 0.9734 | 0.9744 | 0.9794 | 0.9798 | 0.978 | 0.9755 | 11.531","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowkylin%2Fntm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnowkylin%2Fntm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowkylin%2Fntm/lists"}