{"id":13560892,"url":"https://github.com/unixpickle/sgdstore","last_synced_at":"2025-05-02T00:31:42.068Z","repository":{"id":138791332,"uuid":"86104476","full_name":"unixpickle/sgdstore","owner":"unixpickle","description":"Augmented RNN memory via live SGD","archived":false,"fork":false,"pushed_at":"2017-04-25T23:06:56.000Z","size":5858,"stargazers_count":40,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T20:23:28.192Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unixpickle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-03-24T19:37:21.000Z","updated_at":"2025-02-24T15:55:48.000Z","dependencies_parsed_at":"2024-01-14T03:45:03.310Z","dependency_job_id":"3c158bf8-224a-4172-882a-634734502957","html_url":"https://github.com/unixpickle/sgdstore","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixpickle%2Fsgdstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixpickle%2Fsgdstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixpickle%2Fsgdstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixpickle%2Fsgdstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unixpickle","download_url":"https://codeload.github.com/unixpickle/sgdstore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251966431,"owners_count":21672666,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:00:50.578Z","updated_at":"2025-05-02T00:31:37.056Z","avatar_url":"https://github.com/unixpickle.png","language":"Go","readme":"# sgdstore\n\nThis is a memory-augmented neural network that uses a neural network as a storage device. Particularly, a *controller network* provides training examples for a *storage network* at every timestep. The *storage network* is trained on these samples with SGD at every timestep (in a differentiable manner). The controller can then query the storage network by feeding it inputs and seeing the corresponding outputs. The end result is that the storage network serves as a memory bank which is \"written to\" via SGD.\n\n# Hypotheses\n\nNeural networks seem to provide a lot of desirable properties as memory modules:\n\n * They can store a lot of information.\n * They can compress information.\n * They can generalize to new information.\n * They can interpolate between training samples.\n\nIn a sense, a neural network can be seen as a key-value store which tries to generalize to new keys. This seems like the perfect memory structure for a memory-augmented neural network.\n\n# Results\n\nPreliminary results on polynomial approximation looked promising. See [experiments/poly_approx](experiments/poly_approx). After seeing those results, I decided to scale up to a harder meta-learning task.\n\nOn the Omniglot handwriting dataset, the model (controlled by a vanilla RNN) outperformed an LSTM in training time (measured in epochs) and accuracy. See [experiments/omniglot](experiments/omniglot).\n","funding_links":[],"categories":["Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funixpickle%2Fsgdstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funixpickle%2Fsgdstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funixpickle%2Fsgdstore/lists"}