{"id":22121565,"url":"https://github.com/anantzoid/language-modeling-gatedcnn","last_synced_at":"2025-04-08T04:19:50.193Z","repository":{"id":142505775,"uuid":"78639294","full_name":"anantzoid/Language-Modeling-GatedCNN","owner":"anantzoid","description":"Tensorflow implementation of \"Language Modeling with Gated Convolutional Networks\"","archived":false,"fork":false,"pushed_at":"2017-01-16T04:32:21.000Z","size":572,"stargazers_count":271,"open_issues_count":8,"forks_count":98,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-24T09:36:58.524Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anantzoid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-01-11T12:59:12.000Z","updated_at":"2025-03-21T16:06:16.000Z","dependencies_parsed_at":"2023-10-20T17:04:31.347Z","dependency_job_id":"9d0c311a-17e7-46fd-a359-d509acc3a4d3","html_url":"https://github.com/anantzoid/Language-Modeling-GatedCNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anantzoid%2FLanguage-Modeling-GatedCNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anantzoid%2FLanguage-Modeling-GatedCNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anantzoid%2FLanguage-Modeling-GatedCNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anantzoid%2FLanguage-Modeling-GatedCNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anantzoid","download_url":"https://codeload.github.com/anantzoid/Language-Modeling-GatedCNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247773962,"owners_count":20993677,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-01T14:41:48.573Z","updated_at":"2025-04-08T04:19:50.167Z","avatar_url":"https://github.com/anantzoid.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Language Modeling with Gated Convolutional Networks\n\nThis is a Tensorflow implementation of Facebook AI Research Lab's paper: [Language Modeling with Gated Convolutional Networks](https://arxiv.org/abs/1612.08083). This paper applies a convolutional approach to language modelling with a novel Gated-CNN model.\n\n## Architecture\n![Architecture](assets/architecture.png)\n\n## Requirements\n- Download and extract the [Google 1 Billion Word dataset](http://www.statmt.org/lm-benchmark/1-billion-word-language-modeling-benchmark-r13output.tar.gz) in the `data` folder.\n\n- [TensorFlow 0.12.1](https://www.tensorflow.org/)\n\n## Usage\nTo train the model using the default hyperparameters:\n\n```\n$ python main.py\n$ tensorboard --logdir=logs --host=0.0.0.0\n```\nCheck `main.py` for tunable hyperparameter flags.\n\n## TODO\n- [ ] Replace NCE loss with Adaptive Softmax.\n- [ ] Remove restricted training on fixed sized sentences (20, for now) and extend to account for all varied sentence lenghts.\n- [ ] Implement Weight Normalisation for faster convergence.\n- [ ] Train extensively on deeper models to match the results with the paper.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanantzoid%2Flanguage-modeling-gatedcnn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanantzoid%2Flanguage-modeling-gatedcnn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanantzoid%2Flanguage-modeling-gatedcnn/lists"}