{"id":24101392,"url":"https://github.com/yxtay/char-rnn-text-generation","last_synced_at":"2025-05-08T01:11:52.726Z","repository":{"id":82529047,"uuid":"96000631","full_name":"yxtay/char-rnn-text-generation","owner":"yxtay","description":"Character Embeddings Recurrent Neural Network Text Generation Models","archived":false,"fork":false,"pushed_at":"2019-05-17T07:27:57.000Z","size":5042,"stargazers_count":60,"open_issues_count":2,"forks_count":23,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-08T01:11:46.360Z","etag":null,"topics":["chainer","character-embeddings","deep-learning","keras","long-short-term-memory","mxnet","neural-network","pytorch","recurrent-neural-network","tensorflow","text-generation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yxtay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-02T03:40:40.000Z","updated_at":"2025-01-20T10:49:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"439e40de-a8fa-4684-a872-d8cb3ec3779f","html_url":"https://github.com/yxtay/char-rnn-text-generation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yxtay%2Fchar-rnn-text-generation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yxtay%2Fchar-rnn-text-generation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yxtay%2Fchar-rnn-text-generation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yxtay%2Fchar-rnn-text-generation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yxtay","download_url":"https://codeload.github.com/yxtay/char-rnn-text-generation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252978779,"owners_count":21834917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chainer","character-embeddings","deep-learning","keras","long-short-term-memory","mxnet","neural-network","pytorch","recurrent-neural-network","tensorflow","text-generation"],"created_at":"2025-01-10T16:55:27.520Z","updated_at":"2025-05-08T01:11:52.716Z","avatar_url":"https://github.com/yxtay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Character Embeddings Recurrent Neural Network Text Generation Models\n\nInspired by [Andrej Karpathy](https://github.com/karpathy/)'s \n[The Unreasonable Effectiveness of Recurrent Neural Networks](https://karpathy.github.io/2015/05/21/rnn-effectiveness/).\n\nThis repository attempts to replicate the models, with slight modifications, in different python deep learning frameworks.\n\n## Frameworks\n\n- Keras: [`keras_model.py`](keras_model.py)\n- TensorFlow: [`tf_model.py`](tf_model.py)\n- PyTorch: [`pytorch_model.py`](pytorch_model.py)\n- Chainer: [`chainer_model.py`](chainer_model.py)\n- MXNet: [`mxnet_model.py`](mxnet_model.py)\n- CNTK\n- Caffe\n\n## Default Model Specification\n\n| Layer Type | Output Shape  | Param # | Remarks                            |\n|------------|---------------|---------|------------------------------------|\n| Embedding  | (64, 64, 32)  | 3136    | vocab size: 98, embedding size: 32 |\n| Dropout    | (64, 64, 32)  | 0       | dropout rate: 0.0                  |\n| LSTM       | (64, 64, 128) | 82432   | output size: 128                   |\n| Dropout    | (64, 64, 128) | 0       | dropout rate: 0.0                  |\n| LSTM       | (64, 64, 128) | 131584  | output size: 128                   |\n| Dropout    | (64, 64, 128) | 0       | dropout rate: 0.0                  |\n| Dense      | (64, 64, 98)  | 12642   | output size: 98                    |\n\n### Training Specification\n\n- Batch size: 64\n- Sequence length: 64\n- Number of epochs: 32\n- Learning rate: 0.001\n- Max gradient norm: 5.0\n\n## Setup\n\n```bash\n# clone repo\ngit clone git@github.com:yxtay/char-rnn-text-generation.git \u0026\u0026 cd char-rnn-text-generation\n\n# create conda environment\nconda env create -f=environment.yml\n\n# activate environment\nsource activate dl\n```\n\n## Usage\n\n### Training\n\n```\nusage: \u003cframework\u003e_model.py train [-h] --checkpoint-path CHECKPOINT_PATH \n                                  --text-path TEXT_PATH\n                                  [--restore [RESTORE]]\n                                  [--seq-len SEQ_LEN]\n                                  [--embedding-size EMBEDDING_SIZE]\n                                  [--rnn-size RNN_SIZE] \n                                  [--num-layers NUM_LAYERS]\n                                  [--drop-rate DROP_RATE]\n                                  [--learning-rate LEARNING_RATE]\n                                  [--clip-norm CLIP_NORM] \n                                  [--batch-size BATCH_SIZE]\n                                  [--num-epochs NUM_EPOCHS]\n                                  [--log-path LOG_PATH]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --checkpoint-path CHECKPOINT_PATH\n                        path to save or load model checkpoints\n  --text-path TEXT_PATH\n                        path of text file for training\n  --restore [RESTORE]   whether to restore from checkpoint_path or from\n                        another path if specified\n  --seq-len SEQ_LEN     sequence length of inputs and outputs (default: 64)\n  --embedding-size EMBEDDING_SIZE\n                        character embedding size (default: 32)\n  --rnn-size RNN_SIZE   size of rnn cell (default: 128)\n  --num-layers NUM_LAYERS\n                        number of rnn layers (default: 2)\n  --drop-rate DROP_RATE\n                        dropout rate for rnn layers (default: 0.0)\n  --learning-rate LEARNING_RATE\n                        learning rate (default: 0.001)\n  --clip-norm CLIP_NORM\n                        max norm to clip gradient (default: 5.0)\n  --batch-size BATCH_SIZE\n                        training batch size (default: 64)\n  --num-epochs NUM_EPOCHS\n                        number of epochs for training (default: 32)\n  --log-path LOG_PATH   path of log file (default: main.log)\n```\n\nExample:\n\n```bash\npython tf_model.py train \\\n    --checkpoint=checkpoints/tf_tinyshakespeare/model.ckpt \\\n    --text=data/tinyshakespeare.txt\n```\n\nSample logs:\n\n- [`keras_model.log`](logs/keras_model.log)\n- [`tf_model.log`](logs/tf_model.log)\n- [`pytorch_model.log`](logs/pytorch_model.log)\n- [`chainer_model.log`](logs/chainer_model.log)\n- [`mxnet_model.log`](logs/mxnet_model.log)\n\n### Text Generation\n\n```\nusage: \u003cframework\u003e_model.py generate [-h] --checkpoint-path CHECKPOINT_PATH\n                                     (--text-path TEXT_PATH | --seed SEED)\n                                     [--length LENGTH] [--top-n TOP_N]\n                                     [--log-path LOG_PATH]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --checkpoint-path CHECKPOINT_PATH\n                        path to load model checkpoints\n  --text-path TEXT_PATH\n                        path of text file to generate seed\n  --seed SEED           seed character sequence\n  --length LENGTH       length of character sequence to generate (default:\n                        1024)\n  --top-n TOP_N         number of top choices to sample (default: 3)\n  --log-path LOG_PATH   path of log file (default: main.log)\n```\n\nExample:\n\n```bash\npython tf_model.py generate \\\n    --checkpoint=checkpoints/tf_tinyshakespeare/model.ckpt \\\n    --seed=\"KING RICHARD\"\n```\n\nSample output:\n\n```\nKING RICHARDIIIIl II I tell thee,\nAs I have no mark of his confection,\nThe people so see my son.\n\nSEBASTIAN:\nI have men's man in the common to his sounds,\nAnd so she said of my soul, and to him,\nAnd too marry his sun their commanded\nAs thou shalt be alone too means\nAs he should to thy sensess so far to mark of\nthese foul trust them fringer whom, there would he had\nAs the word of merrous and subject.\n\nGLOUCESTER:\nA spack, a service the counsel son and here.\nWhat is a misin the wind and to the will\nAnd shall not streaks of this show into all heard.\n\nKING EDIN YORK:\nI will be suppet on himself tears as the sends.\n\nKING EDWARD IV:\nNo looks and them, and while, a will, when this way.\n\nBAPTHIO:\nA mortain and me to the callant our souls\nAnd the changed and such of the son.\n\nCORIOLANUS:\nI will, so show me with the child to the could sheep\nTo beseence, and shall so so should but hear\nThan him with her fair to be that soul,\nWhishe it is no meach of my lard and\nAnd this, and with my love and the senter'd with marked\nAnd her should\n```\n\n## Benchmarks\n\nBelow are training duration and loss on [`tinyshakespeare.txt`](data/tinyshakespeare.txt).\n\n| Framework  | Duration (s) | Loss    |\n|------------|--------------|---------|\n| Keras      | 5270         | 1.42505 | \n| TensorFlow | 3003         | 1.45795 | \n| PyTorch    | 5868         | 1.32285 | \n| Chainer    | 4954         | 1.22930 | \n| MXNet      | 7348         | 1.34199 | \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyxtay%2Fchar-rnn-text-generation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyxtay%2Fchar-rnn-text-generation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyxtay%2Fchar-rnn-text-generation/lists"}