{"id":23080726,"url":"https://github.com/chen0040/keras-text-to-image","last_synced_at":"2025-08-15T22:31:20.808Z","repository":{"id":50794698,"uuid":"121374432","full_name":"chen0040/keras-text-to-image","owner":"chen0040","description":"Translate text to image in Keras using GAN and Word2Vec as well as recurrent neural networks","archived":false,"fork":false,"pushed_at":"2021-05-29T05:14:49.000Z","size":11442,"stargazers_count":62,"open_issues_count":7,"forks_count":30,"subscribers_count":9,"default_branch":"master","last_synced_at":"2023-02-26T11:02:54.906Z","etag":null,"topics":["dcgan","generative-adversarial-network","keras","text-to-image"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chen0040.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-13T11:25:32.000Z","updated_at":"2023-01-08T17:10:23.000Z","dependencies_parsed_at":"2022-08-24T00:40:30.981Z","dependency_job_id":null,"html_url":"https://github.com/chen0040/keras-text-to-image","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chen0040%2Fkeras-text-to-image","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chen0040%2Fkeras-text-to-image/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chen0040%2Fkeras-text-to-image/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chen0040%2Fkeras-text-to-image/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chen0040","download_url":"https://codeload.github.com/chen0040/keras-text-to-image/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229964387,"owners_count":18152034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dcgan","generative-adversarial-network","keras","text-to-image"],"created_at":"2024-12-16T13:16:08.899Z","updated_at":"2024-12-16T13:16:09.569Z","avatar_url":"https://github.com/chen0040.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# keras-text-to-image\n\nTranslate text to image in Keras using GAN and Word2Vec as well as recurrent neural networks\n\nThe following models are implemented in [keras_text_to_image/library]\n\n* [dcgan.py](keras_text_to_image/library/dcgan.py): this version has a very noisy input with text input (half of the \ninput is pure noise while the other half is generated from glove embedding of the input text)\n* [dcgan_v2.py](keras_text_to_image/library/dcgan_v2.py): this version remove noise as input (the input is just \nglove embedding of the input text)\n* [dcgan_v3.py](keras_text_to_image/library/dcgan_v3.py): this version add a configurable amount of noise as input \ntogether with the glove embedding of the text input\n\n\n\n# Usage\n\nThe sample codes below only generate very small images, but the image size can be increased if you have sufficient\nmemory \n\n### Text-to-Image using GloVe and Deep Convolution GAN\n\nBelow is the [sample codes](demo/dcgan_train.py) to train the DCGan on a set of pokemon samples of pair (image, text)\n\n```python\nimport os \nimport sys \nimport numpy as np\nfrom random import shuffle\n\n\ndef main():\n    seed = 42\n\n    np.random.seed(seed)\n    \n    current_dir = os.path.dirname(__file__)\n    # add the keras_text_to_image module to the system path\n    sys.path.append(os.path.join(current_dir, '..'))\n    current_dir = current_dir if current_dir is not '' else '.'\n\n    img_dir_path = current_dir + '/data/pokemon/img'\n    txt_dir_path = current_dir + '/data/pokemon/txt'\n    model_dir_path = current_dir + '/models'\n\n    img_width = 32\n    img_height = 32\n    img_channels = 3\n    \n    from keras_text_to_image.library.dcgan import DCGan\n    from keras_text_to_image.library.utility.img_cap_loader import load_normalized_img_and_its_text\n\n    image_label_pairs = load_normalized_img_and_its_text(img_dir_path, txt_dir_path, img_width=img_width, img_height=img_height)\n\n    shuffle(image_label_pairs)\n\n    gan = DCGan()\n    gan.img_width = img_width\n    gan.img_height = img_height\n    gan.img_channels = img_channels\n    gan.random_input_dim = 200\n    gan.glove_source_dir_path = './very_large_data'\n\n    batch_size = 16\n    epochs = 1000\n    gan.fit(model_dir_path=model_dir_path, image_label_pairs=image_label_pairs,\n            snapshot_dir_path=current_dir + '/data/snapshots',\n            snapshot_interval=100,\n            batch_size=batch_size,\n            epochs=epochs)\n\n\nif __name__ == '__main__':\n    main()\n\n```\n\nBelow is the [sample codes](demo/dcgan_generate.py) on how to load the trained DCGan model to generate\n3 new pokemon samples from each text description of a pokemon:\n\n```python\nimport os \nimport sys \nimport numpy as np\nfrom random import shuffle\n\n\ndef main():\n    seed = 42\n    np.random.seed(seed)\n\n    current_dir = os.path.dirname(__file__)\n    sys.path.append(os.path.join(current_dir, '..'))\n    current_dir = current_dir if current_dir is not '' else '.'\n    \n    img_dir_path = current_dir + '/data/pokemon/img'\n    txt_dir_path = current_dir + '/data/pokemon/txt'\n    model_dir_path = current_dir + '/models'\n\n    img_width = 32\n    img_height = 32\n    \n    from keras_text_to_image.library.dcgan import DCGan\n    from keras_text_to_image.library.utility.image_utils import img_from_normalized_img\n    from keras_text_to_image.library.utility.img_cap_loader import load_normalized_img_and_its_text\n\n    image_label_pairs = load_normalized_img_and_its_text(img_dir_path, txt_dir_path, img_width=img_width, img_height=img_height)\n\n    shuffle(image_label_pairs)\n\n    gan = DCGan()\n    gan.load_model(model_dir_path)\n\n    for i in range(3):\n        image_label_pair = image_label_pairs[i]\n        normalized_image = image_label_pair[0]\n        text = image_label_pair[1]\n\n        image = img_from_normalized_img(normalized_image)\n        image.save(current_dir + '/data/outputs/' + DCGan.model_name + '-generated-' + str(i) + '-0.png')\n        for j in range(3):\n            generated_image = gan.generate_image_from_text(text)\n            generated_image.save(current_dir + '/data/outputs/' + DCGan.model_name + '-generated-' + str(i) + '-' + str(j) + '.png')\n\n\nif __name__ == '__main__':\n    main()\n\n```\n\n# Configure to run on GPU on Windows\n\n* Step 1: Change tensorflow to tensorflow-gpu in requirements.txt and install tensorflow-gpu\n* Step 2: Download and install the [CUDA® Toolkit 9.0](https://developer.nvidia.com/cuda-90-download-archive) (Please note that\ncurrently CUDA® Toolkit 9.1 is not yet supported by tensorflow, therefore you should download CUDA® Toolkit 9.0)\n* Step 3: Download and unzip the [cuDNN 7.4 for CUDA@ Toolkit 9.0](https://developer.nvidia.com/cudnn) and add the\nbin folder of the unzipped directory to the $PATH of your Windows environment \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchen0040%2Fkeras-text-to-image","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchen0040%2Fkeras-text-to-image","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchen0040%2Fkeras-text-to-image/lists"}