{"id":13419517,"url":"https://github.com/ashnkumar/sketch-code","last_synced_at":"2025-05-14T15:10:25.818Z","repository":{"id":37790353,"uuid":"118181798","full_name":"ashnkumar/sketch-code","owner":"ashnkumar","description":"Keras model to generate HTML code from hand-drawn website mockups. Implements an image captioning architecture to drawn source images.","archived":false,"fork":false,"pushed_at":"2024-04-05T09:48:18.000Z","size":2949,"stargazers_count":5136,"open_issues_count":29,"forks_count":690,"subscribers_count":212,"default_branch":"master","last_synced_at":"2025-04-04T06:01:30.826Z","etag":null,"topics":["augmentation","deep-learning","image-processing","keras","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashnkumar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-19T21:44:13.000Z","updated_at":"2025-04-03T08:01:29.000Z","dependencies_parsed_at":"2024-10-13T06:40:49.770Z","dependency_job_id":"f167502a-6471-497f-b5cc-647468e44764","html_url":"https://github.com/ashnkumar/sketch-code","commit_stats":{"total_commits":10,"total_committers":2,"mean_commits":5.0,"dds":0.09999999999999998,"last_synced_commit":"cda077981047594e6ae09b17820e24c509f595a8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashnkumar%2Fsketch-code","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashnkumar%2Fsketch-code/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashnkumar%2Fsketch-code/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashnkumar%2Fsketch-code/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashnkumar","download_url":"https://codeload.github.com/ashnkumar/sketch-code/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248384688,"owners_count":21094757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["augmentation","deep-learning","image-processing","keras","tensorflow"],"created_at":"2024-07-30T22:01:17.110Z","updated_at":"2025-04-11T11:36:28.427Z","avatar_url":"https://github.com/ashnkumar.png","language":"Python","readme":"# SketchCode\r\n\r\n![](https://img.shields.io/badge/python-3-brightgreen.svg) ![](https://img.shields.io/badge/tensorflow-1.1.0-orange.svg)\r\n\r\n*Generating HTML Code from a hand-drawn wireframe*\r\n\r\n![Preview](https://github.com/ashnkumar/sketch-code/blob/master/header_image.png)\r\n\r\nSketchCode is a deep learning model that takes hand-drawn web mockups and converts them into working HTML code. It uses an [image captioning](https://towardsdatascience.com/image-captioning-in-deep-learning-9cd23fb4d8d2) architecture to generate its HTML markup from hand-drawn website wireframes.\r\n\r\nFor more information, check out this post: [Automating front-end development with deep learning](https://blog.insightdatascience.com/automated-front-end-development-using-deep-learning-3169dd086e82)\r\n\r\nThis project builds on the synthetically generated dataset and model architecture from [pix2code](https://github.com/tonybeltramelli/pix2code) by [Tony Beltramelli](https://github.com/tonybeltramelli) and the [Design Mockups](https://github.com/emilwallner/Screenshot-to-code-in-Keras) project from [Emil Wallner](https://github.com/emilwallner).\r\n\r\n\u003cb\u003eNote:\u003c/b\u003e This project is meant as a proof-of-concept; the model isn't (yet) built to generalize to the variability of sketches seen in actual wireframes, and thus its performance relies on wireframes resembling the core dataset.\r\n\r\n\r\n## Setup\r\n### Prerequisites\r\n\r\n- Python 3 (not compatible with python 2)\r\n- pip\r\n\r\n### Install dependencies\r\n\r\n```sh\r\npip install -r requirements.txt\r\n```\r\n\r\n## Example Usage\r\n\r\nDownload the data and pretrained weights:\r\n```sh\r\n# Getting the data, 1,700 images, 342mb\r\ngit clone https://github.com/ashnkumar/sketch-code.git\r\ncd sketch-code\r\ncd scripts\r\n\r\n# Get the data and pretrained weights\r\nsh get_data.sh\r\nsh get_pretrained_model.sh\r\n```\r\n\r\nConverting an example drawn image into HTML code, using pretrained weights:\r\n```sh\r\ncd src\r\n\r\npython convert_single_image.py --png_path ../examples/drawn_example1.png \\\r\n      --output_folder ./generated_html \\\r\n      --model_json_file ../bin/model_json.json \\\r\n      --model_weights_file ../bin/weights.h5\r\n```\r\n\r\n\r\n## General Usage\r\n\r\nConverting a single image into HTML code, using weights:\r\n```sh\r\ncd src\r\n\r\npython convert_single_image.py --png_path {path/to/img.png} \\\r\n      --output_folder {folder/to/output/html} \\\r\n      --model_json_file {path/to/model/json_file.json} \\\r\n      --model_weights_file {path/to/model/weights.h5}\r\n```\r\n\r\nConverting a batch of images in a folder to HTML:\r\n```sh\r\ncd src\r\n\r\npython convert_batch_of_images.py --pngs_path {path/to/folder/with/pngs} \\\r\n      --output_folder {folder/to/output/html} \\\r\n      --model_json_file {path/to/model/json_file.json} \\\r\n      --model_weights_file {path/to/model/weights.h5}\r\n```\r\n\r\nTrain the model:\r\n```sh\r\ncd src\r\n\r\n# training from scratch\r\n# \u003caugment_training_data\u003e adds Keras ImageDataGenerator augmentation for training images\r\npython train.py --data_input_path {path/to/folder/with/pngs/guis} \\\r\n      --validation_split 0.2 \\\r\n      --epochs 10 \\\r\n      --model_output_path {path/to/output/model}\r\n      --augment_training_data 1\r\n\r\n# training starting with pretrained model\r\npython train.py --data_input_path {path/to/folder/with/pngs/guis} \\\r\n      --validation_split 0.2 \\\r\n      --epochs 10 \\\r\n      --model_output_path {path/to/output/model} \\\r\n      --model_json_file ../bin/model_json.json \\\r\n      --model_weights_file ../bin/pretrained_weights.h5 \\\r\n      --augment_training_data 1\r\n```\r\n\r\nEvalute the generated prediction using the [BLEU score](https://machinelearningmastery.com/calculate-bleu-score-for-text-python/)\r\n```sh\r\ncd src\r\n\r\n# evaluate single GUI prediction\r\npython evaluate_single_gui.py --original_gui_filepath  {path/to/original/gui/file} \\\r\n      --predicted_gui_filepath {path/to/predicted/gui/file}\r\n\r\n# training starting with pretrained model\r\npython evaluate_batch_guis.py --original_guis_filepath  {path/to/folder/with/original/guis} \\\r\n      --predicted_guis_filepath {path/to/folder/with/predicted/guis}\r\n```\r\n\r\n## License\r\n\r\n### The MIT License (MIT)\r\n\r\nCopyright (c) 2018 Ashwin Kumar\u003cash.nkumar@gmail.com@gmail.com\u003e\r\n\r\n\u003e Permission is hereby granted, free of charge, to any person obtaining a copy\r\n\u003e of this software and associated documentation files (the \"Software\"), to deal\r\n\u003e in the Software without restriction, including without limitation the rights\r\n\u003e to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\r\n\u003e copies of the Software, and to permit persons to whom the Software is\r\n\u003e furnished to do so, subject to the following conditions:\r\n\u003e\r\n\u003e The above copyright notice and this permission notice shall be included in\r\n\u003e all copies or substantial portions of the Software.\r\n\u003e\r\n\u003e THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\r\n\u003e IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\r\n\u003e FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\r\n\u003e AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\r\n\u003e LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\r\n\u003e OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\r\n\u003e THE SOFTWARE.\r\n","funding_links":[],"categories":["Python","Python (144)","📦 Legacy \u0026 Inactive Projects","人工智能 \u003ca name=\"AI\"\u003e\u003c/a\u003e","Examples/Notebooks"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashnkumar%2Fsketch-code","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashnkumar%2Fsketch-code","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashnkumar%2Fsketch-code/lists"}