{"id":13401225,"url":"https://github.com/emilwallner/Screenshot-to-code","last_synced_at":"2025-03-14T06:32:18.660Z","repository":{"id":38555486,"uuid":"107119649","full_name":"emilwallner/Screenshot-to-code","owner":"emilwallner","description":"A neural network that transforms a design mock-up into a static website.","archived":false,"fork":false,"pushed_at":"2024-08-16T22:05:07.000Z","size":51643,"stargazers_count":16420,"open_issues_count":21,"forks_count":1557,"subscribers_count":537,"default_branch":"master","last_synced_at":"2024-11-21T08:04:56.736Z","etag":null,"topics":["cnn","cnn-keras","deep-learning","encoder-decoder","floydhub","jupyter","jupyter-notebook","keras","lstm","machine-learning","seq2seq"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emilwallner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-16T11:41:48.000Z","updated_at":"2024-11-21T03:23:39.000Z","dependencies_parsed_at":"2024-06-25T12:14:07.237Z","dependency_job_id":"7e3aeb5e-f057-4728-90f3-a555a603a6e3","html_url":"https://github.com/emilwallner/Screenshot-to-code","commit_stats":{"total_commits":169,"total_committers":8,"mean_commits":21.125,"dds":0.242603550295858,"last_synced_commit":"f7f8ceee38094e990fec5a76e10f50d8b91a5033"},"previous_names":["emilwallner/screenshot-to-code-in-keras"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilwallner%2FScreenshot-to-code","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilwallner%2FScreenshot-to-code/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilwallner%2FScreenshot-to-code/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emilwallner%2FScreenshot-to-code/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emilwallner","download_url":"https://codeload.github.com/emilwallner/Screenshot-to-code/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243538060,"owners_count":20307100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","cnn-keras","deep-learning","encoder-decoder","floydhub","jupyter","jupyter-notebook","keras","lstm","machine-learning","seq2seq"],"created_at":"2024-07-30T19:01:00.084Z","updated_at":"2025-03-14T06:32:18.653Z","avatar_url":"https://github.com/emilwallner.png","language":"HTML","readme":"\u003cimg src=\"/README_images/screenshot-to-code.svg?raw=true\" width=\"800px\"\u003e\n\n---\n\n**A detailed tutorial covering the code in this repository:** [Turning design mockups into code with deep learning](https://emilwallner.medium.com/how-you-can-train-an-ai-to-convert-your-design-mockups-into-html-and-css-cc7afd82fed4).\n\n**Plug:** 👉 Check out my 60-page guide, [No ML Degree](https://www.emilwallner.com/p/no-ml-degree), on how to land a machine learning job without a degree.\n\nThe neural network is built in three iterations. Starting with a Hello World version, followed by the main neural network layers, and ending by training it to generalize. \n\nThe models are based on Tony Beltramelli's [pix2code](https://github.com/tonybeltramelli/pix2code), and inspired by Airbnb's [sketching interfaces](https://airbnb.design/sketching-interfaces/), and Harvard's [im2markup](https://github.com/harvardnlp/im2markup).\n\n**Note:** only the Bootstrap version can generalize on new design mock-ups. It uses 16 domain-specific tokens which are translated into HTML/CSS. It has a 97% accuracy. The best model uses a GRU instead of an LSTM. This version can be trained on a few GPUs. The raw HTML version has potential to generalize, but is still unproven and requires a significant amount of GPUs to train. The current model is also trained on a homogeneous and small dataset, thus it's hard to tell how well it behaves on more complex layouts.\n\nDataset: https://github.com/tonybeltramelli/pix2code/tree/master/datasets\n\nA quick overview of the process: \n\n### 1) Give a design image to the trained neural network\n\n![Insert image](https://i.imgur.com/LDmoLLV.png)\n\n### 2) The neural network converts the image into HTML markup \n\n\u003cimg src=\"/README_images/html_display.gif?raw=true\" width=\"800px\"\u003e\n\n### 3) Rendered output\n\n![Screenshot](https://i.imgur.com/tEAfyZ8.png)\n\n\n## Installation\n\n### FloydHub\n\n[![Run on FloydHub](https://static.floydhub.com/button/button.svg)](https://floydhub.com/run?template=https://github.com/floydhub/pix2code-template)\n\nClick this button to open a [Workspace](https://blog.floydhub.com/workspaces/) on [FloydHub](https://www.floydhub.com/?utm_medium=readme\u0026utm_source=pix2code\u0026utm_campaign=aug_2018) where you will find the same environment and dataset used for the *Bootstrap version*. You can also find the trained models for testing.\n\n### Local\n``` bash\npip install keras tensorflow pillow h5py jupyter\n```\n```\ngit clone https://github.com/emilwallner/Screenshot-to-code.git\ncd Screenshot-to-code/\njupyter notebook\n```\nGo do the desired notebook, files that end with '.ipynb'. To run the model, go to the menu then click on Cell \u003e Run all\n\nThe final version, the Bootstrap version, is prepared with a small set to test run the model. If you want to try it with all the data, you need to download the data here: https://www.floydhub.com/emilwallner/datasets/imagetocode, and specify the correct ```dir_name```.\n\n## Folder structure\n\n``` bash\n  |  |-Bootstrap                           #The Bootstrap version\n  |  |  |-compiler                         #A compiler to turn the tokens to HTML/CSS (by pix2code)\n  |  |  |-resources\t\t\t\t\t\t\t\t\t\t\t\n  |  |  |  |-eval_light                    #10 test images and markup\n  |  |-Hello_world                         #The Hello World version\n  |  |-HTML                                #The HTML version\n  |  |  |-Resources_for_index_file         #CSS,images and scripts to test index.html file\n  |  |  |-html                             #HTML files to train it on\n  |  |  |-images                           #Screenshots for training\n  |-readme_images                          #Images for the readme page\n```\n\n\n## Hello World\n\u003cp align=\"center\"\u003e\u003cimg src=\"/README_images/Hello_world_model.png?raw=true\" width=\"400px\"\u003e\u003c/p\u003e\n\n\n## HTML\n\u003cp align=\"center\"\u003e\u003cimg src=\"/README_images/HTML_model.png?raw=true\" width=\"400px\"\u003e\u003c/p\u003e\n\n\n## Bootstrap\n\u003cp align=\"center\"\u003e\u003cimg src=\"/README_images/Bootstrap_model.png?raw=true\" width=\"400px\"\u003e\u003c/p\u003e\n\n## Model weights\n- [Bootstrap](https://www.floydhub.com/emilwallner/datasets/imagetocode) (The pre-trained model uses GRUs instead of LSTMs)\n- [HTML](https://www.floydhub.com/emilwallner/datasets/html_models)\n\n## Acknowledgments\n- Thanks to IBM for donating computing power through their PowerAI platform\n- The code is largely influenced by Tony Beltramelli's pix2code paper. [Code](https://github.com/tonybeltramelli/pix2code) [Paper](https://arxiv.org/abs/1705.07962)\n- The structure and some of the functions are from Jason Brownlee's [excellent tutorial](https://machinelearningmastery.com/develop-a-caption-generation-model-in-keras/)\n","funding_links":[],"categories":["HTML","Jupyter Notebook","Other Machine Learning Applications","Code to GUI: algorithm to user interface:","前端开发框架及项目","Repos","machine-learning","精选 LessCode 项目","Examples/Notebooks"],"sub_categories":["Others","其他_文本生成、文本对话"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femilwallner%2FScreenshot-to-code","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femilwallner%2FScreenshot-to-code","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femilwallner%2FScreenshot-to-code/lists"}