{"id":23021696,"url":"https://github.com/vinothpandian/metamorph","last_synced_at":"2025-08-14T09:33:05.951Z","repository":{"id":39853352,"uuid":"306470154","full_name":"vinothpandian/MetaMorph","owner":"vinothpandian","description":"MetaMorph is a UI element detector that detects constituent UI elements of freehand lo-fi sketches using DNN based object detection models.","archived":false,"fork":false,"pushed_at":"2024-02-25T17:22:38.000Z","size":755,"stargazers_count":7,"open_issues_count":6,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-14T13:01:36.576Z","etag":null,"topics":["deep-learning","low-fidelity-prototype","object-detection","object-detection-api","sketch-detection-api","ui-detection","ui-element-detector"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vinothpandian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-22T22:13:31.000Z","updated_at":"2024-04-14T13:01:36.577Z","dependencies_parsed_at":"2023-02-08T03:30:28.626Z","dependency_job_id":null,"html_url":"https://github.com/vinothpandian/MetaMorph","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinothpandian%2FMetaMorph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinothpandian%2FMetaMorph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinothpandian%2FMetaMorph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vinothpandian%2FMetaMorph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vinothpandian","download_url":"https://codeload.github.com/vinothpandian/MetaMorph/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229816448,"owners_count":18128577,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","low-fidelity-prototype","object-detection","object-detection-api","sketch-detection-api","ui-detection","ui-element-detector"],"created_at":"2024-12-15T12:18:52.500Z","updated_at":"2024-12-15T12:18:52.592Z","avatar_url":"https://github.com/vinothpandian.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eMetaMorph: AI Assistance to Transform Lo-Fi Sketches to Higher Fidelities\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg alt=\"Version\" src=\"https://img.shields.io/badge/version-1.0.0-blue.svg?cacheSeconds=2592000\" /\u003e\n  \u003ca href=\"#\" target=\"_blank\"\u003e\n    \u003cimg alt=\"License: MIT\" src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" /\u003e\n  \u003c/a\u003e\n  \u003cbr/\u003e\n  \u003ca href=\"#\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Python: 3.6\" src=\"https://img.shields.io/badge/Python-3.6-important\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"#\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Dependency: Tensorflow 1.9\" src=\"https://img.shields.io/badge/Tensorflow-1.9-important\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://python-poetry.org/\" target=\"_blank\"\u003e\n    \u003cimg alt=\"Dependency Manager: Poetry\" src=\"https://img.shields.io/badge/Dependency Manager-Poetry-important\" /\u003e\n  \u003c/a\u003e\n  \u003cbr/\u003e\n  \u003cbr/\u003e\n\u003c/p\u003e\n\n\u003e MetaMorph is an AI tool to detect the constituent UI elements of low fidelity prototype sketches.\n\n---\n\n## Dataset\n\nMetaMorph uses\n\n- [UISketch](https://www.kaggle.com/vinothpandian/uisketch)\n- [Syn dataset](https://www.kaggle.com/vinothpandian/syn-dataset)\n\n---\n\n## Setup and usage\n\nMetaMorph API uses Python 3.6 and Tensorflow Object Detection API (TFOD).\n\nTo install and use MetaMorph API, follow the steps below\n\n- Download the following files to the `models/` directory\n\n  - [frozen_inference_graph.pb](https://blackbox-toolkit.s3.us-east-2.amazonaws.com/files/uisketch/frozen_inference_graph.pb)\n  - [labels.json](https://blackbox-toolkit.s3.us-east-2.amazonaws.com/files/uisketch/labels.json)\n\n- Install poetry\n\n  ```sh\n  pip install poetry\n  ```\n\n- Upgrade pip, as older version of pip causes installation issue with opencv\n\n  ```\n  poetry run pip install --upgrade pip\n  ```\n\n- Install dependencies\n  ```sh\n  poetry install\n  ```\n- Run the API\n  ```sh\n  poetry run python app.py\n  ```\n\n---\n\n## Docker\n\nMetaMorph API can be quickly deployed from docker without any installation or model download steps.\n\nTo use it, pull the image from [dockerhub at vinothpandian/metamorph](https://hub.docker.com/repository/docker/vinothpandian/metamorph)\n\n```\ndocker pull vinothpandian/metamorph:latest\n```\n\nRun it with docker the following command\n\n```sh\ndocker run -p 8000:8000 --name metamorph vinothpandian/metamorph:latest\n```\n\n---\n\n## Development\n\nTo retrain or further improve MetaMorph model\n\n### Installation\n\n- Install Tensorflow Object Detection API using [this installation guide](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1.md).\n- Copy the config file from `configs` folder or setup your own [Tensorflow Object Detection API Model with config](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md)\n\n- Download the [UISketch](https://www.kaggle.com/vinothpandian/syn-dataset) dataset from Kaggle.\n- Generate synthetic data\n\n  ```sh\n  poetry run python syn_datagen.py -d /path/to/uisketch -o /path/to/tfod/object_detection -l no.of.sketches\n  ```\n\n  To check more options on data generation use `poetry run python syn_datagen.py --help`\n\n- Follow the guide from [training and evaluation guide](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_training_and_evaluation.md) by TFOD team.\n\n---\n\n## Citation\n\nIf you use MetaMorph, please use the following citation:\n\n- V. P. S. Pandian, S. Suleri, C. Beecks, M. Jarke. **MetaMorph: AI Assistance to Transform Lo-Fi Sketches to Higher Fidelities.** _Proceedings of the 32st Australian Conference on Human-Computer-Interaction._\n\n```bib\n@inproceedings{Pandian_MetaMorph,\n\ttitle        = {MetaMorph: AI Assistance to Transform Lo-Fi Sketches to Higher Fidelities.},\n\tauthor       = {Pandian, Vinoth Pandian Sermuga and Suleri, Sarah and Beecks, Christian and Jarke, Matthias},\n\tyear         = 2020,\n\tbooktitle    = {Proceedings of the 32st Australian Conference on Human-Computer-Interaction},\n\tpublisher    = {Association for Computing Machinery},\n\taddress      = {New York, NY, USA},\n\tseries       = {OZCHI'20},\n\tdoi          = {10.1145/3441000.3441030},\n\tisbn         = {978-1-4503-8975-4/20/12},\n\turl          = {https://doi.org/10.1145/3441000.3441030},\n\tnumpages     = 10,\n\tkeywords     = {UI Element Dataset, Neural Networks, Deep Learning, Sketch Detection, Sketch Recognition, Artificial Intelligence, Prototyping}\n}\n```\n\nIf you use Syn or UISketch, please use the following citation:\n\n```bib\n@inproceedings{Pandin_Syn,\n\ttitle        = {Syn: Synthetic Dataset for Training UI Element Detector From Lo-Fi Sketches},\n\tauthor       = {Pandian, Vinoth Pandian Sermuga and Suleri, Sarah and Jarke, Matthias},\n\tyear         = 2020,\n\tbooktitle    = {Proceedings of the 25th International Conference on Intelligent User Interfaces Companion},\n\tlocation     = {Cagliari, Italy},\n\tpublisher    = {Association for Computing Machinery},\n\taddress      = {New York, NY, USA},\n\tseries       = {IUI '20},\n\tpages        = {79–80},\n\tdoi          = {10.1145/3379336.3381498},\n\tisbn         = 9781450375139,\n\turl          = {https://doi.org/10.1145/3379336.3381498},\n\tnumpages     = 2,\n\tkeywords     = {Neural Networks, Sketch Detection, Prototyping, Sketch Recognition, UI Element Dataset, Deep Learning}\n}\n```\n\n---\n\n## Authors\n\n👤 **Vinoth Pandian**\n\n- Website: [vinoth.info](https://vinoth.info)\n- Github: [@vinothpandian](https://github.com/vinothpandian)\n- LinkedIn: [@vinothpandian](https://linkedin.com/in/vinothpandian)\n\n👤 **Sarah Suleri**\n\n- Website: [sarahsuleri.info](https://sarahsuleri.info)\n- LinkedIn: [@sarahsuleri](https://linkedin.com/in/sarahsuleri)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinothpandian%2Fmetamorph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvinothpandian%2Fmetamorph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinothpandian%2Fmetamorph/lists"}