{"id":13670540,"url":"https://github.com/reiinakano/arbitrary-image-stylization-tfjs","last_synced_at":"2025-05-16T12:12:05.595Z","repository":{"id":39787961,"uuid":"157377758","full_name":"reiinakano/arbitrary-image-stylization-tfjs","owner":"reiinakano","description":"Arbitrary style transfer using TensorFlow.js","archived":false,"fork":false,"pushed_at":"2022-12-03T03:20:33.000Z","size":74757,"stargazers_count":1202,"open_issues_count":22,"forks_count":213,"subscribers_count":40,"default_branch":"master","last_synced_at":"2025-04-09T10:03:38.220Z","etag":null,"topics":["artists","browser","deep-learning","generative-art","neural-network","style-transfer","tensorflow-js"],"latest_commit_sha":null,"homepage":"https://reiinakano.github.io/arbitrary-image-stylization-tfjs/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reiinakano.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-13T12:36:03.000Z","updated_at":"2025-04-07T15:14:25.000Z","dependencies_parsed_at":"2023-01-23T04:15:39.607Z","dependency_job_id":null,"html_url":"https://github.com/reiinakano/arbitrary-image-stylization-tfjs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reiinakano%2Farbitrary-image-stylization-tfjs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reiinakano%2Farbitrary-image-stylization-tfjs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reiinakano%2Farbitrary-image-stylization-tfjs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reiinakano%2Farbitrary-image-stylization-tfjs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reiinakano","download_url":"https://codeload.github.com/reiinakano/arbitrary-image-stylization-tfjs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254527099,"owners_count":22085919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artists","browser","deep-learning","generative-art","neural-network","style-transfer","tensorflow-js"],"created_at":"2024-08-02T09:00:44.782Z","updated_at":"2025-05-16T12:12:00.579Z","avatar_url":"https://github.com/reiinakano.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","Uncategorized"],"sub_categories":["Uncategorized"],"readme":"# Arbitrary style transfer in TensorFlow.js\n\nThis repository contains an implementation of arbitrary style transfer running fully\ninside the browser using TensorFlow.js.\n\nDemo website: https://reiinakano.github.io/arbitrary-image-stylization-tfjs\n\nBlog post with more details: https://magenta.tensorflow.org/blog/2018/12/20/style-transfer-js/\n\n### Stylize an image\n\n![stylize](readme_img/stylize.jpg)\n\n### Combine styles\n\n![combine](readme_img/combine.jpg)\n\n## FAQ\n\n### What is this?\n\nThis is an implementation of an arbitrary style transfer algorithm\nrunning purely in the browser using TensorFlow.js. As with all neural \nstyle transfer algorithms, a neural network attempts to \"draw\" one \npicture, the Content (usually a photograph), in the style of another, \nthe Style (usually a painting). \n\nAlthough [other browser implementations](https://github.com/reiinakano/fast-style-transfer-deeplearnjs)\nof style transfer exist,\nthey are normally limited to a pre-selected handful of styles, due to\nthe requirement that a separate neural network must be trained for each\nstyle image.\n\nArbitrary style transfer works around this limitation by using a\nseparate *style network* that learns to break down *any* image into \na 100-dimensional vector representing its style. This style vector is \nthen fed into another network, the *transformer network*, along\nwith the content image, to produce the final stylized image.\n\nI have written a [blog post](https://magenta.tensorflow.org/blog/2018/12/20/style-transfer-js/) \nexplaining this project in more detail.\n\n### Is my data safe? Can you see my pictures?\n\nYour data and pictures here never leave your computer! In fact,\nthis is one of the main advantages of running neural networks \nin your browser. Instead of sending us your data, we send *you* \nboth the model *and* the code to run the model. These are then \nrun by your browser.\n\n### What are all these different models?\n\nThe original paper uses an Inception-v3 model \nas the style network, which takes up ~36.3MB \nwhen ported to the browser as a FrozenModel.\n\nIn order to make this model smaller, a MobileNet-v2 was\nused to distill the knowledge from the pretrained Inception-v3 \nstyle network. This resulted in a size reduction of just under 4x,\nfrom ~36.3MB to ~9.6MB, at the expense of some quality.\n\nFor the transformer network, the original paper uses \na model using plain convolution layers. When ported to\nthe browser, this model takes up 7.9MB and is responsible\nfor the majority of the calculations during stylization.\n\nIn order to make the transformer model more efficient, most of the\nplain convolution layers were replaced with depthwise separable \nconvolutions. This reduced the model size to 2.4MB, while\ndrastically improving the speed of stylization.\n\nThis demo lets you use any combination of the models, defaulting\nto the MobileNet-v2 style network and the separable convolution\ntransformer network.\n\n### How big are the models I'm downloading?\n\nThe distilled style network is ~9.6MB, while the separable convolution\ntransformer network is ~2.4MB, for a total of ~12MB. \nSince these models work for any style, you only \nhave to download them once!\n\n### How does style combination work?\n\nSince each style can be mapped to a 100-dimensional \nstyle vector by the style network,\nwe simply take a weighted average of the two to get\na new style vector for the transformer network.\n\nThis is also how we are able to control the strength\nof stylization. We take a weighted average of the style \nvectors of *both* content and style images and use \nit as input to the transformer network.\n\n## Running locally for development\n\nThis project uses [Yarn](https://yarnpkg.com/en/) for dependencies.\n\nTo run it locally, you must install Yarn and run the following command at the repository's root to get all the dependencies.\n\n```bash\nyarn run prep\n```\n\nThen, you can run\n\n```bash\nyarn run start\n```\n\nYou can then browse to `localhost:9966` to view the application.\n\n\n## Credits\n\nThis demo could not have been done without the following:\n\n* Authors of the [arbitrary style transfer](https://arxiv.org/abs/1705.06830) paper.\n* The Magenta repository for [arbitrary style transfer](https://github.com/tensorflow/magenta/tree/master/magenta/models/arbitrary_image_stylization).\n* Authors of [the MobileNet-v2 paper](https://arxiv.org/abs/1801.04381).\n* Authors of the paper describing [neural network knowledge distillation](https://arxiv.org/abs/1503.02531).\n* The [TensorFlow.js library](https://js.tensorflow.org).\n* [Google Colaboratory](https://colab.research.google.com/), with which I was able \nto do all necessary training using a free(!) GPU.\n\nAs a final note, I'd love to hear from people interested \nin making a suite of tools for artistically manipulating images, kind of like \n[Magenta Studio](https://magenta.tensorflow.org/studio)\nbut for images. Please reach out if you're planning to build/are \nbuilding one out!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freiinakano%2Farbitrary-image-stylization-tfjs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freiinakano%2Farbitrary-image-stylization-tfjs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freiinakano%2Farbitrary-image-stylization-tfjs/lists"}