{"id":15664010,"url":"https://github.com/patlevin/pdn-styletransfer","last_synced_at":"2025-08-18T01:33:49.349Z","repository":{"id":51388381,"uuid":"231801043","full_name":"patlevin/pdn-styletransfer","owner":"patlevin","description":"A Paint.NET Plugin that implements Arbitrary Neural Style Transfer","archived":false,"fork":false,"pushed_at":"2021-08-13T11:12:28.000Z","size":70001,"stargazers_count":19,"open_issues_count":3,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-05-06T19:06:46.670Z","etag":null,"topics":["neural-style-transfer","paintdotnet","paintdotnet-plugin"],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/patlevin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-04T17:21:02.000Z","updated_at":"2024-09-27T18:12:54.000Z","dependencies_parsed_at":"2022-09-02T06:44:32.881Z","dependency_job_id":null,"html_url":"https://github.com/patlevin/pdn-styletransfer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/patlevin/pdn-styletransfer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patlevin%2Fpdn-styletransfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patlevin%2Fpdn-styletransfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patlevin%2Fpdn-styletransfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patlevin%2Fpdn-styletransfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/patlevin","download_url":"https://codeload.github.com/patlevin/pdn-styletransfer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/patlevin%2Fpdn-styletransfer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270932567,"owners_count":24670240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["neural-style-transfer","paintdotnet","paintdotnet-plugin"],"created_at":"2024-10-03T13:40:53.428Z","updated_at":"2025-08-18T01:33:49.328Z","avatar_url":"https://github.com/patlevin.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Paint.NET Effect Plugin based on Arbitrary Style Transfer\r\n\r\nThis plugin makes [Neural Style Transfer](https://en.wikipedia.org/wiki/Neural_Style_Transfer) available in the free and popular Pain.NET image editing software.\r\n\r\n## Introduction\r\n\r\nIn neural style transfer, the style of an image is extracted and transferred to another image. The technique has become popular in recent years and can yield very interesting results. Here's an example using the painting “The Starry Night” by Vincent van Gogh:\r\n\r\n![neural style example](StyleTransfer/assets/images/Example.png \"Neural style example\")\r\n\r\nMost implementations use a single style or very limited set of styles and deliver high quality results. This plugin is more of an exploratory tool for art and fun, and uses arbitrary neural style transfer as described in the paper [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://arxiv.org/abs/1705.06830) by Ghiasi, et. al.\r\nThe models where ported from pre-trained versions available in the [Magenta repository](https://github.com/tensorflow/magenta/tree/master/magenta/models/arbitrary_image_stylization). This technique allows interpolating between styles. Instead of blending the original image and the stylised result, the content image style is extracted and can be mixed with the target style at a selectable ratio.\r\n\r\n## Installation\r\n\r\nThe released version contains an installer that copies the plugin and all its required dependencies and data to the Paint.NET-Effects folder automatically.\r\n\r\n(If you just want to try it out, download the installer directly from [**HERE**](https://github.com/patlevin/pdn-styletransfer/releases/download/v1.1/StyleTransferEffect-Installer.exe) - requires the latest Paint.NET 4.2.16+)\r\n\r\n### System requirements\r\n\r\n* Windows 7 or better, **64-bit** version\r\n* **lots** of RAM (16 GB+ recommended, but will work fine with 8GB as well)\r\n* Paint.NET version 4.2 or newer (tested with PDN 4.2.8)\r\n* a fast CPU (the plugin doesn't support GPUs at the moment)\r\n\r\n## Using the plugin\r\n\r\nAfter the installation, the plugin can be found in, and selected from Paint.NET's \"**Effects**\"-menu in the section \"**Artistic**\":\r\n\r\n![plugin location](StyleTransfer/assets/images/plugin_location.png \"Plugin: Selecting from menu\")\r\n\r\n### Selecting presets\r\n\r\n![plugin presets](StyleTransfer/assets/images/plugin_presets.png \"Plugin: Preset Selection\")\r\n\r\nThe first tab of the plugin lets you select a preset from a selection of styles extracted from random images. To avoid copyright issues, none of the style images are contained in the plugin or its source.\r\nAfter selecting a style, a sample image with the style applied is shown. This example image features a movable slider for a nice \"before and after\"-effect:\r\n\r\n![plugin preset example](StyleTransfer/assets/images/plugin_preset_example.png \"Plugin: Preset Example\")\r\n\r\nThe \"Stylize Amount\"-slider lets you adjust the ratio between the selected style and the content image style. Note that this is not a simple alpha-blend, though.\r\nEven selecting zero (\"0\") will change the content image significantly, as the model tries to apply the style extracted from the content to the content itself. This process isn't perfect by any means, so keep that in mind.\r\n\r\n\"Style Extraction\" and \"Style Transfer\" models can be selected as well. The \"High Quality\" setting uses a larger model for extracting and transferring styles. The \"Fast\"-option selects a smaller, \"distilled\" version of the higher quality one that requires less memory and less CPU power. The results will be very different between the models and depending on the selected style and amount, you might even prefer one over the other.\r\n\r\n### Creating custom styles\r\n\r\nThe \"**Custom Style**\"-tab allows you to select your own style images and apply them:\r\n\r\n![plugin custom style](StyleTransfer/assets/images/plugin_custom_style.png \"Plugin: Custom Style\")\r\n\r\nClicking the \"**Select Style Image**\"-button or clicking the style image preview directly, will open a file dialog for image selection. The \"**Style Image Size**\"-slider determines the relative scaling factor of the style image that is used before the style is analyzed. Think of it as how much of the style image the model is allowed to \"see\" before the style is extracted.\r\nThe colored bar below the slider indicates a size range that will probably yield the best-looking results, with the green arrow pointing at the recommended value.\r\nYou can select values outside this range (though RAM and minimum size restrictions still apply), but the results may not be as good.\r\n\r\nThe plugin will display a warning, if you choose a combination of a big style image and large style size:\r\n\r\n![plugin size warning](StyleTransfer/assets/images/plugin_custom_style_warning.png \"Plugin: Size warning\")\r\n\r\nThis happens, because the processing requires a large amount of RAM and once you use more RAM than is physically installed in your computer, your system will crawl to a halt, become unresponsive, and eventually even crash Paint.NET.\r\nThe plugin will not use your settings, if they would result in too much RAM being used to protect the system from running out of memory.\r\n\r\n## How it works\r\n\r\nThe plugin will first determine whether it can process the content image as a whole. Since the AI models use a lot of RAM, not all image sizes can be processed while still fitting in the computer's memory. Big images will therefore be divided into tiles, which are processed separately. This means, that the style transfer doesn't apply to the image as a whole in such cases, resulting in artifacting at the tile boundaries.\r\nIn order to mitigate this effect somewhat, the algorithm creates a set of overlapping tiles that will be blended into each other. This works well in general, but results may vary depending on image size and -content.\r\nThe amount of available RAM also affects the resulting quality as larger tiles means fewer tiles, and thus less potential artifacting...\r\n\r\nThe selected style image also affects the results (duh!), but not necessarily in a way that's obvious. This is actually the beauty of the effect (at least in my eyes), as you never quite get what you'd expect. Part of this comes from the limited training data and -time on the AI models used - specialised models that have been trained on a single image style or a small set of styles, perform much better.\r\n\r\nAnother factor is the amount of information that can be captures during style extraction. The model uses a 100-element style vector and trying to capture the essence of a mega-pixel artwork in just 100 numbers is... challenging, to say the least.\r\n\r\nThis is also where the styleimage size setting comes into play. Selecting a smaller value will remove details and focus on \"the big picture\", while values that are too big might simply fail to capture any meaningful data during the style extraction process.\r\n\r\n## Building from source\r\n\r\nBuilding from source requires Visual Studio 2019 (available for free [here](https://visualstudio.microsoft.com/vs/)).\r\nPresets cannot be recreated from the source code just to avoid any potential copyright issues. All dependencies are included, except for the NuGet-packages.\r\nThe NuGet-packages will be acquired automatically by Visual Studio.\r\nThe included [ONNX runtime](https://microsoft.github.io/onnxruntime/) is a custom build using the CPU-backend with multi-core optimizations enabled.\r\n\r\n## Future improvements\r\n\r\nI tried to pack as much into the plugin as I deemed necessary to make it useful and fun to play with. There are, however, a few things that I skipped, just to get it out the door first:\r\n\r\n* **GPU-support**\r\n* RAM-usage optimisations\r\n* combining multiple styles\r\n* user-created presets\r\n* support for cancel while processing the effect\r\n* advanced options (tiling overlap, blend modes)\r\n* **improved AI models**\r\n\r\n## GPU Support Update\r\n\r\nThe DirectML team has made great progress since I first tested it. \r\nI also tested the CUDA version of the ONNX Runtime, but at over 160 MB it's ridiculously huge in size and impractical for use.\r\n\r\nAs it stands now, DirectML is the way to go. The latest version fully supports the models used in this project and peformance is\r\nwhere it should be:\r\n\r\n| Device   |Style extraction|Content transform| Total    |\r\n|:---------|---------------:|----------------:|---------:|\r\n| CPU¹     | 225ms          | 3393ms          | 3619ms   |\r\n| iGPU²    |  64ms          |  882ms          |  947ms   |\r\n| dGPU³    |  31ms          |  508ms          |  540ms   |\r\n\r\n¹[Intel® Core™ i7-7500U @ 25W](https://ark.intel.com/content/www/us/en/ark/products/95451/intel-core-i7-7500u-processor-4m-cache-up-to-3-50-ghz.html)\r\n²[Intel® HD Graphics 620](https://ark.intel.com/content/www/us/en/ark/products/95451/intel-core-i7-7500u-processor-4m-cache-up-to-3-50-ghz.html)\r\n³[NVIDIA GeForce GTX 950M, ASUS](https://www.techpowerup.com/gpu-specs/asus-gtx-950m-2-gb.b5469)\r\n\r\nNumbers taken from a 10 run average after a single warmup run (see below).\r\n\r\nEven using the rather weak integrated GPU yielded very good results: style transfer is 3.8x faster.\r\nThe equally weak discrete GPU (comparable to the more modern 150MX/250MX/350MX variants found in modern ultrabooks or AMD APUs) achieves an impressive\r\n6.7x speed-up and is about twice as fast as the iGPU.\r\n\r\nThere is one small issue, though that I still need to investigate further. Note that the 10 run averages exclude a warm-up run.\r\nThis first run is about 50% slower due to JIT and DLL loading - that's OK and still significantly faster than using just the CPU.\r\nThe problem arises with the very first(?) run, however. The Intel® iGPU took a shocking 42 seconds to complete the first pass.\r\nI actually had to run it 3 times, because I genuinely thought the app had crashed...\r\nThe dGPU's very first(?) run took \"just\" 12 seconds, which is still unacceptable compared to the CPU performance.\r\n\r\nSubsequent runs of the benchmark app (even after OS restart) displayed the expected behaviour of the first run being about 50% slower than the average.\r\n\r\nUnfortunately I learned that there's no way to pre-compile shaders from another executable as [per the specs](https://github.com/microsoft/DirectX-Specs/blob/master/d3d/ShaderCache.md). The very first run of the models will therefore always be significantly slower if D3D12 is used.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatlevin%2Fpdn-styletransfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpatlevin%2Fpdn-styletransfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpatlevin%2Fpdn-styletransfer/lists"}