{"id":13580589,"url":"https://github.com/EmbarkStudios/texture-synthesis","last_synced_at":"2025-04-06T02:32:01.799Z","repository":{"id":35098980,"uuid":"204427049","full_name":"EmbarkStudios/texture-synthesis","owner":"EmbarkStudios","description":"🎨 Example-based texture synthesis written in Rust 🦀","archived":true,"fork":false,"pushed_at":"2023-01-22T17:30:52.000Z","size":3138,"stargazers_count":1774,"open_issues_count":11,"forks_count":85,"subscribers_count":44,"default_branch":"main","last_synced_at":"2025-03-28T01:41:05.282Z","etag":null,"topics":["cli","hacktoberfest","inpaint","rust","texture-synthesis"],"latest_commit_sha":null,"homepage":"http://embark.rs","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EmbarkStudios.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null}},"created_at":"2019-08-26T08:06:48.000Z","updated_at":"2025-03-27T06:06:27.000Z","dependencies_parsed_at":"2023-02-12T17:16:09.941Z","dependency_job_id":null,"html_url":"https://github.com/EmbarkStudios/texture-synthesis","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":"EmbarkStudios/opensource-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Ftexture-synthesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Ftexture-synthesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Ftexture-synthesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmbarkStudios%2Ftexture-synthesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EmbarkStudios","download_url":"https://codeload.github.com/EmbarkStudios/texture-synthesis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247426175,"owners_count":20937075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","hacktoberfest","inpaint","rust","texture-synthesis"],"created_at":"2024-08-01T15:01:53.341Z","updated_at":"2025-04-06T02:31:59.130Z","avatar_url":"https://github.com/EmbarkStudios.png","language":"Rust","funding_links":[],"categories":["3D-File-Format","Rust","File formats","Computer Vision"],"sub_categories":["Denosing"],"readme":"\u003cdiv align=\"center\"\u003e\n\n# `🎨 texture-synthesis`\n\n[![Embark](https://img.shields.io/badge/embark-open%20source-blueviolet.svg)](http://embark.games)\n[![Embark](https://img.shields.io/badge/discord-ark-%237289da.svg?logo=discord)](https://discord.gg/dAuKfZS)\n[![Crates.io](https://img.shields.io/crates/v/texture-synthesis.svg)](https://crates.io/crates/texture-synthesis)\n[![Docs](https://docs.rs/texture-synthesis/badge.svg)](https://docs.rs/texture-synthesis)\n[![dependency status](https://deps.rs/repo/github/EmbarkStudios/texture-synthesis/status.svg)](https://deps.rs/repo/github/EmbarkStudios/texture-synthesis)\n[![Build Status](https://github.com/EmbarkStudios/texture-synthesis/workflows/CI/badge.svg)](https://github.com/EmbarkStudios/texture-synthesis/actions?workflow=CI)\n\nA light Rust API for _Multiresolution Stochastic Texture Synthesis_ [1], a non-parametric example-based algorithm for image generation.\n\nThe repo also includes multiple code examples to get you started (along with test images), and you can find a compiled binary with a command line interface under the release tab.\n\nAlso see our talk [_More Like This, Please! Texture Synthesis and Remixing from a Single Example_](https://youtu.be/fMbK7PYQux4) which explains this technique and the background more in-depth:\n\n[![Video thumbnail](imgs/docs/video-thumbnail.jpg)](https://www.youtube.com/watch?v=fMbK7PYQux4\u0026t=6m57s)\n\n\u003c/div\u003e\n\n## Maintenance note\n\nWe at Embark are not actively using or developing these crates and would be open to transferring them to a maintainer or maintainers that would be more active. See [#166](https://github.com/EmbarkStudios/texture-synthesis/issues/166).\n\n## Features and examples\n\n### 1. Single example generation\n\n![Imgur](https://i.imgur.com/CsZoSPS.jpg)\n\nGenerate similar-looking images from a single example.\n\n#### API - [01_single_example_synthesis](lib/examples/01_single_example_synthesis.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    //create a new session\n    let texsynth = ts::Session::builder()\n        //load a single example image\n        .add_example(\u0026\"imgs/1.jpg\")\n        .build()?;\n\n    //generate an image\n    let generated = texsynth.run(None);\n\n    //save the image to the disk\n    generated.save(\"out/01.jpg\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --out out/01.jpg generate imgs/1.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/8p6nVYl.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 2. Multi example generation\n\n![Imgur](https://i.imgur.com/rYaae2w.jpg)\n\nWe can also provide multiple example images and the algorithm will \"remix\" them into a new image.\n\n#### API - [02_multi_example_synthesis](lib/examples/02_multi_example_synthesis.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    // create a new session\n    let texsynth = ts::Session::builder()\n        // load multiple example image\n        .add_examples(\u0026[\n            \u0026\"imgs/multiexample/1.jpg\",\n            \u0026\"imgs/multiexample/2.jpg\",\n            \u0026\"imgs/multiexample/3.jpg\",\n            \u0026\"imgs/multiexample/4.jpg\",\n        ])\n        // we can ensure all of them come with same size\n        // that is however optional, the generator doesnt care whether all images are same sizes\n        // however, if you have guides or other additional maps, those have to be same size(s) as corresponding example(s)\n        .resize_input(ts::Dims {\n            width: 300,\n            height: 300,\n        })\n        // randomly initialize first 10 pixels\n        .random_init(10)\n        .seed(211)\n        .build()?;\n\n    // generate an image\n    let generated = texsynth.run(None);\n\n    // save the image to the disk\n    generated.save(\"out/02.jpg\")?;\n\n    //save debug information to see \"remixing\" borders of different examples in map_id.jpg\n    //different colors represent information coming from different maps\n    generated.save_debug(\"out/\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --rand-init 10 --seed 211 --in-size 300x300 -o out/02.png --debug-out-dir out generate imgs/multiexample/1.jpg imgs/multiexample/2.jpg imgs/multiexample/3.jpg imgs/multiexample/4.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/tbz5d57.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 3. Guided Synthesis\n\n![Imgur](https://i.imgur.com/eAiNZBg.jpg)\n\nWe can also guide the generation by providing a transformation \"FROM\"-\"TO\" in a form of guide maps\n\n#### API - [03_guided_synthesis](lib/examples/03_guided_synthesis.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    let texsynth = ts::Session::builder()\n        // NOTE: it is important that example(s) and their corresponding guides have same size(s)\n        // you can ensure that by overwriting the input images sizes with .resize_input()\n        .add_example(ts::Example::builder(\u0026\"imgs/2.jpg\").with_guide(\u0026\"imgs/masks/2_example.jpg\"))\n        // load target \"heart\" shape that we would like the generated image to look like\n        // now the generator will take our target guide into account during synthesis\n        .load_target_guide(\u0026\"imgs/masks/2_target.jpg\")\n        .build()?;\n\n    let generated = texsynth.run(None);\n\n    // save the image to the disk\n    generated.save(\"out/03.jpg\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- -o out/03.png generate --target-guide imgs/masks/2_target.jpg --guides imgs/masks/2_example.jpg -- imgs/2.jpg`\n\n**NOTE:** Note the use of `--` to delimit the path to the example `imgs/2.jpg`, if you don't specify `--`, the path\nto the example will be used as another guide path and there won't be any examples.\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/arTCi2f.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 4. Style Transfer\n\n![Imgur](https://i.imgur.com/o9UxFGO.jpg)\n\nTexture synthesis API supports auto-generation of example guide maps, which produces a style transfer-like effect.\n\n#### API - [04_style_transfer](lib/examples/04_style_transfer.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    let texsynth = ts::Session::builder()\n        // load example which will serve as our style, note you can have more than 1!\n        .add_examples(\u0026[\u0026\"imgs/multiexample/4.jpg\"])\n        // load target which will be the content\n        // with style transfer, we do not need to provide example guides\n        // they will be auto-generated if none were provided\n        .load_target_guide(\u0026\"imgs/tom.jpg\")\n        .guide_alpha(0.8)\n        .build()?;\n\n    // generate an image that applies 'style' to \"tom.jpg\"\n    let generated = texsynth.run(None);\n\n    // save the result to the disk\n    generated.save(\"out/04.jpg\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --alpha 0.8 -o out/04.png transfer-style --style imgs/multiexample/4.jpg --guide imgs/tom.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/1E7eDAb.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 5. Inpaint\n\n![Imgur](https://i.imgur.com/FqvV651.jpg)\n\nWe can also fill-in missing information with inpaint. By changing the seed, we will get different version of the 'fillment'.\n\n#### API - [05_inpaint](lib/examples/05_inpaint.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    let texsynth = ts::Session::builder()\n        // let the generator know which part we would like to fill in\n        // if we had more examples, they would be additional information\n        // the generator could use to inpaint\n        .inpaint_example(\n            \u0026\"imgs/masks/3_inpaint.jpg\",\n            // load a \"corrupted\" example with missing red information we would like to fill in\n            ts::Example::builder(\u0026\"imgs/3.jpg\")\n                // we would also like to prevent sampling from \"corrupted\" red areas\n                // otherwise, generator will treat that those as valid areas it can copy from in the example,\n                // we could also use SampleMethod::Ignore to ignore the example altogether, but we\n                // would then need at least 1 other example image to actually source from\n                // example.set_sample_method(ts::SampleMethod::Ignore);\n                .set_sample_method(\u0026\"imgs/masks/3_inpaint.jpg\"),\n            // Inpaint requires that inputs and outputs be the same size, so it's a required\n            // parameter that overrides both `resize_input` and `output_size`\n            ts::Dims::square(400),\n        )\n        // Ignored\n        .resize_input(ts::Dims::square(200))\n        // Ignored\n        .output_size(ts::Dims::square(100))\n        .build()?;\n\n    let generated = texsynth.run(None);\n\n    //save the result to the disk\n    generated.save(\"out/05.jpg\")\n}\n```\n\n#### CLI\n\nNote that the `--out-size` parameter determines the size for all inputs and outputs when using inpaint!\n\n`cargo run --release -- --out-size 400 --inpaint imgs/masks/3_inpaint.jpg -o out/05.png generate imgs/3.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/WZm2HHL.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 6. Inpaint Channel\n\n![bricks](imgs/bricks.png)\n\nInstead of using a separate image for our inpaint mask, we can instead obtain the information from a specific\nchannel. In this example, the alpha channel is a circle directly in the middle of the image.\n\n#### API - [06_inpaint_channel](lib/examples/06_inpaint_channel.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    let texsynth = ts::Session::builder()\n        // Let the generator know that it is using \n        .inpaint_example_channel(\n            ts::ChannelMask::A,\n            \u0026\"imgs/bricks.png\",\n            ts::Dims::square(400),\n        )\n        .build()?;\n\n    let generated = texsynth.run(None);\n\n    //save the result to the disk\n    generated.save(\"out/06.jpg\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --inpaint-channel a -o out/06.png generate imgs/bricks.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://imgur.com/7IuVN5K.jpg\" width=\"350\" height=\"350\"\u003e\n\n### 7. Tiling texture\n\n![](https://i.imgur.com/nFpCFzy.jpg)\n\nWe can make the generated image tile (meaning it will not have seams if you put multiple images together side-by-side). By invoking inpaint mode together with tiling, we can make an existing image tile.\n\n#### API - [07_tiling_texture](lib/examples/07_tiling_texture.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    // Let's start layering some of the \"verbs\" of texture synthesis\n    // if we just run tiling_mode(true) we will generate a completely new image from scratch (try it!)\n    // but what if we want to tile an existing image?\n    // we can use inpaint!\n\n    let texsynth = ts::Session::builder()\n        // load a mask that specifies borders of the image we can modify to make it tiling\n        .inpaint_example(\n            \u0026\"imgs/masks/1_tile.jpg\",\n            ts::Example::new(\u0026\"imgs/1.jpg\"),\n            ts::Dims::square(400),\n        )\n        //turn on tiling mode!\n        .tiling_mode(true)\n        .build()?;\n\n    let generated = texsynth.run(None);\n\n    generated.save(\"out/07.jpg\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --inpaint imgs/masks/1_tile.jpg --out-size 400 --tiling -o out/07.bmp generate imgs/1.jpg`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://i.imgur.com/foSlREz.jpg\" width=\"600\" height=\"364\"\u003e\n\n### 8. Repeat texture synthesis transform on a new image\n\n![](https://i.imgur.com/WEf6iir.jpg)\n\nWe can re-apply the coordinate transformation performed by texture synthesis onto a new image.\n\n#### API - [08_repeat_transform](lib/examples/08_repeat_transform.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    // create a new session\n    let texsynth = ts::Session::builder()\n        //load a single example image\n        .add_example(\u0026\"imgs/1.jpg\")\n        .build()?;\n\n    // generate an image\n    let generated = texsynth.run(None);\n\n    // now we can apply the same transformation of the generated image\n    // onto a new image (which can be used to ensure 1-1 mapping between multiple images)\n    // NOTE: it is important to provide same number of input images as the \n    // otherwise, there will be coordinates mismatch\n    let repeat_transform_img = generated\n        .get_coordinate_transform()\n        .apply(\u0026[\"imgs/1_bw.jpg\"])?;\n\n    // save the image to the disk\n    // 08 and 08_repeated images should match perfectly\n    repeat_transform_img.save(\"out/08_repeated.jpg\").unwrap();\n    generated.save(\"out/08.jpg\")\n}\n```\n\n#### CLI\n\n1. First, we need to create a transform that can be reused\n\nThe notable bit here is the `--save-transform out/multi.xform` which creates the\nfile that can be used to generate new outputs with.\n\n`cargo run --release -- --rand-init 10 --seed 211 --in-size 300x300 -o\nout/02.png generate --save-transform out/multi.xform imgs/multiexample/1.jpg imgs/multiexample/2.jpg imgs/multiexample/3.jpg\nimgs/multiexample/4.jpg`\n\n2. Next, we use the `repeat` subcommand to repeat transform with different\ninputs\n\nThe important bits here are the use of the `repeat` subcommand instead of\n`generate`, and `--transform out/multi.xform` which tells what transform to\napply to the inputs. The only restriction is that the number of images you\nspecify must match the original number of examples **exactly**. If the input\nimages have different dimensions than the example images, they will be\nautomatically resized for you.\n\n`cargo run --release -- -o out/02-repeated.png repeat --transform\nout/multi.xform imgs/multiexample/1.jpg imgs/multiexample/2.jpg\nimgs/multiexample/4.jpg imgs/multiexample/3.jpg`\n\nAlso note that the normal parameters that are used with `generate` don't apply\nto the `repeat` subcommand and will be ignored.\n\n### 9. Sample masks\n\nSample masks allow you to specify how an example image is sampled during generation.\n\n#### API - [09_sample_masks](lib/examples/09_sample_masks.rs)\n\n```rust\nuse texture_synthesis as ts;\n\nfn main() -\u003e Result\u003c(), ts::Error\u003e {\n    let session = ts::Session::builder()\n        .add_example(\n            ts::Example::builder(\u0026\"imgs/4.png\").set_sample_method(ts::SampleMethod::Ignore),\n        )\n        .add_example(ts::Example::builder(\u0026\"imgs/5.png\").set_sample_method(ts::SampleMethod::All))\n        .seed(211)\n        .output_size(ts::Dims::square(200))\n        .build()?;\n\n    // generate an image\n    let generated = session.run(None);\n\n    // save the image to the disk\n    generated.save(\"out/09.png\")\n}\n```\n\n#### CLI\n\n`cargo run --release -- --seed 211 --out-size 200 --sample-masks IGNORE ALL --out 09_sample_masks.png generate imgs/4.png imgs/5.png`\n\nYou should get the following result with the images provided in this repo:\n\n\u003cimg src=\"https://imgur.com/EqW3rkN.png\" width=\"200\" height=\"200\"\u003e\n\n### 10. Combining texture synthesis 'verbs'\n\nWe can also combine multiple modes together. For example, multi-example guided synthesis:\n\n![](https://i.imgur.com/By64UXG.jpg)\n\nOr chaining multiple stages of generation together:\n\n![](https://i.imgur.com/FzZW3sl.jpg)\n\nFor more use cases and examples, please refer to the presentation [\"More Like This, Please! Texture Synthesis and Remixing from a Single Example\"](https://youtu.be/fMbK7PYQux4)\n\n### Additional CLI functionality\n\nSome functionality is only exposed through the CLI and not built into the library.\n\n#### `flip-and-rotate`\n\nThis subcommand takes each example and performs flip and rotation transformations to it to generate additional example inputs for generation. This subcommand doesn't support target or example guides.\n\nExample: `cargo run --release -- -o out/output.png flip-and-rotate imgs/1.jpg`\n\n## Command line binary\n\n* [Download the binary](https://github.com/EmbarkStudios/texture-synthesis/releases) for your OS.\n* **Or** Install it from source.\n  * [Install Rust](https://www.rust-lang.org/tools/install) - The minimum required version is `1.37.0`\n  * [Clone this repo](https://help.github.com/en/articles/cloning-a-repository)\n  * In a terminal `cd` to the directory you cloned this repository into\n  * Run `cargo install --path=cli`\n  * **Or** if you wish to see the texture as it is being synthesized `cargo install --path=cli --features=\"progress\"`\n* Open a terminal\n* Navigate to the directory where you downloaded the binary, if you didn't just `cargo install` it\n* Run `texture_synthesis --help` to get a list of all of the options and commands you can run\n* Refer to the examples section in this readme for examples of running the binary\n\n## Notes\n\n* By default, generating output will use all of your logical cores\n* When using multiple threads for generation, the output image is not guaranteed to be deterministic with the same inputs. To have 100% determinism, you must use a thread count of one, which can by done via\n  * CLI - `texture-synthesis --threads 1`\n  * API - `SessionBuilder::max_thread_count(1)`\n\n## Limitations\n\n* Struggles with complex semantics beyond pixel color (unless you guide it)\n* Not great with regular textures (seams can become obvious)\n* Cannot infer new information from existing information (only operates on what’s already there)\n* Designed for single exemplars or very small datasets (unlike Deep Learning based approaches)\n\n## Additional Dependencies\n\nIf you're compiling for Linux, you'll need to have `libxkbcommon` development libraries installed. For ubuntu this is `libxkbcommon-x11-dev`.\n\n## Links/references\n\n[1] [Opara \u0026 Stachowiak] [\"More Like This, Please! Texture Synthesis and Remixing from a Single Example\"](https://youtu.be/fMbK7PYQux4)\n\n[2] [Harrison] Image Texture Tools\n\n[3] [Ashikhmin] Synthesizing Natural Textures\n\n[4] [Efros \u0026 Leung] Texture Synthesis by Non-parametric Sampling\n\n[5] [Wey \u0026 Levoy] Fast Texture Synthesis using Tree-structured Vector Quantization\n\n[6] [De Bonet] Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images\n\n[7] All the test images in this repo are from [Unsplash](https://unsplash.com/)\n\n## Contributing\n\n[![Contributor Covenant](https://img.shields.io/badge/contributor%20covenant-v1.4-ff69b4.svg)](../CODE_OF_CONDUCT.md)\n\nWe welcome community contributions to this project.\n\nPlease read our [Contributor Guide](CONTRIBUTING.md) for more information on how to get started.\n\n## License\n\nLicensed under either of\n\n* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e)\n* MIT license ([LICENSE-MIT](LICENSE-MIT) or \u003chttp://opensource.org/licenses/MIT\u003e)\n\nat your option.\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEmbarkStudios%2Ftexture-synthesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEmbarkStudios%2Ftexture-synthesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEmbarkStudios%2Ftexture-synthesis/lists"}