Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Jake-Shadle/xwin

A utility for downloading and packaging the Microsoft CRT & Windows SDK headers and libraries needed for compiling and linking programs targeting Windows.
https://github.com/Jake-Shadle/xwin

cross-compilation msvc msvcrt rust windows windows-sdk

Last synced: 3 months ago
JSON representation

A utility for downloading and packaging the Microsoft CRT & Windows SDK headers and libraries needed for compiling and linking programs targeting Windows.

Awesome Lists containing this project

README 

        






# `xwin`



**A utility for downloading and packaging the [Microsoft CRT](https://docs.microsoft.com/en-us/cpp/c-runtime-library/crt-library-features?redirectedfrom=MSDN&view=msvc-160) headers and libraries, and [Windows SDK](https://en.wikipedia.org/wiki/Microsoft_Windows_SDK) headers and libraries needed for compiling and linking programs targeting Windows.**



[![Crates.io](https://img.shields.io/crates/v/xwin.svg)](https://crates.io/crates/xwin)

[![Docs](https://docs.rs/xwin/badge.svg)](https://docs.rs/xwin)

[![dependency status](https://deps.rs/repo/github/Jake-Shadle/xwin/status.svg)](https://deps.rs/repo/github/Jake-Shadle/xwin)

[![Build status](https://github.com/Jake-Shadle/xwin/workflows/CI/badge.svg)](https://github.com/Jake-Shadle/xwin/actions)






## Introduction



The goal of this project is to create a root directory for both the CRT and Windows SDK that each contain all of the necessary includes and libraries needed for an application to compile and link from a non-Windows platform, using a native cross compiling toolchain like clang/LLVM. This includes adding symlinks to correct numerous casing issues in the Windows SDK so that the files generated by this program can function on a case-sensitive file system.



See this [blog post](https://jake-shadle.github.io/xwin/) for an in depth walk-through of how xwin can be used.



## Installation



### From source



`cargo install xwin --locked`



#### Features



`xwin` provides two feature toggles used to decide which TLS implementation to use



* `rustls` (default) - Uses [`rustls`](https://github.com/rustls/rustls) for TLS

* `native-tls` - Uses [`native-tls`](https://github.com/sfackler/rust-native-tls) for TLS. Note that on platforms where OpenSSL is used it is always built from source.



### From tarball



You can download a prebuilt binary from the [Releases](https://github.com/Jake-Shadle/xwin/releases).



* `x86_64-unknown-linux-musl`

* `x86_64-apple-darwin`

* `aarch64-apple-darwin`



## Usage



### Common



* `--accept-license` - Doesn't display the prompt to accept the license. You can also set the `XWIN_ACCEPT_LICENSE=1` environment variable

* `--arch ` - The architectures to include [default: `x86_64`]  [possible values: `x86`, `x86_64`, `aarch`, `aarch64`]. Note that I haven't fully tested aarch/64 nor x86 so there _might_ be issues with them, please file an issue if you encounter problems with them.

* `--cache-dir ` - Specifies the cache directory used to persist downloaded items to disk. Defaults to `./.xwin-cache` if not specified.

* `-L, --log-level ` - The log level for messages, only log messages at or above the level will be emitted [default: info] [possible values: off, error, warn, info, debug, trace].

* `--variant ...` - The variants to include [default: desktop]  [possible values: desktop, onecore, spectre]. Note that I haven't fully tested any variant except `desktop`, please file an issue if you try to use one of the others and run into issues. Note that there is another `store` variant that hasn't even been implemented due to it being weird and me not having a real project targeting it.

* `--channel ` - The product channel to use [default: release]

* `--manifest-version ` - The manifest version to retrieve  [default: 17].

* `--manifest` - Specifies a top level manifest to use, rather than downloading it from Microsoft. This can be used to ensure the output is reproducible.

* `--sdk-version` - The specific SDK version to use. If not specified the latest SDK version in the manifest is used.

* `--crt-version` - The specific CRT version to use. If not specified the latest CRT version in the manifest is used.

* `--timeout` - Specifies a timeout for long a single HTTP get request is allowed to take. The default is 60s.



### Env vars



* `https_proxy` - Environment variable that specifies the HTTPS proxy to use.



### `xwin download`



This downloads the top level manifest and any vsix, msi, or cab files that are needed that aren't already in the download cache.



### `xwin unpack`



Decompresses all of the downloaded package contents to disk. `download` is run automatically.



### `xwin splat`



Fixes the packages to prune unneeded files and adds symlinks to address file casing issues and then spalts the final artifacts into directories. This is the main command you will want to run as it also `download`s and `unpack`s automatically, providing the desired headers at the path specified to `--output` (`./.xwin-cache/splat`).



#### Splat options



* `--copy` - Copies files from the unpack directory to the splat directory instead of moving them, which preserves the original unpack directories but increases overall execution time and disk usage.

* `--disable-symlinks` - By default, symlinks are added to both the CRT and `WindowsSDK` to address casing issues in general usage. For example, if you are compiling C/C++ code that does `#include `, it will break on a case-sensitive file system, as the actual path in the `WindowsSDK` is `Windows.h`. This also applies even if the C/C++ you are compiling uses correct casing for all CRT/SDK includes, as the internal headers also use incorrect casing in most cases

* `--include-debug-libs` - The MSVCRT includes (non-redistributable) debug versions of the various libs that are generally uninteresting to keep for most usage

* `--include-debug-symbols` - The MSVCRT includes PDB (debug symbols) files for several of the libraries that are generally uninteresting to keep for most usage

* `--preserve-ms-arch-notation` - By default, we convert the MS specific `x64`, `arm`, and `arm64` target architectures to the more canonical `x86_64`, `aarch`, and `aarch64` of LLVM etc when creating directories/names. Passing this flag will preserve the MS names for those targets

* `--use-winsysroot-style` - Use the /winsysroot layout, so that clang-cl's /winsysroot flag can be used with the output, rather than needing both -vctoolsdir and -winsdkdir. You will likely also want to use --preserve-ms-arch-notation and --disable-symlinks for use with clang-cl on Windows.

* `--output` - The root output directory. Defaults to `./.xwin-cache/splat` if not specified

* `--map` - An optional [map](#map-file) file used to configure what files are splatted, and any additional symlinks to create.



This moves all of the unpacked files which aren't pruned to their canonical locations under a root directory, for example here is what an `x86_64` `Desktop` splat looks like. `unpack` is run automatically as needed.



```txt

.xwin-cache/splat

├── crt

│  ├── include

│  │  ├── cliext

│  │  ├── CodeAnalysis

│  │  ├── cvt

│  │  ├── experimental

│  │  ├── Manifest

│  │  └── msclr

│  │     └── com

│  └── lib

│     └── x86_64

└── sdk

   ├── include

   │  ├── cppwinrt

   │  │  └── winrt

   │  │     └── impl

   │  ├── shared

   │  │  ├── ndis

   │  │  └── netcx

   │  │     └── shared

   │  │        └── net

   │  │           └── wifi

   │  ├── ucrt

   │  │  └── sys

   │  ├── um

   │  │  ├── alljoyn_c

   │  │  ├── gl

   │  │  ├── qcc

   │  │  │  └── windows

   │  │  └── winsqlite

   │  └── winrt

   │     └── wrl

   │        └── wrappers

   └── lib

      ├── ucrt

      │  └── x86_64

      └── um

         └── x86_64

```



### `xwin minimize`



This is an advanced command that performs a `splat` before performing a build on a cargo manifest using strace to capture all of the headers and libraries that are used throughout the build and dumping them to a [map](#map-file). This command can also output the final splat to disk, or the map file can be used with `splat` to only splat the files and symlinks described in it.



Note that currently the build is always done with the `/vctoolsdir` and `/winsdkdir` options, so it is expected these are the same options used when compiling C/C++ code in your normal environment. If that is not the case please open an issue.



At the end of the command, a printout of the amount and size of the original versus minimized files is done, eg.



```txt

  crt headers: 73(2.6MiB) / 384(18.4MiB) => 14.00%

  crt libs: 5(28.0MiB) / 26(81.1MiB) => 34.58%

  sdk headers: 180(9.6MiB) / 4435(304.7MiB) => 3.15%

  sdk libs: 29(69.8MiB) / 456(169.9MiB) => 41.06%

```



#### Requirements



* Linux host - This _might_ work on other platforms, but it's not guaranteed, nor tested

* `cargo` - This is the singular supported build tool.

* `-pc-windows-msvc` - The target you are building for needs to be installed (eg. via `rustup target add`)

* `clang-cl` - This is used as the C/C++ compiler

* `llvm-lib` - This is used as the archiver

* `lld-link` - This is used as the linker

* `strace` - This is used to capture the syscalls made by the lld and clang compiler



#### Minimize options



Note all of the [splat options](#splat-options) also apply to minimize.



* `--map` - The path to the [map](#map-file) to output the minimized results to. Default to `./.xwin-cache/xwin-map.toml` if not specified.

* `--minimize-output` - The root directory where only the minimized files are splatted to. If not specified only the `--map` file is written in addition to the normal splat

* `--preserve-strace` - By default the `strace` output is written to disk in a temporary location that is deleted once the build is finished, passing this option allows it to be persisted. The path is written out before the build starts.



## Map file



As noted in [minimize](#xwin-minimize), there are many restrictions on it to make my life easier, but that make it unsuitable for those who don't use cargo/rust. It's possible for others to come up with their own versions of minimize that can output the same format that `splat` understands to still get the benefits of `xwin` without cargo/rust.



The format is extremely simple



```txt

├── crt

│  ├── headers

│  │  ├── filter - Array of relative paths to keep

│  │  └── symlinks

│  │     └──  - The same path as one of the filters

│  │        └──  - Array of symlinks to create in the same directory as the parent path

│  ├── libs *

└── sdk *

```



### Example



See [docs/example-map.toml](docs/example-map.toml) for a real world example.



## Container



[xwin.dockerfile](xwin.dockerfile) is an example Dockerfile that can be used a container image capable of building and testing Rust crates targeting `x86_64-pc-windows-msvc`.



### Thanks



Special thanks to  for the inspiration and [@mdsteele](https://github.com/mdsteele) for publishing several Rust crates around msi/cab files that were needed in this project



### License



This contribution is dual licensed under EITHER OF



* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or )

* MIT license ([LICENSE-MIT](LICENSE-MIT) or )



at your option.