https://github.com/wireapp/core-crypto
MLS/Proteus multiplexer abstraction with encrypted persistent storage in Rust
https://github.com/wireapp/core-crypto
Last synced: 3 months ago
JSON representation
MLS/Proteus multiplexer abstraction with encrypted persistent storage in Rust
- Host: GitHub
- URL: https://github.com/wireapp/core-crypto
- Owner: wireapp
- License: gpl-3.0
- Created: 2021-12-17T16:46:47.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2026-02-16T17:29:09.000Z (4 months ago)
- Last Synced: 2026-02-16T22:33:53.851Z (4 months ago)
- Language: Rust
- Homepage: https://wireapp.github.io/core-crypto/
- Size: 197 MB
- Stars: 42
- Watchers: 14
- Forks: 10
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
[](https://github.com/wireapp/core-crypto/actions/workflows/scheduled.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/pipeline.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/interop.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/test-android.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/test-ios.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/test-jvm.yml)
[](https://github.com/wireapp/core-crypto/actions/workflows/test-ts.yml)
This repository is part of the source code of Wire. You can find more information at [wire.com](https://wire.com) or by
contacting opensource@wire.com.
You can find the published source code at [github.com/wireapp/wire](https://github.com/wireapp/wire).
For licensing information, see the attached LICENSE file and the list of third-party licenses at
[wire.com/legal/licenses/](https://wire.com/legal/licenses/).
No license is granted to the Wire trademark and its associated logos, all of which will continue to be owned exclusively
by Wire Swiss GmbH. Any use of the Wire trademark and/or its associated logos is expressly prohibited without the
express prior written consent of Wire Swiss GmbH.
- [Wire CoreCrypto](#wire-corecrypto)
- [Documentation](#documentation)
- [Building](#building)
- [General Requirements](#general-requirements)
- [Pre-commit](#pre-commit)
- [JVM](#jvm)
- [Android](#android)
- [iOS](#ios)
- [MacOS](#macos)
- [Linux](#linux)
- [WASM](#wasm)
- [Bindings](#bindings)
- [Testing](#testing)
- [Rust Unit/Integration Tests](#rust-unitintegration-tests)
- [All Ciphersuites](#all-ciphersuites)
- [Keystore Wasm tests](#keystore-wasm-tests)
- [Platform-specific tests for WASM/Web](#platform-specific-tests-for-wasmweb)
- [Platform-specific tests for Kotlin/JVM](#platform-specific-tests-for-kotlinjvm)
- [Platform-specific tests for Android](#platform-specific-tests-for-android)
- [Swift/iOS](#swiftios)
- [Interop](#interop)
- [End-to-end-identity (E2EI) testing](#end-to-end-identity-e2ei-testing)
- [Preparing the container runtime environment](#preparing-the-container-runtime-environment)
- [On Linux with Docker](#on-linux-with-docker)
- [On Linux with Podman](#on-linux-with-podman)
- [On macOS with Docker](#on-macos-with-docker)
- [On macOS with Podman](#on-macos-with-podman)
- [Choosing the OIDC identity provider](#choosing-the-oidc-identity-provider)
- [Running all tests at once](#running-all-tests-at-once)
- [Running specific tests](#running-specific-tests)
- [Manually invoking tests](#manually-invoking-tests)
- [Formatting and Linting](#formatting-and-linting)
- [Requirements](#requirements)
- [Swift](#swift)
- [Kotlin](#kotlin)
- [Markdown](#markdown)
- [Toml](#toml)
- [Benchmarks](#benchmarks)
- [Executing Benches](#executing-benches)
- [Git workflow](#git-workflow)
- [Publishing](#publishing)
- [Versioning](#versioning)
- [Making a new release](#making-a-new-release)
- [Consider when making a release from a release branch](#consider-when-making-a-release-from-a-release-branch)
- [Docs home](https://wireapp.github.io/core-crypto/)
- [Architecture](https://wireapp.github.io/core-crypto/ARCHITECTURE.html)
- [Publish Manually](https://wireapp.github.io/core-crypto/MANUAL_PUBLISH.html)
- rust:
- GNU make: (min version: 4.3)
- nextest: : `cargo install --locked cargo-nextest`
- Ensure your git is configured to
[sign commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
- Install the [`pre-commit` framework](https://pre-commit.com/)
- Run `pre-commit install` to initialize the pre-commit hooks
- Install [SDKMAN!](https://sdkman.io/): `curl -s "https://get.sdkman.io" | bash`
- Install Java 17: `sdk install java 17.0.17-tem`
- Install Kotlin: `sdk install kotlin`
```sh
make jvm # make the JVM target
make jvm-test # make and test the JVM target
```
[Install Android SDK](https://developer.android.com/studio) and Build-Tools for API level 30+
> [!IMPORTANT]
> If you are building on macOS you'll need to setup `$ANDROID_SDK_ROOT` path variable manually:
>
> ```sh
> export ANDROID_SDK_ROOT=~/Android/Sdk
> ```
[Install the Android NDK](https://developer.android.com/studio/projects/install-ndk). Make sure to set the
`ANDROID_NDK_HOME` variable to point to the NDK installation.
Install android rust targets:
```sh
rustup target add x86_64-linux-android aarch64-linux-android armv7-linux-androideabi
```
Build:
```sh
make android
```
Install Xcode & its command-line tools: [https://developer.apple.com/xcode/](https://developer.apple.com/xcode/).
Install iOS rust targets:
```sh
rustup target add aarch64-apple-ios aarch64-apple-ios-sim
```
Build:
```sh
make ios
# Additionally, if you want to export a .XCFramework:
make ios-create-xcframework
```
Install macOS rust targets:
```sh
rustup target add aarch64-apple-darwin
```
> [!NOTE]
> If cross-compiling from macOS, you'll need to install
> [https://github.com/messense/homebrew-macos-cross-toolchains](https://github.com/messense/homebrew-macos-cross-toolchains).
Install Linux targets:
```sh
rustup target add x86_64-unknown-linux-gnu
```
Make sure you have all prerequisites:
- Install the `wasm32-unknown-unknown` toolchain: `rustup target add wasm32-unknown-unknown`
- Install node.js (recommended way is via [Volta](https://volta.sh/))
- Install Bun (follow the instructions on [Bun's website](https://bun.sh/))
- Install [wasm-bindgen-cli](https://github.com/wasm-bindgen/wasm-bindgen):
```sh
wasm_bindgen_version="$(
cargo metadata --format-version 1 |
jq -r '.packages[] | select (.name == "wasm-bindgen") | .version'
)"
cargo install wasm-bindgen-cli --version $wasm_bindgen_version
```
It is important to ensure that the `wasm-bindgen-cli` version always precisely matches the `wasm-bindgen` version in
`Cargo.lock` (as shown by `cargo info wasm-bindgen`), because otherwise the wasm tests will not run.
- Install [chromedriver](https://getwebdriver.com/chromedriver)
```sh
bunx @puppeteer/browsers install --path ~/bin chrome-headless-shell
bunx @puppeteer/browsers install --path ~/bin chromedriver
```
Build:
```sh
make ts # make the typescript target
make ts-test # make the typescript target and run tests
```
Build bindings for Android, JVM, iOS and WASM
```sh
# builds bindings and targets for the JVM (macOS / Linux)
make jvm
# builds bindings and targets for Android
make android
# builds iOS framework
make ios-create-xcframework
# builds wasm binary & TS bindings
make ts
```
### Rust Unit/Integration Tests
```sh
cargo nextest run
```
> [!WARNING]
> This takes quite a while.
```sh
cargo nextest run --features test-all-cipher
```
Sometimes for the Keystore it is valuable to run Rust unit/integration tests on the WASM target.
- Ensure you are set up to [build wasm](#wasm)
Then, just use `cargo test`:
```sh
cargo test --target wasm32-unknown-unknown -p core-crypto-keystore
```
Unfortunately it appears that nextest doesn't work well with the wasm runner, so we're stuck with the basic test runner.
### Platform-specific tests for WASM/Web
```sh
make ts-test
```
Note the `CC_TEST_LOG_LEVEL` environment variable. At 1 it emits browser console logs; at 2 it also emits CoreCrypto
logs.
### Platform-specific tests for Kotlin/JVM
```sh
make jvm-test
```
### Platform-specific tests for Android
```sh
make android-test
```
Currently works on macOS only.
```sh
make ios-test
```
1. Make sure you've successfully run each platform-specific test suite that is supported on your machine, since the
setup required for this test is a superset of the requirements for each platform.
1. Install Chrome and `chromedriver`
1. `make interop-test`
### End-to-end-identity (E2EI) testing
#### Preparing the container runtime environment
Some tests require a working container runtime, so make sure to prepare one before running all tests. Platform-specific
instructions follow below.
Make sure to start the Docker service if it is not already running:
```sh
systemctl start docker.service
```
```sh
# start socket activation, which will cause Podman to start once
# anything connects to the socket:
systemctl --user start podman.socket
# check that socket activation works
podman version
# if the above didn't work, depending on the distribution and installed packages,
# it may be necessary to configure the DOCKER_HOST variable to point to Podman's socket
export DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock
```
Note: Docker under macOS requires Docker Desktop, which must run as a GUI application.
```sh
# install docker and docker-desktop
brew install docker docker-desktop
# start the Docker daemon by launching docker-desktop as a GUI application
# check if everything went fine
docker version
```
```sh
# install Podman
brew install podman
# install podman-mac-helper
sudo podman-mac-helper install
# create new VM based on machine-os:5.5; note that we're explicitly specifying
# an older image version because the newest one seems to be broken
podman machine init --image docker://quay.io/podman/machine-os:5.5
# start the machine
podman machine start
# if everything went well, this should print server version `5.5.x`
podman version
# symlink docker to podman (test scripts and code assume existence
# of the `docker` command)
ln -s /opt/homebrew/bin/podman /opt/homebrew/bin/docker
```
#### Choosing the OIDC identity provider
Choose the OIDC identity provider to use in tests by setting the `TEST_IDP` variable:
```sh
# use Keycloak
export TEST_IDP=keycloak
# or Authelia
export TEST_IDP=authelia
```
#### Running all tests at once
Simply execute the `run-e2ei-tests.sh` script:
```sh
bash scripts/run-e2ei-tests.sh
```
The script will take care of cleaning up processes and containers that are started during tests.
`run-e2ei-tests.sh` forwards its arguments to `cargo nextest`, which can be used to run a specific test, or any subset
of tests, e.g.
```sh
bash scripts/run-e2ei-tests.sh alg::p256
```
First, you need to start `test-wire-server`:
```sh
$ cargo run --locked --bin test-wire-server
[...]
127.0.0.1:20530
```
Note the IP and port printed by `test-wire-server` and export that as `TEST_WIRE_SERVER_ADDR`:
```sh
export TEST_WIRE_SERVER_ADDR=127.0.0.1:20530
```
Now that the environment is ready, you can run a specific test, or any subset of tests, e.g.
```sh
cargo nextest run --locked --ignore-default-filter -p wire-e2e-identity alg::p256
```
Once you are done with testing, terminate the IdP container that has been started:
```sh
# if you're using Keycloak
docker kill keycloak && docker rm keycloak
# if you're using Authelia
docker kill authelia.local && docker rm authelia.local
```
as well as the `test-wire-server` instance.
For all languages we provide make rules for formatting and checking.
- `make fmt`
- `make rust-fmt`
- `make swift-fmt`
- `make kotlin-fmt`
- `make ts-fmt`
- `make check`
- `make rust-check`
- `make swift-check`
- `make kotlin-check`
- `make ts-check`
We're using [swift-format](https://github.com/swiftlang/swift-format) to format swift files and use
[swiftlint](https://github.com/realm/SwiftLint) for linting.
We're using [ktlint](https://github.com/pinterest/ktlint) to format and lint kotlin files.
We're using `mdformat` for consistent formatting of our markdown files. Install it with the following extensions
- `mdformat-gfm`
- `mdformat-frontmatter`
- `mdformat-footnote`
- `mdformat-gfm-alerts`
- `mdformat-toc`
We're using [taplo](https://taplo.tamasfe.dev/) to format .toml files.
There are benches implemented in [`crypto/benches`](crypto/benches/) for several operations on mls groups with varying
sizes or proteus. Parameters like minimum or maximum group sizes and step sizes are defined in
[`crypto/benches/utils/mod.rs`](crypto/benches/utils/mod.rs).
To execute the benches, e.g. for creating commits, run
```bash
cargo bench --bench=commit -- --quick
```
where `commit` is the name of the bench specified in [`crypto/Cargo.toml`](crypto/Cargo.toml), and the corresponding
file in [`crypto/benches`](crypto/benches/). In case you're interested in higher accuracy, and willing to trade it for
execution speed, omit the `--quick` flag. If you need reporting plots, remove the `.without_plots()` call in
[`crypto/benches/utils/mod.rs`](crypto/benches/utils/mod.rs). The reports generated by criterion will be located in
`target/criterion`.
- The `main` branch is used as the everyday development branch.
- No merge commits. Always rebase on top of `main`.
- Release branches are named `release/`, e.g. `release/1.x`, `release/2.x`.
- Release branches contain fixes relevant to their specific release series and are never merged to `main`.
- Release branches always branch off their first major release tag. For example, the output of
`git merge-base main release/2.x` must be a commit pointed to by tag `v2.0.0`.
- Release branches are created lazily, that is, only when the first fix needs to be applied and released for a specific
release series.
- Use [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) -- those are picked up by the changelog
generator.
- If there is a JIRA ticket related to the change, you should mention it in either the PR title or the commit(s), with
the following format: `[TICKET_ID]`.
- Sign your [commits](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
and [tags](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-tags).
- Remove branches from the remote once you don't need them anymore.
The versioning scheme used is [SemVer AKA Semantic Versioning](https://semver.org).
1. Make a branch based on `main` to prepare for release (`git checkout -b prepare-release/X.Y.Z`)
1. Run `sh scripts/update-versions.sh X.Y.Z` to update the versions of
- all workspace member crates
- `package.json`
- `crypto-ffi/bindings/gradle.properties` Make sure the result of the script run is correct.
1. Generate a fresh `e2e-identity/README.md.test`:
```bash
TEST_IDP=authelia bash scripts/run-e2ei-tests.sh demo_should_succeed
```
If there are non-trivial differences between `e2e-identity/README.md` and the generated file, update
`e2e-identity/README.md` and commit the changes.
1. Generate the relevant changelog section:
```bash
git cliff --bump --unreleased
```
and add it to the top of `CHANGELOG.md`. Make sure the version number generated by `git cliff` matches the release
version.
1. If there are any release highlights, add them as the first subsection below release title:
```markdown
## v1.0.2 - 2024-08-16
### Highlights
- foo
- bar
- baz
```
1. In [index.md](docs/index.md), copy the commented-out table row from the bottom of the file to the appropriate place
in the table, ordering by version number, descending. Search and replace the first 5 occurrences of `x.x.x` with
`X.Y.Z`.
1. Make sure the changes look reasonable and complete; you can use the previous release as a reference
1. Push your `prepare-release/X.Y.Z` branch and create a PR for it
1. Get it reviewed, then merge it into `main` and remove the `prepare-release/X.Y.Z` branch from the remote
1. Now, pull your local `main`: `git checkout main && git pull`
1. Create the release tag: `git tag -s vX.Y.Z`
1. Push the new tag: `git push origin tag vX.Y.Z`
1. Create a new release on github, copying the relevant section from `CHANGELOG.md`
1. Voilà !
#### Consider when making a release from a release branch
1. Isolate the changes to [index.md](docs/index.md) and `CHANGELOG.md` from the release commit itself
1. After the release is finished, cherry-pick the changes to [index.md](docs/index.md) and `CHANGELOG.md` and get them
into `main`
1. For release series `4.x` and newer, docs upload happens automatically. If you released from the series `3.x` or
older, you need to trigger docs upload manually:
1. On GitHub, go to the [docs workflow](https://github.com/wireapp/core-crypto/actions/workflows/docs.yml)
1. Click the `Run workflow` button
1. In the `Use workflow from` dropdown, choose `release/5.x`, in `Tag to checkout` provide your release tag