{"id":13599812,"url":"https://github.com/rusticstuff/simdutf8","last_synced_at":"2025-12-12T16:51:23.403Z","repository":{"id":41164373,"uuid":"359921939","full_name":"rusticstuff/simdutf8","owner":"rusticstuff","description":"SIMD-accelerated UTF-8 validation for Rust.","archived":false,"fork":false,"pushed_at":"2025-11-22T09:42:12.000Z","size":3160,"stargazers_count":570,"open_issues_count":15,"forks_count":31,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-11-22T10:26:18.912Z","etag":null,"topics":["aarch64","arm64","avx2","neon","rust","rust-crate","simd","simd-extensions","sse41","unicode","utf-8","wasm"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rusticstuff.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-Apache","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-04-20T18:58:50.000Z","updated_at":"2025-11-22T09:42:17.000Z","dependencies_parsed_at":"2025-12-07T11:06:59.405Z","dependency_job_id":null,"html_url":"https://github.com/rusticstuff/simdutf8","commit_stats":{"total_commits":747,"total_committers":11,"mean_commits":67.9090909090909,"dds":"0.029451137884872858","last_synced_commit":"1b7edda61c30bb3f1bb84ff1dc5442a6776d4a30"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/rusticstuff/simdutf8","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticstuff%2Fsimdutf8","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticstuff%2Fsimdutf8/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticstuff%2Fsimdutf8/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticstuff%2Fsimdutf8/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rusticstuff","download_url":"https://codeload.github.com/rusticstuff/simdutf8/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticstuff%2Fsimdutf8/sbom","scorecard":{"id":790740,"data":{"date":"2025-08-11","repo":{"name":"github.com/rusticstuff/simdutf8","commit":"bffc12f73e520f835a38bdd95fbbb3b6f896d9f1"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.1,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 1/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/cross.yml:1","Warn: no topLevel permission defined: .github/workflows/wasm.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:38: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:39: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:81: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:82: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:121: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:122: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:138: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:164: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:165: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:174: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:175: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:189: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:190: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:205: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:206: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cross.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cross.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cross.yml:50: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cross.yml:52: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cross.yml:74: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cross.yml:76: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/cross.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/wasm.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/wasm.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/wasm.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/wasm.yml:48: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/wasm.yml:49: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/wasm.yml:72: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/wasm.yml:73: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/wasm.yml:90: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/wasm.yml:91: update your workflow using https://app.stepsecurity.io/secureworkflow/rusticstuff/simdutf8/wasm.yml/main?enable=pin","Info:   0 out of  14 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of  16 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":10,"reason":"project is fuzzed","details":["Info: RustCargoFuzzer integration found: fuzzing/fuzz/targets/simd.rs:3"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE-Apache:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 9 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"22 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: RUSTSEC-2021-0139","Warn: Project is vulnerable to: RUSTSEC-2021-0145 / GHSA-g98v-hv3f-hcfr","Warn: Project is vulnerable to: RUSTSEC-2024-0375","Warn: Project is vulnerable to: RUSTSEC-2022-0095 / GHSA-jqwc-c49r-4w2x","Warn: Project is vulnerable to: RUSTSEC-2022-0096 / RUSTSEC-2022-0101 / GHSA-7f6x-jwh5-m9r4 / GHSA-88xq-w8cq-xfg7","Warn: Project is vulnerable to: GHSA-wwq9-3cpr-mm53","Warn: Project is vulnerable to: RUSTSEC-2024-0402","Warn: Project is vulnerable to: RUSTSEC-2020-0168","Warn: Project is vulnerable to: RUSTSEC-2024-0436","Warn: Project is vulnerable to: RUSTSEC-2024-0370","Warn: Project is vulnerable to: RUSTSEC-2024-0006 / GHSA-r7qv-8r2h-pg27","Warn: Project is vulnerable to: GHSA-55f3-3qvg-8pv5","Warn: Project is vulnerable to: RUSTSEC-2022-0099 / GHSA-gwc9-348x-qwv2","Warn: Project is vulnerable to: RUSTSEC-2022-0100","Warn: Project is vulnerable to: RUSTSEC-2022-0076 / RUSTSEC-2022-0102 / GHSA-44mr-8vmm-wjhg","Warn: Project is vulnerable to: RUSTSEC-2022-0097 / GHSA-h84q-m8rr-3v9q","Warn: Project is vulnerable to: RUSTSEC-2022-0075 / RUSTSEC-2022-0098 / GHSA-wh6w-3828-g9qf","Warn: Project is vulnerable to: RUSTSEC-2023-0090","Warn: Project is vulnerable to: RUSTSEC-2023-0093","Warn: Project is vulnerable to: RUSTSEC-2023-0092 / GHSA-ch89-5g45-qwc7","Warn: Project is vulnerable to: RUSTSEC-2023-0091","Warn: Project is vulnerable to: RUSTSEC-2024-0438 / GHSA-c2f5-jxjv-2hh8"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-23T07:27:43.781Z","repository_id":41164373,"created_at":"2025-08-23T07:27:43.782Z","updated_at":"2025-08-23T07:27:43.782Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27686527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-12T02:00:06.775Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aarch64","arm64","avx2","neon","rust","rust-crate","simd","simd-extensions","sse41","unicode","utf-8","wasm"],"created_at":"2024-08-01T17:01:12.297Z","updated_at":"2025-12-12T16:51:23.395Z","avatar_url":"https://github.com/rusticstuff.png","language":"Rust","readme":"[![CI](https://github.com/rusticstuff/simdutf8/actions/workflows/ci.yml/badge.svg)](https://github.com/rusticstuff/simdutf8/actions/workflows/ci.yml)\n[![crates.io](https://img.shields.io/crates/v/simdutf8.svg)](https://crates.io/crates/simdutf8)\n[![docs.rs](https://docs.rs/simdutf8/badge.svg)](https://docs.rs/simdutf8)\n\n# simdutf8 – High-speed UTF-8 validation\n\nBlazingly fast API-compatible UTF-8 validation for Rust using SIMD extensions, based on the implementation from\n[simdjson](https://github.com/simdjson/simdjson). Originally ported to Rust by the developers of [simd-json.rs](https://simd-json.rs), but now heavily improved.\n\n## Status\nThis library has been thoroughly tested with sample data as well as fuzzing and there are no known bugs.\n\n## Features\n* `basic` API for the fastest validation, optimized for valid UTF-8\n* `compat` API as a fully compatible replacement for `std::str::from_utf8()`\n* 🆕 AVX 512 support on modern x86/x86-64 CPUs since Rust 1.89\n* Supports AVX 2 and SSE 4.2 implementations on x86 and x86-64\n* ARM64 (aarch64) SIMD is supported since Rust 1.61\n* WASM (wasm32) SIMD is supported\n* 🆕 armv7 NEON support with the `armv7_neon` feature on nightly Rust\n* x86-64: Up to 23 times faster than the std library on valid non-ASCII, up to four times faster on ASCII\n* aarch64: Up to eleven times faster than the std library on valid non-ASCII, up to four times faster on ASCII (Apple Silicon)\n* Faster than the original simdjson implementation\n* Selects the fastest implementation at runtime based on CPU support (on x86)\n* Falls back to the excellent std implementation if SIMD extensions are not supported\n* Written in pure Rust\n* No-std support\n\n## Quick start\nAdd the dependency to your Cargo.toml file:\n```toml\n[dependencies]\nsimdutf8 = \"0.1.5\"\n```\n\nUse `simdutf8::basic::from_utf8()` as a drop-in replacement for `std::str::from_utf8()`.\n\n```rust\nuse simdutf8::basic::from_utf8;\n\nprintln!(\"{}\", from_utf8(b\"I \\xE2\\x9D\\xA4\\xEF\\xB8\\x8F UTF-8!\").unwrap());\n```\n\nIf you need detailed information on validation failures, use `simdutf8::compat::from_utf8()`\ninstead.\n\n```rust\nuse simdutf8::compat::from_utf8;\n\nlet err = from_utf8(b\"I \\xE2\\x9D\\xA4\\xEF\\xB8 UTF-8!\").unwrap_err();\nassert_eq!(err.valid_up_to(), 5);\nassert_eq!(err.error_len(), Some(2));\n```\n\n## APIs\n\n### Basic flavor\nUse the `basic` API flavor for maximum speed. It is fastest on valid UTF-8, but only checks\nfor errors after processing the whole byte sequence and does not provide detailed information if the data\nis not valid UTF-8. `simdutf8::basic::Utf8Error` is a zero-sized error struct.\n\n### Compat flavor\nThe `compat` flavor is fully API-compatible with `std::str::from_utf8()`. In particular, `simdutf8::compat::from_utf8()`\nreturns a `simdutf8::compat::Utf8Error`, which has `valid_up_to()` and `error_len()` methods. The first is useful for\nverification of streamed data. The second is useful e.g. for replacing invalid byte sequences with a replacement character.\n\nIt also fails early: errors are checked on the fly as the string is processed and once\nan invalid UTF-8 sequence is encountered, it returns without processing the rest of the data.\nThis comes at a slight performance penalty compared to the `basic` API even if the input is valid UTF-8.\n\n## Implementation selection\n\n### X86\nThe fastest implementation is usually selected at runtime using the `std::is_x86_feature_detected!` macro. The AVX 512 \nimplementation requires Rust 1.89 or late and is only selected if the CPU support the VBMI2 features to avoid throttling \nhappening with CPUs before Intels Ice Lake microarchitecture.\n\nIf you compile with `RUSTFLAGS=\"-C target-cpu=native\"` on a recent x86-64 machine whichs support AVX 512 with Rust 1.89 or later,\nthe AVX 512 implementation is selected at compile-time and runtime selection is disabled.\n\nFor no-std support (compiled with `--no-default-features`) the implementation is always selected at compile time based on\nthe targeted CPU. Use `RUSTFLAGS=\"-C target-feature=+avx2\"` for the AVX 2 implementation or `RUSTFLAGS=\"-C target-feature=+sse4.2\"`\nfor the SSE 4.2 implementation. For AVX 512 use `RUSTFLAGS=\"-C target-feature=+avx512f,+avx512bw,+avx512vbmi,+avx512vbmi2\"` with \nRust 1.89 or later.\n\n### ARM64\nThe SIMD implementation is used automatically since Rust 1.61.\n\n### ARMv7\nRequires a recent nightly Rust compiler. The `armv7_neon` feature needs to be enabled. The fasted implementation is selected at\nruntime using the `std::arch::is_arm_feature_detected!` macro unless the CPU target features are enabled, e.g. with\n`RUSTFLAGS=\"-C target-feature=+neon\"`. Some targets such as `thumbv7neon-linux-androideabi` and `thumbv7neon-unknown-linux-gnueabihf`\nhave NEON enabled by default.\n\n### WASM32\nFor wasm32 support, the implementation is selected at compile time based on the presence of the `simd128` target feature.\nUse `RUSTFLAGS=\"-C target-feature=+simd128\"` to enable the WASM SIMD implementation.  WASM, at\nthe time of this writing, doesn't have a way to detect SIMD through WASM itself.  Although this capability\nis available in various WASM host environments (e.g., [wasm-feature-detect] in the web browser), there is no portable\nway from within the library to detect this.\n\n[wasm-feature-detect]: https://github.com/GoogleChromeLabs/wasm-feature-detect\n\n#### Building/Targeting WASM\nSee [this document](./wasm32-development.md) for more details.\n\n### Access to low-level functionality\n\nIf you want to be able to call a SIMD implementation directly, use the `public_imp` feature flag. The validation implementations are then accessible in the `simdutf8::{basic, compat}::imp` hierarchy. Traits\nfacilitating streaming validation are available there as well.\n\n## Optimisation flags\nDo not use [`opt-level = \"z\"`](https://doc.rust-lang.org/cargo/reference/profiles.html), which prevents inlining and makes\nthe code quite slow.\n\n## Minimum Supported Rust Version (MSRV)\nThis crate's minimum supported Rust version is 1.38.0.\n\n## Benchmarks\nThe benchmarks have been done with [criterion](https://bheisler.github.io/criterion.rs/book/index.html), the tables\nare created with [critcmp](https://github.com/BurntSushi/critcmp). Source code and data are in the\n[bench directory](https://github.com/rusticstuff/simdutf8/tree/main/bench).\n\nThe naming schema is id-charset/size. _0-empty_ is the empty byte slice, _x-error/66536_ is a 64KiB slice where the very\nfirst character is invalid UTF-8. Library versions are simdutf8 v0.1.2 and simdjson v0.9.2. When comparing\nwith simdjson simdutf8 is compiled with `#inline(never)`.\n\nConfigurations:\n* X86-64: PC with an AMD Ryzen 7 PRO 3700 CPU (Zen2) on Linux with Rust 1.52.0\n* Aarch64: Macbook Air with an Apple M1 CPU (Apple Silicon) on macOS with Rust rustc 1.54.0-nightly (881c1ac40 2021-05-08).\n\n### simdutf8 basic vs std library on x86-64 (AMD Zen2)\n![image](https://user-images.githubusercontent.com/3736990/117568104-1c00f900-b0bf-11eb-938f-4c253d192480.png)\nSimdutf8 is up to 23 times faster than the std library on valid non-ASCII, up to four times on pure ASCII.\n\n### simdutf8 basic vs std library on aarch64 (Apple Silicon)\n![image](https://user-images.githubusercontent.com/3736990/117568160-42bf2f80-b0bf-11eb-86a4-9aeee4cee87d.png)\nSimdutf8 is up to to eleven times faster than the std library on valid non-ASCII, up to four times faster on\npure ASCII.\n\n### simdutf8 basic vs simdjson on x86-64\n![image](https://user-images.githubusercontent.com/3736990/117568231-80bc5380-b0bf-11eb-8e90-1dcc6d966ebd.png)\nSimdutf8 is faster than simdjson on almost all inputs.\n\n### simdutf8 basic vs simdutf8 compat UTF-8 on x86-64\n![image](https://user-images.githubusercontent.com/3736990/117568270-af3a2e80-b0bf-11eb-8ec4-e5a0a4ad7210.png)\nThere is a small performance penalty to continuously checking the error status while processing data, but detecting\nerrors early provides a huge benefit for the _x-error/66536_ benchmark.\n\n## Technical details\nFor inputs shorter than 64 bytes validation is delegated to `core::str::from_utf8()` except for the direct-access\nfunctions in `simdutf8::{basic, compat}::imp`.\n\nThe SIMD implementation is mostly similar to the one in simdjson except that it is has additional optimizations\nfor the pure ASCII case. Also it uses prefetch with AVX 2 on x86 which leads to slightly better performance with\nsome Intel CPUs on synthetic benchmarks.\n\nFor the compat API, we need to check the error status vector on each 64-byte block instead of just aggregating it. If an\nerror is found, the last bytes of the previous block are checked for a cross-block continuation and then\n`std::str::from_utf8()` is run to find the exact location of the error.\n\nCare is taken that all functions are properly inlined up to the public interface.\n\n## Thanks\n* to the authors of simdjson for coming up with the high-performance SIMD implementation and in particular to Daniel Lemire\n  for his feedback. It was very helpful.\n* to the authors of the simdjson Rust port who did most of the heavy lifting of porting the C++ code to Rust.\n\n\n## License\nThis code is dual-licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) and the [MIT License](https://opensource.org/licenses/MIT).\n\nIt is based on code distributed with simd-json.rs, the Rust port of simdjson, which is dual-licensed under\nthe MIT license and Apache 2.0 license as well.\n\nsimdjson itself is distributed under the Apache License 2.0.\n\n## References\nJohn Keiser, Daniel Lemire, [Validating UTF-8 In Less Than One Instruction Per Byte](https://arxiv.org/abs/2010.03090), Software: Practice and Experience 51 (5), 2021\n","funding_links":[],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusticstuff%2Fsimdutf8","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frusticstuff%2Fsimdutf8","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusticstuff%2Fsimdutf8/lists"}