Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/stevenroose/rust-base64-compat

base64 crate for Rust that supports rustc v0.19.0 and newer
https://github.com/stevenroose/rust-base64-compat

Last synced: 3 months ago
JSON representation

base64 crate for Rust that supports rustc v0.19.0 and newer

Awesome Lists containing this project

README 

        
[base64-compat](https://crates.io/crates/base64-compat)

===



[![](https://img.shields.io/crates/v/base64-compat.svg)](https://crates.io/crates/base64-compat) [![Docs](https://docs.rs/base64-compat/badge.svg)](https://docs.rs/base64-compat) [![Build](https://travis-ci.org/stevenroose/rust-base64-compat.svg?branch=master)](https://travis-ci.org/stevenroose/rust-base64-compat) [![codecov](https://codecov.io/gh/stevenroose/rust-base64-compat/branch/master/graph/badge.svg)](https://codecov.io/gh/stevenroose/rust-base64-compat)



It's base64. What more could anyone want?



Perhaps a stable API and compatibility with not-last-summer Rust versions?



This project is a fork of [rust-base64](https://github.com/marshallpierce/rust-base64/) with stable support for older Rust versions.



----------



This library's goals are to be *correct* and *fast*. It's thoroughly tested and widely used. It exposes functionality at multiple levels of abstraction so you can choose the level of convenience vs performance that you want, e.g. `decode_config_slice` decodes into an existing `&mut [u8]` and is pretty fast (2.6GiB/s for a 3 KiB input), whereas `decode_config` allocates a new `Vec` and returns it, which might be more convenient in some cases, but is slower (although still fast enough for most purposes) at 2.1 GiB/s.



Example

---



Cargo.toml line: `base64-compat = "1.0.0"`



```rust

extern crate base64;



use base64::{encode, decode};



fn main() {

    let a = b"hello world";

    let b = "aGVsbG8gd29ybGQ=";



    assert_eq!(encode(a), b);

    assert_eq!(a, &decode(b).unwrap()[..]);

}

```



See the [docs](https://docs.rs/base64-compat) for all the details.



Rust version compatibility

---



The minimum required Rust version is 1.19.0.



Developing

---



Benchmarks are in `benches/`. Running them requires nightly rust, but `rustup` makes it easy:



```bash

rustup run nightly cargo bench

```



Decoding is aided by some pre-calculated tables, which are generated by:



```bash

cargo run --example make_tables > src/tables.rs.tmp && mv src/tables.rs.tmp src/tables.rs

```



Profiling

---



On Linux, you can use [perf](https://perf.wiki.kernel.org/index.php/Main_Page) for profiling. Then compile the benchmarks with `rustup nightly run cargo bench --no-run`.



Run the benchmark binary with `perf` (shown here filtering to one particular benchmark, which will make the results easier to read). `perf` is only available to the root user on most systems as it fiddles with event counters in your CPU, so use `sudo`. We need to run the actual benchmark binary, hence the path into `target`. You can see the actual full path with `rustup run nightly cargo bench -v`; it will print out the commands it runs. If you use the exact path that `bench` outputs, make sure you get the one that's for the benchmarks, not the tests. You may also want to `cargo clean` so you have only one `benchmarks-` binary (they tend to accumulate).



```bash

sudo perf record target/release/deps/benchmarks-* --bench decode_10mib_reuse

```



Then analyze the results, again with perf:



```bash

sudo perf annotate -l

```



You'll see a bunch of interleaved rust source and assembly like this. The section with `lib.rs:327` is telling us that 4.02% of samples saw the `movzbl` aka bit shift as the active instruction. However, this percentage is not as exact as it seems due to a phenomenon called *skid*. Basically, a consequence of how fancy modern CPUs are is that this sort of instruction profiling is inherently inaccurate, especially in branch-heavy code.



```text

 lib.rs:322    0.70 :     10698:       mov    %rdi,%rax

    2.82 :        1069b:       shr    $0x38,%rax

         :                  if morsel == decode_tables::INVALID_VALUE {

         :                      bad_byte_index = input_index;

         :                      break;

         :                  };

         :                  accum = (morsel as u64) << 58;

 lib.rs:327    4.02 :     1069f:       movzbl (%r9,%rax,1),%r15d

         :              // fast loop of 8 bytes at a time

         :              while input_index < length_of_full_chunks {

         :                  let mut accum: u64;

         :

         :                  let input_chunk = BigEndian::read_u64(&input_bytes[input_index..(input_index + 8)]);

         :                  morsel = decode_table[(input_chunk >> 56) as usize];

 lib.rs:322    3.68 :     106a4:       cmp    $0xff,%r15

         :                  if morsel == decode_tables::INVALID_VALUE {

    0.00 :        106ab:       je     1090e 

```



Fuzzing

---



This uses [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz). See `fuzz/fuzzers` for the available fuzzing scripts. To run, use an invocation like these:



```bash

cargo +nightly fuzz run roundtrip

cargo +nightly fuzz run roundtrip_no_pad

cargo +nightly fuzz run roundtrip_random_config -- -max_len=10240

cargo +nightly fuzz run decode_random

```



License

---



This project is a fork of the [rust-base64](https://github.com/marshallpierce/rust-base64/) project,

which is dual-licensed under MIT and Apache 2.0.

This project is thus also dual-licensed under MIT and Apache 2.0.



All copyrights of code in this project belong to the original

copyright holders of the original rust-base64 project.