https://github.com/typst/unscanny

Painless string scanning.
https://github.com/typst/unscanny

parsing scanning tokenization

Last synced: 5 days ago
JSON representation

Painless string scanning.

• Host: GitHub
• URL: https://github.com/typst/unscanny
• Owner: typst
• License: other
• Created: 2022-04-16T12:19:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-05-24T10:21:59.000Z (over 1 year ago)
• Last Synced: 2024-10-14T01:18:24.737Z (about 1 month ago)
• Topics: parsing, scanning, tokenization
• Language: Rust
• Homepage:
• Size: 15.6 KB
• Stars: 40
• Watchers: 3
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE-APACHE

Awesome Lists containing this project

README

        
# unscanny

[![Crates.io](https://img.shields.io/crates/v/unscanny.svg)](https://crates.io/crates/unscanny)

[![Documentation](https://docs.rs/unscanny/badge.svg)](https://docs.rs/unscanny)

Painless string scanning.

```toml

[dependencies]

unscanny = "0.1"

```

Basically, you'll want to use this crate if it's too much pain to solve your

problem with a bare `chars()` iterator. Speaking more broadly, `unscanny` is

useful in these situations:

- You need to parse simple flat grammars (dates, times, custom stuff, ...) and

  want an interface that's a bit more convenient to use than a simple char

  iterator.

- You're hand-writing a tokenizer.

The `Scanner` keeps an internal cursor, allows you to peek around it, advance it

beyond chars or other patterns and easily slice substrings before and after the

cursor.

# Example

Recognizing and parsing a simple comma separated list of floats.

```rust

let mut s = Scanner::new(" +12 , -15.3, 14.3  ");

let mut nums = vec![];

while !s.done() {

    s.eat_whitespace();

    let start = s.cursor();

    s.eat_if(['+', '-']);

    s.eat_while(char::is_ascii_digit);

    s.eat_if('.');

    s.eat_while(char::is_ascii_digit);

    nums.push(s.from(start).parse::().unwrap());

    s.eat_whitespace();

    s.eat_if(',');

}

assert_eq!(nums, [12.0, -15.3, 14.3]);

```

## Safety

This crate internally uses unsafe code for better performance. However, all

unsafe code is documented with justification for why it's safe, all accesses are

checked in debug mode and everything is tested.

## License

This crate is dual-licensed under the MIT and Apache 2.0 licenses.