https://github.com/jakeroggenbuck/plrs

The multi-tool of lexical analysis and tokenization. Make parsers in less time for many use cases.
https://github.com/jakeroggenbuck/plrs

language lexer python rust

Last synced: 5 days ago
JSON representation

The multi-tool of lexical analysis and tokenization. Make parsers in less time for many use cases.

• Host: GitHub
• URL: https://github.com/jakeroggenbuck/plrs
• Owner: JakeRoggenbuck
• License: mit
• Created: 2021-07-29T09:02:44.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-04-09T04:36:46.000Z (7 months ago)
• Last Synced: 2024-10-31T11:44:55.511Z (15 days ago)
• Topics: language, lexer, python, rust
• Language: Rust
• Homepage: https://pypi.org/project/plrs/
• Size: 57.6 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# plrs ![Version](https://img.shields.io/pypi/v/plrs)

The multi-tool of lexical analysis and tokenization.

# Install

```

pip install plrs

```

# Build

```

maturin build

```

# Examples

- [zipfs-law](https://github.com/JakeRoggenbuck/zipfs-law)

# API

#### Global variables

```

EOF_TOKEN

```

#### Classes

```

Tokens

Settings

Token

  - part

  - token

  - set_part

  - set_token

  - __str__

  - __repr__

Lexer

  - new

  - char_forward

  - skip_over_char_set

  - next

```

#### Functions

```

is_char_symbol

is_char_operator

is_char_whitespace

is_char_numeric

is_single_quote

is_double_quote

ends_token

is_part_numeric

tokenize

```