Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mathematicator-core/tokenizer
Tokenizer that can convert string (user input / LaTeX) to numbers and operators.
https://github.com/mathematicator-core/tokenizer
binary-tree calculator latex math math-formulas math-language math-tokenizer tokenizer
Last synced: about 1 month ago
JSON representation
Tokenizer that can convert string (user input / LaTeX) to numbers and operators.
- Host: GitHub
- URL: https://github.com/mathematicator-core/tokenizer
- Owner: mathematicator-core
- License: mit
- Created: 2019-07-25T20:28:59.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-07-10T04:57:48.000Z (6 months ago)
- Last Synced: 2024-08-09T20:51:31.128Z (5 months ago)
- Topics: binary-tree, calculator, latex, math, math-formulas, math-language, math-tokenizer, tokenizer
- Language: PHP
- Homepage: https://baraja.cz
- Size: 122 KB
- Stars: 8
- Watchers: 3
- Forks: 3
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Funding: .github/funding.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
PHP Math Tokenizer
[](https://github.com/mathematicator-core/tokenizer/actions?query=workflow%3A%22Integrity+check%22)
[](https://codecov.io/gh/mathematicator-core/tokenizer)
[](https://packagist.org/packages/mathematicator-core/tokenizer)
[](https://packagist.org/packages/mathematicator-core/tokenizer)
[](./LICENSE)
[](https://phpstan.org/)
Mathematicator Tokenizer is a simple library for converting string math formulas to
a stream of tokens with LaTeX support.
## Installation
```
composer require mathematicator-core/tokenizer
```
## Features
- Convert all your math formulas to a stream of tokens
- Convert user math input to LaTeX
- Render the tokens tree map for debug
## Usage
Inject the `Tokenizer` service via [DI](https://doc.nette.org/en/3.0/dependency-injection)
and tokenize your query.
```php
use \Mathematicator\Tokenizer\Tokenizer;
$tokenizer = new Tokenizer(/* some dependencies */);
// Convert math formula to an array of tokens:
$tokens = $tokenizer->tokenize('(5+3)*(2/(7+3))');
// Now you can convert tokens to a more useful format:
$objectTokens = $tokenizer->tokensToObject($tokens);
var_dump($objectTokens); // Return typed tokens with meta data
// Render to LaTeX
echo $tokenizer->tokensToLatex($objectTokens);
// Render to debug tree (extremely fast):
echo $tokenizer->renderTokensTree($objectTokens);
```
### Configuration
The tokenizer uses automatic configuration based on DI.
Just use the DI container and the service will be fully available.
## Mathematicator Framework tools structure
The biggest advantage is that you can choose which layer best fits
your needs and start build on the top of it, immediately, without the need
to create everything by yourself. Our tools are tested for bugs
and tuned for performance, so you can save a significant amount
of your time, money, and effort.
Framework tend to be modular as much as possible, so you should be able
to create an extension on each layer and its sublayers.
**Mathematicator framework layers** ordered from the most concrete
one to the most abstract one:
Search
Modular search engine layer that calls its sublayers
and creates user interface.
Vizualizator
Elegant graphic visualizer that can render to
SVG, PNG, JPG and Base64.
Extensions:
Mandelbrot set generator
Calculator
Modular advance calculations layer.
Extensions:
Integral Solver
,
Statistics
Engine
Core logic layer that maintains basic controllers,
DAOs, translator, common exceptions, routing etc.
Tokenizer
Tokenizer that can convert string (user input / LaTeX) to numbers
and operators.
Numbers
Fast & secure storage for numbers with arbitrary precision.
It supports Human string and LaTeX output and basic conversions.
**Third-party packages:**
⚠️ Not guaranteed!
REST API
Install the whole pack as a REST API service
on your server (Docker ready) or
access it via public cloud REST API.
## Contribution
> Please help to improve this documentation by sending a Pull request.
### Tests
All new contributions should have its unit tests in `/tests` directory.
Before you send a PR, please, check all tests pass.
This package uses [Nette Tester](https://tester.nette.org/).
You can run tests via command:
```bash
composer test
````
For benchmarking, we use [phpbench](https://github.com/phpbench/phpbench).
You can run benchmarks this way:
```bash
composer global require phpbench/phpbench @dev # only the first time
phpbench run
````
Before PR, please run complete code check via command:
```bash
composer cs:install # only first time
composer fix # otherwise pre-commit hook can fail
````