https://github.com/Intsights/fastzy
Python library for fast fuzzy search over a big file written in Rust
https://github.com/Intsights/fastzy
distance levenshtein mbleven-algorithm python
Last synced: 5 months ago
JSON representation
Python library for fast fuzzy search over a big file written in Rust
- Host: GitHub
- URL: https://github.com/Intsights/fastzy
- Owner: Intsights
- License: mit
- Created: 2020-01-11T17:04:06.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-03-21T12:11:47.000Z (over 1 year ago)
- Last Synced: 2024-10-12T06:24:00.403Z (12 months ago)
- Topics: distance, levenshtein, mbleven-algorithm, python
- Language: Rust
- Homepage:
- Size: 238 KB
- Stars: 43
- Watchers: 4
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Python library for fast fuzzy search over a big file written in Rust
[](https://pypi.org/project/fastzy/)
## Table of Contents
- [Table of Contents](#table-of-contents)
- [About The Project](#about-the-project)
- [Built With](#built-with)
- [Performance](#performance)
- [Installation](#installation)
- [Usage](#usage)
- [License](#license)
- [Contact](#contact)
## About The Project
Fastzy is a library written in Rust that can search through a file looking for text based on its distance (Levenshtein). For measuring the Levenshtein distance, the library uses mbleven's algorithm. In situations where the requested distance exceeds 3, where mbleven is slower, Wagner-Fischer is used instead of mbleven. This library loads the whole file into memory, and creates a lightweight index based on the length of the lines. The result is that only potential lines are looked up, opposed to a large number of lines.
### Built With
* [mbleven](https://github.com/fujimotos/mbleven)
* [Pyo3](https://github.com/PyO3/pyo3)
### Performance
| Library | Function | Time |
| ------------- | ------------- | ------------- |
| [polyleven](https://github.com/ztane/python-Levenshtein) | polyleven.levenshtein('text') | 8.48s |
| [fastzy](https://github.com/Intsights/fastzy) | fastzy.search('text) | 0.003s |
### Installation
```sh
pip3 install fastzy
```
## Usage
```python
import fastzy
# open a file and index it in memory
searcher = fastzy.Searcher(
file_path='input_text_file.txt',
separator='',
)
# search for the input text 'text' with the distance of 1
searcher.search(
pattern='text',
max_distance=1,
)
['test', 'texts', 'next']
```
## License
Distributed under the MIT License. See `LICENSE` for more information.
## Contact
Gal Ben David - gal@intsights.com
Project Link: [https://github.com/Intsights/fastzy](https://github.com/Intsights/fastzy)