https://github.com/guenthermi/fast_minh
Python package for fast MinHash calculation and operations
https://github.com/guenthermi/fast_minh
jaccard lsh minhash search similarity
Last synced: about 1 month ago
JSON representation
Python package for fast MinHash calculation and operations
- Host: GitHub
- URL: https://github.com/guenthermi/fast_minh
- Owner: guenthermi
- License: apache-2.0
- Created: 2024-04-20T09:52:04.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-08-04T14:00:27.000Z (9 months ago)
- Last Synced: 2025-02-01T06:27:05.903Z (3 months ago)
- Topics: jaccard, lsh, minhash, search, similarity
- Language: C++
- Homepage:
- Size: 19.5 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fast_minh
A small Python package for calculating MinHash values, computing approaximated Jaccard similarity, and building LSH indices of MinHash values to perform fast approximated set similarity search.
## Installation
```bash
pip install fast-minh
```
## Usage
**Calculating MinHash values:**
To calculate min hash values, you can create a `HashFamily` object which initializes a set of hash fuctions (default: 128).
After that you can use the `HashFamily.minh` function to obtain a set of MinHash values for a given set of strings:
```python
hf = HashFamily()
mh = hf.minh(['test', 'it', 'out'])
```
**Calculate an approximated Jaccard coefficient:**
After calculating multiple minhash values for different sets with the same hash family, you can use the `jaccard` function to determine and approximated similarity score:
```python
from fast_minh import minh, jaccard
hf = HashFamily()
mh1 = hf.minh(['test', 'it', 'out'])
mh2 = hf.minh(['test', 'it', 'again'])
sim = jaccard(mh1, mh2)
```
**MinHash LSH Index:**
To find similar sets of text values fast, you can use an MinHash LSH index.
You can insert sets with the `LshIndex.insert` function and retrieve similar candidates with the `LshIndex.find` method:
```python
from fast_minh import LshIndex
lsh = LshIndex(1, 3)
input_key = 'Key'
input_set = ['A', 'set', 'of', 'multiple', 'tokens']
lsh.insert(input_key, input_set)
out = lsh.find(input_set)
```