https://github.com/kwang2049/easy-elasticsearch
Using business-level retrieval system (BM25) with Python in just a few lines.
https://github.com/kwang2049/easy-elasticsearch
bm25 docker elasticsearch information-retrieval
Last synced: over 1 year ago
JSON representation
Using business-level retrieval system (BM25) with Python in just a few lines.
- Host: GitHub
- URL: https://github.com/kwang2049/easy-elasticsearch
- Owner: kwang2049
- License: apache-2.0
- Created: 2021-05-09T13:57:24.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2023-02-03T14:38:10.000Z (over 3 years ago)
- Last Synced: 2025-03-19T02:38:27.595Z (over 1 year ago)
- Topics: bm25, docker, elasticsearch, information-retrieval
- Language: Python
- Homepage:
- Size: 38.1 KB
- Stars: 31
- Watchers: 1
- Forks: 6
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Easy Elasticsearch
This repository contains a high-level encapsulation for using [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) with python in just a few lines.
## Installation
Via pip:
```bash
pip install easy-elasticsearch
```
Via git repo:
```bash
git clone https://github.com/kwang2049/easy-elasticsearch
pip install -e .
```
## Usage
To utilize the elasticsearch service, one can select from 3 ways:
- (1) Start an ES service manually and then indicate the `host` and `port_http` (please refere to [download_and_run.sh](easy_elasticsearch/examples/download_and_run.sh));
- (2) Or leave `host=None` by default to start a docker container itself;
- (3) Or leava `host=None` and setting `service_type=executable` to download an ES executable and start it in the back end.
Finally, just either call its ```rank``` or ```score``` function for retrieval or calculating BM25 scores.
```python
from easy_elasticsearch import ElasticSearchBM25
pool = {
'id1': 'What is Python? Is it a programming language',
'id2': 'Which Python version is the best?',
'id3': 'Using easy-elasticsearch in Python is really convenient!'
}
bm25 = ElasticSearchBM25(pool, port_http='9222', port_tcp='9333') # By default, when `host=None` and `mode="docker"`, a ES docker container will be started at localhost.
query = "What is Python?"
rank = bm25.query(query, topk=10) # topk should be <= 10000
scores = bm25.score(query, document_ids=['id2', 'id3'])
print(query, rank, scores)
bm25.delete_index() # delete the one-trial index named 'one_trial'
bm25.delete_container() # remove the docker container'
```
Another example for retrieving Quora questions can be found in [easy_elasticsearch/examples/quora.py](easy_elasticsearch/examples/quora.py):
```bash
python -m easy_elasticsearch.examples.quora --mode docker
```
or
```bash
python -m easy_elasticsearch.examples.quora --mode executable
```