https://github.com/experimaestro/experimaestro-ir

IR module for experimaestro
https://github.com/experimaestro/experimaestro-ir

Last synced: 7 months ago
JSON representation

IR module for experimaestro

• Host: GitHub
• URL: https://github.com/experimaestro/experimaestro-ir
• Owner: experimaestro
• License: gpl-3.0
• Created: 2019-11-08T14:42:25.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2025-09-16T13:23:03.000Z (10 months ago)
• Last Synced: 2025-09-16T15:37:12.332Z (10 months ago)
• Language: Python
• Size: 1.66 MB
• Stars: 13
• Watchers: 2
• Forks: 4
• Open Issues: 14
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

          
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)

[![Documentation Status](https://readthedocs.org/projects/experimaestro-ir/badge/?version=latest)](https://experimaestro-ir.readthedocs.io/en/latest/?badge=latest)

[![PyPI version](https://badge.fury.io/py/experimaestro-ir.svg)](https://badge.fury.io/py/experimaestro-ir)

# Information Retrieval for experimaestro

Information Retrieval module for [experimaestro](https://experimaestro-python.readthedocs.io/)

The full documentation can be read at [IR@experimaestro](https://experimaestro-ir.readthedocs.io/).

You can find experiments built on top of XPMIR on the [xpmir github workspace](https://github.com/orgs/xpmir/repositories).

Finally, you can find the [roadmap](https://github.com/experimaestro/experimaestro-ir/issues/9).

## Install

Base experimaestro-IR can be installed with `pip install xpmir`.

Functionalities can be added by installing optional dependencies:

- `pip install xpmir[neural]` to install neural-IR packages (torch, etc.)

- `pip install xpmir[anserini]` to install Anserini related packages

For the development version, you can:

- If you just want the development version: install with `pip install git+https://github.com/experimaestro/experimaestro-ir.git`

- If you want to edit the code: clone and then do a `pip install -e .` within the directory

## What's inside?

- Collection management (using datamaestro)

    - Interface for the [IR datasets library](https://ir-datasets.com/)

    - Splitting IR datasets

    - Shuffling training triplets

- Representation

    - Word Embeddings

    - HuggingFace transformers

- Indices

    - dense: [FAISS](https://github.com/facebookresearch/faiss) interface

    - sparse: [xpmir-rust library](https://github.com/experimaestro/experimaestro-ir-rust)

- Standard Indexing and Retrieval

    - Anserini

- Learning to Rank

    - Pointwise

    - Pairwise

    - Distillation

- Neural IR

    - Cross-Encoder

    - Splade

    - DRMM

    - ColBERT

- Paper reproduction:

    - *MonoBERT* (Passage Re-ranking with BERT. Rodrigo Nogueira and Kyunghyun Cho. 2019)

    - (alpha) *DuoBERT* (Multi-Stage Document Ranking with BERT. Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, Jimmy Lin. 2019)

    - (beta) *Splade v2* (SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval, Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. SIGIR 2021)

    - (planned) ANCE

- Pre-trained models

    - [HuggingFace](https://huggingface.co) [integration](https://experimaestro-ir.readthedocs.io/en/latest/pretrained.html) (direct, through the Sentence Transformers library)

## Thanks

Some parts of the code have been adapted from [OpenNIR](https://github.com/Georgetown-IR-Lab/OpenNIR)