{"id":21429589,"url":"https://github.com/mayank-02/boolean-retrieval-model","last_synced_at":"2025-04-19T16:08:56.434Z","repository":{"id":188401409,"uuid":"319950875","full_name":"mayank-02/boolean-retrieval-model","owner":"mayank-02","description":"Implementation of Boolean Model of Information Retrieval","archived":false,"fork":false,"pushed_at":"2024-09-03T12:49:46.000Z","size":21,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-29T09:51:15.914Z","etag":null,"topics":["boolean-model","information-retrieval","python","unranked-retrieval"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mayank-02.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-09T12:32:52.000Z","updated_at":"2025-02-12T19:20:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"5c0ff502-6929-48b9-8cdc-40a9d29cc84c","html_url":"https://github.com/mayank-02/boolean-retrieval-model","commit_stats":null,"previous_names":["mayank-02/boolean-retrieval-model"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fboolean-retrieval-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fboolean-retrieval-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fboolean-retrieval-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fboolean-retrieval-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mayank-02","download_url":"https://codeload.github.com/mayank-02/boolean-retrieval-model/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249733763,"owners_count":21317633,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boolean-model","information-retrieval","python","unranked-retrieval"],"created_at":"2024-11-22T22:18:23.666Z","updated_at":"2025-04-19T16:08:56.403Z","avatar_url":"https://github.com/mayank-02.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Boolean Retrieval Model\n\nThe class `BooleanModel.py` implements a toy search engine to illustrate the boolean retrieval model for text documents.\n\nThe program asks you to enter a search query, and then returns all documents matching the query (exact match), in no particular order (unranked retrieval).\n\nThe document corpus consists of documents, which are short stories downloaded from [here](https://www.rong-chang.com/qa2/).\n\n## Getting Started\n\n- Install Python 3.6+\n- Install all pip requirements from the `requirements.txt`:\n\n```bash\n$ python3 -m pip install -r requirements.txt\n```\n\n- To download stopwords used for the model, open your terminal or command prompt and enter following commands:\n\n```bash\n$ python3\n\u003e\u003e\u003e import nltk\n\u003e\u003e\u003e nltk.download('stopwords')\n```\n\n## Usage\n\n```python\n# Import boolean model\nfrom BooleanModel import BooleanModel\n\n# Create a model on your corpus of documents by passing it's path as an argument\nmodel = BooleanModel(\"./corpus/*\")\n\n# Query on it as many times as you like\nresults = model.query(\"book\")\n\n# results = ['Freeway Chase Ends at Newsstand.txt', 'A Festival of Books.txt']\n\n# Querying on a word which is not in the corpus\nresults = model.query(\"pikachu\")\n\n# Warning: pikachu was not found in the corpus!\n# results = []\n```\n\n### Queries\n\n#### Supported Queries\n\n- Single term =\u003e `ash`\n- AND =\u003e `ash \u0026 may`\n- OR =\u003e `ash | may \u0026 brown`\n- Parenthesis =\u003e `( ash | may ) \u0026 brown`\n- NOT =\u003e `( ~ash | may ) \u0026 brown`\n\n\u003e Precedence: NOT (~) \u003e AND (\u0026) \u003e OR (|)\n\n#### Unsupported Queries\n\n- NOT operator on an  intermediate result =\u003e `~( ash | may ) \u0026 brown`\n- Spaces between NOT operator and operand =\u003e `~ ash \u0026 may`\n\n## Methodology\n\n1. Preprocessing to build standard inverted index\n   - Remove special characters\n   - Remove digits\n   - Tokenize\n   - Lowercasing\n   - Stemming using `PorterStemmer`\n   - Add unique words and their postings to the index\n\n2. Refer to [this](https://nlp.stanford.edu/IR-book/html/htmledition/an-example-information-retrieval-problem-1.html) for the internals of boolean model and query evaluation\n\n## Note\n\n- In case of start byte invalid errors, check for character encodings of the documents in corpus. (Currently, `utf-8` is used.)\n\n## Authors\n\n[Mayank Jain](https://github.com/mayank-02)\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayank-02%2Fboolean-retrieval-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmayank-02%2Fboolean-retrieval-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayank-02%2Fboolean-retrieval-model/lists"}