{"id":31649296,"url":"https://github.com/vladimircuriel/information-retrieval-system","last_synced_at":"2026-05-08T14:11:57.447Z","repository":{"id":313741383,"uuid":"1052304500","full_name":"vladimircuriel/information-retrieval-system","owner":"vladimircuriel","description":"An NLP-based information retrieval system that indexes document collections, understands natural language queries, and returns relevance-ranked results using modern ranking algorithms.","archived":false,"fork":false,"pushed_at":"2025-09-08T05:54:04.000Z","size":764,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-08T07:23:56.272Z","etag":null,"topics":["ai","nlp","nltk","sklearn","spicy","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vladimircuriel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-07T20:18:54.000Z","updated_at":"2025-09-08T05:54:07.000Z","dependencies_parsed_at":"2025-09-08T07:23:57.955Z","dependency_job_id":"0a46cb99-2362-4781-85c7-e9e0496e96d8","html_url":"https://github.com/vladimircuriel/information-retrieval-system","commit_stats":null,"previous_names":["vladimircuriel/information-retrieval-system"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/vladimircuriel/information-retrieval-system","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Finformation-retrieval-system","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Finformation-retrieval-system/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Finformation-retrieval-system/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Finformation-retrieval-system/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vladimircuriel","download_url":"https://codeload.github.com/vladimircuriel/information-retrieval-system/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Finformation-retrieval-system/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278740831,"owners_count":26037480,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","nlp","nltk","sklearn","spicy","streamlit"],"created_at":"2025-10-07T07:42:00.208Z","updated_at":"2025-10-07T07:42:08.058Z","avatar_url":"https://github.com/vladimircuriel.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\r\n   Classes Information Retrieval System\r\n\u003c/h1\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/Python-3776AB?style=for-the-badge\u0026logo=python\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/NLTK-FF4500?style=for-the-badge\u0026logo=nltk\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/scikit--learn-F7931E?style=for-the-badge\u0026logo=scikitlearn\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/NLP-006B6B?style=for-the-badge\u0026logo=natural-language-processing\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/Streamlit-FF4B4B?style=for-the-badge\u0026logo=streamlit\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/Poetry-4E227A?style=for-the-badge\u0026logo=poetry\u0026logoColor=white\" /\u003e  \r\n  \u003cimg src=\"https://img.shields.io/badge/SciPy-8CAAE6?style=for-the-badge\u0026logo=scipy\u0026logoColor=white\" /\u003e  \r\n\u003c/div\u003e\r\n\r\n---\r\n\r\n**Information Retrieval System** is a Python-powered information-retrieval engine that lets you index entire document collections, interpret free-form queries, and return relevance-ranked answers—all with no external service required.\r\n\r\nUnlike a simple CSV search that scans each row for literal query matches, IR model builds a TF-IDF vector index and applies NLP preprocessing (tokenization, lemmatization, stop-word removal) so that queries and documents are compared in a high-dimensional space using cosine similarity—yielding fast, relevance-ranked results rather than unranked, exact‐match hits, and allowing seamless extension to more advanced ranking algorithms like BM25 or neural embeddings.\r\n\r\n## Table of Contents\r\n\r\n- [Features](#features)\r\n- [Application](#application)\r\n- [Installation](#installation)\r\n\r\n## Features\r\n\r\n- **Local Indexing**: Builds a TF-IDF vector index over your document collection (any pandas DataFrame column) on initialization—no external services involved.  \r\n- **Custom NLP Pipeline**: Leverages `query_processing()` for tokenization, normalization, stop-word removal and lemmatization to turn free-form queries into polished search inputs.  \r\n- **Fast Similarity Search**: Transforms queries into TF-IDF vectors and computes cosine similarity against your corpus to generate relevance scores in milliseconds.  \r\n- **Top-N Ranking**: Returns a configurable number of results, sorted by descending score, including document ID, metadata fields (e.g. Major, Course Title), cleaned description, and relevance score.  \r\n- **Persistence \u0026 Reuse**: Keeps the fitted `TfidfVectorizer` and index in memory or disk (your choice) so subsequent searches are instant and consistent across runs.  \r\n- **Schema-Agnostic**: Simply point the system at any DataFrame and column name—no fixed schema required, making it easy to index PDFs, CSVs or custom data sources.  \r\n- **Extensible Scoring**: Core TF-IDF + cosine similarity can be augmented with additional ranking algorithms (BM25, neural embeddings) as your needs evolve.  \r\n\r\n## Application\r\n\r\n![ScreenShot - 12AM-48M@2x](https://github.com/user-attachments/assets/1496d230-0226-4807-bb54-127721c04d10)\r\n\r\n## Installation\r\n\r\n### Prerequisites\r\n\r\n- **Docker**\r\n\r\n### Steps\r\n\r\n1. **Clone the repository**:\r\n\r\n```bash\r\ngit clone https://github.com/vladimircuriel/information-retrieval-system\r\n```\r\n\r\n2. **Navigate to the project directory**:\r\n\r\n```bash\r\ncd information-retrieval-system\r\n```\r\n   \r\n3. **Run the commands**:\r\n\r\n```bash\r\ndocker build -t system:latest .\r\n```\r\n\r\n```bash\r\ndocker run -p 8501:8501 system:latest\r\n```\r\n4. **Access the application**:\r\n\r\nOpen your browser and visit `http://localhost:8501` to access the user interface.\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimircuriel%2Finformation-retrieval-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvladimircuriel%2Finformation-retrieval-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimircuriel%2Finformation-retrieval-system/lists"}