{"id":26201614,"url":"https://github.com/sivkri/nlp-llm-textmining_genesearch","last_synced_at":"2025-03-12T03:22:41.023Z","repository":{"id":281693290,"uuid":"946117432","full_name":"sivkri/NLP-LLM-TextMining_GeneSearch","owner":"sivkri","description":"Text Mining (PubMed Search) with NLP \u0026 LLM","archived":false,"fork":false,"pushed_at":"2025-03-10T17:01:01.000Z","size":32,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-10T17:36:59.264Z","etag":null,"topics":["ensemble-machine-learning","llm","minilm","named-entity-recognition","ner","nlp","nlpmodels","nltk","pubmed","pubmed-abstracts","spacy","textmining"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sivkri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-10T16:29:32.000Z","updated_at":"2025-03-10T17:03:32.000Z","dependencies_parsed_at":"2025-03-10T17:49:18.075Z","dependency_job_id":null,"html_url":"https://github.com/sivkri/NLP-LLM-TextMining_GeneSearch","commit_stats":null,"previous_names":["sivkri/nlp-llm-textmining_genesearch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sivkri%2FNLP-LLM-TextMining_GeneSearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sivkri%2FNLP-LLM-TextMining_GeneSearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sivkri%2FNLP-LLM-TextMining_GeneSearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sivkri%2FNLP-LLM-TextMining_GeneSearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sivkri","download_url":"https://codeload.github.com/sivkri/NLP-LLM-TextMining_GeneSearch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243148025,"owners_count":20243903,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ensemble-machine-learning","llm","minilm","named-entity-recognition","ner","nlp","nlpmodels","nltk","pubmed","pubmed-abstracts","spacy","textmining"],"created_at":"2025-03-12T03:22:40.316Z","updated_at":"2025-03-12T03:22:41.018Z","avatar_url":"https://github.com/sivkri.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Text Mining (PubMed Search) with NLP \u0026 LLM 🚀\n\nThis project uses **NLP, LLM models, and text mining** to search **PubMed** for research articles related to specific **genes**. \nIt integrates keyword expansion using **spaCy, NLTK, and MiniLM** for enhanced search accuracy.\n\nThis project leverages **NLP, LLM models, and text mining** to efficiently search **PubMed** for research articles related to specific genes. Even though Arabidopsis search is shown in the **colab notebook**. It can be extended to other organism as well. It integrates **keyword expansion** using **spaCy, NLTK, and MiniLM** to enhance search accuracy and improve literature retrieval.\n\n## 🌟 Features  \n✅ Retrieve PubMed articles using **gene symbols \u0026 TAIR IDs**  \n✅ Expand search queries using **NLP models (spaCy, NLTK, MiniLM)**  \n✅ Remove duplicate PubMed IDs \u0026 provide **clear logs**\n✅ Fetch **detailed abstracts, DOI links, and matched keywords**\n\n## Usage\n**Prerequistes**\n- **NCBI API Key**: Required for accessing PubMed efficiently\n- **Hugging Face Token**: Needed for NLP model authentication (store in Colab secrets)\n\n## Steps to Run\n1️⃣ Enter **TAIR IDs or Gene Symbols** when prompted (*supports expansion to all organisms*)\n\n2️⃣ Enter the **number of articles** to retrieve per search\n\n3️⃣ The script will process **original and expanded queries** to retrieve PubMed articles\n\n\n## 📊 Execution Flow\n1. **User Input:** TAIR IDs / Gene Symbols\n2. **Data Processing:** Extract TAIR IDs \u0026 Gene Symbols\n3. **ID Conversion:** Convert TAIR IDs to Gene Symbols (if applicable)\n4. **Keyword Expansion:** Use **NLP models** to generate related terms\n5. **PubMed Querying:** Search using both **original \u0026 expanded keywords**\n6. **Data Extraction:** Fetch article titles, abstracts, DOIs, and publication years\n7. **Duplicate Removal:** Ensure **unique PubMed IDs** in results\n8. **Final Output:** Display cleaned \u0026 structured search results\n\n## 🛠️ Installation \u0026 Execution\n1️⃣ **Clone the Repository**\n   ```bash\n   git clone https://github.com/sivkri/NLP-LLM-TextMining_GeneSearch.git\n   cd NLP-LLM-TextMining_GeneSearch\n   ```\n\n2️⃣ **Run in Google Colab**\n\nSimply upload and execute the provided Colab Notebook\n```NLP-LLM-TextMining_GeneSearch.ipynb ```\n\n##💡**Example Output**\n ```bash\nEnter comma-separated TAIR IDs or gene symbols: AT4G26080, mapkkk18  \nEnter the number of articles to retrieve per search (e.g., 5, 10, 15): 5  \n✅ Results for AT4G26080: 0 (original) | 0 (expanded)  \n✅ Results for mapkkk18: 5 (original) | 5 (expanded)  \n✅ Results for ABI1: 5 (original) | 5 (expanded)  \n⚠️ 10 duplicate PubMed ID(s) removed.  \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsivkri%2Fnlp-llm-textmining_genesearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsivkri%2Fnlp-llm-textmining_genesearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsivkri%2Fnlp-llm-textmining_genesearch/lists"}