{"id":49260600,"url":"https://github.com/man4ish/bioinfo-lora-finetuning","last_synced_at":"2026-04-25T07:04:03.863Z","repository":{"id":321874256,"uuid":"1087486055","full_name":"man4ish/bioinfo-lora-finetuning","owner":"man4ish","description":"Fine-tune a lightweight LLM (TinyLlama-1.1B) for bioinformatics instruction-response tasks using LoRA. Compare model performance before and after fine-tuning on domain-specific prompts. Fully compatible with Apple M1/M4 (Metal GPU) and demonstrates end-to-end fine-tuning, inference, and adapter merging.","archived":false,"fork":false,"pushed_at":"2025-11-01T03:36:11.000Z","size":12,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-01T04:18:44.449Z","etag":null,"topics":["bioinformatics","deep-learning","fine-tuning","instruction-tuning","llm","lora","machine-learning","natural-language-processing","python","tinyllama","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/man4ish.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-01T02:24:57.000Z","updated_at":"2025-11-01T03:05:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"49a554e5-425e-4473-8862-ddd39c7c2ac5","html_url":"https://github.com/man4ish/bioinfo-lora-finetuning","commit_stats":null,"previous_names":["man4ish/bioinfo-lora-finetuning"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/man4ish/bioinfo-lora-finetuning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/man4ish%2Fbioinfo-lora-finetuning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/man4ish%2Fbioinfo-lora-finetuning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/man4ish%2Fbioinfo-lora-finetuning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/man4ish%2Fbioinfo-lora-finetuning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/man4ish","download_url":"https://codeload.github.com/man4ish/bioinfo-lora-finetuning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/man4ish%2Fbioinfo-lora-finetuning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32253252,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T04:23:17.126Z","status":"ssl_error","status_checked_at":"2026-04-25T04:21:53.360Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","deep-learning","fine-tuning","instruction-tuning","llm","lora","machine-learning","natural-language-processing","python","tinyllama","transformers"],"created_at":"2026-04-25T07:03:59.282Z","updated_at":"2026-04-25T07:04:03.854Z","avatar_url":"https://github.com/man4ish.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bioinformatics LLM Fine-Tuning with LoRA\n\nThis repository demonstrates fine-tuning a lightweight LLM (TinyLlama-1.1B) on bioinformatics instruction-response prompts using Low-Rank Adaptation (LoRA). The goal is to adapt a general instruction-tuned model to answer domain-specific questions and showcase the improvements before and after fine-tuning.\n\n---\n\n## Objective\n- Fine-tune TinyLlama for bioinformatics instructions  \n- Run locally on Apple Silicon (M4 Max, MPS backend)  \n- Compare outputs before and after LoRA fine-tuning  \n\n---\n\n## Repository Structure\n\n```\n\nbioinfo-lora-finetuning-demo/\n├── data/\n│   └── bioinfo_train.jsonl          # Instruction-response dataset\n├── src/\n│   ├── lora_train.py                # LoRA fine-tuning script\n│   ├── lora_infer_before.py         # Baseline inference\n│   ├── lora_infer_after.py          # Inference using LoRA adapter\n│   └── merge_lora.py                # Merge adapter into base model\n├── results/\n│   ├── sample_outputs_before.txt\n│   ├── sample_outputs_after.txt\n│   └── merged-model/\n├── requirements.txt\n└── README.md\n\n````\n\n---\n\n## Environment Setup\n\n1. Clone the repository and create a virtual environment:\n```bash\ngit clone \u003cyour-repo-url\u003e\ncd bioinfo-lora-finetuning-demo\npython -m venv venv\nsource venv/bin/activate\n````\n\n2. Install dependencies:\n\n```bash\npip install --upgrade pip setuptools wheel\npip install -r requirements.txt\n```\n\nNote: On macOS M4, if you encounter `sentencepiece` build errors, use:\n\n```bash\npip install sentencepiece --prefer-binary\n```\n\n---\n\n## Dataset\n\n* `data/bioinfo_train.jsonl` contains bioinformatics instruction-response pairs in JSON Lines format:\n\n```json\n{\"instruction\": \"Explain what a FASTQ file is.\", \"output\": \"A FASTQ file stores sequencing reads with quality scores...\"}\n{\"instruction\": \"What is SNP annotation?\", \"output\": \"SNP annotation links single-nucleotide polymorphisms to genes and predicts functional impacts.\"}\n```\n\n* You can expand this dataset to hundreds of examples for improved results.\n\n---\n\n## Fine-Tuning\n\n```bash\npython src/lora_train.py \\     \n  --dataset_path data/bioinfo_train.jsonl \\\n  --epochs 3 \\\n  --batch_size 2 \\\n  --gradient_accumulation 4 \\\n  --lr 2e-4 \\\n  --output_dir results/lora-adapter\n```\n\n* Saves LoRA adapter to `./results/lora-adapter`\n* Uses MPS GPU if available\n* Training logs show decreasing loss over steps\n\n---\n\n## Inference\n\n### Baseline (Before Fine-Tuning)\n\n```bash\npython src/lora_infer_before.py\n```\n\n### Fine-Tuned Model (After LoRA)\n\n```bash\npython src/lora_infer_after.py\n```\n\nExample comparison:\n\n| Prompt         | Baseline                            | LoRA Fine-Tuned                                                                                                     |\n| -------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------- |\n| Explain FASTQ  | “A text file containing sequences.” | “A FASTQ file stores sequencing reads with quality scores, used in genome sequencing and bioinformatics pipelines.” |\n| SNP annotation | “A process in genomics.”            | “SNP annotation links single-nucleotide polymorphisms to genes and predicts functional impact.”                     |\n\n---\n\n## Merge LoRA Adapter\n\nTo create a standalone fine-tuned model:\n\n```bash\npython src/merge_lora.py\n```\n\n* Output: `./results/merged-model/`\n* Load without PEFT adapters:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nmodel = AutoModelForCausalLM.from_pretrained(\"./results/merged-model\").to(\"mps\")\ntokenizer = AutoTokenizer.from_pretrained(\"./results/merged-model\")\n```\n\n---\n\n## Results\n\n* Train time: ~1 min per epoch on M4 Max\n* Loss decrease: ~10 → 0.3\n* Output improvement: Domain-specific answers with bioinformatics terms\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fman4ish%2Fbioinfo-lora-finetuning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fman4ish%2Fbioinfo-lora-finetuning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fman4ish%2Fbioinfo-lora-finetuning/lists"}