https://github.com/thiraput01/qwenmed
Qwen3 fine-tuned on medical datasets with reasoning data
https://github.com/thiraput01/qwenmed
llm parameter-efficient-fine-tuning unsloth
Last synced: about 1 year ago
JSON representation
Qwen3 fine-tuned on medical datasets with reasoning data
- Host: GitHub
- URL: https://github.com/thiraput01/qwenmed
- Owner: Thiraput01
- Created: 2025-06-21T17:31:41.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-21T18:28:04.000Z (about 1 year ago)
- Last Synced: 2025-06-21T18:34:49.365Z (about 1 year ago)
- Topics: llm, parameter-efficient-fine-tuning, unsloth
- Language: Jupyter Notebook
- Homepage: https://huggingface.co/Thiraput01/QwenMed-1.7B-Reasoning
- Size: 0 Bytes
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# QwenMed
QwenMed is a Large Language Model (LLM) fine-tuned on the Qwen3-1.7B model using Unsloth for QLoRA for medical question answering.
---
## Features
* **Medical Question Answering**: Provides responses to medical queries.
* **Reasoning Capability**: Can generate detailed thought processes (Chain-of-Thought) before providing a concise answer.
* **Efficient Fine-tuning**: Unsloth library for PEFT.
* **Hugging Face Integration**: Models are pushed to and loaded from Hugging Face Hub.
## Usage
The `notebooks/QwenMed_inference.ipynb` notebook demonstrates how to load the model and perform inference.
You can switch between non-thinking and reasoning modes to observe different response styles.
**For Non-Thinking**
```python
from unsloth import FastLanguageModel
import torch
from transformers import TextStreamer
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Thiraput01/QwenMed-1.7B-Reasoning",
max_seq_length = 2048,
load_in_4bit = True,
)
messages = [
{"role" : "user", "content" : "I'm having chest pain, what could be the cause?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
enable_thinking = False, # Disable thinking
)
_ = model.generate(
**tokenizer(text, return_tensors = "pt").to("cuda"),
max_new_tokens = 256,
temperature = 0.7, top_p = 0.8, top_k = 20,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)
```
**For Thinking (Reasoning) inference**
```python
from unsloth import FastLanguageModel
import torch
from transformers import TextStreamer
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Thiraput01/QwenMed-1.7B-Reasoning",
max_seq_length = 2048,
load_in_4bit = True,
)
messages = [
{"role" : "user", "content" : "I'm having a headace, what do you think?"}
]
text = tokenizer.apply_chat_template(
messages,
tokenize = False,
add_generation_prompt = True,
enable_thinking = True, # Enable thinking
)
_ = model.generate(
**tokenizer(text, return_tensors = "pt").to("cuda"),
max_new_tokens = 2048,
temperature = 0.6, top_p = 0.95, top_k = 20,
streamer = TextStreamer(tokenizer, skip_prompt = True),
)
```
# Training
The `notebooks/QwenMed_train.ipynb` notebook provides the process for fine-tuning the QwenMed model.
# Dataset Used
- [FreedomIntelligence/medical-o1-reasoning-SFT](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT)
- [Laurent1/MedQuad-MedicalQnADataset_128tokens_max](https://huggingface.co/datasets/Laurent1/MedQuad-MedicalQnADataset_128tokens_max)
# Training Configuration
- Base Model: `unsloth/Qwen3-1.7B-unsloth-bnb-4bit`
- Epochs: `2`
- Learning Rate: `5e-5`
- Scheduler: `CosineAnnealingLR`
- LoRA Parameters: `r=32, alpha=64`
- Batch Size: `2`
- Gradient Accumulation Steps = `8`
# Results
**Training results**
**Validation results**
