https://github.com/muavia1/deepseek-math-finetuned-for-math-meme-correcction
This project fine-tunes the DeepSeek-Math-7B model using LoRA (Low-Rank Adaptation) to correct mathematical memes and equations efficiently. By leveraging 4-bit quantization and PEFT (Parameter-Efficient Fine-Tuning), the model improves mathematical reasoning while maintaining a lightweight footprint.
https://github.com/muavia1/deepseek-math-finetuned-for-math-meme-correcction
deepseek fine-tuning finetuning-llms lora qlora
Last synced: about 2 months ago
JSON representation
This project fine-tunes the DeepSeek-Math-7B model using LoRA (Low-Rank Adaptation) to correct mathematical memes and equations efficiently. By leveraging 4-bit quantization and PEFT (Parameter-Efficient Fine-Tuning), the model improves mathematical reasoning while maintaining a lightweight footprint.
- Host: GitHub
- URL: https://github.com/muavia1/deepseek-math-finetuned-for-math-meme-correcction
- Owner: Muavia1
- Created: 2025-03-25T11:28:43.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2025-03-25T12:10:30.000Z (2 months ago)
- Last Synced: 2025-03-25T13:22:14.060Z (2 months ago)
- Topics: deepseek, fine-tuning, finetuning-llms, lora, qlora
- Language: Jupyter Notebook
- Homepage: https://medium.com/@muaviaijaz8/fine-tuning-deepseek-math-7b-with-qlora-for-math-meme-correction-ce333cf345fb
- Size: 14.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DeepSeek-Math-FineTuned-For-Math-Meme-Correcction
This project fine-tunes the DeepSeek-Math-7B model using LoRA (Low-Rank Adaptation) to enhance its ability to correct mathematical memes and equations. By applying parameter-efficient fine-tuning (PEFT) and 4-bit quantization, the model effectively identifies and corrects incorrect mathematical expressions while maintaining computational efficiency.
🚀 Features
✅ Fine-Tuned LLM: DeepSeek-Math-7B adapted for math meme correction
✅ Efficient Training: LoRA fine-tuning on attention layers for lightweight adaptation
✅ Quantization: 4-bit bitsandbytes compression for reduced memory usage
✅ Custom Dataset: Math meme corrections in CSV format
✅ Hugging Face Trainer: Used for structured and optimized training
📉 Training Performance
LoRA applied to query, key, value, and projection layers
Significant loss reduction during training, improving equation correction accuracy
Example correction:
Input: (10/5) + 3 = 8?
Output: Incorrect! Solve brackets first: (10/5) = 2, then add 3 → Correct answer: 5