Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/opencodeiiita/finetuning_llama
Fine-Tuning LLaMA for Indian Laws
https://github.com/opencodeiiita/finetuning_llama
llm opencode24 python pytorch tensorboard transformers
Last synced: 19 days ago
JSON representation
Fine-Tuning LLaMA for Indian Laws
- Host: GitHub
- URL: https://github.com/opencodeiiita/finetuning_llama
- Owner: opencodeiiita
- Created: 2024-12-13T16:01:58.000Z (27 days ago)
- Default Branch: main
- Last Pushed: 2024-12-15T18:51:37.000Z (25 days ago)
- Last Synced: 2024-12-15T19:33:58.440Z (25 days ago)
- Topics: llm, opencode24, python, pytorch, tensorboard, transformers
- Language: Jupyter Notebook
- Homepage:
- Size: 5.86 KB
- Stars: 0
- Watchers: 1
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Finetuning_Llama
Fine-tuning large language models like LLaMA has transformed the way we adapt pre-trained models for specialized tasks. This repository focuses on parameter-efficient fine-tuning techniques such as LoRA and QLoRA to adapt the LLaMA2-7B model to Indian legal text datasets.
## Problem Statement
You are tasked with fine-tuning the LLaMA2-7B model on a dataset related to Indian laws to make it capable of generating context-aware legal insights. The challenge is to leverage advanced fine-tuning techniques like LoRA/QLoRA to optimize the training process while keeping computational requirements minimal. Demonstrate your skills in model tuning and deployment!
## Instructions
- Refer to articles, research papers, and official documentation for guidance on techniques and best practices.
- Do not alter any pre-written code or comments.
- Write code only in the provided space and document your steps with comments for better understanding.
- Use Google Colab or similar GPU-enabled environments for training and testing the model.
- Help
- For any queries or support, feel free to reach out via email at [email protected] or [email protected] or join the discussion on the project’s Discord server.
## Contributions
- Contributions are welcome! Follow these steps:
- Fork this repository and clone it to your local device.
- Work on individual tasks in a separate branch.
- Push your updates to the forked repo and create a Pull Request (PR).
- Your PR will be reviewed, and upon approval, merged into the main repository.
## Resources
- Dataset: Indian Law Dataset (https://huggingface.co/datasets/jizzu/llama2_indian_law_v2)
- Parameter-Efficient Fine-Tuning: LoRA Paper (https://arxiv.org/pdf/1902.00751)
- Hugging Face Transformers Documentation: Link(https://huggingface.co/docs/transformers/index)
## Happy Fine-Tuning!