{"id":20229248,"url":"https://github.com/rahulunair/sql_llm","last_synced_at":"2025-04-10T17:51:06.819Z","repository":{"id":204570059,"uuid":"712157496","full_name":"rahulunair/sql_llm","owner":"rahulunair","description":"Finetune an LLM to generate SQL from text on Intel GPUs (XPUs) using QLoRA","archived":false,"fork":false,"pushed_at":"2023-12-04T05:38:25.000Z","size":1652,"stargazers_count":11,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-07-26T15:38:51.884Z","etag":null,"topics":["bigdl","intel-gpu","intel-gpu-max","llama","llm-finetuning","qlora","xpu"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rahulunair.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-30T22:58:16.000Z","updated_at":"2024-06-27T04:37:31.000Z","dependencies_parsed_at":"2023-11-27T22:39:46.046Z","dependency_job_id":null,"html_url":"https://github.com/rahulunair/sql_llm","commit_stats":null,"previous_names":["rahulunair/sql_llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulunair%2Fsql_llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulunair%2Fsql_llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulunair%2Fsql_llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rahulunair%2Fsql_llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rahulunair","download_url":"https://codeload.github.com/rahulunair/sql_llm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224586264,"owners_count":17335947,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigdl","intel-gpu","intel-gpu-max","llama","llm-finetuning","qlora","xpu"],"created_at":"2024-11-14T07:34:59.076Z","updated_at":"2024-11-14T07:34:59.556Z","avatar_url":"https://github.com/rahulunair.png","language":"Jupyter Notebook","readme":"## Text-to-SQL Generation Using Fine-tuned LLMs on Intel GPUs(XPUs) and QLoRA.\n\n\u003cimg src=\"https://github.com/rahulunair/sql_llm/assets/786476/8353bb33-bda7-47fe-bbc2-0214ce1e2395\" width=\"350\"\u003e\n\nThis repository includes code for fine-tuning a Language Model for text-to-SQL tasks and for generating SQL queries with the fine-tuned model. Both the fine-tuning and generation processes leverage QLoRA, a Quantized Low-Rank Parameter Efficient finetuning method, enabled by [Intel's BigDL](https://github.com/intel-analytics/BigDL/tree/main/python/llm/example/GPU/QLoRA-FineTuning) library on Intel GPUs.\n\n![lora_adapters_v2(1)](https://github.com/rahulunair/sql_llm/assets/786476/c30d7fb4-2051-428c-9c55-fc4130cb11bc)\n\n### Prerequisites\n\n- Python 3.x\n- PyTorch\n- Transformers library\n- Datasets library\n- Intel Extension for PyTorch (IPEX)\n- Intel BigDL-LLM[XPU]\n\n### Installation\n\n1. Clone this repo.\n\n```bash\ngit clone https://github.com/your_username/your_repository.git\n```\n\n2. Install required python packages\n\n```bash\npip install -r requirements\n```\n\n3. Install Intel BigDL llm package\n\n```bash\npip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu\n```\n\n### File Descriptions\n\n- **finetune.py** : Contains code for fine-tuning a pre-trained Language Model on text-to-SQL tasks.\n- **generate.py** : Contains code for generating SQL queries using a fine-tuned model.\n\n### Fine-Tuning a Model (finetune.py)\n\nTo finetune a model, run the `finetune.py` script\n\n```bash\npython finetune.py\n```\n\n```bash\n============================================================\nTraining Parameters:\nFoundation model:         NousResearch/CodeLlama-7b-hf\nModel save path:          ./final_model\nDevice used:              xpu\nIntel GPU:                Intel(R) Data Center GPU Max 1100\nBatch size per device:    32\nGradient accum. steps:    4\nWarmup steps:             100\nSave steps:               20\nEvaluation steps:         20\nMax steps:                300\nLearning rate:            0.0003\nMax gradient norm:        0.3\nSave total limit:         3\nLogging steps:            20\n============================================================\n```\n\u003cimg src=\"https://github.com/rahulunair/sql_llm/assets/786476/225935e6-b36a-4633-8bb6-b2ab8c32ef6a\" width=\"600\"\u003e\n\nHere is how the loss chart looks at the end of 300 steps of finetuning:\n\nAs you can see the loss has a big drop in the intial steps and training loss gradually tapers to around 0.6:\n\n\u003cimg width=\"600\" alt=\"loss_chart\" src=\"https://github.com/rahulunair/sql_llm/assets/786476/0c86bf02-93d6-47da-be34-b09d39e6ffea\"\u003e\n\n#### Key Features:\n\n- Downloads a pre-trained model based on the given base model ID.\n- Tokenizes the input questions, context, and answers.\n- Fine-tunes the model using the tokenized data and qLoRA.\n- Saves the fine-tuned model.\n\n#### Configuration:\n\n- BASE_MODEL: The pre-trained model to use for fine-tuning.\n- MODEL_PATH: Path to save the fine-tuned model.\n- DEVICE: Device to run the model on.\n\n### SQL Query Generation (generate.py)\n\nTo generate SQL queries using the fine-tuned model, run the generate.py script.\n\n#### Key Features:\n\n- Uses either the base model or a fine-tuned model for SQL query generation.\n- Loads sample data and generates SQL queries for each sample.\n\n#### Configuration:\n\n- BASE_MODEL: The base model to use for inference.\n- MODEL_PATH: Path to the fine-tuned model.\n- LORA_CHECKPOINT: Latest checkpoint for the fine-tuned model.\n- TEST_DATA: Path to the test data file.\n\nFollowing a 15-minute training session, the finetuned model demonstrates enhanced proficiency in generating SQL queries that more accurately reflect the given questions, compared to the base model. With additional training steps, we can anticipate further improvements in the model's response accuracy:\n\nFinetuned model generation:\n\n\u003cimg width=\"600\" src=\"https://github.com/rahulunair/sql_llm/assets/786476/1edf44ea-557b-4156-be2b-bf2716e2b4a5\"\u003e\n\nBase model generation:\n\n\u003cimg width=\"600\" src=\"https://github.com/rahulunair/sql_llm/assets/786476/f873f868-73e9-4bcf-861f-94cdefe34fc1\"\u003e\n\n### Default Configurations\n\n#### Model\n\n- Default base model for fine-tuning: **openlm-research/open_llama_3b**\n- Model path for saving the fine-tuned LoRA adaptor (incase of interruptions): `./saved_model`\n- Path for saving task based (here it is text to sql) LoRA adaptors: `./lora_models`\n\n#### Dataset\n\n- Default dataset for fine-tuning: **b-mc2/sql-create-context**\n\n\n### Contributing\n\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulunair%2Fsql_llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frahulunair%2Fsql_llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frahulunair%2Fsql_llm/lists"}