{"id":18839675,"url":"https://github.com/eliashornberg/epfllama","last_synced_at":"2026-05-05T02:38:08.086Z","repository":{"id":257278088,"uuid":"857800165","full_name":"eliashornberg/EPFLLaMA","owner":"eliashornberg","description":"EPFLLaMA: A lightweight language model fine-tuned on EPFL curriculum content. Specialized for STEM education and multiple-choice question answering. Implements advanced techniques like SFT, DPO, and quantization.","archived":false,"fork":false,"pushed_at":"2024-09-15T16:44:01.000Z","size":29430,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-29T20:39:26.587Z","etag":null,"topics":["artificial-intelligence","direct-preference-optimization","large-language-models","lora","natural-language-processing","pytorch","supervised-finetuning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eliashornberg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-15T16:36:37.000Z","updated_at":"2024-09-17T07:42:18.000Z","dependencies_parsed_at":"2024-09-15T18:21:29.766Z","dependency_job_id":null,"html_url":"https://github.com/eliashornberg/EPFLLaMA","commit_stats":null,"previous_names":["eliashornberg/epfllama"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eliashornberg/EPFLLaMA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliashornberg%2FEPFLLaMA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliashornberg%2FEPFLLaMA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliashornberg%2FEPFLLaMA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliashornberg%2FEPFLLaMA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eliashornberg","download_url":"https://codeload.github.com/eliashornberg/EPFLLaMA/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eliashornberg%2FEPFLLaMA/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32633437,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"online","status_checked_at":"2026-05-05T02:00:06.033Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","direct-preference-optimization","large-language-models","lora","natural-language-processing","pytorch","supervised-finetuning"],"created_at":"2024-11-08T02:43:46.975Z","updated_at":"2026-05-05T02:38:08.052Z","avatar_url":"https://github.com/eliashornberg.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EPFLLaMA: A Lightweight LLM Finetuned on EPFL Curriculum\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"753\" alt=\"EPFLLama\" src=\"https://github.com/user-attachments/assets/4d681a49-d658-46d5-afdc-f8b2504f807c\"\u003e\n\u003c/p\u003e\n\n\n\n## Project Overview\n\nEPFLLaMA is a project that enhances the TinyLlama model through Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), utilizing datasets from student annotations and Stack Exchange. The project aims to create an AI tutor specialized in EPFL course content, with a particular focus on Multiple-Choice Question Answering (MCQA) tasks.\n\n### Key Features\n\n- Specialized for EPFL curriculum content\n- Enhanced performance on MCQA tasks\n- Utilizes advanced techniques: SFT, DPO, and Chain-of-Thought prompting\n- Incorporates quantization for reduced memory footprint\n\n## Project Structure\n```\nEPFLLAMA\n│\n├── README.md\n│\n├── data\n│   ├── all_datasets\n│   │   ├── MCQA_DPO.jsonl\n│   │   ├── MCQA_unique_data.jsonl\n│   │   ├── merged_DPO_test.jsonl\n│   │   ├── merged_DPO_train.jsonl\n│   │   └── sft_2000.jsonl\n│   │\n│   ├── annotated\n│   │   ├── 383057.json\n│   │   └── 384928.json\n│   │\n│   └── annotation_scripts\n│       ├── DPO_Annotation.py\n│       └── MCQA_Annotation.py\n│\n├── model\n│   ├── dataset_example\n│   │   ├── dpo_preference_example.jsonl\n│   │   ├── mcqa_example.jsonl\n│   │   └── MCQA_sft_test.jsonl\n│   │\n│   ├── models\n│   │   ├── model_base.py\n│   │   └── model_dpo.py\n│   │\n│   ├── Create_Loss_Plots.ipynb\n│   ├── data_processing.ipynb\n│   ├── Evaluate.ipynb\n│   ├── evaluator.py\n│   ├── main_config.yaml\n│   ├── requirements.txt\n│   ├── Training.ipynb\n│   └── utils.py\n│\n├── pdfs\n│   ├── litterature_reviews\n│   │   ├── 383057.pdf\n│   │   └── 384928.pdf\n│   │\n│   ├── progress_report\n│   │   ├── ab-eh-me.pdf\n│   │\n│   ├── project_proposal\n│   │   ├── ab_eh_me.pdf\n│   │\n│   └── project_report\n│       ├── ab-eh-me.pdf\n\n```\n## Data Collection and Preparation\n\nThe project utilizes various data sources:\n\n1. Student-annotated data from EPFL curricula\n2. Stack Exchange datasets (Data Science, Computer Science, Physics, Mathematics)\n3. GPT-generated preference pairs\n\nData collection scripts can be found in `model/models/data_processing.ipynb` and annotation scripts in `data/annotation_scripts`.\n\n## Model Architecture\n\nEPFLLaMA is based on the TinyLlama architecture, a compact and efficient language model with 1.1 billion parameters. It incorporates:\n\n- 22 layers with 32 attention heads each\n- Grouped-query attention mechanism\n- RoPE (Rotary Positional Embedding)\n- SwiGLU activation function\n\n## Training Process\n\nThe training process involves two main phases:\n\n1. **Supervised Fine-Tuning (SFT)**: Using the SFTTrainer from the trl library.\n2. **Direct Preference Optimization (DPO)**: Implementing the DPO loss function to align the model with human preferences.\n\nAdditionally, the project explores:\n\n- Parameter-Efficient Fine-Tuning (PEFT)\n- Low-Rank Adaptation (LoRA)\n- Quantization techniques\n\n## Model Improvements\n\nThe project implements one main improvement:\n\n**Quantization**: Reduces the model size while maintaining performance, using techniques like LLM.int8().\n\n## Results and Evaluation\n\nThe EPFLLaMA model demonstrates:\n\n- Improved performance on MCQA tasks compared to baselines\n- Robust performance across various technical subjects\n- Effective adaptation for educational purposes\n\nDetailed results and analysis can be found in the project report (`pdfs/project_report/ab-eh-me.pdf`).\n\n## Getting Started\n\nTo use or contribute to this project:\n\n1. Clone the repository\n2. Install dependencies: `pip install -r model/models/requirements.txt`\n3. Explore the Jupyter notebooks in `model/models/` for training and evaluation\n\nTo use our pre-trained models:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load the base model\nmodel = AutoModelForCausalLM.from_pretrained(\"Ali7538/EPFLLaMA\")\ntokenizer = AutoTokenizer.from_pretrained(\"Ali7538/EPFLLaMA\")\n\n# For MCQA tasks\nmcqa_model = AutoModelForCausalLM.from_pretrained(\"Ali7538/EPFLLaMA_MCQA\")\nmcqa_tokenizer = AutoTokenizer.from_pretrained(\"Ali7538/EPFLLaMA_MCQA\")\n\n# For faster inference with the quantized model\nquantized_model = AutoModelForCausalLM.from_pretrained(\"Ali7538/EPFLLaMA_MCQA_Quantized\")\nquantized_tokenizer = AutoTokenizer.from_pretrained(\"Ali7538/EPFLLaMA_MCQA_Quantized\")\n```\n\n## Contributors\n\n- Elias Hörnberg\n- Ali Bakly\n- Othmane Sqalli Houssaini\n\n## Acknowledgments\n\nThis project was developed as part of the CS-552 course at EPFL. Special thanks to the course staff and the NLP lab for providing resources and guidance.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliashornberg%2Fepfllama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feliashornberg%2Fepfllama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feliashornberg%2Fepfllama/lists"}