{"id":25212883,"url":"https://github.com/pranavdarshan/autograder","last_synced_at":"2026-02-23T02:36:55.266Z","repository":{"id":243673528,"uuid":"813098251","full_name":"PranavDarshan/AutoGrader","owner":"PranavDarshan","description":"The above project is used to evaluate handwritten exam answer scripts. We have fine tuned Llama2 for this task and also created a RAG pipeline to display the page of the textbook where the answer to that question is present. Finally, we have deployed this model on AWS SageMaker and created a Lambda function to handle requests from the REST API.","archived":false,"fork":false,"pushed_at":"2025-02-09T11:46:22.000Z","size":10867,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-09T12:27:24.262Z","etag":null,"topics":["aws","lambda-functions","llama2","rag","sagemaker"],"latest_commit_sha":null,"homepage":"https://ieeexplore.ieee.org/document/10817016","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PranavDarshan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-10T13:31:22.000Z","updated_at":"2025-02-09T11:46:25.000Z","dependencies_parsed_at":"2024-08-08T18:08:25.052Z","dependency_job_id":null,"html_url":"https://github.com/PranavDarshan/AutoGrader","commit_stats":null,"previous_names":["pranavdarshan/autograder"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PranavDarshan%2FAutoGrader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PranavDarshan%2FAutoGrader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PranavDarshan%2FAutoGrader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PranavDarshan%2FAutoGrader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PranavDarshan","download_url":"https://codeload.github.com/PranavDarshan/AutoGrader/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238133809,"owners_count":19421909,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","lambda-functions","llama2","rag","sagemaker"],"created_at":"2025-02-10T15:17:59.700Z","updated_at":"2026-02-23T02:36:55.205Z","avatar_url":"https://github.com/PranavDarshan.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AutoGrader\nThe above project is used to evaluate handwritten exam answer scripts. We have fine tuned Llama2 for this task and also created a RAG pipeline to display the page of the textbook where the answer to that question is present. Finally, we have deployed this model on AWS SageMaker and created a Lambda function to handle requests from the REST API. This project is fine tuned to evaluate answer scripts for the course Operating Systems only and every question is graded from 1-5. \nWe have created a whole ML Workflow from scratch. \n\u003cp align=center\u003e\n\u003cimg src=https://github.com/PranavDarshan/AutoGrader/blob/main/assets/llmworkflow.png width=500, height=300\u003e\n\u003c/p\u003e\n\n# Step 1: Dataset Generation\n\nDataset is generated by collecting questions from the course Operating Systems and the answers to each of the questions given by students and the corresponding score 1-5 is collected from the teacher. The dataset contains of around 116 entries and has 7 coloumns in which contain: \n1. Question\n2. Answer given by the student\n3. Grade given by the techer for the corresponding answer\n4. Temp - It is the input format for the LLM model excluding Llama2\n5. Text - It is the input format for fine tuning the LLM model Llama2\n6. context- This is created for the RAG pipeline\n7. text - This is the updated input format for fine tuning the LLM model Llama2 after introducing the RAG pipeline  \n\nThe dataset is found in the [Dataset.csv](https://github.com/PranavDarshan/AutoGrader/blob/main/Dataset.csv)\n\n# Step 2: Fine-Tuning Process\n\nIn this section, we outline the steps required to fine-tune the Llama 2 model with 7 billion parameters on a T4 GPU. You can use a free GPU on platforms like Google Colab or Kaggle for this process.\n\nDue to the limited 16 GB of VRAM on the Colab T4 GPU, full fine-tuning is not possible. Instead, we employ parameter-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation) or QLoRA (Quantized Low-Rank Adaptation) to manage VRAM usage effectively.\n\nSteps for Fine-Tuning\n\n1.Setup the Environment: Ensure you have access to a T4 GPU on Google Colab or Kaggle.\n\n2.Install Dependencies: Use the Hugging Face ecosystem of libraries including transformers, accelerate, peft, trl, and 3.bitsandbytes.\n\n4.Utilize [QLoRA](https://arxiv.org/abs/2305.14314) Technique: Fine-tune the model in 4-bit precision to optimize VRAM usage and maintain model performance.\n\nBy leveraging these tools and techniques, we can efficiently fine-tune the Llama 2–7b model despite hardware constraints.\n\n[read more](https://www.datacamp.com/tutorial/fine-tuning-llama-2)\n\n\n# Step 3: Creating a RAG Pipeline\n\nCheck RAG folder for more information \n\n# Step 4: Deploying Fine Tuned Model on AWS SageMaker\n\nLLaMA-2 is a massive model with billions of parameters. Running it effectively requires significant computational resources, especially memory and potentially GPUs. Most local machines may not have enough power to handle the model efficiently. Due to this reason we deploy our model on AWS SageMaker.\nCheck the [AWS-Deployment](https://github.com/PranavDarshan/AutoGrader/tree/main/AWS-Deployment) folder for more information. \n\n1. Pull the Model from HuggingFace\n2. Create an Endpoint\n3. Create a Lambda Function\n4. Create a REST API Trigger for Lambda Function\n\n# Step 5: Handwriting to Text Conversion\nHandwriting to Text Conversion is done using google vision. Already integrated in UI-UX, also present individually in vision-ocr folder. Fill in your details in vision_ocr.json.json and it is ready to use.\n\n# Step 6: Integrating with UI/UX\nAfter acquiring AWS api_url, the webapp can be run by following the steps in readme present in [UI-UX](https://github.com/PranavDarshan/AutoGrader/tree/main/UI-UX#integration-with-uiux) folder.\n# Results\n\u003cp align=center\u003e\n\u003cimg src=https://github.com/PranavDarshan/AutoGrader/blob/main/assets/ModelResult.jpg width=700, height=300\u003e\n\u003cimg src=https://github.com/PranavDarshan/AutoGrader/blob/main/assets/RAG.jpg width=700, height=700\u003e\n\u003c/p\u003e\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpranavdarshan%2Fautograder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpranavdarshan%2Fautograder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpranavdarshan%2Fautograder/lists"}