{"id":19291583,"url":"https://github.com/debapriya-source/llama-3.1-8b-instruct","last_synced_at":"2025-02-23T23:54:24.599Z","repository":{"id":250675264,"uuid":"835132639","full_name":"Debapriya-source/llama-3.1-8B-Instruct","owner":"Debapriya-source","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-19T14:44:21.000Z","size":9,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-05T20:11:38.579Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Debapriya-source.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-29T08:20:06.000Z","updated_at":"2024-08-19T14:44:24.000Z","dependencies_parsed_at":"2024-08-01T20:59:24.023Z","dependency_job_id":"3ed4f29b-d49f-4cab-993e-66aaa2eba7a2","html_url":"https://github.com/Debapriya-source/llama-3.1-8B-Instruct","commit_stats":null,"previous_names":["debapriya-source/nlp-projects"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-8B-Instruct","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-8B-Instruct/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-8B-Instruct/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-8B-Instruct/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Debapriya-source","download_url":"https://codeload.github.com/Debapriya-source/llama-3.1-8B-Instruct/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240395739,"owners_count":19794573,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:26:19.371Z","updated_at":"2025-02-23T23:54:24.563Z","avatar_url":"https://github.com/Debapriya-source.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Llama-3.1 model test offline\n\n## Introduction\nThe latest Llama🦙 (Large Language Model Meta AI) 3.1 is a powerful AI model developed by Meta AI that has gained significant attention in the natural language processing (NLP) community. It is the most capable open-source llm till date. In this blog, I will guide you through the process of cloning the Llama 3.1 model from Hugging Face🤗 and running it on your local machine using Python. After which you can integrate it in any AI project. \n\n---\n\n## Prerequisites\n\n- Python 3.8 or higher installed on your local machine\n- Hugging Face Transformers library installed (`pip install transformers`)\n- Git installed on your local machine\n- A Hugging Face account\n\n---\n\n\n## Step 1: Get access to the model\n- Click here [https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) to open the official hugging face repository of Meta's Llama-3.1-8B-Instruct (you can use other llama 3.1 models in the same way).\n\n![Meta-llama-3.1-8b-Instruct hugging face model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o0lpys2y71s4cte30ex7.png)\n\n- At the beginning you should be seeing this:\n\n![Meta-llama-3.1-8b-Instruct model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gky1nxjvf28r46cuiwtp.png)\n\n- Submit the form below to get access of the model\n\n![access to meta llama 3.1 model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/00byphoel5c8s0nfa7xw.png)\n\n- Once you see **\"You have been granted access to this model\"**, you are good to go...\n\n![gated model in hugging face](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nsttr9arstf3s7r2lp2t.png)\n\n\n\n\n## Step 2: Create an ACCESS_TOKEN\n- Go to **\"Settings\"** (Bottom right corner of the below image):\n\n![hugging face settings](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8mfn10g0l952jn7drses.png)\n\n- Go to **\"Access Tokens\"** click **\"Create new token\"**(upper right corner of the image):\n\n![create hugging face token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o43nhk25cg8e57i6u1r3.png)\n\n- Give read and write permissions and **select the repo** as shown:\n\n![create hugging face token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g15k9gxmp8vi8gdbx3cw.png)\n\n\n- Copy the token and place it somewhere safe and secure as it will be needed in the future.(**note:** once you copy it you cannot copy it again, so if you anyhow forget the key, you have to create a new one to begin with :))\n\n![huggingface token](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/175lizyojzlbkzy95pxw.png)\n\n\n---\n\n\n## Step 3: Clone the LLaMA 3.1 Model\n\nNow run the following command on your favorite terminal. \nThe `ACCESS_TOKEN` is the one you copied and the `\u003chuggingface-user-name\u003e` is the username of your hugging face account.\n\n```Bash\ngit clone https://\u003chuggingface-user-name\u003e:\u003cACCESS_TOKEN\u003e@huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct\n```\n\nThis can take a lot of time depending on your internet speed.\n\n## Step 4: Install Required Libraries\n\nOnce the cloning is done, go to the cloned folder and install all the dependencies from the `requirements.txt`. (you can create an virtual-environment using conda(**recommended**) or virtualenv)\nYou can find out the requirements file in my GitHub provided in the **resources** section below.\n\nUsing **conda**:\n```Bash\ncd Meta-Llama-3.1-8B-Instruct\nconda install --yes --file requirements.txt\n```\nUsing **pip**:\n```Bash\ncd Meta-Llama-3.1-8B-Instruct\npip install -r requirements.txt\n```\n\n---\n\n## Step 5: Run the Llama 3.1 Model\nCreate a new Python file (e.g., test.py) and paste the location of the model repository you just cloned as the `model_id` (such as, `\"D:\\\\Codes\\\\NLP\\\\Meta-Llama-3.1-8B-Instruct\"`). Here is an example:\n\n```Python\nimport transformers\nimport torch\n\n## Here you paste your cloned repos location\nmodel_id = \"D:\\\\Codes\\\\NLP\\\\Meta-Llama-3.1-8B-Instruct\" \n\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=model_id,\n    model_kwargs={\"torch_dtype\": torch.bfloat16},\n    device_map=\"auto\",\n)\n\nmessages = [\n    {\"role\": \"user\", \"content\": \"Who are you?\"},\n]\n\noutputs = pipeline(\n    messages,\n    max_new_tokens=256,\n)\nprint(outputs[0][\"generated_text\"][-1])\n```\nYou can set `device_map=cuda` if you want use the gpu also.\n\nStep 6: Run the Python Script\n\n```Bash\npython test.py\n```\n### Output\n\n![llama-3.1 output](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s2145xgibavplg0bkjvp.png)\n\n---\n\n\n## Issues you can face\n\n- `OSError: [WinError 126] fbgemm.dll` \n   - To solve this error make sure you have **Visual Studio** installed.\n     - In case you don't have it, [click here](https://visualstudio.microsoft.com/vs/community/) and install it.\n     - Then restart the computer.\n- If there is still any errors with pytorch versions, use [anaconda](https://www.anaconda.com/) or [miniconda](https://docs.anaconda.com/miniconda/miniconda-install/) to configure a new environment with suitable python version and dependencies.\n- If you are facing any other issue or error feel free to comment below.\n\n---\n\n\u003e ## Resources\n\u003e For more details on llama 3.1 check out: [https://ai.meta.com/blog/meta-llama-3-1/](https://ai.meta.com/blog/meta-llama-3-1/)\n\n\u003e My implementation [https://github.com/Debapriya-source/llama-3.1-8B-Instruct.git](https://github.com/Debapriya-source/llama-3.1-8B-Instruct.git) \n\n---\n\n## Conclusion\nIn this blog, we have successfully cloned the LLaMA-3.1-8B-Instruct model from Hugging Face and run it on our local machine using Python. You can now experiment with the model by modifying the prompt, adjusting hyperparameters, or integrate with your upcoming projects. Happy coding!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebapriya-source%2Fllama-3.1-8b-instruct","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdebapriya-source%2Fllama-3.1-8b-instruct","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebapriya-source%2Fllama-3.1-8b-instruct/lists"}