{"id":22615495,"url":"https://github.com/sovit-123/chat_llama","last_synced_at":"2026-05-20T10:04:25.677Z","repository":{"id":188482194,"uuid":"678825564","full_name":"sovit-123/chat_llama","owner":"sovit-123","description":"Chat with LLama 2 models (locally)","archived":false,"fork":false,"pushed_at":"2023-10-21T14:14:18.000Z","size":35,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-12T22:03:56.903Z","etag":null,"topics":["chatbot","chatgpt","gpt","llama2","llms","nlp"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sovit-123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-15T13:22:00.000Z","updated_at":"2024-03-06T13:12:59.000Z","dependencies_parsed_at":"2023-08-15T14:44:51.369Z","dependency_job_id":"290e5a89-769c-4e3b-b17d-31529c118263","html_url":"https://github.com/sovit-123/chat_llama","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"b68bd3dd3609907f74caedfc5aaf74092c651d3d"},"previous_names":["sovit-123/chat_llama"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sovit-123/chat_llama","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sovit-123%2Fchat_llama","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sovit-123%2Fchat_llama/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sovit-123%2Fchat_llama/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sovit-123%2Fchat_llama/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sovit-123","download_url":"https://codeload.github.com/sovit-123/chat_llama/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sovit-123%2Fchat_llama/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261556511,"owners_count":23176674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","chatgpt","gpt","llama2","llms","nlp"],"created_at":"2024-12-08T19:07:45.381Z","updated_at":"2026-05-20T10:04:25.645Z","avatar_url":"https://github.com/sovit-123.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# README\n\n## Model From\n\n* The repository uses the 4 bit quantized **Llama 2 7B Chat GPTQ** model.\n\n* Model credits - https://huggingface.co/TheBloke/Llama-2-7b-Chat-GPTQ\n* License: As mentioned in the [original HuggingFace repository](https://huggingface.co/TheBloke/Llama-2-7b-Chat-GPTQ#original-model-card-metas-llama-2-7b-chat).\n\n## Steps to Run\n\n* Clone this repository\n\n```\ngit clone https://github.com/sovit-123/chat_llama.git\n```\n\n```\ncd chat_llama\n```\n\n* Install requirements\n\n```\npip install -r requirements.txt\n```\n\n* Clone the **LLama 2 7b Chat GPTQ** repository so that the model stays in the local folder instead of cache. They are quite huge.\n\n```\ngit clone https://huggingface.co/TheBloke/Llama-2-7b-Chat-GPTQ\n```\n\n* Run\n\n```\npython run.py --prompt \"python function to add two numbers\"\n```\n\nThe output:\n\n````\nGreat! I'm happy to help you with your Python function. To add two numbers in Python, you can use the `+` operator. Here's an example:\n```\ndef add_numbers(num1, num2):\n    return num1 + num2\n\n# Example usage\nresult = add_numbers(3, 5)\nprint(result) # Output: 8\n```\n````\n\n## Advanced Prompting (All results from a 10 GB RTX 3080 GPU)\n\nSuppose that you have some content in the `prompt.txt` file. A sample file is already provided with the repository. This contains a part of the DETR paper. You can replace it with your own content. Here are some prompting techniques:\n\n* Instead of a prompt, provide the text file path and set the agenda as `summarize`:\n\n```\npython run.py --prompt data/prompt.txt --system-agenda \"summarize\"\n```\n\nOutput:\n\n```\nSummary:\nThe paper presents a new method for object detection called DEtection TRansformer (DETR). DETR views object detection as a direct set prediction problem, eliminating the need for hand-designed components like non-maximum suppression or anchor generation. DETR uses a transformer encoder-decoder architecture and a set-based global loss to force unique predictions via bipartite matching. The model directly predicts the final set of detections in parallel, without requiring specialized libraries or post-processing steps. DETR demonstrates accuracy and runtime performance on par with the well-established and highly-optimized Faster R-CNN baseline on the challenging COCO object detection dataset. Additionally, DETR can be easily generalized to produce panoptic segmentation in a unified manner. The paper shows that DETR significantly outperforms competitive baselines and provides insights into the design choices that contribute to its success.\n```\n\n* The text file can be quite large and prompting the entire content to the model can result in CUDA OOM. By default the script considers only the first 11000 characters. We can increase or decrease the limit using the `--limit` argument. Higher limit gives better outputs. Set `--limit -1` to consider the all the content of the file.\n  * Following is an example to do so. No output shown as I get CUDA OOM on 10 GB GPU.\n\n```\npython run.py --prompt data/prompt.txt --system-agenda \"summarize\" --limit -1\n```\n\n* By default the model generates 512 new tokens. We can set the number of tokens using `--new-tokens`\n\n```\npython run.py --prompt data/prompt.txt --system-agenda \"summarize in 1000 words\" --new-tokens 2048\n```\n\nOutput:\n\n```\nSummary: End-to-End Object Detection with Transformers\nThe paper presents a new method for object detection that views it as a direct set prediction problem. The proposed method, called DEtection TRansformer (DETR), uses a transformer encoder-decoder architecture and a set-based global loss to directly predict the final set of detections without the need for manual feature engineering or post-processing steps. DETR demonstrates accuracy and runtime performance on par with the well-established and highly-optimized Faster R-CNN baseline on the challenging COCO object detection dataset. Additionally, DETR can be easily generalized to produce panoptic segmentation, a related task that involves assigning semantic labels to pixels in an image.\nKey contributions:\n* Proposes a new method for object detection that views it as a direct set prediction problem.\n* Introduces a transformer encoder-decoder architecture and a set-based global loss to directly predict the final set of detections.\n* Demonstrates equality of performance with the well-established and highly-optimized Faster R-CNN baseline on the challenging COCO object detection dataset.\n* Shows that DETR can be easily generalized to produce panoptic segmentation, a related task that involves assigning semantic labels to pixels in an image.\n* Provides a detailed analysis of the components of the DETR model and their impact on performance.\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsovit-123%2Fchat_llama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsovit-123%2Fchat_llama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsovit-123%2Fchat_llama/lists"}