{"id":15433166,"url":"https://github.com/simonw/llm-gpt4all","last_synced_at":"2025-04-04T17:10:32.818Z","repository":{"id":179970477,"uuid":"664386158","full_name":"simonw/llm-gpt4all","owner":"simonw","description":"Plugin for LLM adding support for the GPT4All collection of models","archived":false,"fork":false,"pushed_at":"2024-05-10T23:13:38.000Z","size":58,"stargazers_count":246,"open_issues_count":29,"forks_count":24,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-28T16:11:09.607Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"simonw"}},"created_at":"2023-07-09T20:24:11.000Z","updated_at":"2025-03-22T22:36:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"4f46ca75-00fb-4a53-80fa-db7877dbdf4e","html_url":"https://github.com/simonw/llm-gpt4all","commit_stats":{"total_commits":27,"total_committers":5,"mean_commits":5.4,"dds":"0.18518518518518523","last_synced_commit":"363559a3accd49c5c0757a1bc843e0376c902bf2"},"previous_names":["simonw/llm-gpt4all"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fllm-gpt4all","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fllm-gpt4all/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fllm-gpt4all/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fllm-gpt4all/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonw","download_url":"https://codeload.github.com/simonw/llm-gpt4all/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247217222,"owners_count":20903009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T18:32:15.983Z","updated_at":"2025-04-04T17:10:32.801Z","avatar_url":"https://github.com/simonw.png","language":"Python","funding_links":["https://github.com/sponsors/simonw"],"categories":[],"sub_categories":[],"readme":"# llm-gpt4all\n\n[![PyPI](https://img.shields.io/pypi/v/llm-gpt4all.svg)](https://pypi.org/project/llm-gpt4all/)\n[![Changelog](https://img.shields.io/github/v/release/simonw/llm-gpt4all?include_prereleases\u0026label=changelog)](https://github.com/simonw/llm-gpt4all/releases)\n[![Tests](https://github.com/simonw/llm-gpt4all/workflows/Test/badge.svg)](https://github.com/simonw/llm-gpt4all/actions?query=workflow%3ATest)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/simonw/llm-gpt4all/blob/main/LICENSE)\n\nPlugin for [LLM](https://llm.datasette.io/) adding support for the [GPT4All](https://gpt4all.io/) collection of models.\n\n## Installation\n\nInstall this plugin in the same environment as LLM.\n```bash\nllm install llm-gpt4all\n```\nAfter installing the plugin you can see a new list of available models like this:\n\n```bash\nllm models list\n```\nThe output will include something like this:\n```\ngpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM (installed)\ngpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM (installed)\ngpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM (installed)\ngpt4all: mistral-7b-openorca - Mistral OpenOrca, 3.83GB download, needs 8GB RAM (installed)\ngpt4all: Meta-Llama-3-8B-Instruct - Llama 3 Instruct, 4.34GB download, needs 8GB RAM (installed) (aliases: l3)\ngpt4all: all-MiniLM-L6-v2 - SBert, 43.82MB download, needs 1GB RAM\ngpt4all: nomic-embed-text-v1 - Nomic Embed Text v1, 261.58MB download, needs 1GB RAM\ngpt4all: nomic-embed-text-v1 - Nomic Embed Text v1.5, 261.58MB download, needs 1GB RAM\ngpt4all: replit-code-v1_5-3b-newbpe-q4_0 - Replit, 1.82GB download, needs 4GB RAM\ngpt4all: mpt-7b-chat - MPT Chat, 3.54GB download, needs 8GB RAM\ngpt4all: orca-2-7b - Orca 2 (Medium), 3.56GB download, needs 8GB RAM\ngpt4all: rift-coder-v0-7b-q4_0 - Rift coder, 3.56GB download, needs 8GB RAM\ngpt4all: mpt-7b-chat-newbpe-q4_0 - MPT Chat, 3.64GB download, needs 8GB RAM\ngpt4all: em_german_mistral_v01 - EM German Mistral, 3.83GB download, needs 8GB RAM\ngpt4all: ghost-7b-v0 - Ghost 7B v0.9.1, 3.83GB download, needs 8GB RAM\ngpt4all: Nous-Hermes-2-Mistral-7B-DPO - Nous Hermes 2 Mistral DPO, 3.83GB download, needs 8GB RAM\ngpt4all: gpt4all-falcon-newbpe-q4_0 - GPT4All Falcon, 3.92GB download, needs 8GB RAM\ngpt4all: gpt4all-13b-snoozy-q4_0 - Snoozy, 6.86GB download, needs 16GB RAM\ngpt4all: wizardlm-13b-v1 - Wizard v1.2, 6.86GB download, needs 16GB RAM\ngpt4all: orca-2-13b - Orca 2 (Full), 6.86GB download, needs 16GB RAM\ngpt4all: nous-hermes-llama2-13b - Hermes, 6.86GB download, needs 16GB RAM\ngpt4all: starcoder-newbpe-q4_0 - Starcoder, 8.37GB download, needs 4GB RAM\n```\nFurther details on these models can be found [in this Observable notebook](https://observablehq.com/@simonw/gpt4all-models).\n\n## Usage\n\n### Model Installation and Prompting\n\nYou can execute a model using the name displayed in the `llm models list` output. The model file will be downloaded the first time you attempt to run it.\n\n```bash\nllm -m orca-mini-3b-gguf2-q4_0 '3 names for a pet cow'\n```\nThe first time you run this you will see a progress bar for the model file download:\n```\n 31%|█████████▋                        | 1.16G/3.79G [00:26\u003c01:02, 42.0MiB/s]\n```\nOn subsequent uses the model output will be displayed immediately.\n\nNote that the models will be downloaded to `~/.cache/gpt4all`.\n\n### Model options\n\nRun `llm models --options` for a list of available model options, which should include:\n\n```\ngpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM (installed)\n  max_tokens: int\n    The maximum number of tokens to generate.\n  temp: float\n    The model temperature. Larger values increase creativity but decrease\n    factuality.\n  top_k: int\n    Randomly sample from the top_k most likely tokens at each generation\n    step. Set this to 1 for greedy decoding.\n  top_p: float\n    Randomly sample at each generation step from the top most likely\n    tokens whose probabilities add up to top_p.\n  repeat_penalty: float\n    Penalize the model for repetition. Higher values result in less\n    repetition.\n  repeat_last_n: int\n    How far in the models generation history to apply the repeat penalty.\n  n_batch: int\n    Number of prompt tokens processed in parallel. Larger values decrease\n    latency but increase resource requirements.\n```\nUse them like this:\n\n```bash\nllm -m mistral-7b-instruct-v0 -o max_tokens 2 'hi'\n```\n```\nHello!\n```\n\n### Chatting\n\nTo chat with a model, avoiding the need to load it into memory for every message, use `llm chat`:\n\n```bash\nllm chat -m orca-mini-3b-gguf2-q4_0\n```\n```\nChatting with orca-mini-3b-gguf2-q4_0\nType 'exit' or 'quit' to exit\nType '!multi' to enter multiple lines, then '!end' to finish\n\u003e hi\n Hello! How can I assist you today?\n\u003e three jokes about a pelican and a chicken who are friends\n Sure, here are three jokes about a pelican and a chicken who are friends:\n\n1. Why did the pelican cross the road? To get to the other side where the chicken was waiting for him!\n2. What do you call a group of chickens playing basketball? A flock of feathers!\n3. Why did the chicken invite the pelican over for dinner? Because it had nothing else to eat and needed some extra protein in its diet!\n```\n\n### Removing Models\n\nTo remove a downloaded model, delete the `.gguf` file from `~/.cache/gpt4all`.\n\n## Development\n\nTo set up this plugin locally, first checkout the code. Then create a new virtual environment:\n```bash\ncd llm-gpt4all\npython3 -m venv venv\nsource venv/bin/activate\n```\nNow install the dependencies and test dependencies:\n```bash\npip install -e '.[test]'\n```\nTo run the tests:\n```bash\npytest\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fllm-gpt4all","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonw%2Fllm-gpt4all","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fllm-gpt4all/lists"}