{"id":15649603,"url":"https://github.com/marella/gpt4all-j","last_synced_at":"2025-04-30T07:48:53.437Z","repository":{"id":153901947,"uuid":"629175983","full_name":"marella/gpt4all-j","owner":"marella","description":"Python bindings for the C++ port of GPT4All-J model.","archived":false,"fork":false,"pushed_at":"2023-05-15T00:28:47.000Z","size":3406,"stargazers_count":37,"open_issues_count":1,"forks_count":10,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-25T20:19:40.109Z","etag":null,"topics":["ai","gpt-j","gpt4all","gpt4all-j","llm","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marella.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-17T19:29:18.000Z","updated_at":"2025-02-16T14:28:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"27376b1f-2a9c-4a5e-b199-3e1a666b7a26","html_url":"https://github.com/marella/gpt4all-j","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marella%2Fgpt4all-j","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marella%2Fgpt4all-j/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marella%2Fgpt4all-j/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marella%2Fgpt4all-j/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marella","download_url":"https://codeload.github.com/marella/gpt4all-j/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246326759,"owners_count":20759437,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","gpt-j","gpt4all","gpt4all-j","llm","python"],"created_at":"2024-10-03T12:30:25.291Z","updated_at":"2025-03-31T13:31:17.344Z","avatar_url":"https://github.com/marella.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [GPT4All-J](https://github.com/marella/gpt4all-j) [![PyPI](https://img.shields.io/pypi/v/gpt4all-j)](https://pypi.org/project/gpt4all-j/) [![tests](https://github.com/marella/gpt4all-j/actions/workflows/tests.yml/badge.svg)](https://github.com/marella/gpt4all-j/actions/workflows/tests.yml)\n\nPython bindings for the [C++ port][gptj.cpp] of GPT4All-J model.\n\n\u003e Please migrate to [`ctransformers`](https://github.com/marella/ctransformers) library which supports more models and has more features.\n\n## Installation\n\n```sh\npip install gpt4all-j\n```\n\nDownload the model from [here](https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin).\n\n## Usage\n\n```py\nfrom gpt4allj import Model\n\nmodel = Model('/path/to/ggml-gpt4all-j.bin')\n\nprint(model.generate('AI is going to'))\n```\n\n[Run in Google Colab](https://colab.research.google.com/drive/1bd38-i1Qlx6_MvJyCTJOy7t8eHSNnqAx)\n\nIf you are getting `illegal instruction` error, try using `instructions='avx'` or `instructions='basic'`:\n\n```py\nmodel = Model('/path/to/ggml-gpt4all-j.bin', instructions='avx')\n```\n\nIf it is running slow, try building the C++ library from source. [Learn more](https://github.com/marella/gpt4all-j#c-library)\n\n### Parameters\n\n```py\nmodel.generate(prompt,\n               seed=-1,\n               n_threads=-1,\n               n_predict=200,\n               top_k=40,\n               top_p=0.9,\n               temp=0.9,\n               repeat_penalty=1.0,\n               repeat_last_n=64,\n               n_batch=8,\n               reset=True,\n               callback=None)\n```\n\n#### `reset`\n\nIf `True`, context will be reset. To keep the previous context, use `reset=False`.\n\n```py\nmodel.generate('Write code to sort numbers in Python.')\nmodel.generate('Rewrite the code in JavaScript.', reset=False)\n```\n\n#### `callback`\n\nIf a callback function is passed, it will be called once per each generated token. To stop generating more tokens, return `False` inside the callback function.\n\n```py\ndef callback(token):\n    print(token)\n\nmodel.generate('AI is going to', callback=callback)\n```\n\n## LangChain\n\n[LangChain](https://python.langchain.com/) is a framework for developing applications powered by language models. A LangChain LLM object for the GPT4All-J model can be created using:\n\n```py\nfrom gpt4allj.langchain import GPT4AllJ\n\nllm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin')\n\nprint(llm('AI is going to'))\n```\n\nIf you are getting `illegal instruction` error, try using `instructions='avx'` or `instructions='basic'`:\n\n```py\nllm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin', instructions='avx')\n```\n\nIt can be used with other LangChain modules:\n\n```py\nfrom langchain import PromptTemplate, LLMChain\n\ntemplate = \"\"\"Question: {question}\n\nAnswer:\"\"\"\n\nprompt = PromptTemplate(template=template, input_variables=['question'])\n\nllm_chain = LLMChain(prompt=prompt, llm=llm)\n\nprint(llm_chain.run('What is AI?'))\n```\n\n### Parameters\n\n```py\nllm = GPT4AllJ(model='/path/to/ggml-gpt4all-j.bin',\n               seed=-1,\n               n_threads=-1,\n               n_predict=200,\n               top_k=40,\n               top_p=0.9,\n               temp=0.9,\n               repeat_penalty=1.0,\n               repeat_last_n=64,\n               n_batch=8,\n               reset=True)\n```\n\n## C++ Library\n\nTo build the C++ library from source, please see [gptj.cpp][gptj.cpp]. Once you have built the shared libraries, you can use them as:\n\n```py\nfrom gpt4allj import Model, load_library\n\nlib = load_library('/path/to/libgptj.so', '/path/to/libggml.so')\n\nmodel = Model('/path/to/ggml-gpt4all-j.bin', lib=lib)\n```\n\n## License\n\n[MIT](https://github.com/marella/gpt4all-j/blob/main/LICENSE)\n\n[gptj.cpp]: https://github.com/marella/gptj.cpp\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarella%2Fgpt4all-j","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarella%2Fgpt4all-j","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarella%2Fgpt4all-j/lists"}