{"id":13452992,"url":"https://github.com/withcatai/catai","last_synced_at":"2025-10-07T23:32:57.121Z","repository":{"id":148261362,"uuid":"618875095","full_name":"withcatai/catai","owner":"withcatai","description":"Run AI ✨ assistant locally! with simple API for Node.js 🚀","archived":false,"fork":false,"pushed_at":"2024-06-21T17:21:54.000Z","size":21383,"stargazers_count":477,"open_issues_count":2,"forks_count":40,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-10-07T14:42:28.380Z","etag":null,"topics":["ai","ai-assistant","catai","chatbot","chatgpt","chatui","dalai","ggmlv3","gguf","llama-cpp","llm","local-llm","localai","node-llama-cpp","nodejs","openai","vicuna","vicuna-installation-guide","wizardlm"],"latest_commit_sha":null,"homepage":"https://withcatai.github.io/catai/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/withcatai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"ido-pluto"}},"created_at":"2023-03-25T15:53:57.000Z","updated_at":"2025-10-04T15:41:40.000Z","dependencies_parsed_at":"2023-08-12T22:41:18.592Z","dependency_job_id":"9432592d-fd18-49e0-a00a-1b2097fe1343","html_url":"https://github.com/withcatai/catai","commit_stats":{"total_commits":185,"total_committers":6,"mean_commits":"30.833333333333332","dds":0.07567567567567568,"last_synced_commit":"64f7e68be71864bfc2a2804dd72775dc533a0f4e"},"previous_names":["withcatai/catai","ido-pluto/catai"],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/withcatai/catai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/withcatai%2Fcatai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/withcatai%2Fcatai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/withcatai%2Fcatai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/withcatai%2Fcatai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/withcatai","download_url":"https://codeload.github.com/withcatai/catai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/withcatai%2Fcatai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278861087,"owners_count":26058641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-assistant","catai","chatbot","chatgpt","chatui","dalai","ggmlv3","gguf","llama-cpp","llm","local-llm","localai","node-llama-cpp","nodejs","openai","vicuna","vicuna-installation-guide","wizardlm"],"created_at":"2024-07-31T08:00:30.241Z","updated_at":"2025-10-07T23:32:57.082Z","avatar_url":"https://github.com/withcatai.png","language":"TypeScript","funding_links":["https://github.com/sponsors/ido-pluto"],"categories":["Open-Source Local LLM Projects","Chatbots","chatgpt"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n    \u003cimg alt=\"CatAI Logo\" src=\"docs/demo/logo.png\" width=\"360px\"/\u003e\n    \u003ch1\u003eCatAI\u003c/h1\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![Build](https://github.com/withcatai/catai/actions/workflows/release.yml/badge.svg)](https://github.com/withcatai/catai/actions/workflows/build.yml)\n[![License](https://badgen.net/badge/color/MIT/green?label=license)](https://www.npmjs.com/package/catai)\n[![License](https://badgen.net/badge/color/TypeScript/blue?label=types)](https://www.npmjs.com/package/catai)\n[![Version](https://badgen.net/npm/v/catai)](https://www.npmjs.com/package/catai)\n\n\u003c/div\u003e\n\u003cbr /\u003e\n\nRun GGUF models on your computer with a chat ui.\n\n\u003e Your own AI assistant runs locally on your computer.\n\nInspired\nby [Node-Llama-Cpp](https://github.com/withcatai/node-llama-cpp), [Llama.cpp](https://github.com/ggerganov/llama.cpp)\n\n## Installation \u0026 Use\n\nMake sure you have [Node.js](https://nodejs.org/en/) (**download current**) installed.\n\n```bash\nnpm install -g catai\n\ncatai install meta-llama-3-8b-q4_k_m\ncatai up\n```\n\n![catai](docs/demo/chat.gif)\n\n## Features\n- Auto detect programming language 🧑‍💻\n- Click on user icon to show original message 💬\n- Real time text streaming ⏱️\n- Fast model downloads 🚀\n\n## CLI\n\n```\nUsage: catai [options] [command]\n\nOptions:\n  -V, --version                    output the version number\n  -h, --help                       display help for command\n\nCommands:\n  install|i [options] [models...]  Install any GGUF model\n  models|ls [options]              List all available models\n  use [model]                      Set model to use\n  serve|up [options]               Open the chat website\n  update                           Update server to the latest version\n  active                           Show active model\n  remove|rm [options] [models...]  Remove a model\n  uninstall                        Uninstall server and delete all models\n  node-llama-cpp|cpp [options]     Node llama.cpp CLI - recompile node-llama-cpp binaries\n  help [command]                   display help for command\n```\n\n### Install command\n\n```\nUsage: cli install|i [options] [models...]\n\nInstall any GGUF model\n\nArguments:\n  models                Model name/url/path\n\nOptions:\n  -t --tag [tag]        The name of the model in local directory\n  -l --latest           Install the latest version of a model (may be unstable)\n  -b --bind [bind]      The model binding method\n  -bk --bind-key [key]  key/cookie that the binding requires\n  -h, --help            display help for command\n```\n\n### Cross-platform\n\nYou can use it on Windows, Linux and Mac.\n\nThis package uses [node-llama-cpp](https://github.com/withcatai/node-llama-cpp) which supports the following platforms:\n\n- darwin-x64\n- darwin-arm64\n- linux-x64\n- linux-arm64\n- linux-armv7l\n- linux-ppc64le\n- win32-x64-msvc\n\n\n### Good to know\n- All download data will be downloaded at `~/catai` folder by default.\n- The download is multi-threaded, so it may use a lot of bandwidth, but it will download faster!\n\n## Web API\n\nThere is also a simple API that you can use to ask the model questions.\n\n```js\nconst response = await fetch('http://127.0.0.1:3000/api/chat/prompt', {\n    method: 'POST',\n    body: JSON.stringify({\n        prompt: 'Write me 100 words story'\n    }),\n    headers: {\n        'Content-Type': 'application/json'\n    }\n});\n\nconst data = await response.text();\n```\n\nFor more information, please read the [API guide](https://github.com/withcatai/catai/blob/main/docs/api.md)\n\n## Development API\n\nYou can also use the development API to interact with the model.\n\n```ts\nimport {createChat, downloadModel, initCatAILlama, LlamaJsonSchemaGrammar} from \"catai\";\n\n// skip downloading the model if you already have it\nawait downloadModel(\"meta-llama-3-8b-q4_k_m\");\n\nconst llama = await initCatAILlama();\nconst chat = await createChat({\n    model: \"meta-llama-3-8b-q4_k_m\"\n});\n\nconst fullResponse = await chat.prompt(\"Give me array of random numbers (10 numbers)\", {\n    grammar: new LlamaJsonSchemaGrammar(llama, {\n        type: \"array\",\n        items: {\n            type: \"number\",\n            minimum: 0,\n            maximum: 100\n        },\n    }),\n    topP: 0.8,\n    temperature: 0.8,\n});\n\nconsole.log(fullResponse); // [10, 2, 3, 4, 6, 9, 8, 1, 7, 5]\n```\n\n(For the full list of model, run `catai models`)\n\n### Node-llama-cpp@beta low level integration\n\nYou can use the model with [node-llama-cpp@beta](https://github.com/withcatai/node-llama-cpp/pull/105)\n\nCatAI enables you to easily manage the models and chat with them.\n\n```ts\nimport {downloadModel, getModelPath, initCatAILlama, LlamaChatSession} from 'catai';\n\n// download the model, skip if you already have the model\nawait downloadModel(\n    \"https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q2_K.gguf?download=true\",\n    \"llama3\"\n);\n\n// get the model path with catai\nconst modelPath = getModelPath(\"llama3\");\n\nconst llama = await initCatAILlama();\nconst model = await llama.loadModel({\n    modelPath\n});\n\nconst context = await model.createContext();\nconst session = new LlamaChatSession({\n    contextSequence: context.getSequence()\n});\n\nconst a1 = await session.prompt(\"Hi there, how are you?\");\nconsole.log(\"AI: \" + a1);\n```\n\n## Configuration\n\nYou can edit the configuration via the web ui.\n\nMore information [here](https://github.com/withcatai/catai/blob/main/docs/configuration.md)\n\n## Contributing\n\nContributions are welcome!\n\nPlease read our [contributing guide](./CONTRIBUTING.md) to get started.\n\n## License\n\nThis project uses [Llama.cpp](https://github.com/ggerganov/llama.cpp) to run models on your computer.\nSo any license applied to Llama.cpp is also applied to this project.\n\n\n\u003cbr /\u003e\n\n\u003cdiv align=\"center\" width=\"360\"\u003e\n    \u003cimg alt=\"Star please\" src=\"docs/demo/star.please.png\" style=\"border-radius: 12px\" width=\"360px\" margin=\"auto\" /\u003e\n    \u003cbr/\u003e\n    \u003cp align=\"right\"\u003e\n        \u003ci\u003eIf you like this repo, star it ✨\u003c/i\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003c/p\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwithcatai%2Fcatai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwithcatai%2Fcatai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwithcatai%2Fcatai/lists"}