{"id":13625614,"url":"https://github.com/tangledgroup/llama-cpp-wasm","last_synced_at":"2025-04-09T13:06:47.430Z","repository":{"id":221532570,"uuid":"754556353","full_name":"tangledgroup/llama-cpp-wasm","owner":"tangledgroup","description":"WebAssembly (Wasm) Build and Bindings for llama.cpp","archived":false,"fork":false,"pushed_at":"2024-07-23T13:43:47.000Z","size":6310,"stargazers_count":248,"open_issues_count":8,"forks_count":22,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-02T08:39:26.598Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tangledgroup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-08T10:03:26.000Z","updated_at":"2025-04-01T18:38:45.000Z","dependencies_parsed_at":"2024-02-12T09:37:46.262Z","dependency_job_id":"634bf466-cc8d-4f14-b195-ef32c29a79f1","html_url":"https://github.com/tangledgroup/llama-cpp-wasm","commit_stats":null,"previous_names":["tangledgroup/llama-cpp-wasm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tangledgroup%2Fllama-cpp-wasm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tangledgroup%2Fllama-cpp-wasm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tangledgroup%2Fllama-cpp-wasm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tangledgroup%2Fllama-cpp-wasm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tangledgroup","download_url":"https://codeload.github.com/tangledgroup/llama-cpp-wasm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045231,"owners_count":21038553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T21:01:58.369Z","updated_at":"2025-04-09T13:06:47.411Z","avatar_url":"https://github.com/tangledgroup.png","language":"JavaScript","funding_links":[],"categories":["others"],"sub_categories":[],"readme":"# llama-cpp-wasm\n\n**WebAssembly** (Wasm) Build and Bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) and supported by [Tangled Group, Inc](https://tangledgroup.com).\n\n![llama-cpp-wasm](docs/img/run-llama-cpp-in-browser-twitter-fs8.png)\n\n## Online Demos\n\nhttps://tangledgroup.github.io/llama-cpp-wasm/\n\n\n## Build\n\n```bash\ngit clone https://github.com/tangledgroup/llama-cpp-wasm.git\ncd llama-cpp-wasm\n./build-single-thread.sh\n./build-multi-thread.sh\n```\n\nOnce build is complete you can find `llama.cpp` built in `dist/llama-st` and `dist/llama-mt` directory.\n\n\n## Deploy\n\nBasically, you can copy/paste `dist/llama-st` or `dist/llama-mt` directory after build to your project and use as vanilla JavaScript library/module.\n\n\n**index.html**\n\n```html\n\u003c!DOCTYPE html\u003e\n\u003chtml lang=\"en\"\u003e\n  \u003cbody\u003e\n    \u003clabel for=\"prompt\"\u003ePrompt:\u003c/label\u003e\n    \u003cbr/\u003e\n\n    \u003ctextarea id=\"prompt\" name=\"prompt\" rows=\"25\" cols=\"80\"\u003eSuppose Alice originally had 3 apples, then Bob gave Alice 7 apples, then Alice gave Cook 5 apples, and then Tim gave Alice 3x the amount of apples Alice had. How many apples does Alice have now? Let’s think step by step.\u003c/textarea\u003e\n    \u003cbr/\u003e\n\n    \u003clabel for=\"result\"\u003eResult:\u003c/label\u003e\n    \u003cbr/\u003e\n\n    \u003ctextarea id=\"result\" name=\"result\" rows=\"25\" cols=\"80\"\u003e\u003c/textarea\u003e\n    \u003cbr/\u003e\n    \n    \u003cscript type=\"module\" src=\"example.js\"\u003e\u003c/script\u003e\n  \u003c/body\u003e\n\u003c/html\u003e\n```\n\n\n**example.js**\n\n```javascript\n// import { LlamaCpp } from \"./llama-st/llama.js\";\nimport { LlamaCpp } from \"./llama-mt/llama.js\";\n\nconst onModelLoaded = () =\u003e { \n  console.debug('model: loaded');\n  const prompt = document.querySelector(\"#prompt\").value;\n  document.querySelector(\"#result\").value = prompt;\n\n  app.run({\n    prompt: prompt,\n    ctx_size: 4096,\n    temp: 0.1,\n    no_display_prompt: true,\n  });\n}\n\nconst onMessageChunk = (text) =\u003e {\n  console.log(text);\n  document.querySelector('#result').value += text;\n};\n\nconst onComplete = () =\u003e {\n  console.debug('model: completed');\n};\n\nconst models = [\n  'https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat-GGUF/resolve/main/qwen2-beta-0_5b-chat-q8_0.gguf',\n  'https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat-GGUF/resolve/main/qwen1_5-1_8b-chat-q8_0.gguf',\n  'https://huggingface.co/stabilityai/stablelm-2-zephyr-1_6b/resolve/main/stablelm-2-zephyr-1_6b-Q4_1.gguf',\n  'https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf',\n  'https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q4_K_M.gguf'\n];\n\nconst model = models[2]; // stablelm-2-zephyr-1_6b\n\nconst app = new LlamaCpp(\n  model,\n  onModelLoaded,          \n  onMessageChunk,       \n  onComplete,\n);\n```\n\n\n## Run Example\n\nFirst generate self-signed certificate.\n\n```bash\nopenssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout key.pem -out cert.pem\n```\n\n### Run Single Thread Example\n\n```bash\nnpx http-server -S -C cert.pem\n```\n\n### Run Multi-threading Example\n\nCopy `docs/server.js` to your working directory.\n\n```bash\nnpm i express\nnode server.js\n```\n\nThen open in browser: https://127.0.0.1:8080/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftangledgroup%2Fllama-cpp-wasm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftangledgroup%2Fllama-cpp-wasm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftangledgroup%2Fllama-cpp-wasm/lists"}