{"id":22160527,"url":"https://github.com/rozek/node-red-flow-wizardlm","last_synced_at":"2026-03-19T22:01:35.392Z","repository":{"id":163928853,"uuid":"639362611","full_name":"rozek/node-red-flow-wizardlm","owner":"rozek","description":"Node-RED Flow (and web page example) for the WizardLM AI model","archived":false,"fork":false,"pushed_at":"2023-07-27T15:16:37.000Z","size":340,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-26T18:44:50.786Z","etag":null,"topics":["ai-chatbot","node-red","node-red-flow","wizardlm","wizardlm-7b"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rozek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-05-11T10:12:49.000Z","updated_at":"2023-10-08T06:47:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"a32a345b-2989-4d7d-86ec-9902f4700eab","html_url":"https://github.com/rozek/node-red-flow-wizardlm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rozek/node-red-flow-wizardlm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rozek%2Fnode-red-flow-wizardlm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rozek%2Fnode-red-flow-wizardlm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rozek%2Fnode-red-flow-wizardlm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rozek%2Fnode-red-flow-wizardlm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rozek","download_url":"https://codeload.github.com/rozek/node-red-flow-wizardlm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rozek%2Fnode-red-flow-wizardlm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28936182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T08:53:31.997Z","status":"ssl_error","status_checked_at":"2026-01-31T08:51:38.521Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-chatbot","node-red","node-red-flow","wizardlm","wizardlm-7b"],"created_at":"2024-12-02T04:08:26.598Z","updated_at":"2026-01-31T09:32:26.060Z","avatar_url":"https://github.com/rozek.png","language":"HTML","readme":"# node-red-flow-wizardlm #\n\nNode-RED Flow (and web page example) for the WizardLM AI model\n\n\u003e Nota bene: if you are interested in serving LLMs from a Node-RED server, you may also be interested in [node-red-flow-openai-api](https://github.com/rozek/node-red-flow-openai-api), a set of flows which implement a relevant subset of OpenAI APIs and may act as a drop-in replacement for OpenAI in [LangChain](https://github.com/hwchase17/langchainjs) or similar tools and may directly be used from within [Flowise](https://github.com/rozek/Flowise), the no-code environment for LangChain\n\nThis repository contains a function node for [Node-RED](https://nodered.org/) which can be used to run the [WizardLM model]([https://github.com/nlpxucan/WizardLM](https://github.com/nlpxucan/WizardLM)) using [llama.cpp](https://github.com/rozek/llama.cpp) within a Node-RED flow. **Inference is done on the CPU** (without requiring any special harware) and still completes within a few seconds on a reasonably powerful computer.\n\n![WizardLM HTTP Flow](./WizardLM-HTTP-Flow.png)\n\nAdditionally, this repo also contains function nodes to tokenize a prompt or to calculate embeddings based on the WizardLM model.\n\nHaving the inference, tokenization and embedding calculation as a self-contained function node gives you the possibility to create your own user interface or even use it as part of an autonomous agent.\n\n\u003e Nota bene: these flows do not contain the actual model. You will have to download your own copy directly from [Huggingface](https://huggingface.co/TheBloke/wizardLM-7B-GGML) (use file \"wizardLM-7B.GGML.q4_0.bin\").\n\nIf you like, you may also check out similar nodes and flows for other AI models as there are\n\n* [LLaMA](https://github.com/rozek/node-red-flow-llama)\n* [Stanford Alpaca, trained with GPT-3](https://github.com/rozek/node-red-flow-alpaca)\n* [Stanford Alpaca, trained with GPT-4](https://github.com/rozek/node-red-flow-gpt4-x-alpaca)\n* [Nomic AI GPT4All (filtered version)](https://github.com/rozek/node-red-flow-gpt4all-filtered)\n* [Nomic AI GPT4All (unfiltered version)](https://github.com/rozek/node-red-flow-gpt4all-unfiltered)\n* [Nomic AI GPT4All-J](https://github.com/rozek/node-red-flow-gpt4all-j)\n* [Vicuna](https://github.com/rozek/node-red-flow-vicuna)\n* [OpenLLaMA](https://github.com/rozek/node-red-flow-openllama)\n\n\u003e Just a small note: if you like this work and plan to use it, consider \"starring\" this repository (you will find the \"Star\" button on the top right of this page), so that I know which of my repositories to take most care of.\n\n## Installation ##\n\nStart by creating a subfolder called `ai` within the installation folder of your Node-RED server. This subfolder will later store both the executable and the actual model. Using such a subfolder helps keeping the folder structure of your server clean if you decide to play with other AI models as well.\n\n### Building the Executable ###\n\nThe actual \"heavy lifting\" is done by [llama.cpp](https://github.com/rozek/llama.cpp). Simply follow the instructions found in section [Usage](https://github.com/rozek/llama.cpp#usage) of the llama.cpp docs to build the `main` executable for your platform.\n\nAfterwards, rename \n\n* `main` to `llama`,\n* `tokenization` to `llama-tokenization` and\n* `embedding` to `llama-embeddings`\n\nand copy them into the subfolder `ai` you created before.\n\n### Preparing the Model ###\n\nOnce you got the actual WizardLM model file `wizardLM-7B.ggml.q4_0.bin`, rename it to `ggml-wizardlm-7b-q4_0.bin` and move (or copy) it into the same subfolder `ai` where you already placed the `llama` executable.\n\n\u003e Nota bene: right now, the function node supports the given model file only - but this may easily be changed in the function source\n\n### Importing the Nodes ###\n\nFinally, open the Flow Editor of your Node-RED server and import the contents of [WizardLM-Function.json](./WizardLM-Function.json). After deploying your changes, you are ready to run WizardLM inferences directly from within Node-RED.\n\nAdditionally, you may also import the contents of [WizardLM-Tokenization.json](./WizardLM-Tokenization.json) if you want to tokenize prompts, or of [WizardLM-Embeddings.json](./WizardLM-Embeddings.json) if you want to calculate embeddings for a given text.\n\n## Usage ##\n\nAll function nodes expect their parameters as properties of the msg object. The prompt itself (or the input text to tokenize or calculate embeddings from) is expected in `msg.payload` and will later be replaced by the function result.\n\nAll properties (except prompt or input text) are optional. If given, they should be strings (even if they contain numbers), this makes it simpler to extract them from an HTTP request.\n\n### Inference Node ###\n\nInference supports the following properties:\n\n* `payload` - this is the actual prompt \n* `seed` - seed value for the internal pseudo random number generator (integer, default: -1, use random seed for \u003c= 0)\n* `threads` - number of threads to use during computation (integer ≧ 1, default: 4)\n* `context` - size of the prompt context (0...2048, default: 512)\n* `keep` - number of tokens to keep from the initial prompt (integer ≧ -1, default: 0, -1 = all)\n* `predict` - number of tokens to predict (integer ≧ -1, default: 128, -1 = infinity)\n* `topk` - top-k sampling limit (integer ≧ 1, default: 40)\n* `topp` - top-p sampling limit (0.0...1.0, default: 0.9)\n* `temperature` - temperature (0.0...2.0, default: 0.8)\n* `batches` - batch size for prompt processing (integer ≧ 1, default: 8)\n\n### Tokenization Node ###\n\nTokenization supports the following properties:\n\n* `payload` - this is the actual input text \n* `threads` - number of threads to use during computation (integer ≧ 1, default: 4)\n* `context` - size of the prompt context (0...2048, default: 512)\n\n### Embeddings Node ###\n\nEmbeddings calculation supports the following properties:\n\n* `payload` - this is the actual input text  \n* `seed` - seed value for the internal pseudo random number generator (integer, default: -1, use random seed for \u003c= 0)\n* `threads` - number of threads to use during computation (integer ≧ 1, default: 4)\n* `context` - size of the prompt context (0...2048, default: 512)\n\n## Inference Example ##\n\nThe file [WizardLM-HTTP-Endpoint.json](./WizardLM-HTTP-Endpoint.json) contains an example which uses the WizardLM function node to answer HTTP requests. The prompt itself and any inference parameters have to be passed as query parameters, the result of the inference will then be returned in the body of the HTTP response.\n\n\u003e Nota bene: the screenshot from above shows a modified version of this flow including an authentication node from the author's [Node-RED Authorization Examples](https://github.com/rozek/node-red-authorization-examples), the flow in [WizardLM-HTTP-Endpoint.json](./WizardLM-HTTP-Endpoint.json) comes without any authentication.\n\nThe following parameters are supported (most of them will be copied into a `msg` property of the same name):\n\n* `prompt` - will be copied into `msg.payload`\n* `seed` - will be copied into `msg.seed`\n* `threads` - will be copied into `msg.threads`\n* `context` - will be copied into `msg.context`\n* `keep` - will be copied into `msg.keep`\n* `predict` - will be copied into `msg.predict`\n* `topk` - will be copied into `msg.topk`\n* `topp` - will be copied into `msg.topp`\n* `temperature` - will be copied into `msg.temperature`\n* `batches` - will be copied into `msg.batches`\n\nIn order to install this flow, simply open the Flow Editor of your Node-RED server and import the contents of [WizardLM-HTTP-Endpoint.json](./WizardLM-HTTP-Endpoint.json)\n\n### Web Page ###\n\nThe file [WizardLM.html](./WizardLM.html) contains a trivial web page which can act as a user interface for the HTTP endpoint.\n\n![WizardLM Screenshot](./WizardLM-Screenshot.png)\n\nIdeally, this page should be served from the same Node-RED server that also accepts the HTTP requests for WizardLM, but this is not strictly necessary.\n\nThe input fields `Base URL`, `User Name` and `Password` can be used if web server and Node-RED server are at different locations: just enter the base URL of your Node-RED HTTP endpoint (without the trailing `llama`) and, if that server requires basic authentication, your user name and your password in the related input fields before you send your first prompt - otherwise, just leave all these fields empty.\n\nThe input fields `Seed`, `Temperature`, `Prediction Length`, `Context Length`, `top K pick` and `top P pick` may be used to customize some of the parameters described above - if left empty, their \"placeholders\" show the respective default values.\n\nThe largest field will show a transcript of your current dialog with the inference node.\n\nFinally, the bottommost input field may be used to enter a prompt - if one is present, the \"Send\" button becomes enabled: press it to submit your prompt, then wait for a response.\n\n\u003e Nota bene: **inference is still done on the Node-RED server**, not within your browser!\n\n## License ##\n\n[MIT License](LICENSE.md)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frozek%2Fnode-red-flow-wizardlm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frozek%2Fnode-red-flow-wizardlm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frozek%2Fnode-red-flow-wizardlm/lists"}