{"id":19758827,"url":"https://github.com/daq-tools/node-blue","last_synced_at":"2025-02-28T02:35:03.164Z","repository":{"id":171308331,"uuid":"640412319","full_name":"daq-tools/node-blue","owner":"daq-tools","description":"Node-RED without a mouse / Node-RED as a library / Write Node-RED flows in YAML / Now it all makes sense","archived":false,"fork":false,"pushed_at":"2025-02-20T03:16:16.000Z","size":125,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-20T04:24:06.901Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daq-tools.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-14T02:07:31.000Z","updated_at":"2025-02-20T03:16:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"900616fe-b3b3-4eff-bd4f-eabf74eab772","html_url":"https://github.com/daq-tools/node-blue","commit_stats":null,"previous_names":["daq-tools/node-blue"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Fnode-blue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Fnode-blue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Fnode-blue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daq-tools%2Fnode-blue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daq-tools","download_url":"https://codeload.github.com/daq-tools/node-blue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241090243,"owners_count":19907912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T03:26:09.522Z","updated_at":"2025-02-28T02:35:03.127Z","avatar_url":"https://github.com/daq-tools.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node-BLUE\n\n[Node-BLUE] is a friendly wrapper around [Node-RED].\n\n_Node-RED without a mouse / Node-RED as a library / Write Node-RED flows in YAML /\nWrite Node-RED user-defined functions in Python / Efficient test harnesses for Node-RED /\nTrue headless Node-RED / Improved development iteration times / Node-RED without needing\nto take the red pill / Node-BLUE is Node-RED on rails / Now it all makes sense_\n\n:::{todo}\nThis list should make it into the »Features« section.\n:::\n\n:::{attention}\nHere be dragons.\n:::\n\n\n## About\n\nThe idea is to embed [Node-RED] into Python programs, in order to leverage it\nfor a number of use cases, like system automation, software testing, parallel\nexecution, etc.\n\nThe project has similar intentions like [node-red-embedded-start], offering to\ninteract programmatically with the Node-RED API, with a few bells and whistles.\n\nThe other idea is to extend the Node-RED ecosystem by leveraging other programming\nlanguages and their ecosystems natively, beyond what JavaScript/NPM can do. We made\na start with [Python], by using the excellent [JSPyBridge] package.\n\n\n## Synopsis\n\n\n### Command-line use\n\nInstall [Node-BLUE] and [HTTPie], and their prerequisites.\n```shell\npip install httpie https://github.com/daq-tools/node-blue\nnode-blue setup\nnpm install\n```\n\nStart Node-BLUE with a Node-RED flow defining an HTTP/HTML endpoint/responder.\n```shell\n# Launch Node-BLUE/Node-RED.\nnode-blue launch --flow=examples/flows/http-html-templating.json\nnode-blue launch --flow=https://github.com/daq-tools/node-blue/raw/main/examples/flows/http-html-templating.json\n\n# Run an example HTTP request.\nhttp http://localhost:1880/hello-form name=Hotzenplotz\n```\n\nLaunch a flow which defines a pipeline to converge data from [MQTT] to [CrateDB].\n```shell\n# Launch Node-BLUE/Node-RED.\nnode-blue launch --flow=examples/flows/mqtt-to-cratedb.yaml\n```\n\n\n### Library use\n\n```python\nimport asyncio\nfrom node_blue.core import NodeBlue\n\nasync def launch_blue():\n    \"\"\"\n    Launch a Node-BLUE instance.\n    \"\"\"\n    \n    # Configure Node-BLUE instance with Node-RED flow.\n    blue = NodeBlue(flow=\"https://github.com/daq-tools/node-blue/raw/main/examples/flows/http-html-templating.json\")\n\n    # Start Node-RED instance.\n    blue.start().wait_started()\n\n    # Wait until termination.\n    await blue.forever()\n\nif __name__ == \"__main__\":\n    asyncio.run(launch_blue())\n```\n\n\n## Examples\n\n### Introduction\n\nUnless otherwise noted, all subsequent examples can be exercised using MQTT. You will\nstart the [Mosquitto] MQTT broker, subscribe to a topic, and publish a JSON message,\nwhich will be processed by a Node-RED flow definition.\n\nAll flow definitions implement the same rule, and as such, can be triggered by using\nthe same MQTT messages, outlined below. For demonstration purposes, the transformation\nrule is very simple: It applies a unit conversion formula to a numeric input parameter,\neffectively converting value units, from Fahrenheit to Celsius. In order to not mix\nthings up, the outcome will be published to a different MQTT topic.\n\nThis effectively demonstrates message routing and republishing capabilities in both\nterms of content transformation and topic rewriting. The procedures can be applied\nto any message broker system, we just used MQTT here, because it is so convenient\nto operate.\n\n```shell\ndocker run --name=mosquitto -it --rm --publish=1883:1883 eclipse-mosquitto:2.0 \\\n  mosquitto -c /mosquitto-no-auth.conf\n```\n\nSubscribe to the broker to observe the outcome.\n```shell\nmosquitto_sub -t 'testdrive/#' -v\n```\n\nRun an example MQTT publish.\n```shell\necho '{\"temperature\": 42.42}' | mosquitto_pub -t 'testdrive/imperial' -l\n```\n\n### JSON5 flow format\n\nStart Node-BLUE with a Node-RED flow defining an MQTT topic rewrite and message transformation\nstep. Note that this example uses the [JSON5] format, which has a few advantages over regular\nJSON, like inline comments, and multi-line strings.\n```shell\nwget https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-unit-rewriting.json5\nnode-blue launch --flow=mqtt-unit-rewriting.json5\n```\nThis flow snippet demonstrates JSON5 capabilities on behalf of a sensible example.\n```json5\n// The transformation node element uses a function written in JavaScript to convert from\n// Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.\n// The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`\n// data sink node without further ado.\n{\n  \"id\": \"transformation.55460a\",\n  \"type\": \"function\",\n  \"func\": \"\\\n    if (msg.topic.endsWith('imperial')) { \\\n      msg.payload.temperature = Number.parseFloat(((Number.parseFloat(msg.payload.temperature) - 32) * 5 / 9).toFixed(2)); \\\n      msg.topic = msg.topic.replace('imperial', 'metric'); \\\n      return msg; \\\n    } \\\n  \",\n  \"wires\": [\n    [\n      \"sink.3539af\",\n    ]\n  ]\n}\n```\n\n### YAML flow format\n\nYou can also use the [YAML] format for writing flow recipes. The authors believe it offers\nthe best conciseness and convenience, with even better multi-line code blocks, without needing\nany sorts of line-continuation characters. YAML is already used by the\n[node-red-contrib-flow-manager] and [node-red-contrib-yaml-storage] plugins.\n```shell\nnode-blue launch --flow=https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-unit-rewriting.yaml\n```\nWith this example flow snippet, you will immediately recognize how convenient it is,\nespecially for user-defined functions.\n```yaml\n# The transformation node element uses a function written in JavaScript to convert from\n# Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.\n# The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`\n# data sink node without further ado.\n- id: transformation.55460a\n  type: function\n  func: |-\n    if (msg.topic.endsWith('imperial')) {\n      msg.payload.temperature = Number.parseFloat(((Number.parseFloat(msg.payload.temperature) - 32) * 5 / 9).toFixed(2));\n      msg.topic = msg.topic.replace('imperial', 'metric');\n      return msg;\n    }\n  wires: [\n    [\n      \"sink.3539af\",\n    ]\n  ]\n```\n\n### Python user-defined functions\n\nThis time, let's use the [Python] language, to define a user-defined function within the\nflow step.\n```shell\nwget https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-routing-python.yaml\nnode-blue launch --flow=examples/flows/mqtt-routing-python.yaml\n```\n\n```yaml\n# The transformation node element uses a function written in Python to convert from\n# Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.\n# The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`\n# data sink node without further ado.\n- id: transformation.66571b\n  type: python-function\n  func: |-\n    if msg.topic.endswith(\"imperial\"):\n      msg.payload.temperature = round((float(msg.payload.temperature) - 32) * 5 / 9, 2)\n      msg.topic = msg.topic.replace(\"imperial\", \"metric\")\n      send(msg)\n    done()\n  wires: [\n    [\n      \"sink.3539af\",\n    ]\n  ]\n```\n\n\n## References\n\nSee also the IBM Data management article about [Implementing ETL flows with\nNode-RED] by [Ondrej Lehota].\n\n\n## Etymology\n\nTo use the name »Node-BLUE« for this project was obvious. However, we discovered\nthat there has been another project called Node-BLUE, referenced below. On the\nother hand, because it has been archived two years ago already, we think there\nwill not be much harm to reuse that name now.\n\n- https://www.npmjs.com/package/node-blue\n- https://github.com/node-blue/node-blue\n\n\n## Acknowledgements\n\nThis project bundles a few significant pieces of software and technologies,\nwhich a few bright minds invented the other day, and countless authors\ncontributed to. We are only listing inventors of the major application-level\ncomponents, remember there may be several layers of operating systems beneath,\nand silicon either.\n\n- [Andy Stanford-Clark], [Arlen Nipper], and co-authors for [MQTT].\n- [Brendan Eich] and contributors for [JavaScript].\n- [@extremeheat] and contributors for [JSPyBridge].\n- [Guido van Rossum] and contributors for [Python].\n- [Lars Bak] and contributors for [V8].\n- [Nick O'Leary] and contributors for [Node-RED]. \n- [Roger Light]  and contributors for [Mosquitto].\n- [Ryan Dahl] and contributors for [Node.js] and [Deno].\n\nThank you!\n\n\n[Andy Stanford-Clark]: https://stanford-clark.com/\n[Arlen Nipper]: https://github.com/anipper\n[Brendan Eich]: https://en.wikipedia.org/wiki/Brendan_Eich\n[Deno]: https://github.com/denoland\n[@extremeheat]: https://github.com/extremeheat\n[Guido van Rossum]: https://github.com/gvanrossum\n[HTTPie]: https://httpie.io/\n[Implementing ETL flows with Node-RED]: https://developer.ibm.com/articles/implementing-etl-flows-with-node-red/\n[JavaScript]: https://en.wikipedia.org/wiki/JavaScript\n[JSON5]: https://json5.org/\n[JSPyBridge]: https://github.com/extremeheat/JSPyBridge\n[Lars Bak]: https://en.wikipedia.org/wiki/Lars_Bak_(computer_programmer)\n[Mosquitto]: https://mosquitto.org/\n[MQTT]: https://en.wikipedia.org/wiki/MQTT\n[Nick O'Leary]: https://github.com/knolleary\n[Node.js]: https://github.com/nodejs\n[Node-BLUE]: https://github.com/daq-tools/node-blue\n[Node-RED]: https://github.com/node-red/node-red\n[node-red-contrib-flow-manager]: https://flows.nodered.org/node/node-red-contrib-flow-manager\n[node-red-contrib-yaml-storage]: https://flows.nodered.org/node/node-red-contrib-yaml-storage\n[node-red-embedded-start]: https://www.npmjs.com/package/node-red-embedded-start\n[Ondrej Lehota]: https://github.com/barshociaj\n[Python]: https://en.wikipedia.org/wiki/Python_(programming_language)\n[Roger Light]: https://github.com/ralight\n[Ryan Dahl]: https://github.com/ry\n[V8]: https://en.wikipedia.org/wiki/V8_(JavaScript_engine)\n[YAML]: https://en.wikipedia.org/wiki/YAML\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaq-tools%2Fnode-blue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaq-tools%2Fnode-blue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaq-tools%2Fnode-blue/lists"}