{"id":18924650,"url":"https://github.com/dash-os/tcl-cluster","last_synced_at":"2026-02-07T09:31:52.888Z","repository":{"id":71821446,"uuid":"83360961","full_name":"Dash-OS/tcl-cluster","owner":"Dash-OS","description":"Tcl Cluster Comm Package","archived":false,"fork":false,"pushed_at":"2017-10-23T04:15:16.000Z","size":187,"stargazers_count":3,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-08-02T16:39:34.605Z","etag":null,"topics":["beacon","comm","discovery","tcl","tcp","udp","unix-sockets"],"latest_commit_sha":null,"homepage":null,"language":"Tcl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dash-OS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-02-27T21:55:30.000Z","updated_at":"2022-05-18T06:02:13.000Z","dependencies_parsed_at":"2023-07-15T10:46:23.678Z","dependency_job_id":null,"html_url":"https://github.com/Dash-OS/tcl-cluster","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/Dash-OS/tcl-cluster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dash-OS%2Ftcl-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dash-OS%2Ftcl-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dash-OS%2Ftcl-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dash-OS%2Ftcl-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dash-OS","download_url":"https://codeload.github.com/Dash-OS/tcl-cluster/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dash-OS%2Ftcl-cluster/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29191399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T07:37:03.739Z","status":"ssl_error","status_checked_at":"2026-02-07T07:37:03.029Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beacon","comm","discovery","tcl","tcp","udp","unix-sockets"],"created_at":"2024-11-08T11:07:39.685Z","updated_at":"2026-02-07T09:31:52.870Z","avatar_url":"https://github.com/Dash-OS.png","language":"Tcl","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cluster\n\n**UNFINISHED (But Working)**\n\n`cluster` is a tcl package which provides a simple light-weight framework for\nproviding inter-process and inter-machine discovery and communications.  \n\n`cluster`'s evaluation hooks provide powerful extensibility, allowing the programmer to modify\nthe general behavior of a cluster to fit their security and/or communications handling needs.\n\n**Package Dependencies**\n\n- [TclUDP](https://sourceforge.net/projects/tcludp/) Extension (v1.0.11 or higher)\n\n**Supported Platforms**\n\n- Linux / Unix\n  - Recommended [tuapi](http://chiselapp.com/user/rkeene/repository/tuapi/home)\n- OS X *Initial Support*\n\nIt is actually relatively easy to support other platforms or not require tuapi.  Just need to modify a few of the initial procs to give data such as LAN IP's and MAC Address.\n\n\u003e **Note:** All of the platform-specific functionality is implemented in [utils/platform.tcl](https://github.com/Dash-OS/cluster-comm/blob/master/utils/platform.tcl).  It should not be difficult to add support for other\nplatforms.\n\n## Key Concepts\n\n`cluster` aims to provide a lightweight modular framework for cluster discovery, communications,\nand coordination.  `cluster` members automatically discover each other and provide their preferred\nprotocols for communication.  \n\nIn a local environment, it's as simple as having both run `[cluster join]` and awaiting\ndiscovery.  We can then pass messages and commands between our cluster members.\n\n#### Local and/or Remote Capable\n\nBy default, our beacons will be set with a UDP TTL of 0.  This means that our beacons\nwill only be heard locally on the system.  We may set the ttl value used by our beacons\nusing the `-remote` configuration parameter.  Anything higher than 0 will begin to open\nour beacons further out from the machine.\n\nWhen `-remote` is set to 0, communications will be ignored from any communications originating\nfrom outside of our localhost.  You may also use hooks to provide this type of security based on\nthe context of the cluster.\n\n#### Lightweight Binary Wire Protocol\n\nIn order to make the cluster communication as fast and light-weight as possible, a special\nbinary wire protocol is utilized (bpacket) which resembles [Protocol Buffers](https://developers.google.com/protocol-buffers/)\nwhich a few modifications.  In general this should be anywhere from 30-60% smaller than using JSON or\nsimilar (depending on the type of packet being sent).\n\n#### Customizable \u0026 Extendable\n\n`cluster` provides hooks that allow you each member to intercept evaluation at different\nparts of the communications process.  This allows you to add security, new features, and/or\nrun tasks whenever needed.  We do not automatically execute any code within your interp, it is\nup to you to add such functionality if needed (examples below).\n\n#### Reliable Multi-Protocol Negotiation\n\nEach member in the cluster advertises what protocols it knows how to use as well as the priority\nof those protocols. Other members use this to establish direct channels of communication when needed.\nShould a protocol fail for any reason, the next will be attempted (and so on).\n\n#### Custom Protocol Handlers\n\nIt is extremely easy to provide new protocols that cluster can utilize.  Simply follow the\ngeneral template provided by the included protocols.  Out of the box we support [UDP](https://sourceforge.net/projects/tcludp/),\n[TCP](https://www.tcl.tk/man/tcl8.6/TclCmd/socket.htm), and [Unix Sockets](https://sourceforge.net/projects/tcl-unixsockets/).  \n\n#### Protocol Self-Healing\n\nPart of the `cluster` protocol provides capabilities for services to assist each other in\nself-healing when something goes wrong with one of a services preferred protocol handlers\nwithout its knowledge.\n\n## Simple Service Discovery\n\nBelow we see a simple example of using `cluster` where we simply join the cluster\nand register a hook to inform us whenever a new service has been discovered.\n\nRunning this on two different shells (on the same system for now), you should\nsee the two shells discovered each other.  In each example, the `$service` variable\nis a reference to the service object which can be used to communicate with or get\nfurther context about the service.\n\n\n```tcl\npackage require cluster\n\nset cluster [cluster join -tags [list service_one]]\n\n$cluster hook service discovered {\n  puts \"New Service Discovered: $service\"\n}\n\n# Enter the event loop and allow cluster to work\nvwait _forever_\n\n```\n\n```tcl\npackage require cluster\n\nset cluster [cluster join -tags [list service_two]]\n\n$cluster hook service discovered {\n  puts \"New Service Discovered: $service\"\n}\n\n# Enter the event loop and allow cluster to work\nvwait _forever_\n```\n\n## Channels / Hooks / Encryption\n\nChannels provide an additional layer of extensibility to your cluster communications.\nMembers of the cluster will ignore any data they receive that are on channels they have\nnot subscribed to.\n\nWhen combined with hooks this provides us with the capability to add some simple add-ons\nto how our cluster will work.\n\nFor example, say we wanted to encrypt some of the data that was being transmitted, but only\non a specific channel.  This way only members which know the encryption key would attempt to\nread it.\n\nAll we would need to do is add a hook for the channel we want which would encrypt before sending\nand decrypt before reading.  The \"raw\" key of a payload tells us to save raw bytes rather than\nutf-8 encoded strings.\n\n```tcl\n$cluster hook channel 5 send {\n  if { [dict exists $payload data] } {\n    dict set payload raw [encrypt [dict get $payload data]]\n    dict unset payload data\n  }\n}\n\n$cluster hook channel 5 receive {\n  if {[dict exists $data raw]} {\n    dict set payload data [decrypt [dict get $payload raw]]\n    dict unset payload raw\n  }\n}\n```\n\n## Queries\n\nQueries allow us to send a command to the cluster and collect the results.  They\nprovide a special object that lives for a short period to coordinate the responses\nautomatically for you.\n\n```tcl\n\npackage require cluster\n\nset cluster [cluster join -tags [list service_one]]\n\nset var \"Hello,\"\n\n$cluster hook query {\n  if { ! [my local] } { throw error \"Only Local can Query\" }\n  uplevel #0 [list try $data]\n}\n\n$cluster hook service discovered {\n  puts \"New Service Discovered: $service\"\n}\n\n# Enter the event loop and allow cluster to work\nvwait _forever_\n\n```\n\n```tcl\n\npackage require cluster\n\nset cluster [cluster join -tags [list service_two]]\n\nset var \"World!\"\n\n$cluster hook query {\n  if { ! [my local] } { throw error \"Only Local can Query\" }\n  uplevel #0 [list try $data]\n}\n\n$cluster hook service discovered {\n  puts \"New Service Discovered: $service\"\n}\n\n# Enter the event loop and allow cluster to work\nvwait _forever_\n\n```\n\n```tcl\n\npackage require cluster\n\nset cluster [cluster join -tags [list service_three]]\n\n# Our QueryResponse will be called with events that occur during\n# the queries lifecycle.\nproc QueryResponse { event } {\n  lassign $event action query\n  switch -- $action {\n    response {\n      # A Service has provided a response.  Return the result so\n      # we can collect the results when completed.\n      return [$query result]\n    }\n    done {\n      # Our query is complete - all services have responded.\n      set results [$query results]\n      puts \"Query Completed!  Results Are:\"\n      # Hello, World!\n      puts [dict values $results]\n    }\n    timeout {\n      # Our query has timed out\n    }\n  }\n}\n\nproc RunQuery {} {\n  # Query all the services on the localhost, collect the results, run QueryResponse\n  # for events, return the value of ::var\n  $::cluster query -collect -resolve localhost -command QueryResponse -query { set ::var }   \n}\n\n# Give a few seconds for all the services to join, then run the query\nafter 5000 RunQuery\n\n# Enter the event loop and allow cluster to work\nvwait _forever_\n\n```\n\n## `cluster` API Reference\n\nThe top-level API is used to join a given cluster.\n\n#### cluster join *?..configuration?*\n\nAll of the configuration options are optional.  It is valid to simply call cluster join\nto utilize the default values.\n\n| Argument Name     |  Type   |  Required  |  Default  |  Description   |\n| ------------- | ------  | ---------- | --------- | -------------- |\n| -address      | IP      | No         | 230.230.230.230 | The Broadcast IP Address to use for the cluster    |\n| -port         | Port    | No         | 23000           | The Broadcast Port to use for the cluster |\n| -ttl          | Seconds | No         | 600             | How many seconds should a service live if unseen? |\n| -heartbeat    | MS      | No         | 120000          | At what interval should we send heartbeats to the cluster? |\n| -channels     | List    | No         | 0 1 2           | A list of communication channels that we should join |\n| -remote       | Integer | No         | 0               | Should we listen outside of localhost? TTL. |\n| -tags         | List    | No         |                 | A list of tags that we want to broadcast to the cluster. |\n| -protocols    | List    | No         | t c             | What protocols do we want to support for this member?  |\n| -$proto_id    | Dict    | Yes*       |                 | This is only required when providing custom protocols. |\n\n\u003e More Information on each argument coming soon...\n\n---\n\n## `$cluster` Commands Reference\n\nOnce we have joined the cluster, we will use the reference that was returned to coordinate\ncommunication and requests.\n\n#### $cluster heartbeat *?props tags channel?*\n\nSend a heartbeat to the cluster, informing them of your presence on the cluster.  This is\nhandled automatically but can be sent manually if desired.  We have a few optional arguments\nthat may be included.\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $cluster heartbeat_after *milliseconds*\n\nReschedule the heartbeat so that it will occur after the given time.  This is useful\nfor when we want to cause a heartbeat to occur sooner than the normal time but dont\nwant to accidentally dispatch tons of heartbeats when multiple services call it, etc.\n\n---\n\n#### $cluster discover\n\nSend a discovery probe to the cluster.  This requests that all members of the cluster\nreport to you.  Services may report back at randomly delayed intervals. For the most part\nthis should never really be required as it is handled internally.\n\n---\n\n#### $cluster broadcast\n\nThis command allows you to broadcast a command to all members of the cluster.  It\nexpects a single argument (*payload*) which should be a properly formatted payload\ndict.\n\n---\n\n#### $cluster send *...args*\n\nWhen we want to send data to our cluster, the `[cluster send]` command allows us to do so.  It includes\nvarious optional arguments that assist us with defining which services should receive the payload.  Cluster\nwill determine from the final list of services what the best method will be to efficiently handle the dispatch.\n\n\u003e **Note:** If multiple arguments are provided which resolve to services, the final list will be the merged result of unique services.\n\n##### $cluster send arguments\n\n| Argument                  | Type |  Description   \n|-------------------------- |---|--------------\n| **`-services`**           | List | A list of services to send the message to.\n| **`-resolver`**           | [resolver](#cluster-resolver) | See `[$cluster resolver]` for help on available arguments\n| **`-resolve`**            | [resolve](#cluster-resolve) | See `[$cluster resolve]` for help on available arguments  \n| **`-broadcast`**          | Boolean | A Boolean as-to whether a UDP Multi-Cast should be used\n| **`-protocols`**          | List | A list of protocols that should be attempted\n| **`-channel`**            | Number | A channel to send the payload to\n| **`-ruid`**               | String | A \"Request Unique ID\" to send with the payload\n| **`-data`**               | String | Any data that should be sent with the payload\n\n\u003e **Tip:** If using -broadcast but also resolving services, the broadcast will include a filter so that only the given services\n\u003e will actually read the given payload. In general this type of logic is automatically handled for you.\n\n\u003e More Information on each argument coming soon...\n\n---\n\n#### $cluster query *...args*\n\n - **-id**\n - **-collect**\n - **-resolve**\n - **-command**\n - **-query**\n - **-timeout**\n\n---\n\n#### $cluster resolve\n\nThis allows us to \"search\" for matching services which meet a specific criteria.  This\nis used to aid in sending queries and events to the cluster.\n\n```tcl\n# Resolve services by running a search against each $arg to return the\n# filtered services which match every arg. Resolution is a simple \"tag-based\"\n# search which matches against a services given tags.\n# set services [$cluster resolve localhost my_service]\n```\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $cluster resolver\n\nA more advanced version of resolve which allows us to add additional logic to the\nresolution process.\n\n```tcl\n\n# resolver is a more powerful option than resolve which allows adding of some added logic.\n# Each argument will define which services we want to match against.\n#\n# Additionally, we can specify boolean-type modifiers which will change the behavior.  These\n# are applied IN ORDER so for example if we run -match after -has then has will not use match\n# but any queries after will.\n#\n# Modifiers:\n#  -equal (default)\n#   Items will use equality to test for success\n#  -match\n#   Items will use string match to test for success\n#  -regexp\n#   Items will use regexp to test for success on each item\n\n# -has [list]\n#   The service must match all items in the list\n# -not [list]\n#   The service must NOT match any of the items given\n# -exact [list]\n#   The service must have every item in the list and no others\n# -some [list]\n#   The service must match at least one item in the list\n#\n# Examples:\n\nset services [ $cluster resolver -match -has [list *wait] -equal -some [list one two three] ]\n\n```\n\u003e More Information Coming Soon...\n\n---\n\n#### $cluster resolve_self\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $cluster tags *?modifier ...tags?* *?modifier ...tags?*\n\nWhen sent without arguments, this will respond with the current tags that have been\nsent to the members of the cluster.  Otherwise we can use this command to add, remove,\nor replace the tags that we wish to associate ourselves with.  \n\nWhen tags are modified, they will be included on the next heartbeat which is sent to the\ncluster.  Other than that, they are not included unless requested through discovery or\ndirect queries.\n\nThink of tags as a way of determining which members of the cluster are in various states\nand/or provide various services.\n\nThe command itself provides a syntax for modifying the current tags where each argument is\ntaken and handled in the order given.  Modifiers allow us to conduct various operations on the\ntags so that we can easily manipulate them if needed.\n\n\u003e **Note:** tags are what we are generally using while resolving services during send and query operations.\n\n##### Tag Modifiers\n\n| Modifier                  |  Description   \n|--------------------------|--------------\n| **`-map`**                |  Similar to `[string map]`, this will replace a given tag with another if it exists.  \n| **`-mappend`**            |  Similar to `-map` except that it will add the second tag even if the first does not exist.  It will also only add the new tag if it does not already exist.\n| **`-remove`**             |  Removes the given tags if they exist.\n| **`-replace`**            |  Replaces the entire list of tags with the given tags then switches back to append for any further arguments given.\n| **`-append`** (Default)   |  (-lappend is synonymous), adds the given tag to the list of tags.\n\n```tcl\n\n$cluster tags tag0 tag1\n#  tag0 tag1\n\n$cluster tags -append tag0 tag2 -map [list tag2 tag3] -remove tag3 -append tag4 tag5\n#  tag0 tag1 tag4 tag5\n\n# In this example the -map does nothing since tag10 does not exist in our list of tags.\n$cluster tags -map [list tag10 tag12]\n#  tag0 tag1 tag4 tag5\n\n$cluster tags -mappend [list tag0 tag5] [list tag6 tag8] [list tag3 tag8]\n#  tag1 tag4 tag5 tag8\n\n```\n\n\u003e **Tip:** You can think of `[-mappend]` as a shortcut for `[$cluster tags -remove tag1 -append tag2]`\n\n---\n\n#### $cluster services\n\nReturns a list of references to each service that we currently know about.\n\n---\n\n#### $cluster known_services\n\nReturns the number of services we currently know about.\n\n---\n\n#### $cluster config\n\nReturns the current configuration object that our cluster and services utilize\nto coordinate their lifecycles.\n\n---\n\n#### $cluster ttl\n\nReturns the current time-to-live value that is used.  This determines how long we will\nallow a discovered service to remain in memory before removing it (if we have not received\na heartbeat or communication from it).\n\n\u003e There are times this value is reduced.  When we attempt to open a channel with a service and it\n\u003e fails for any reason, we will dispatch a \"ping\" request.  This will tell our cluster that we have\n\u003e failed to communicate with a service.  All services will then expect a heartbeat from the given service\n\u003e or they will remove that service on their next evaluation (heartbeat).  \n\n---\n\n#### $cluster uuid\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $cluster hook\n\n\u003e More Information Coming Soon...\n\n---\n\n## `$service` Commands Reference\n\n#### $service resolve\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service resolver\n\n\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service query\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service send\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service info\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service ip\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service props\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service tags\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service local\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service hid\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service sid\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service last_seen\n\n\u003e More Information Coming Soon...\n\n---\n\n#### $service proto_props\n\n\u003e More Information Coming Soon...\n\n---\n\n## Hooks\n\nHooks provide a means to customize the evaluation environment with ***short snippets of\nmutations and additions*** to the regular lifecycle of the package.  Hooks are always\nevaluated within the hooks given context.  \n\nWe can modify the behavior at each level by either changing values before they are sent\nor received, or throwing an error to cancel any further handling.\n\n### Transmission Hooks\n\n#### `$cluster hook evaluate receive`\n\n#### `$cluster hook evaluate send`\n\n#### `$cluster hook event`\n\n### Channel Hooks\n\n#### `$cluster hook channel receive \u003cchannels\u003e`\n\n#### `$cluster hook channel send \u003cchannels\u003e`\n\n### Query Hooks\n\n#### `$cluster hook query`\n\n### Protocol Hooks\n\n#### `$cluster hook protocol \u003cproto\u003e receive`\n\n#### `$cluster hook protocol \u003cproto\u003e send`\n\n### Service Hooks\n\n#### `$cluster hook service evaluate`\n\n#### `$cluster hook service validate`\n\n#### `$cluster hook service discovered`\n\n#### `$cluster hook service lost`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdash-os%2Ftcl-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdash-os%2Ftcl-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdash-os%2Ftcl-cluster/lists"}