Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aurora-is-near/remaphore

Admin tool employing NATS to coordinate processes on distributed infrastructure.
https://github.com/aurora-is-near/remaphore

Last synced: 10 days ago
JSON representation

Admin tool employing NATS to coordinate processes on distributed infrastructure.

Awesome Lists containing this project

README 

        
# remaphore

 

Admin tool employing NATS to coordinate processes on distributed infrastructure.



Tasks on widely distributed machines often have to be coordinated, without opening

systems up to abuse or creating complex workflow setups.



remaphore is a "remote semaphore" that communicates via NATS message broker infrastructure

and can be used to coordinate tasks.



### Example: Upload a file to S3 and trigger mass download after upload is finished.



Server 1 (uploader):



  `$ aws s3 cp bigfile s3://bucket/ && remaphore -s -u upload_done`

  

Server 2-n (downloaders):



  `$ remaphore -u upload_done && aws s3 cp s3://bucket/ bigfile`

  

In the above example the first server will send a remaphore message to a central

message subject when the upload has finished successfully. The other servers wait

for this message to then, and only then, start the s3 download.



**Only nodes that are connected while a message is sent can receive this message.**



## Advanced usage



remaphore exchanges signed messages over a public topic on a nats infrastructure.

One remaphore instances sends the message, the others receive it. How messages are

sent and how to react to them is a matter of configuration:



### Sending



  ```

  remaphore -s [options] [message]  

  -D string

    	Specify destination to match

  -p string

    	Use public key for sending

  -u string

    	UUID

  -v string

    	Verb to send

  ```



`-s` enables send and forget mode. The message will be sent and remaphore will

return immediately afterwards. Additional arguments control how messages are constructed.



`-D` defines a pattern that matches destination nodes. Each node contains a

`destination` in its configuration in reverse dot-segmented format, aka `tld.domain.host...`. This

destination can be matched either precisely, or by wildcard (*). Be aware that

only one wildcard per segment is allowed. The end of the pattern can be (**) to

match any number of segments after. Example:



Destination `com.crypto.host.us` can be matched by `com.crypto.host.us`, `com.crypto.*.us` or

`com.crypto.**`.



The default destination used is "**" which reaches all nodes.



`-p` can select a different public key for sending the message. A node can have

multiple identities configured that have different permissions. 



`-u` defines a (possibly unique) value in the message that receivers can match for.



`-v` defines the "verb" of a message. Each recipient has a list of peers and which

verbs those peers may use in messages. This allows authenticated access control in more complicated

scenarios.



The remainder of the commandline is considered the message payload to be sent.



### Receiving



  ```

  remaphore [options] [script]

  -D string

    	Specify destination to match

  -d	Do not match for destination

  -o	Exit after one matching message received

  -p string

    	Match for public key.

  -t duration

    	Timeout for operation

  -u string

    	UUID filter for

  -v string

    	-v [,verb...]: Verb to match 

  ```



Receiving nodes connect to the public message topic and read all messages sent.

The messages are by default filtered to be intended for the recipient node (destination).



Additional filtering can be configured:



`-D` changes the destination to match for.



`-d` disable matching for destination (dangerous).



`-u` match uuid string in message. remaphore will exit successfully on the first

matching message (implies `-o`).



`-v` match one or more verbs in a message. Only defined peers can send messages

for a specific verb. This allows for access control.



`-o` instructs remaphore to exit successfully after the first matching message

has been received. Otherwise remaphore will continue listening forever - unless

a timeout `-t` has been configured.



`-t` configures a timeout (duration format, `1h2m3s4ms`). After the timeout has

been reached, remaphore exits.



**Exit Codes**: Remaphore will return exit code 0 if it has received a message, exit code 1

if no message was received before timeout, and other exit codes on error.



Optionally remaphore can execute a script/command that is defined on the

commandline. It will be called as `cmd $verb $payload` and with these environment

variables set:



```

    REMAPHORE_SENDER    sender's public key as base58.

    REMAPHORE_VERB      the verb of the message.

    REMAPHORE_TIME      Unix timestamp when the message was sent.

    REMAPHORE_UUID      The message UUID.

    REMAPHORE_MSG       The payload of the message.

```

Only if `-d` has not been defined.

```

    REMAPHORE_DESTMATCH The matched destination.

```



By default, the output of the command will not be processed, unless the

sender requested a reply.



### Request&Response



```

  remaphore -r [options] [message]

  -t duration

  	Timeout for operation  

  ...

```



remaphore senders can request a response from the recipient(s). This behaviour

is enabled by the `-r` switch which is mutually exclusive to `-s`. In addition

to the same parameters known from send operations, the `-t` timeout parameter

is worth mentioning in this context.



remaphore maintains a list of suspected responders and will return as soon as

all responders have answered or the timeout has been reached.



Responses are written to stdout in the following format:



 `destination,exit-code,output`



One line per destination/responder.



If the reply contains newlines it will be surrounding by begin/end tags of this form:



```

--> randomchars

response

response

response

--< randomchars

```



`-->` begins a response while `--<` terminates it. The randomchars in the separator

remain the same for all responses for a request.



Responses are never interleaved.



### Additional functions



`-C` will print an example config file to stdout.



`-c string` allows specifying a config file other than the default one in `/etc/remaphore/remaphore.conf`.



`-S string` allows specifying a different NATS subject to communicate on. Needs to be

set for both sender and recipients.



## Configuration



The default configuration file is located in `/etc/remaphore/remaphore.conf`.



It looks similar to this:



```

server: nats://natsserver:4222

credentials: /path/to/credentials/file

subject: remaphore

default_identity: 3v96V3EgjiuXjmdkb5a4RjjtqfLoZCD657uyqrYZ1Xam

destination: com.crypto.us.left

allow_skew: 5s



[ Identities ]

3v9... g4xm... [ping]



[ Peers ]

com.crypto.us.right 5v22... [ping] 

```



`server` defines the NATS url to connect to. Multiple server entries can be present

for failover use.



`credentials` is the path to the nats credentials file. It is not optional.



`subject` is the default subject root to communicate on. Leave unchanged unless you understand.



`default_identity` is the public key of the default sender identity to use.



`destination` is the local destination name. It should be globally unique.



`allow_skew` defines the maximum delta between local time and time encoded in message. Messages outside the delta are ignored.



`[ Identities ]` introduces the list of locally configured identities. Each

identity consists of `publickey privatekey [verbs...]`.



Public- and private key are base58 encoded ed25519 keys. They are used to sign

messages. The `[verbs]` part is a list of comma-separated verbs that this identity can

use. When sending a message, the verb will select the identity used for sending. `[*]` means

*all verbs*.



`[ Peers ]` begins the list of known peers. Only peers configured in this list

will be able communicate to the receiving node. Each line consists of:



`destination publickey [verbs...]` 



The destination is the node's destination value, the publickey is used to authenticate

messages sent by that peer. Verbs define the verbs for which the peer may send

messages.



## Closing notes



remaphore can be started (either sending or receiving) concurrently

on the same host/same configuration. Each receiving instance will

process all messages. Be aware that only messages that are sent WHILE

the receiving node is waiting for messages will be received. Past messages

are lost in the void.



To control critical processes, make sure to define a verb so that

the senders that can trigger the receiver are limited to authorized

parties only.



In general, listeners should use the `-t` timeout option to prevent

unlimited lingering. `-t 24h` is a good safeguard.



Be aware that the remaphore configuration needs to be readable for

all users that need to send or receive messages via remaphore. This

should be used with care. It is advisable to create a group that has

read-access to the /etc/remaphore directory in exclusion of everybody else.

Multiple remaphore configuration files (and nats credentials) can be used

to limit the powers of users.



remaphore is an automation tool. Do not use it for chatting or file transfer

directly.