Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/alfredodeza/remoto

Remote commands (or processes calls) in Python.
https://github.com/alfredodeza/remoto

Last synced: 3 months ago
JSON representation

Remote commands (or processes calls) in Python.

Awesome Lists containing this project

README 

        
remoto

======

A very simplistic remote-command-executor using connections to hosts (``ssh``,

local, containers, and several others are supported) and Python in the remote

end.



All the heavy lifting is done by execnet, while this minimal API provides the

bare minimum to handle easy logging and connections from the remote end.



``remoto`` is a bit opinionated as it was conceived to replace helpers and

remote utilities for ``ceph-deploy``, a tool to run remote commands to configure

and setup the distributed file system Ceph. `ceph-medic

`_ uses remoto as well to inspect Ceph

clusters.



Example Usage

-------------

The usage aims to be extremely straightforward, with a very minimal set of

helpers and utilities for remote processes and logging output.



The most basic example will use the ``run`` helper to execute a command on the

remote end. It does require a logging object, which needs to be one that, at

the very least, has both ``error`` and ``debug``. Those are called for

``stderr`` and ``stdout`` respectively.



This is how it would look with a basic logger passed in::



    >>> conn = remoto.Connection('hostname')

    >>> run(conn, ['ls', '-a'])

    INFO:hostname:Running command: ls -a

    DEBUG:hostname:.

    DEBUG:hostname:..

    DEBUG:hostname:.bash_history

    DEBUG:hostname:.bash_logout

    DEBUG:hostname:.bash_profile

    DEBUG:hostname:.bashrc

    DEBUG:hostname:.lesshst

    DEBUG:hostname:.pki

    DEBUG:hostname:.ssh

    DEBUG:hostname:.vim

    DEBUG:hostname:.viminfo



The ``run`` helper will display the ``stderr`` and ``stdout`` as ``ERROR`` and

``DEBUG`` respectively.



For other types of usage (like checking exit status codes, or raising upon

them) ``remoto`` does provide them too.



Remote Commands

===============



``process.run``

---------------

Calling remote commands can be done in a few different ways. The most simple

one is with ``process.run``::



    >>> from remoto.process import run

    >>> from remoto import connection

    >>> Connection = connection.get('ssh')

    >>> conn = Connection('myhost')

    >>> run(conn, ['whoami'])

    INFO:myhost:Running command: whoami

    DEBUG:myhost:root



Note however, that you are not capturing results or information from the remote

end. The intention here is only to be able to run a command and log its output.

It is a *fire and forget* call.



``process.check``

-----------------

This callable, allows the caller to deal with the ``stderr``, ``stdout`` and

exit code. It returns it in a 3 item tuple::



    >>> from remoto.process import check

    >>> check(conn, ['ls', '/nonexistent/path'])

    ([], ['ls: cannot access /nonexistent/path: No such file or directory'], 2)



Note that the ``stdout`` and ``stderr`` items are returned as lists with the ``\n``

characters removed.



This is useful if you need to process the information back locally, as opposed

to just firing and forgetting (while logging, like ``process.run``).



Remote Functions

================

There are two supported ways to execute functions on the remote side. The

library that ``remoto`` uses to connect (``execnet``) only supports a few

backends *natively*, and ``remoto`` has extended this ability for other backend

connections like kubernetes.



The remote function capabilities are provided by ``LegacyModuleExecute`` and

``JsonModuleExecute``. By default, both ``ssh`` and ``local`` connection will

use the legacy execution class, and everything else will use the ``legacy``

class. The ``ssh`` and ``local`` connections can still be forced to use the new

module execution by setting::



    conn.remote_import_system = 'json'



``json``

--------

The default module for ``docker``, ``kubernetes``, ``podman``, and

``openshift``. It does not require any magic on the module to be executed,

however it is worth noting that the library *will* add the following bit of

magic when sending the module to the remote end for execution::



    if __name__ == '__main__':

        import json, traceback

        obj = {'return': None, 'exception': None}

        try:

            obj['return'] = function_name(*a)

        except Exception:

            obj['exception'] = traceback.format_exc()

        try:

            print(json.dumps(obj).decode('utf-8'))

        except AttributeError:

            print(json.dumps(obj))



This allows the system to execute ``function_name`` (replaced by the real

function to be executed with its arguments), grab any results, serialize them

with ``json`` and send them back for local processing.



If you had a function in a module named ``foo`` that looks like this::



    import os



    def listdir(path):

        return os.listdir(path)



To be able to execute that ``listdir`` function remotely you would need to pass

the module to the connection object and then call that function::



    >>> import foo

    >>> conn = Connection('hostname')

    >>> remote_foo = conn.import_module(foo)

    >>> remote_foo.listdir('.')

    ['.bash_logout',

     '.profile',

     '.veewee_version',

     '.lesshst',

     'python',

     '.vbox_version',

     'ceph',

     '.cache',

     '.ssh']



Note that functions to be executed remotely **cannot** accept objects as

arguments, just normal Python data structures, like tuples, lists and

dictionaries. Also safe to use are ints and strings.



``legacy``

----------

When using the ``legacy`` execution model (the default for ``local`` and

``ssh`` connections), modules are required to add the following to the end of

that module::



    if __name__ == '__channelexec__':

        for item in channel:

            channel.send(eval(item))



This piece of code is fully compatible with the ``json`` execution model, and

would not cause conflicts.



Automatic detection for ssh connections

---------------------------------------

There is automatic detection for the need to connect remotely (via SSH) or not

that it is infered by the hostname of the current host (vs. the host that is

connecting to).



If the local host has the same as the remote hostname, a local connection (via

`Popen`) will be opened and that will be used instead of `ssh`, and avoiding

the issues of being able to ssh into the same host.



Automatic detection for using `sudo`

------------------------------------

This magical detection can be enabled by using the `detect_sudo` flag in the

`Connection` class. It is disabled by default.



When enabled, it will prefix any command with `sudo`. This is useful for

libraries that need super user permissions and want to avoid passing `sudo`

everywhere, which can be non-trivial if dealing with `root` users that are

connecting via SSH.