Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jprjr/lua-resty-exec

Run external programs in OpenResty without spawning a shell or blocking
https://github.com/jprjr/lua-resty-exec

exec lua-nginx lua-resty openresty stdout

Last synced: 19 days ago
JSON representation

Run external programs in OpenResty without spawning a shell or blocking

Awesome Lists containing this project

README 

        
# lua-resty-exec



A small Lua module for executing processes. It's primarily

intended to be used with OpenResty, but will work in regular Lua applications

as well. When used with OpenResty, it's completely non-blocking (otherwise it

falls back to using LuaSocket and does block).



It's similar to (and inspired by)

[lua-resty-shell](https://github.com/juce/lua-resty-shell), the primary

difference being this module uses sockexec, which doesn't spawn a shell -

instead you provide an array of argument strings, which means you don't need

to worry about shell escaping/quoting/parsing rules.



Additionally, as of version 2.0.0, you can use `resty.exec.socket` to access a

lower-level interface that allows two-way communication with programs. You can

read and write to running applications!



This requires your web server to have an active instance of

[sockexec](https://github.com/jprjr/sockexec) running.



## Changelog



* `3.0.0`

  * new field returned: `unknown` - if this happens please send me a bug!

* `2.0.0`

  * New `resty.exec.socket` module for using a duplex connection

  * `resty.exec` no longer uses the `bufsize` argument

  * `resty.exec` now accepts a `timeout` argument, specify in milliseconds, defaults to 60s

  * This is a major revision, please test thoroughly before upgrading!

* No changelog before `2.0.0`



## Installation



`lua-resty-exec` is available on [luarocks](https://luarocks.org/modules/jprjr/lua-resty-exec)

as well as [opm](https://opm.openresty.org/), you can install it with `luarocks install

lua-resty-exec` or `opm get jprjr/lua-resty-exec`.



If you're using this outside of OpenResty, you'll also need the LuaSocket

module installed, ie `luarocks install luasocket`.



Additionally, you'll need `sockexec` running, see [its repo](https://github.com/jprjr/sockexec)

for instructions.



## `resty.exec` Usage



```lua

local exec = require'resty.exec'

local prog = exec.new('/tmp/exec.sock')

```



Creates a new `prog` object, using `/tmp/exec.sock` for its connection to

sockexec.



From there, you can use `prog` in a couple of different ways:



### ez-mode



```lua

local res, err = prog('uname')



-- res = { stdout = "Linux\n", stderr = nil, exitcode = 0, termsig = nil }

-- err = nil



ngx.print(res.stdout)

```



This will run `uname`, with no data on stdin.



Returns a table of output/error codes, with `err` set to any errors

encountered.



### Setup argv beforehand



```lua

prog.argv = { 'uname', '-a' }

local res, err = prog()



-- res = { stdout = "Linux localhost 3.10.18 #1 SMP Tue Aug 2 21:08:34 PDT 2016 x86_64 GNU/Linux\n", stderr = nil, exitcode = 0, termsig = nil }

-- err = nil



ngx.print(res.stdout)

```



### Setup stdin beforehand



```lua

prog.stdin = 'this is neat!'

local res, err = prog('cat')



-- res = { stdout = "this is neat!", stderr = nil, exitcode = 0, termsig = nil }

-- err = nil



ngx.print(res.stdout)

```



### Call with explicit argv, stdin data, stdout/stderr callbacks



```lua

local res, err = prog( {

    argv = 'cat',

    stdin = 'fun!',

    stdout = function(data) print(data) end,

    stderr = function(data) print("error:", data) end

} )



-- res = { stdout = nil, stderr = nil, exitcode = 0, termsig = nil }

-- err = nil

-- 'fun!' is printed

```



Note: here `argv` is a string, which is fine if your program doesn't need

any arguments.



### Setup stdout/stderr callbacks



If you set `prog.stdout` or `prog.stderr` to a function, it will be called for

each chunk of stdout/stderr data received.



Please note that there's no guarantees of stdout/stderr being a complete

string, or anything particularly sensible for that matter!



```lua

prog.stdout = function(data)

    ngx.print(data)

    ngx.flush(true)

end



local res, err = prog('some-program')



```



### Treat timeouts as non-errors



By default, `sockexec` treats a timeout as an error. You can disable this by

setting the object's `timeout_fatal` key to false. Examples:



```lua

-- set timeout_fatal = false on the prog objects

prog.timeout_fatal = false



-- or, set it at calltime:

local res, err = prog({argv = {'cat'}, timeout_fatal = false})

```



### But I actually want a shell!



Not a problem! You can just do something like:



```lua

local res, err = prog('bash','-c','echo $PATH')

```



Or if you want to run an entire script:



```lua

prog.stdin = script_data

local res, err = prog('bash')



-- this is roughly equivalent to running `bash < script` on the CLI

```



### Daemonizing processes



I generally recommend against daemonizing processes - I think it's far

better to use some kind of message queue and/or supervision system, so

you can monitor processes, take actions on failure, and so on.



That said, if you want to spin off some process, you could use

`start-stop-daemon`, ie:



```lua

local res, err = prog('start-stop-daemon','--pidfile','/dev/null','--background','--exec','/usr/bin/sleep', '--start','--','10')

```



will spawn `sleep 10` as a detached background process.



If you don't want to deal with `start-stop-daemon`, I have a small utility

for spawning a background program called [idgaf](https://github.com/jprjr/idgaf), ie:



```lua

local res, err = prog('idgaf','sleep','10')

```



This will basically accomplish the same thing `start-stop-daemon` does without

requiring a billion flags.



## `resty.exec.socket` Usage



```lua

local exec_socket = require'resty.exec.socket'



-- you can specify timeout in milliseconds, optional

local client = exec_socket:new({ timeout = 60000 })



-- every new program instance requires a new

-- call to connect

local ok, err = client:connect('/tmp/exec.sock')



-- send program arguments, only accepts a table of

-- arguments

client:send_args({'cat'})



-- send data for stdin

client:send('hello there')



-- receive data

local data, typ, err = client:receive()



-- `typ` can be one of:

--    `stdout`   - data from the program's stdout

--    `stderr`   - data from the program's stderr

--    `exitcode` - the program's exit code

--    `termsig`  - if terminated via signal, what signal was used



-- if `err` is set, data and typ will be nil

-- common `err` values are `closed` and `timeout`

print(string.format('Received %s data: %s',typ,data)

-- will print 'Received stdout data: hello there'



client:send('hey this cat process is still running')

data, typ, err = client:receive()

print(string.format('Received %s data: %s',typ,data)

-- will print 'Received stdout data: hey this cat process is still running'



client:send_close() -- closes stdin

data, typ, err = client:receive()

print(string.format('Received %s data: %s',typ,data)

-- will print 'Received exitcode data: 0'



data, typ, err = client:receive()

print(err) -- will print 'closed'

```



### `client` object methods:



* **`ok, err = client:connect(path)`**



Connects via unix socket to the path given. If this is running

in nginx, the `unix:` string will be prepended automatically.



* **`bytes, err = client:send_args(args)`**



Sends a table of arguments to sockexec and starts the program.



* **`bytes, err = client:send_data(data)`**



Sends `data` to the program's standard input



* **`bytes, err = client:send(data)`**



Just a shortcut to `client:send_data(data)`



* **`bytes, err = client:send_close()`**



Closes the program's standard input. You can also send an empty

string, like `client:send_data('')`



* **`data, typ, err = client:receive()`**



Receives data from the running process. `typ` indicates the type

of data, which can be `stdout`, `stderr`, `termsig`, `exitcode`



`err` is typically either `closed` or `timeout`



* **`client:close()`**



Forcefully closes the client connection



* **`client:getfd()`**



A getfd method, useful if you want to monitor the underlying socket

connection in a select loop



## Some example nginx configs



Assuming you're running sockexec at `/tmp/exec.sock`



```

$ sockexec /tmp/exec.sock

```



Then in your nginx config:



```nginx

location /uname-1 {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        local data,err = prog('uname')

        if(err) then

            ngx.say(err)

        else

            ngx.say(data.stdout)

        end

    }

}

location /uname-2 {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        prog.argv = { 'uname', '-a' }

        local data,err = prog()

        if(err) then

            ngx.say(err)

        else

            ngx.say(data.stdout)

        end

    }

}

location /cat-1 {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        prog.stdin = 'this is neat!'

        local data,err = prog('cat')

        if(err) then

            ngx.say(err)

        else

            ngx.say(data.stdout)

        end

    }

}

location /cat-2 {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        local data,err = prog({argv = 'cat', stdin = 'awesome'})

        if(err) then

            ngx.say(err)

        else

            ngx.say(data.stdout)

        end

    }

}

location /slow-print {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        prog.stdout = function(v)

            ngx.print(v)

            ngx.flush(true)

        end

        prog('/usr/local/bin/slow-print')

    }

    # look in `/misc` of this repo for `slow-print`

}

location /shell {

    content_by_lua_block {

        local prog = require'resty.exec'.new('/tmp/exec.sock')

        local data, err = prog('bash','-c','echo $PATH')

        if(err) then

            ngx.say(err)

        else

            ngx.say(data.stdout)

        end

    }

}



```



## License



MIT license (see `LICENSE`)