https://github.com/flonatel/pipexec
Handling pipe of commands like a single command
https://github.com/flonatel/pipexec
Last synced: 3 months ago
JSON representation
Handling pipe of commands like a single command
- Host: GitHub
- URL: https://github.com/flonatel/pipexec
- Owner: flonatel
- License: gpl-2.0
- Created: 2014-03-16T18:34:35.000Z (almost 12 years ago)
- Default Branch: master
- Last Pushed: 2024-10-16T19:20:13.000Z (about 1 year ago)
- Last Synced: 2024-12-08T00:28:10.530Z (about 1 year ago)
- Language: C
- Size: 659 KB
- Stars: 516
- Watchers: 13
- Forks: 19
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
- awesome-tools - flonatel/pipexec - Handling pipe of commands like a single command (Process Manager / Selfhosted)
README
pipexec
=======
Build a network of processes and connecting pipes - and have them act like a single process.
[](https://github.com/flonatel/pipexec/actions/workflows/compile-check.yml)
[](https://github.com/flonatel/pipexec/actions/workflows/codeql-analysis.yml)
[](https://github.com/flonatel/pipexec/releases/latest)
[](https://repology.org/project/pipexec/versions)
[](#license)
[](https://github.com/flonatel/pipexec/issues)
# Introduction and Purpose #
*pipexec* has two major use cases.
## Use Case 1: Handling Arbitrary Pipes between Processes ##
### Basics ###
When it comes to pipes in shells many tutorials introduce
stdin, stdout and stderr which
map to file descriptors 0, 1 and 2 respectively.
If you want to know how many lines contains the word *bird* in
chapter 1 and 2 of your text, you can use a command like:
$ cat Chap1.txt Chap2.txt | grep bird | wc -l
And pictures like this are shown to explain what happens internally:
### Advanced ###
The more advanced sections provide information how to use constructs
like 2>&1 to redirect stderr to
stdout. And then you might come to the sections for the
pros and nerds. There is explained that you can build up a whole tree
of processes like
$ find / 1> >(grep .txt) 2> >(wc >/tmp/w.log)
### The Hidden Universe of File-Descriptors, Processes and Pipes ###
Nobody will tell you:
1. stdin, stdout and stderr are
artificial definitions.
2. Also the relation to file descriptors 0, 1 and 2 is artificial.
3. There are more than three file descriptors. On a typical Linux
system each process has by default 1024 - which can be increased if
needed.
4. From starting up processes and generating pipes between them there
is mostly no limitation on system level; shells only support this in a
very limited way.
This is the moment when *pipexec* drops in: with *pipexec* you can
start up any kind of processes and build up pipes between them as you want.
#### Cyclic ####
$ pipexec -- [ A /usr/bin/cmd1 ] [ B /usr/bin/cmd2 ] "{A:1>B:0}" "{B:1>A:0}"
gives
#### Complex ####
*pipexec* supports any directed graph of processes and pipes like
## Use Case 2: Handle Bunch of Processes like one single Process ##
Most systems to start and run processes during system start-up time do
not support pipe. If you need to run a pipe of programs from an
/etc/init.d script you are mostly lost.
Depending on your distribution you can be happy if it starts up - but
when it comes to stopping, at least the current Debian
start-stop-daemon and RHEL 6 daemon function fail.
Also here *pipexec* comes in: it starts up processes piped together,
acts like a single process and supports pid file handling.
# Usage
$ ./pipexec -h
pipexec version 2.5.5
(c) 2014-2015,2022 by Andreas Florath
License GPLv2+: GNU GPL version 2 or later .
Usage: pipexec [options] -- process-pipe-graph
Options:
-h display this help
-k kill all child processes when one
terminates abnormally
-l logfd set fd which is used for logging
-p pidfile specify a pidfile
-s sleep_time time to wait before a restart
process-pipe-graph is a list of process descriptions
and pipe descriptions.
process description: '[ NAME /path/to/proc ]'
pipe description: '{NAME1:fd1>NAME2:fd2}'
Example:
$ pipexec -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}'
-rw-r--r-- 1 florath florath 18025 Mar 16 19:36 LICENSE
Be sure to escape pipe descriptions. Brackets for the command '[]' must be
separated by space! Definitions for pipes '{}' must not contain spaces!
It is possible to specify a fd for logging.
$ pipexec -l 2 -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}'
2014-05-15 16:30:35;pipexec;23978;pipexec version 2.4
2014-05-15 16:30:35;pipexec;23978;Number of commands in command line [2]
2014-05-15 16:30:35;pipexec;23978;Number of pipes in command line [1]
2014-05-15 16:30:35;pipexec;23978;[LS] command_info path [/bin/ls]
2014-05-15 16:30:35;pipexec;23978;[GREP] command_info path [/bin/grep]
2014-05-15 16:30:35;pipexec;23978;{0} Pipe [LS] [1] > [GREP] [0]
2014-05-15 16:30:35;pipexec;23978;Cannot set restart flag - process will terminate
2014-05-15 16:30:35;pipexec;23978;Start all [2] children
[...]
Or
$ pipexec -l 7 -- [ LS /bin/ls -l ] [ GREP /bin/grep LIC ] '{LS:1>GREP:0}' 7>/tmp/pipexec.log
-rw-r--r-- 1 florath florath 18025 Mar 16 19:53 LICENSE
$ head -2 /tmp/pipexec.log
2014-05-15 16:30:35;pipexec;23978;pipexec version 2.4
2014-05-15 16:30:35;pipexec;23978;Number of commands in command line [2]
# Installation #
## From Packages ##
The following Linux distributions include the package. You can
install pipexec with the distribution's package manager:
* [Debian](https://packages.debian.org/search?keywords=pipexec)
* [Ubuntu](https://packages.ubuntu.com/search?keywords=pipexec)
* [Alpine Linux](https://pkgs.alpinelinux.org/packages?name=pipexec)
* [Archlinux](https://aur.archlinux.org/packages/pipexec)
* [Raspbian](https://www.raspbian.org/)
* [Kali](https://pkg.kali.org/pkg/pipexec)
* [AOS](http://aos.ion.nu/buildwiki/pipexec.html)
* [Linux Mint](https://community.linuxmint.com/software/view/pipexec)
## From Source ##
[Download the latest tar ball](https://github.com/flonatel/pipexec/releases)
$ tar -xf pipexec-X.Y.Z.tar.xz
$ mkdir PIPEXECBUILD
$ cd PIPEXECBUILD
$ ${PWD}/../pipexec-X.Y.Z/configure
$ make
There will be three binaries in the bin directory: pipexec, ptee and
peet. You can copy / install them as you need.
# Copyright #
copyright 2015, 2022, 2024 by Andreas Florath
License: see LICENSE file