https://github.com/dspinellis/dgsh

Shell supporting pipelines to and from multiple processes
https://github.com/dspinellis/dgsh

pipeline shell unix unix-shell

Last synced: 5 days ago
JSON representation

Shell supporting pipelines to and from multiple processes

• Host: GitHub
• URL: https://github.com/dspinellis/dgsh
• Owner: dspinellis
• License: other
• Created: 2013-01-13T11:23:10.000Z (almost 12 years ago)
• Default Branch: master
• Last Pushed: 2024-05-03T17:14:50.000Z (8 months ago)
• Last Synced: 2024-12-15T12:06:18.250Z (12 days ago)
• Topics: pipeline, shell, unix, unix-shell
• Language: C
• Homepage: http://www.spinellis.gr/sw/dgsh/
• Size: 2.5 MB
• Stars: 331
• Watchers: 17
• Forks: 25
• Open Issues: 16
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Citation: CITATION.cff

Awesome Lists containing this project

README

        
## dgsh: The Directed Graph Shell

[![Build Status](https://travis-ci.org/dspinellis/dgsh.svg?branch=master)](https://travis-ci.org/dspinellis/dgsh)

The directed graph shell, *dgsh*, allows the expressive expression of efficient big data set and streams processing pipelines using existing Unix tools as well as custom-built components. It is a Unix-style shell allowing the specification of pipelines with non-linear scatter-gather operations. These form a directed acyclic process graph, which is typically executed by multiple processor cores, thus increasing the operation's processing throughput.

You can find a complete introduction, reference documentation,

and illustrated examples in the suite's

[web site](http://www.spinellis.gr/sw/dgsh/).

See also,

a [quick video overview](https://youtu.be/crqzO4YanwA) and

the associated (open access) paper,

[Extending Unix pipelines to DAGs](http://dx.doi.org/10.1109/TC.2017.2695447),

published in the *IEEE Transactions on Computers*, 66(9):1547–1561, 2017.