https://github.com/mjpost/cachepipe

A package for caching shell commands.
https://github.com/mjpost/cachepipe

Last synced: about 1 month ago
JSON representation

A package for caching shell commands.

• Host: GitHub
• URL: https://github.com/mjpost/cachepipe
• Owner: mjpost
• Created: 2011-05-20T20:50:49.000Z (over 13 years ago)
• Default Branch: master
• Last Pushed: 2012-10-22T02:02:55.000Z (about 12 years ago)
• Last Synced: 2023-04-09T20:57:25.216Z (over 1 year ago)
• Language: Perl
• Homepage:
• Size: 116 KB
• Stars: 4
• Watchers: 1
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README

Awesome Lists containing this project

README

        
CachePipe is a simple-to-use caching pipeline.  By this we mean:

- *simple-to-use*: It is a simple wrapper around existing commands, and

  doesn't require you to learn an extensive system

- *caching*: commands are cached based on the contents of an

   explicitly-provided dependency list, and future invocations will

   not re-run the command if any of them hav changed

- *pipeline*: CachePipe is originally designed to be used as part of a

   long pipeline of many different steps

CachePipe uses git-style SHA-1 hashes of the dependencies and the

command invocation itself to determine when changes have been made.

This improves upon existing caching systems, which make use of

unreliable indicators of command completion such as timestamps or even

file existence.  It is implemented as a Perl module, and provides both

a Perl API and a command-line interface.

CachePipe supports a number of options.  See below for more detail.

-- EXAMPLE -----------------------------------------------------------

# Put this in your init file

    export CACHEPIPE=/path/to/cachepipe

    export PERL5LIB+=$CACHEPIPE

    . $CACHEPIPE/bashrc

You can now do the following:

- From the command line:

    $ cachecmd copy-file "cp a b" a b

    [copy-file] rebuilding...

      dep=a [CHANGED]

      dep=b [NOT FOUND]

      cmd=cp a b

      took 0 seconds (0s)

    $ cachecmd copy-file "cp a b" a b

    [copy-file] cached, skipping...

- From a Perl script:

    use CachePipe;

    my $pipe = new CachePipe();

    $pipe->cmd("copy-file","cp a b",["a","b"]);

-- OPTIONS -----------------------------------------------------------

You can tell the command to build the cache files without actually

running the command using the --cache-only flag between the command

name and the command, i.e.

    $ cachecmd copy-file --cache-only "cp a b" a b

This will compute the hash over all the dependencies and the command

as if the current state were the desired state.  Note that all the

dependencies must exist.

If you have already run a command, you can easily run it again:

    $ cachecmd copy-file --rerun

Sometimes you wan to just recompute the hashes of a run, as if it had

just been successfully completed, but without actually re-running the

command.  If the command has already been run once, you can type:

    $ cachecmd copy-file --cache-only

This flag also works with a new command, e.g.,

    $ cachecmd copy-file --cache-only "cp a b" a b

-- AUTHORS -----------------------------------------------------------

This was all Adam Lopez' idea.

Matt Post 

Adam Lopez