Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/richie765/psync

Bidirectional File Synchronizer written in Perl
https://github.com/richie765/psync

file-syncing perl

Last synced: about 2 months ago
JSON representation

Bidirectional File Synchronizer written in Perl

Awesome Lists containing this project

README 

        
psync

=====



Bidirectional File Synchronizer written in Perl.



Usage

=====



```

psync [-dv] [options]  

    -v --verbose     Verbose

    -d --debug       Debug output

    --tag   Metadata Tag

```



Two roots are required, all options are optional.



Simply specify two roots that you need to synchronize. A root can be

on the local filesystem e.g. `/home/user/path` or remote over ssh e.g. `user@host:/path`



Remote Syncronization

=====================

To synchronize with a remote location, the `psync` script needs to be in the path 

on the remote system. The name of the script must be the same as the one used to initiate the 

sync.



Keep in mind that when the `psync` script is updated, the remote script also needs to be updated.



I recommend only using SSH Public Key Authentication to

avoid any password interactions. Make sure SSH authentication between the systems works

properly before using `psync`.



Multidirectional sync

======================

Using the `--tag` option it is possible to synchronize the same root to a multitude of other roots

in any way you desire. You just have to specify a unique `tagname` for each unique

pair of roots (leg). Not specifying a `tagname` or using the same `tagname` on different legs 

will result in unexpected syncronization problems.



Metadata

========

Metadata that is needed to sync properly is stored in the `.psync` folder in each

root. 



When using the `--tag` option, additional metadata will be stored for each leg

in the same `.psync` folder.



Keep in mind that the metadata of both roots of a leg have to remain in sync. Normally this

will be the case so you don't need to worry about that. But when one of the roots is replaced

by an older backup, the metadata becomes out of sync. This can result in some

unexpected behavior including possible loss of some data. One way around this problem

is to start using a new `tagname` after the leg became out of sync.

This will cause both roots to be merged and no data will be lost (conflicts may occur though).

If you expect only small differences between the two roots you can simply use the same `tagname`

(or no `tagname`) as before, monitor the changes and make manual changes if needed. Make sure

you make a backup of both roots before syncing.



Examples

========

Just synchronize two roots on localhost



    psync /home/user/root1 /home/user/root2



Synchronize a local root with a remote root



    psync /home/user/root user@remote:/home/user/root



Synchronize two remote roots



    psync user@remote1:/home/user/root user@remote2:/home/user/root



Three-way sync



    psync --tag leg1 /home/user/root user@remote1:/home/user/root

    psync --tag leg2 /home/user/root user@remote2:/home/user/root



Design Guidelines

=================

This script was developed with the following ideas in mind, please maintain those ideas

when submitting pull-requests.



* The script should be a single file so it can be copied to other systems easily.

* The module dependencies should be kept low, only core and a few non-exotic modules are used.

* It should work with most Perl versions commonly in use (currently v5.10 is required)



TODO and Known Issues

=====================

`psync` has been working for me without major problems for over a year so it should be fairly stable.



* Only works on *NIX (only tested on OS X)

* Empty directories and deleted directories are not syncronized. Directories are only created when needed to copy a file and never deleted.

* Timestamps are not preserved or synchronized

* Possibly implement multidirectional (more then 2 roots) sync, probably as a separate script

* Improve code documentation

* Conflicts (simultaneous changes in both roots) are not handled by the script. They are just displayed to be handled manually directly on the filesystem.

* Eventhough `psync` is reasonably fast, it was not built for speed or efficiency rather it was made to be simple and reliable.

* Build in check to ensure local and remote `psync` scripts are compatible.

* Improve readability of the script output.

* A temp file (in the format `..`) could be left behind, probably when psync is interrupted during a transfer. This temp file will be synced on the next run (will be fixed in a while)

* Implement a lockfile to enforce only one sync at a time



Internals

=========



### Comparing of files

When a file is changed on both roots and still has an equal size, it will be compared. A file will also be compared after being copied to ensure its integrety. Comparing is done using a SHA512 digest.