https://github.com/TWCable/grabbit

Grabbit - Fast Content Sync tool for AEM/CQ
https://github.com/TWCable/grabbit

aem aem-tools groovy spring-batch

Last synced: 29 days ago
JSON representation

Grabbit - Fast Content Sync tool for AEM/CQ

• Host: GitHub
• URL: https://github.com/TWCable/grabbit
• Owner: TWCable
• License: apache-2.0
• Created: 2015-02-27T00:24:35.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2020-05-19T15:40:32.000Z (over 4 years ago)
• Last Synced: 2024-10-30T19:04:26.238Z (about 1 month ago)
• Topics: aem, aem-tools, groovy, spring-batch
• Language: Groovy
• Homepage:
• Size: 4.32 MB
• Stars: 125
• Watchers: 30
• Forks: 64
• Open Issues: 89
• Metadata Files:
  • Readme: README.adoc
  • Contributing: CONTRIBUTING.md
  • License: LICENSE.txt

Awesome Lists containing this project

• jimsghstars - TWCable/grabbit - Grabbit - Fast Content Sync tool for AEM/CQ (Groovy)

README

        
= Grabbit

:docsDir: docs

image:https://travis-ci.org/TWCable/grabbit.svg?branch=master[title = "Build Status", link = "https://travis-ci.org/TWCable/grabbit"] image:https://badge.waffle.io/TWCable/grabbit.png?label=ready&title=Ready[title = "Stories in Ready", link = "https://waffle.io/TWCable/grabbit"]

image:https://api.bintray.com/packages/twcable/aem/Grabbit/images/download.svg[title = "Download", link = "https://bintray.com/twcable/aem/Grabbit/_latestVersion"]

== Project Purpose

The purpose of Grabbit is to provide a fast and reliable way of copying content from one Sling (specifically Adobe CQ/AEM) instance to another.

Existing solutions have been tried and found insufficient for very large data sets (GB-TB), especially over a network. CQ's .zip packages are extremely space inefficient, causing a lot of extra I/O. http://jackrabbit.apache.org/filevault/usage.html[`vlt rcp`] and Mark Adamcin's http://adamcin.net/net.adamcin.recap/[`recap`] use essentially the same mechanism: WebDAV using XML, doing an HTTP handshake for every node and many sets of properties, which means that any latency whatsoever on the network hurts performance enormously.

Grabbit creates a stream of data using https://developers.google.com/protocol-buffers/[Google's Protocol Buffers] aka "ProtoBuf". Protocol Buffers are an extremely efficient (in terms of CPU, memory and wire size) binary protocol that includes compression.

Moreover, by doing a continuous stream, we avoid the latency issues. Depending on the size and nature of the data, as well as network latency, we have so far seen speed improvements ranging from 2 to 10 times that of Recap/vlt.

NOTE: "Grabbit" obviously refers to this "grabbing" content from one CQ/AEM instance and copying it to another. However it also refers to "Jackrabbit," the reference JCR implementation that the content is being copied to and from.

image:https://api.bintray.com/packages/twcable/aem/Grabbit/images/download.svg[title = "Download", link = "https://bintray.com/twcable/aem/Grabbit/_latestVersion"]

== Table of Contents

* General Information

** link:{docsDir}/RELEASE_NOTES.md[Releases]

** link:{docsDir}/AEMSupport.adoc[Installation & Version Support]

* Using Grabbit

** link:{docsDir}/GeneralLayout.adoc[General Layout]

** link:{docsDir}/Running.adoc[Running]

** link:{docsDir}/Monitoring.adoc[Monitoring / Validating the Content Sync]

** link:{docsDir}/Cleaning.adoc[Cleaning Grabbit Job Repository]

* Grabbit Development

** link:{docsDir}/GettingStarted.adoc[Getting Started]

** link:{docsDir}/Building.adoc[Building from Source]

** link:{docsDir}/RELEASING.adoc[Releasing A New Version]

* link:{docsDir}/LibraryAttribution.adoc[Library Attribution]

* link:{docsDir}/LicenseInfo.adoc[License]