Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/twcable/grabbit
Grabbit - Fast Content Sync tool for AEM/CQ
https://github.com/twcable/grabbit
aem aem-tools groovy spring-batch
Last synced: 2 months ago
JSON representation
Grabbit - Fast Content Sync tool for AEM/CQ
- Host: GitHub
- URL: https://github.com/twcable/grabbit
- Owner: TWCable
- License: apache-2.0
- Created: 2015-02-27T00:24:35.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2020-05-19T15:40:32.000Z (over 4 years ago)
- Last Synced: 2024-09-30T04:01:37.374Z (3 months ago)
- Topics: aem, aem-tools, groovy, spring-batch
- Language: Groovy
- Homepage:
- Size: 4.32 MB
- Stars: 125
- Watchers: 30
- Forks: 64
- Open Issues: 89
-
Metadata Files:
- Readme: README.adoc
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
Awesome Lists containing this project
README
= Grabbit
:docsDir: docsimage:https://travis-ci.org/TWCable/grabbit.svg?branch=master[title = "Build Status", link = "https://travis-ci.org/TWCable/grabbit"] image:https://badge.waffle.io/TWCable/grabbit.png?label=ready&title=Ready[title = "Stories in Ready", link = "https://waffle.io/TWCable/grabbit"]
image:https://api.bintray.com/packages/twcable/aem/Grabbit/images/download.svg[title = "Download", link = "https://bintray.com/twcable/aem/Grabbit/_latestVersion"]
== Project Purpose
The purpose of Grabbit is to provide a fast and reliable way of copying content from one Sling (specifically Adobe CQ/AEM) instance to another.
Existing solutions have been tried and found insufficient for very large data sets (GB-TB), especially over a network. CQ's .zip packages are extremely space inefficient, causing a lot of extra I/O. http://jackrabbit.apache.org/filevault/usage.html[`vlt rcp`] and Mark Adamcin's http://adamcin.net/net.adamcin.recap/[`recap`] use essentially the same mechanism: WebDAV using XML, doing an HTTP handshake for every node and many sets of properties, which means that any latency whatsoever on the network hurts performance enormously.
Grabbit creates a stream of data using https://developers.google.com/protocol-buffers/[Google's Protocol Buffers] aka "ProtoBuf". Protocol Buffers are an extremely efficient (in terms of CPU, memory and wire size) binary protocol that includes compression.
Moreover, by doing a continuous stream, we avoid the latency issues. Depending on the size and nature of the data, as well as network latency, we have so far seen speed improvements ranging from 2 to 10 times that of Recap/vlt.
NOTE: "Grabbit" obviously refers to this "grabbing" content from one CQ/AEM instance and copying it to another. However it also refers to "Jackrabbit," the reference JCR implementation that the content is being copied to and from.
image:https://api.bintray.com/packages/twcable/aem/Grabbit/images/download.svg[title = "Download", link = "https://bintray.com/twcable/aem/Grabbit/_latestVersion"]
== Table of Contents
* General Information
** link:{docsDir}/RELEASE_NOTES.md[Releases]
** link:{docsDir}/AEMSupport.adoc[Installation & Version Support]* Using Grabbit
** link:{docsDir}/GeneralLayout.adoc[General Layout]
** link:{docsDir}/Running.adoc[Running]
** link:{docsDir}/Monitoring.adoc[Monitoring / Validating the Content Sync]
** link:{docsDir}/Cleaning.adoc[Cleaning Grabbit Job Repository]* Grabbit Development
** link:{docsDir}/GettingStarted.adoc[Getting Started]
** link:{docsDir}/Building.adoc[Building from Source]
** link:{docsDir}/RELEASING.adoc[Releasing A New Version]* link:{docsDir}/LibraryAttribution.adoc[Library Attribution]
* link:{docsDir}/LicenseInfo.adoc[License]