https://github.com/FINRAOS/herd

Herd is a managed data lake for the cloud. The Herd unified data catalog helps separate storage from compute in the cloud. Manage petabytes of data and make it accessible for data processing and analytical purposes by any cloud compute platform.
https://github.com/FINRAOS/herd

Last synced: about 2 months ago
JSON representation

Herd is a managed data lake for the cloud. The Herd unified data catalog helps separate storage from compute in the cloud. Manage petabytes of data and make it accessible for data processing and analytical purposes by any cloud compute platform.

• Host: GitHub
• URL: https://github.com/FINRAOS/herd
• Owner: FINRAOS
• License: apache-2.0
• Created: 2015-09-22T17:21:37.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2022-10-01T16:03:56.000Z (over 3 years ago)
• Last Synced: 2025-08-15T22:28:36.060Z (5 months ago)
• Language: Java
• Homepage: http://finraos.github.io/herd/
• Size: 221 MB
• Stars: 138
• Watchers: 40
• Forks: 41
• Open Issues: 126
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-java - Herd

README

          
## Overview [![Build Status](https://travis-ci.org/FINRAOS/herd.svg?branch=master)](https://travis-ci.org/FINRAOS/herd) [![Maven Central](https://img.shields.io/maven-central/v/org.finra.herd/herd.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:%22org.finra.herd%22%20AND%20a:%22herd%22)

Herd is big data governance for the cloud. The herd unified data catalog helps separate compute from storage in the cloud. Herd job orchestration manages your ETL and analytics processes while tracking all data in the catalog. Here is a quick summary of features:

- Unified Data Catalog

A centralized, auditable catalog for operational usage and data governance.

- Track Lineage

Capture data ancestry for regulatory, forensic, and analytical purposes

- Manage Clusters

Create and launch clusters; load data into clusters from catalog entries

- Orchestrate Jobs

Orchestrate clusters and catalog services to automate processing jobs

Find out more about herd features on our [GitHub project page](http://finraos.github.io/herd/#get_involved)

## Quick Start

The best way to start learning about herd is through these links. The demo installation process is quick and easy - you can have herd up and running in AWS in 10-15 minutes and start registering data immediately afterwards.

- [What is herd?](https://github.com/FINRAOS/herd/wiki/what-is-herd)

- [Demo Installation](https://github.com/FINRAOS/herd/wiki/demo-install)

- [Quick Start to Registering Data](https://github.com/FINRAOS/herd/wiki/quick-start-to-registering-data)

## Get Involved

We are actively seeking organizations and individuals that are interested in adopting herd and contributing to the development effort. Find out more in the [contributions section](http://finraos.github.io/herd/#get_involved) of our GitHub project page. If you have any questions or discussion topics, post them on [GitHub Issues](https://github.com/FINRAOS/herd/issues) or email us at herd@finra.org.

## License

Herd is licensed under [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0)