Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/jfrazee/nifi-provenance-reporting-bundle

NiFi provenance reporting tasks
https://github.com/jfrazee/nifi-provenance-reporting-bundle

Last synced: about 1 month ago
JSON representation

NiFi provenance reporting tasks

Host: GitHub
URL: https://github.com/jfrazee/nifi-provenance-reporting-bundle
Owner: jfrazee
License: apache-2.0
Created: 2016-04-03T23:50:12.000Z (about 8 years ago)
Default Branch: master
Last Pushed: 2023-09-21T17:18:58.000Z (9 months ago)
Last Synced: 2024-02-11T22:47:25.881Z (5 months ago)
Language: Java
Homepage:
Size: 423 KB
Stars: 13
Watchers: 5
Forks: 9
Open Issues: 11
Metadata Files:
- Readme: README.md
- License: LICENSE

Lists

awesome-nifi - jfrazee/nifi-provenance-reporting-bundle - NiFi provenance reporting tasks (Processors and Bundles / Mailing List Best Of)
awesome-nifi - jfrazee/nifi-provenance-reporting-bundle - NiFi provenance reporting tasks (Processors and Bundles / Mailing List Best Of)
awesome-nifi - jfrazee/nifi-provenance-reporting-bundle - NiFi provenance reporting tasks (Processors and Bundles / Mailing List Best Of)

README

[![Build Status](https://travis-ci.org/jfrazee/nifi-provenance-reporting-bundle.svg?branch=master)](https://travis-ci.org/jfrazee/nifi-provenance-reporting-bundle)

# nifi-provenance-reporting-bundle

NiFi provenance reporting tasks.

## Table of Contents

- [Installation](#installation)
- [Tasks](#tasks)
- [ElasticsearchProvenanceReporter](#elasticsearchprovenancereporter)
- [HttpProvenanceReporter](#httpprovenancereporter)
- [Todo](#todo)

## Installation

```sh
$ mvn clean package
$ cp nifi-provenance-reporting-nar/target/nifi-provenance-reporting-nar-0.0.2-SNAPSHOT.nar $NIFI_HOME/lib
$ nifi restart
```

## Tasks

### ElasticsearchProvenanceReporter

Reporting task to write provenance events to an Elasticsearch index.

#### Reporting Task Properties

#### Example Event

### HttpProvenanceReporter

Reporting task to POST provenance events to an HTTP web service.

#### Reporting Task Properties

This reporting task can be configured to POST provenance events to an arbitrary web service. Here is an example of using it with Solr (10s commits):

## Todo

- Add batching support.
- Additional adapters:
- HDFS
- Tinkerpop
- NiFi site-to-site
- Optional inclusion of FlowFile contents.
- Create provenance event for runs of the reporting task.
- Example schemas/mappings for data sources (Elasticsearch mapping, Solr schema, JSON schema).
- Add testing.

## License