{"id":14988014,"url":"https://github.com/apache/nifi-minifi-cpp","last_synced_at":"2025-05-16T15:08:54.453Z","repository":{"id":37502941,"uuid":"56750161","full_name":"apache/nifi-minifi-cpp","owner":"apache","description":"Apache NiFi - MiNiFi C++","archived":false,"fork":false,"pushed_at":"2025-04-25T15:29:03.000Z","size":46792,"stargazers_count":152,"open_issues_count":12,"forks_count":94,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-05-08T16:21:30.496Z","etag":null,"topics":["cpp","hacktoberfest","minifi","nifi"],"latest_commit_sha":null,"homepage":"https://nifi.apache.org/minifi/","language":"C++","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-04-21T07:00:06.000Z","updated_at":"2025-04-25T15:29:07.000Z","dependencies_parsed_at":"2024-03-28T09:24:59.614Z","dependency_job_id":"8110c870-170a-48d9-8ac8-b2d9b35e2121","html_url":"https://github.com/apache/nifi-minifi-cpp","commit_stats":{"total_commits":1773,"total_committers":60,"mean_commits":29.55,"dds":0.7879300620417371,"last_synced_commit":"fafb1f9550541b597611aef8fd33992c7ae56a5a"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fnifi-minifi-cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fnifi-minifi-cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fnifi-minifi-cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fnifi-minifi-cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/nifi-minifi-cpp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254553958,"owners_count":22090417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","hacktoberfest","minifi","nifi"],"created_at":"2024-09-24T14:15:57.406Z","updated_at":"2025-05-16T15:08:49.401Z","avatar_url":"https://github.com/apache.png","language":"C++","readme":"\u003c!--\n  Licensed to the Apache Software Foundation (ASF) under one or more\n  contributor license agreements.  See the NOTICE file distributed with\n  this work for additional information regarding copyright ownership.\n  The ASF licenses this file to You under the Apache License, Version 2.0\n  (the \"License\"); you may not use this file except in compliance with\n  the License.  You may obtain a copy of the License at\n      http://www.apache.org/licenses/LICENSE-2.0\n  Unless required by applicable law or agreed to in writing, software\n  distributed under the License is distributed on an \"AS IS\" BASIS,\n  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n  See the License for the specific language governing permissions and\n  limitations under the License.\n--\u003e\n\n[\u003cimg src=\"https://nifi.apache.org/assets/images/minifi/minifi-logo.svg\" width=\"300\" height=\"126\" alt=\"Apache NiFi MiNiFi\"/\u003e](https://nifi.apache.org/minifi/)\n\n# Apache NiFi -  MiNiFi - C++ [![MiNiFi-CPP CI](https://github.com/apache/nifi-minifi-cpp/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/apache/nifi-minifi-cpp/actions/workflows/ci.yml?query=workflow%3A%22MiNiFi-CPP+CI%22+branch%3Amain)\n\nMiNiFi is a child project effort of Apache NiFi.  This repository is for a native implementation in C++.\n\n## Table of Contents\n\n- [Features](#features)\n- [Caveats](#caveats)\n- [Getting Started](#getting-started)\n  - [System Requirements](#system-requirements)\n  - [Bootstrapping](#bootstrapping)\n  - [Building For Other Distros](#building-for-other-distros)\n  - [Installation](#installation)\n  - [Configuring](#configuring)\n  - [Running](#running)\n  - [Deploying](#deploying)\n  - [Cleaning](#cleaning)\n  - [Extensions](#extensions)\n  - [Security](#security)\n- [Operations](#operations)\n- [Issue Tracking](#issue-tracking)\n- [Documentation](#documentation)\n- [Examples](#examples)\n- [License](#license)\n\n## Features\n\nApache NiFi - MiNiFi C++ is a complementary data collection approach that supplements the core tenets of [NiFi](http://nifi.apache.org/) in dataflow management, focusing on the collection of data at the source of its creation.  The C++ implementation is an additional implementation to the one in Java with the aim of an even smaller resource footprint.\n\nSpecific goals for MiNiFi comprise:\n- small and lightweight footprint\n- central management of agents\n- generation of data provenance\n- integration with NiFi for follow-on dataflow management and full chain of custody of information\n\nPerspectives of the role of MiNiFi should be from the perspective of the agent acting immediately at, or directly adjacent to, source sensors, systems, or servers.\n\n### Expression Language\n\nA subset of the Apache NiFi [Expression Language](EXPRESSIONS.md) is supported.\n\n\n### Processors\n\nMiNiFi - C++ supports the following C++ processors:\n\nThe following table lists the base set of processors.\n\n| Extension Set | Processors                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |\n|---------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| **Base**      | [AppendHostInfo](PROCESSORS.md#appendhostinfo)\u003cbr/\u003e[AttributesToJSON](PROCESSORS.md#attributestojson)\u003cbr/\u003e[DefragmentText](PROCESSORS.md#defragmenttext)\u003cbr/\u003e[ExecuteProcess](PROCESSORS.md#executeprocess)\u003cbr/\u003e[ExtractText](PROCESSORS.md#extracttext)\u003cbr/\u003e[FetchFile](PROCESSORS.md#fetchfile)\u003cbr/\u003e[GenerateFlowFile](PROCESSORS.md#generateflowfile)\u003cbr/\u003e[GetFile](PROCESSORS.md#getfile)\u003cbr/\u003e[GetTCP](PROCESSORS.md#gettcp)\u003cbr/\u003e[HashContent](PROCESSORS.md#hashcontent)\u003cbr/\u003e[InvokeHTTP](PROCESSORS.md#invokehttp)\u003cbr/\u003e[ListenSyslog](PROCESSORS.md#listensyslog)\u003cbr/\u003e[ListenTCP](PROCESSORS.md#listentcp)\u003cbr/\u003e[ListenUDP](PROCESSORS.md#listenudp)\u003cbr/\u003e[ListFile](PROCESSORS.md#listfile)\u003cbr/\u003e[LogAttribute](PROCESSORS.md#logattribute)\u003cbr/\u003e[PutFile](PROCESSORS.md#putfile)\u003cbr/\u003e[PutTCP](PROCESSORS.md#puttcp)\u003cbr/\u003e[PutUDP](PROCESSORS.md#putudp)\u003cbr/\u003e[ReplaceText](PROCESSORS.md#replacetext)\u003cbr/\u003e[RetryFlowFile](PROCESSORS.md#retryflowfile)\u003cbr/\u003e[RouteOnAttribute](PROCESSORS.md#routeonattribute)\u003cbr/\u003e[RouteText](PROCESSORS.md#routetext)\u003cbr/\u003e[SegmentContent](PROCESSORS.md#segmentcontent)\u003cbr/\u003e[SplitContent](PROCESSORS.md#splitcontent)\u003cbr/\u003e[SplitText](PROCESSORS.md#splittext)\u003cbr/\u003e[TailFile](PROCESSORS.md#tailfile)\u003cbr/\u003e[UpdateAttribute](PROCESSORS.md#updateattribute) |\n\nThe next table outlines CMAKE flags that correspond with MiNiFi extensions. Extensions that are enabled by default ( such as RocksDB ), can be disabled with the respective CMAKE flag on the command line.\n\n| Extension Set                    | Processors and Controller Services                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | CMAKE Flag                   |\n|----------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------|\n| Archive Extensions               | [ApplyTemplate](PROCESSORS.md#applytemplate)\u003cbr/\u003e[BinFiles](PROCESSORS.md#binfiles)\u003cbr/\u003e[CompressContent](PROCESSORS.md#compresscontent)\u003cbr/\u003e[ManipulateArchive](PROCESSORS.md#manipulatearchive)\u003cbr/\u003e[MergeContent](PROCESSORS.md#mergecontent)\u003cbr/\u003e[FocusArchiveEntry](PROCESSORS.md#focusarchiveentry)\u003cbr/\u003e[UnfocusArchiveEntry](PROCESSORS.md#unfocusarchiveentry)                                                                                                                                                                                                                                                                          | -DBUILD_LIBARCHIVE=ON        |\n| AWS                              | [AWSCredentialsService](CONTROLLERS.md#awscredentialsservice)\u003cbr/\u003e[PutS3Object](PROCESSORS.md#puts3object)\u003cbr/\u003e[DeleteS3Object](PROCESSORS.md#deletes3object)\u003cbr/\u003e[FetchS3Object](PROCESSORS.md#fetchs3object)\u003cbr/\u003e[ListS3](PROCESSORS.md#lists3)                                                                                                                                                                                                                                                                                                                                                                                               | -DENABLE_AWS=ON              |\n| Azure                            | [AzureStorageCredentialsService](CONTROLLERS.md#azurestoragecredentialsservice)\u003cbr/\u003e[PutAzureBlobStorage](PROCESSORS.md#putazureblobstorage)\u003cbr/\u003e[DeleteAzureBlobStorage](PROCESSORS.md#deleteazureblobstorage)\u003cbr/\u003e[FetchAzureBlobStorage](PROCESSORS.md#fetchazureblobstorage)\u003cbr/\u003e[ListAzureBlobStorage](PROCESSORS.md#listazureblobstorage)\u003cbr/\u003e[PutAzureDataLakeStorage](PROCESSORS.md#putazuredatalakestorage)\u003cbr/\u003e[DeleteAzureDataLakeStorage](PROCESSORS.md#deleteazuredatalakestorage)\u003cbr/\u003e[FetchAzureDataLakeStorage](PROCESSORS.md#fetchazuredatalakestorage)\u003cbr/\u003e[ListAzureDataLakeStorage](PROCESSORS.md#listazuredatalakestorage) | -DENABLE_AZURE=ON            |\n| CivetWeb                         | [ListenHTTP](PROCESSORS.md#listenhttp)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | -DENABLE_CIVET=ON            |\n| Couchbase                        | [CouchbaseClusterService](CONTROLLERS.md#couchbaseclusterservice)\u003cbr/\u003e[PutCouchbaseKey](PROCESSORS.md#putcouchbasekey)\u003cbr/\u003e[GetCouchbaseKey](PROCESSORS.md#getcouchbasekey)                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | -DENABLE_COUCHBASE=ON        |\n| Elasticsearch                    | [ElasticsearchCredentialsControllerService](CONTROLLERS.md#elasticsearchcredentialscontrollerservice)\u003cbr/\u003e[PostElasticsearch](PROCESSORS.md#postelasticsearch)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | -DENABLE_ELASTICSEARCH=ON    |\n| ExecuteProcess (Linux and macOS) | [ExecuteProcess](PROCESSORS.md#executeprocess)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | -DENABLE_EXECUTE_PROCESS=ON  |\n| Google Cloud Platform            | [DeleteGCSObject](PROCESSORS.md#deletegcsobject)\u003cbr\u003e[FetchGCSObject](PROCESSORS.md#fetchgcsobject)\u003cbr\u003e[GCPCredentialsControllerService](CONTROLLERS.md#gcpcredentialscontrollerservice)\u003cbr\u003e[ListGCSBucket](PROCESSORS.md#listgcsbucket)\u003cbr\u003e[PutGCSObject](PROCESSORS.md#putgcsobject)                                                                                                                                                                                                                                                                                                                                                           | -DENABLE_GCP=ON              |\n| Grafana Loki                     | [PushGrafanaLokiREST](PROCESSORS.md#pushgrafanalokirest)\u003cbr\u003e[PushGrafanaLokiGrpc](PROCESSORS.md#pushgrafanalokigrpc)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | -DENABLE_GRAFANA_LOKI=ON     |\n| Kafka                            | [PublishKafka](PROCESSORS.md#publishkafka)\u003cbr\u003e[ConsumeKafka](PROCESSORS.md#consumekafka)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | -DENABLE_KAFKA=ON       |\n| Kubernetes (Linux)               | [KubernetesControllerService](CONTROLLERS.md#kubernetescontrollerservice)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | -DENABLE_KUBERNETES=ON       |\n| Lua Scripting                    | [ExecuteScript](PROCESSORS.md#executescript)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | -DENABLE_LUA_SCRIPTING=ON    |\n| MQTT                             | [ConsumeMQTT](PROCESSORS.md#consumemqtt)\u003cbr/\u003e[PublishMQTT](PROCESSORS.md#publishmqtt)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | -DENABLE_MQTT=ON             |\n| OPC                              | [FetchOPCProcessor](PROCESSORS.md#fetchopcprocessor)\u003cbr/\u003e[PutOPCProcessor](PROCESSORS.md#putopcprocessor)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | -DENABLE_OPC=ON              |\n| OpenCV                           | [CaptureRTSPFrame](PROCESSORS.md#capturertspframe)\u003cbr/\u003e[MotionDetector](PROCESSORS.md#motiondetector)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | -DENABLE_OPENCV=ON           |\n| PDH (Windows)                    | [PerformanceDataMonitor](PROCESSORS.md#performancedatamonitor)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | -DENABLE_PDH=ON              |\n| ProcFs (Linux)                   | [ProcFsMonitor](PROCESSORS.md#procfsmonitor)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | -DENABLE_PROCFS=ON           |\n| Python Scripting                 | [ExecuteScript](PROCESSORS.md#executescript)\u003cbr\u003e[ExecutePythonProcessor](PROCESSORS.md#executepythonprocessor)\u003cbr/\u003e**Custom Python Processors**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | -DENABLE_PYTHON_SCRIPTING=ON |\n| SMB (Windows)                    | [FetchSmb](PROCESSORS.md#fetchsmb)\u003cbr/\u003e[ListSmb](PROCESSORS.md#listsmb)\u003cbr/\u003e[PutSmb](PROCESSORS.md#putsmb)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | -DENABLE_SMB=ON              |\n| SFTP                             | [FetchSFTP](PROCESSORS.md#fetchsftp)\u003cbr/\u003e[ListSFTP](PROCESSORS.md#listsftp)\u003cbr/\u003e[PutSFTP](PROCESSORS.md#putsftp)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | -DENABLE_SFTP=ON             |\n| SQL                              | [ExecuteSQL](PROCESSORS.md#executesql)\u003cbr/\u003e[PutSQL](PROCESSORS.md#putsql)\u003cbr/\u003e[QueryDatabaseTable](PROCESSORS.md#querydatabasetable)\u003cbr/\u003e                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | -DENABLE_SQL=ON              |\n| Splunk                           | [PutSplunkHTTP](PROCESSORS.md#putsplunkhttp)\u003cbr/\u003e[QuerySplunkIndexingStatus](PROCESSORS.md#querysplunkindexingstatus)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | -DENABLE_SPLUNK=ON           |\n| Systemd (Linux)                  | [ConsumeJournald](PROCESSORS.md#consumejournald)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | -DENABLE_SYSTEMD=ON          |\n| Windows Event Log (Windows)      | [ConsumeWindowsEventLog](PROCESSORS.md#consumewindowseventlog)\u003cbr/\u003e[TailEventLog](PROCESSORS.md#taileventlog)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | -DENABLE_WEL=ON              |\n\n Please see our [Python guide](extensions/python/PYTHON.md) on how to write Python processors and use them within MiNiFi C++.\n\n## Caveats\n* We follow semver with regards to API compatibility, but no ABI compatibility is provided. See [semver's website](https://semver.org/) for more information\n* Build and usage currently only supports Windows, Linux and macOS environments. MiNiFi C++ can be built and run through the Windows Subsystem for Linux but we provide no support for this platform.\n* Provenance events generation is supported and are persisted using RocksDB. Volatile repositories can be used on systems without persistent storage.\n* If MiNiFi C++ is built with the OPC-UA extension enabled, it bundles [open62541](https://open62541.org/), which is available under the Mozilla Public License Version 2.0, a Category B license under [ASF 3rd party license policy](https://www.apache.org/legal/resolved.html#category-b).\n* If MiNiFi C++ is built using one of the following flags: -DMINIFI_INCLUDE_VC_REDIST_MERGE_MODULES=ON, -DMINIFI_INCLUDE_VC_REDIST_DLLS=ON, -DMINIFI_INCLUDE_UCRT_DLLS=ON, and packaged on Windows, the resulting MSI may not be publicly redistributed under the Apache license, because it contains Microsoft redistributable DLLs, which fall under Category X of the [ASF 3rd party license policy](https://www.apache.org/legal/resolved.html#category-x).\n\n## System Requirements\n\n### To build\n\n#### Utilities\n* CMake 3.24 or greater\n* gcc 11 or greater\n* bison 3.0.x+ (3.2 has been shown to fail builds)\n* flex 2.6 or greater\n\n##### External Projects\n\nThe following utilities are needed to build external projects, when bundled\nversions of OpenSSL, cURL, or zlib are used:\n\n* patch\n* autoconf\n* automake\n* libtool\n\n**NOTE:** if Expression Language support is enabled, FlexLexer must be in the include path and the version must be compatible with the version of flex used when generating lexer sources. On macOS and FreeBSD, make sure that the system version is overridden with the correct flex version, which may require merging your CPPFLAGS into CFLAGS and CXXFLAGS. Lexer source generation is automatically performed during CMake builds. To re-generate the sources, remove:\n\n * build/el-generated/Parser.cpp\n * build/el-generated/Parser.hpp\n * build/el-generated/Scanner.cpp\n * build/el-generated/location.hh\n * build/el-generated/position.hh\n * build/el-generated/stack.hh\n\nand rebuild.\n\n#### System Libraries / Development Headers Required\n* Python 3 and development headers -- Required if Python support is enabled\n* perl -- Required for OpenSSL configuration\n* NASM -- Required for OpenSSL only on Windows\n* jom (optional) -- for parallel build of OpenSSL on Windows\n\n**NOTE:** On Windows if Strawberry Perl is used the `${StrawberryPerlRoot}\\c\\bin` directory should not be part of the PATH environment variable as the patch executable in this directory interferes with git's patch executable. Alternatively [scoop](https://scoop.sh/) package manager can also be used to install Strawberry Perl using the command `scoop install perl` that does not pollute the PATH variable. Also on Windows CMake's CPack is used for MSI generation, building WIX files and calling WIX toolset tools to create an MSI. If Chocolatey package manager is used its CPack can conflict with CMake, so make sure that CMake's CPack is found in the %PATH% before that.\n\n#### Bundled Thirdparty Dependencies\n\nThe [NOTICE](NOTICE) file lists all the bundled thirdparty dependencies that are built and linked statically to MiNiFi or one of its extensions. The licenses of these projects can be found in the [LICENSE](LICENSE) file.\n\n#### CentOS 7\n\nAdditional environmental preparations are required for CentOS 7 support. Before\nbuilding, install and enable the devtoolset-10 SCL:\n\n```\n$ sudo yum install centos-release-scl\n$ sudo yum install devtoolset-10\n$ scl enable devtoolset-10 bash\n```\n\nFinally, it is required to add the `-lrt` compiler flag by using the\n`-DCMAKE_CXX_FLAGS=-lrt` flag when invoking cmake.\n\nOn all distributions please use -DUSE_SHARED_LIBS=OFF to statically link zlib, libcurl, and OpenSSL.\n\n#### Windows\n  Build and Installation has been tested with Windows 10 using Visual Studio 2022.\n  You can build and create an MSI via the CPACK command. This requires the installation of the WiX\n  toolset (http://wixtoolset.org/). To do this, open up a prompt into your build directory and\n  type 'cpack' . The CPACK command will automatically generate and provide you a path to the distributable\n  msi file. See [Windows.md](Windows.md) for more details.\n\n### To run\n\n#### System Libraries Required\n* Python 3 -- Required if Python support is enabled\n\nThe needed dependencies can be installed with the following commands for:\n\n##### rpm-based Linux Distributions\n\n**NOTE:** it is recommended to use the newest compiler using the latest devtoolset-*/gcc-toolset-*/llvm-toolset-* packages from the Software Collections (SCL).\n\n```\n# ~/Development/code/apache/nifi-minifi-cpp on git:master\ndnf install cmake \\\n  gcc gcc-c++ \\\n  git \\\n  bison \\\n  flex \\\n  patch \\\n  autoconf \\\n  automake \\\n  libtool \\\n  libuuid libuuid-devel \\\n  openssl-devel \\\n  bzip2-devel \\\n  xz-devel \\\n  doxygen \\\n  zlib-devel\n# (Optional) for building Python support\ndnf install python36-devel\n# (Optional) for building docker image\ndnf install docker\n# (Optional) for system integration tests\ndnf install docker python-virtualenv\n```\n\n##### Aptitude based Linux Distributions\n```\n# ~/Development/code/apache/nifi-minifi-cpp on git:master\napt install cmake \\\n  gcc g++ \\\n  git \\\n  bison \\\n  flex \\\n  patch \\\n  ca-certificates \\\n  autoconf \\\n  automake \\\n  libtool \\\n  libcurl4-openssl-dev \\\n  uuid-dev uuid \\\n  libssl-dev \\\n  libbz2-dev liblzma-dev \\\n  doxygen \\\n  zlib1g-dev\n# (Optional) for building Python support\napt install libpython3-dev\n# (Optional) for building docker image\napt install docker.io\n# (Optional) for system integration tests\napt install docker.io python-virtualenv\n```\n\n##### macOS Using Homebrew (with XCode Command Line Tools installed)\n```\n# ~/Development/code/apache/nifi-minifi-cpp on git:master\nbrew install cmake \\\n  flex \\\n  patch \\\n  autoconf \\\n  automake \\\n  libtool \\\n  ossp-uuid \\\n  openssl \\\n  python \\\n  xz \\\n  bzip2 \\\n  doxygen \\\n  zlib\nbrew install curl\nbrew link curl --force\n# (Optional) for building docker image/running system integration tests\n# Install docker using instructions at https://docs.docker.com/docker-for-mac/install/\nsudo pip install virtualenv\n# It is recommended that you install bison from source as HomeBrew now uses an incompatible version of Bison\n```\n\n\n## Getting Started\n\n### Bootstrapping\n\nMiNiFi C++ offers bootstrap scripts that will bootstrap the cmake and build process for you without the need to install dependencies yourself.\n\n#### Python based bootstrapping (recommended)\n##### Linux\nPrerequisites:\n- [python](https://docs.python.org/)\n- [venv](https://docs.python.org/3/library/venv.html)\n\n```bash\n./bootstrap/py_bootstrap.sh\n```\n\n#### macOS\nPrerequisites:\n- [python](https://docs.python.org/)\n- [venv](https://docs.python.org/3/library/venv.html)\n- [Homebrew](https://brew.sh/)\n```bash\n./bootstrap/py_bootstrap.sh\n```\n\n#### Windows\nPrerequisites:\n- [python](https://docs.python.org/)\n- [venv](https://docs.python.org/3/library/venv.html)\n- [chocolatey](https://chocolatey.org/)\n```dos\n.\\bootstrap\\py_bootstrap.bat\n```\n\nThis will set up a virtual environment in the bootstrap folder, and guide you through the build process.\n\n#### Shell based bootstrapping (linux and macOS)\n- Please run the command `bootstrap.sh` from the root of the MiNiFi C++ source tree.\n\n- Per the table, below, you will be presented with a menu guided bootstrap process. You may enable and disable extensions ( further defined below ). Once you are finished selecting the features\n  you wish to build, enter P to continue with the process. CMAKE dependencies will be resolved for your distro. You may enter command line options -n to force yes to all prompts\n  (including the package installation prompts ) and -b to automatically run make once the cmake process is complete. Alternatively, you may include the package argument to bootstrap, -p,\n  which will run make package.\n\n- If you provide -b or -p to bootstrap.sh, you do not need to follow the Building section, below. If you do not provide these arguments you may skip the cmake .. section from Building.\n\n- Using the Release build profile is recommended to reduce binary size. (~200 MB vs ~30 MB)\n\n  ```\n  # ~/Development/code/apache/nifi-minifi-cpp on git:master\n  $ ./bootstrap.sh\n  # CMAKE Build dir exists, should we overwrite your build directory before we begin?\n    If you have already bootstrapped, bootstrapping again isn't necessary to run make [ Y/N ] Y\n    ****************************************\n     Select MiNiFi C++ Features to toggle.\n    ****************************************\n    A. Persistent Repositories .....Enabled\n    C. libarchive features .........Enabled\n    D. Python Scripting support ....Enabled\n    E. Expression Language support .Enabled\n    F. Kafka support ...............Enabled\n    K. Bustache Support ............Disabled\n    L. Lua Scripting Support .......Enabled\n    M. MQTT Support ................Enabled\n    O. SFTP Support ................Disabled\n    S. AWS Support .................Enabled\n    T. OpenCV Support ..............Disabled\n    U. OPC-UA Support...............Enabled\n    V. SQL Support..................Enabled\n    X. Azure Support ...............Enabled\n    Y. Systemd Support .............Enabled\n    AA. Splunk Support .............Enabled\n    AB. Kubernetes Support .........Enabled\n    AC. Google Cloud Support .......Enabled\n    AD. ProcFs Support .............Enabled\n    AE. Prometheus Support .........Enabled\n    AF. Elasticsearch Support ......Enabled\n    ****************************************\n                Build Options.\n    ****************************************\n    1. Enable Tests ................Enabled\n    2. Enable all extensions\n    4. Use Shared Dependency Links .Enabled\n    5. Build Profile ...............RelWithDebInfo Debug MinSizeRel Release\n    6. Create ASAN build ...........Disabled\n    7. Treat warnings as errors.....Disabled\n    P. Continue with these options\n    Q. Quit\n    * Extension cannot be installed due to\n      version of cmake or other software, or\n      incompatibility with other extensions\n\n    Enter choice [A-Z or AA-AF or 1-7]\n  ```\n\n- Bootstrap now saves state between runs. State will automatically be saved. Provide -c or --clear to clear this state. The -i option provides a guided menu install with the ability to change\nadvanced features.\n\n### Building\n\n#### Build MiNiFi using Standalone CMake\n\n- From your source checkout, create a directory to perform the build (e.g. build) and cd into that directory.\n  ```\n  # ~/Development/code/apache/nifi-minifi-cpp on git:master\n  $ mkdir build\n  # ~/Development/code/apache/nifi-minifi-cpp on git:master\n  $ cd build\n  ```\n\n- Perform a `cmake ..` to generate the project files\n  - Optionally disable or enable extensions. Please visit our guide [extensions guide](Extensions.md) for flags or our wiki entry on\n    [customizing builds](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74685143) for more information on this topic.\n  ```\n  # ~/Development/code/apache/nifi-minifi-cpp on git:master\n  $ cmake ..\n  ...\n  -- Configuring done\n  -- Generating done\n  -- Build files have been written to: /Users/apiri/Development/code/apache/nifi-minifi-cpp/build\n  ```\n\n- Perform a build\n  ```\n  # ~/Development/code/apache/nifi-minifi-cpp on git:master\n  $ make -j$(nproc)\n  Scanning dependencies of target gmock_main\n  Scanning dependencies of target gmock\n  Scanning dependencies of target minifi\n  Scanning dependencies of target gtest\n  Scanning dependencies of target yaml-cpp\n  [  1%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/gtest/CMakeFiles/gtest.dir/src/gtest-all.cc.o\n  [  3%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock.dir/gtest/src/gtest-all.cc.o\n  [  3%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock.dir/src/gmock-all.cc.o\n  [  6%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock_main.dir/gtest/src/gtest-all.cc.o\n  [  6%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/gmock-1.7.0/CMakeFiles/gmock_main.dir/src/gmock-all.cc.o\n  [  7%] Building CXX object libminifi/CMakeFiles/minifi.dir/src/Configure.cpp.o\n\n  ...\n\n  [ 97%] Linking CXX executable minifi\n  [ 97%] Built target minifiexe\n  [ 98%] Building CXX object thirdparty/yaml-cpp-yaml-cpp-0.5.3/test/CMakeFiles/run-tests.dir/node/node_test.cpp.o\n  [100%] Linking CXX executable run-tests\n  [100%] Built target run-tests\n  ```\n\n#### Create MiNiFi Package using Standalone CMake\n\n- Create a binary assembly located in your build directory with suffix .tar.gz\n  ```\n  ~/Development/code/apache/nifi-minifi-cpp/build\n  $ make package\n  Run CPack packaging tool for source...\n  CPack: Create package using TGZ\n  CPack: Install projects\n  CPack: - Install directory: ~/Development/code/apache/nifi-minifi-cpp\n  CPack: Create package\n  CPack: - package: ~/Development/code/apache/nifi-minifi-cpp/build/nifi-minifi-cpp-0.99.0.tar.gz generated.\n  ```\n\n- Create a source assembly located in your build directory with suffix -source.tar.gz\n  ```\n  ~/Development/code/apache/nifi-minifi-cpp/build\n  $ make package_source\n  Run CPack packaging tool for source...\n  CPack: Create package using TGZ\n  CPack: Install projects\n  CPack: - Install directory: ~/Development/code/apache/nifi-minifi-cpp\n  CPack: Create package\n  CPack: - package: ~/Development/code/apache/nifi-minifi-cpp/build/nifi-minifi-cpp-0.99.0-source.tar.gz generated.\n  ```\n\n#### Build MiNiFi \u0026 Create MiNiFi Package using Conan v2\n\nBuilding MiNiFi and creating MiNiFi package supporting a portion of the extensions has been tested with Conan version 2 using VS code as an alternative to standalone CMake. By building MiNiFi using prebuilt conan packages for the external libraries as an alternative to CMake building the sources of those external libraries, we maybe able to speed up MiNiFi builds. Additionally, by creating a MiNiFi package as a conan package, it should be easier to integrate MiNiFi library and its executables into other C++\nproject infrastructure to build out data pipelines on the edge. For instance, once we create the MiNiFi conan package, we can upload it to jfrog, conancenter or some other supported conan cloud repository and then download the prebuilt MiNiFi conan package to our new C++ project by adding it to our conanfile. For more details on the conan commands to build MiNiFi and create a MiNiFi conan package, see [CONAN.md](CONAN.md).\n\n### Building a docker image\n\n#### Building your own custom image\nYou can create a custom docker image using the cmake configuration to specify which extensions should be included in the final image. Use `-DDOCKER_BUILD_ONLY=ON` to skip local environment checks of cmake.\n```\n~/Development/code/apache/nifi-minifi-cpp/build\n$ cmake -DENABLE_AWS=ON -DENABLE_KAFKA=ON -DENABLE_MQTT=ON -DENABLE_AZURE=ON -DENABLE_SQL=ON -DDOCKER_BUILD_ONLY=ON ..\n$ make docker\n```\n\n#### Building the minimal/cloud image\nThere is also on option to build the minimal image which only contains the most common extensions used in cloud environments. That image includes the standard MiNiFi processors and the AWS, Azure and Kafka extensions. This is also the image which is released to the [apache/nifi-minifi-cpp](https://hub.docker.com/r/apache/nifi-minifi-cpp) repository on dockerhub.\n```\n~/Development/code/apache/nifi-minifi-cpp/build\n$ cmake -DDOCKER_BUILD_ONLY=ON ..\n$ make docker-minimal\n```\n\n#### Executing integration tests with your docker image\nYou can execute system integration tests using the docker image built locally on a docker daemon running locally. The image shall contain every extension tested in the test suite for all scenarios to be executed (currently that includes AWS, Azure, Kafka, MQTT, SQL extensions).\n```\n~/Development/code/apache/nifi-minifi-cpp/build\n$ make docker-verify\n```\n\n### Building For Other Distros\nSince only glibc and libstdc++ is dynamically linked to MiNiFi, the binary built with RHEL devtoolset should be distro-agnostic, and work on any relatively modern distro.\n\n| Distro        | command     | Output File                                |\n|---------------|:------------|:-------------------------------------------|\n| centos 7      | make centos | nifi-minifi-cpp-centos-$VERSION.tar.gz     |\n| Rockylinux 8  | make rocky  | nifi-minifi-cpp-rockylinux-$VERSION.tar.gz |\n\nYou can avoid the requirement of an up-to-date compiler when generating the build system by adding `-DDOCKER_BUILD_ONLY=ON` to the cmake command line. This disables all cmake targets except the docker build and test ones.\n\n### Installation\nAfter [building](#building) MiNiFi C++, extract the generated binary package 'nifi-minifi-cpp-$VERSION.tar.gz' at your desired installation path.\n```shell\n$ MINIFI_PACKAGE=\"$(pwd)\"/build/nifi-minifi-cpp-*.tar.gz\n$ pushd /opt\n$ sudo tar xvzf \"$MINIFI_PACKAGE\"\n$ cd nifi-minifi-cpp-*\n```\n\n### Configuring\nThe 'conf' directory in the installation root contains a template config.yml document, minifi.properties, and minifi-log.properties. Please see our [Configuration document](CONFIGURE.md) for details on how to configure agents.\n\n### Installing as a service\n\nMiNiFi can also be installed as a system service using minifi.sh:\n\n    $ ./bin/minifi.sh install\n\n### Running\nAfter completing the [installation](#installing-as-a-service), the application can be run by issuing the following command from the installation directory:\n\n    $ ./bin/minifi.sh start\n\nBy default, this will make use of a config.yml located in the conf directory. This configuration file location can be altered by adjusting the property `nifi.flow.configuration.file` in minifi.properties located in the conf directory.\n\n### Stopping\n\nMiNiFi can then be stopped by issuing:\n\n    $ ./bin/minifi.sh stop\n\n### Running as a docker container\nYou can use the officially released image pulled from the [apache/nifi-minifi-cpp](https://hub.docker.com/r/apache/nifi-minifi-cpp) repository on dockerhub or you can use your locally built image.\nThe container can be run with a specific configuration by mounting the locally edited configuration files to your docker container.\n```\n$ docker run -v ~/Development/apache/nifi-minifi-cpp/conf/config.yml:/opt/minifi/minifi-current/conf/config.yml -v ~/Development/apache/nifi-minifi-cpp/conf/minifi.properties:/opt/minifi/minifi-current/conf/minifi.properties apache/nifi-minifi-cpp\n```\n\n### Deploying\nMiNiFi C++ comes with a deployment script. This will build and package minifi. Additionally, a file named build_output will be\ncreated within the build directory that contains a manifest of build artifacts.\n\n    $ deploy.sh \u003cbuild identifier\u003e\n\nThe build identifier will be carried with the deployed binary for the configuration you specify. By default all extensions will be built.\n\nOn Windows it is suggested that MSI be used for installation.\n\n### Cleaning\nRemove the build directory created above.\n```\n# ~/Development/code/apache/nifi-minifi-cpp on git:master\n$ rm -rf ./build\n```\n\n\n### Extensions\n\nPlease see [Extensions.md](Extensions.md) on how to build and run conditionally built dependencies and extensions.\n\n### Security\n\nFor securing a MiNiFi agent's configuration files it comes with a tool called `encrypt-config`. Its documentation is available [here](https://cwiki.apache.org/confluence/display/MINIFI/Securing+MiNiFi+configuration+files).\n\n### Recommended Antivirus Exclusions\n\nAntivirus software can take a long time to scan directories and the files within them. Additionally, if the antivirus software locks files or directories during a scan, those resources are unavailable to MiNiFi processes, causing latency or unavailability of these resources in a MiNiFi instance. To prevent these performance and reliability issues from occurring, it is highly recommended to configure your antivirus software to skip scans on the following MiNiFi C++ directories:\n\n- content_repository\n- flowfile_repository\n- provenance_repository\n\n## Operations\nSee our [operations documentation for additional information on how to manage instances](OPS.md)\n\n## Monitoring\nSee our [metrics documentation for information about self published metrics](METRICS.md)\n\n## Issue Tracking\nSee https://issues.apache.org/jira/projects/MINIFICPP/issues for the issue tracker.\n\n## Documentation\nSee https://nifi.apache.org/minifi for the latest documentation.\n\n## Examples\nSee our [examples page](examples/README.md) for flow examples.\n\n## Contributing\nSee our [Contribution Guide](CONTRIB.md).\n\n## License\nExcept as otherwise noted this software is licensed under the\n[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html)\n\nFor additional information regarding the source of included projects and\nthe corresponding licenses, you may visit the following [website](https://cwiki.apache.org/confluence/display/MINIFI/Licensing+Information)\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n  http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fnifi-minifi-cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Fnifi-minifi-cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fnifi-minifi-cpp/lists"}