{"id":19347803,"url":"https://github.com/devopscorner/nifi","last_synced_at":"2025-09-08T03:41:49.499Z","repository":{"id":63754129,"uuid":"568373366","full_name":"devopscorner/nifi","owner":"devopscorner","description":"Production Grade Nifi \u0026 Nifi Registry. Deploy for VM (Virtual Machine) with Terraform + Ansible, Helm \u0026 Helmfile for Kubernetes (EKS)","archived":false,"fork":false,"pushed_at":"2025-07-21T12:59:39.000Z","size":13126,"stargazers_count":13,"open_issues_count":7,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-21T14:50:41.463Z","etag":null,"topics":["ansible","data-science","data-structures","docker","docker-compose","dockerhub","ecr","eks","eks-cluster","etl","kubernetes","machine-learning","ml","mlops","nifi","nifi-registry","terraform","vpn","vpn-client"],"latest_commit_sha":null,"homepage":"https://devopscorner.id","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devopscorner.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-20T10:33:31.000Z","updated_at":"2025-07-21T12:59:43.000Z","dependencies_parsed_at":"2023-02-14T00:30:56.542Z","dependency_job_id":"908a05bd-ca1a-4090-b37d-86c3d8cda24e","html_url":"https://github.com/devopscorner/nifi","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/devopscorner/nifi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devopscorner%2Fnifi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devopscorner%2Fnifi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devopscorner%2Fnifi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devopscorner%2Fnifi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devopscorner","download_url":"https://codeload.github.com/devopscorner/nifi/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devopscorner%2Fnifi/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266626115,"owners_count":23958344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","data-science","data-structures","docker","docker-compose","dockerhub","ecr","eks","eks-cluster","etl","kubernetes","machine-learning","ml","mlops","nifi","nifi-registry","terraform","vpn","vpn-client"],"created_at":"2024-11-10T04:18:05.182Z","updated_at":"2025-07-23T06:05:33.073Z","avatar_url":"https://github.com/devopscorner.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DevOpsCorner Nifi\n\n![all contributors](https://img.shields.io/github/contributors/devopscorner/nifi)\n![tags](https://img.shields.io/github/v/tag/devopscorner/nifi?sort=semver)\n![download all](https://img.shields.io/github/downloads/devopscorner/nifi/total.svg)\n![download latest](https://img.shields.io/github/downloads/devopscorner/nifi/0.11/total)\n![view](https://views.whatilearened.today/views/github/devopscorner/nifi.svg)\n![clone](https://img.shields.io/badge/dynamic/json?color=success\u0026label=clone\u0026query=count\u0026url=https://raw.githubusercontent.com/devopscorner/nifi/master/clone.json?raw=True\u0026logo=github)\n![issues](https://img.shields.io/github/issues/devopscorner/nifi)\n![pull requests](https://img.shields.io/github/issues-pr/devopscorner/nifi)\n![forks](https://img.shields.io/github/forks/devopscorner/nifi)\n![stars](https://img.shields.io/github/stars/devopscorner/nifi)\n[![license](https://img.shields.io/github/license/devopscorner/nifi)](https://img.shields.io/github/license/devopscorner/nifi)\n\nContainer Nifi \u0026 Nifi Registry\n\n- Provisioning with Terraform \u0026 Ansible\n- EC2 Docker Compose \u0026 Kubernetes (EKS)\n- Python3 \u0026 Libraries\n- Unified Machine Learning (ML) Framework with [Ivy](https://github.com/unifyai/ivy)\n\n---\n\n## Introduction to NIFI\n\n### NIFI Usecase\n\n- Apache NiFi Tutorial: What is NiFi? Architecture \u0026 Installation (guru99.com), go to [this](https://www.guru99.com/apache-nifi-tutorial.html) link\n- An Introduction to Apache NiFi, Use Cases and Best Practices (capgemini.github.io), go to [this](https://capgemini.github.io/development/introduction-nifi-best-practices) link\n- Apache NiFi - Designing a flow for a real use case - Digital Hub, go to [this](https://scc-digitalhub.github.io/platform/nifi-tutorial-gtfs) link\n\n## Available Tags\n\n| Containers                                                                  | Pulls                                                                                                                                                                                      |\n| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| Container `devopscorner/nifi` [Tags](docs/README-Nifi.md)                   | [![nifi pulls](https://img.shields.io/docker/pulls/devopscorner/nifi.svg?label=nifi%20pulls\u0026logo=docker)](https://hub.docker.com/r/devopscorner/nifi/)                                     |\n| Container `devopscorner/nifi-registry` [Tags](docs/README-Nifi-Registry.md) | [![nifi-registry pulls](https://img.shields.io/docker/pulls/devopscorner/nifi-registry.svg?label=nifi-registry%20pulls\u0026logo=docker)](https://hub.docker.com/r/devopscorner/nifi-registry/) |\n\n## Features\n\n- Nifi (latest: 1.18.0)\n- Nifi Registry (latest: 1.18.0)\n- Python-3.10.8\n- Python Libraries\n  - cffi\n  - awscli\n  - PyYaml\n  - Jinja2\n  - httplib2\n  - six\n  - requests\n  - boto3\n  - pandas\n  - beautifulsoup4\n  - lxml\n  - scrapy\n  - SQLAlchemy\n  - psycopg2-binary\n  - ivy-core\n  - checkov\n- Unix Tools\n  - git\n  - curl\n  - wget\n  - vim\n  - nano\n  - zip\n  - unzip\n  - ping\n  - netstat\n\n---\n\n## Prerequirements\n\n- Ansible (`ansible`)\n- Docker (`docker`)\n- Docker Compose (`docker-compose`)\n- AWS Cli version 2 (`aws`)\n- Terraform Cli (`terraform`)\n- Terraform Environment (`tfenv`)\n\n## Documentation\n\n- Index Documentation, go to [this](docs/README.md) link\n- Ansible Documentation, go to [this](docs/README-Ansible.md) link\n- Deploy Ansible Nifi, go to [this](docs/Deploy-Ansible.md) link\n- Docker-Compose Nifi, go to [this](docs/Docker-Compose-Nifi.md) link\n- EKS Nifi, go to [this](docs/EKS-Nifi.md) link\n\n## Ansible Features\n\nAnsible Services Pattern:\n\n- Ansible Static \u0026 Dynamic Inventory\n\n- Playbooks Group by Services\n\n  - Datadog\n  - Docker\n  - Java\n  - Nifi\n\n- Roles by Ansible Galaxy\n  - Adminer\n  - Amazon-AWS\n  - AWS-CLI\n  - CertBot\n  - Datadog\n  - Docker\n  - Java\n  - Kubectl\n\n## Terraform Features\n\nMulti Environment Workspace:\n\n- Remote State Terraform (S3 \u0026 DynamoDB)\n\n- Core Infrastructure\n\n  - VPC\n  - Subnet EC2 \u0026 EKS\n  - Security Group\n  - NAT Gateway\n  - Internet Gateway\n  - VPC Peers Single CIDR\n  - VPC Peers Multi CIDR\n\n- Resources Other Infra\n  - Budget\n  - AWS Elastic Computing (EC2)\n    - Jumphost\n    - PostgreSQL (PSQL)\n    - Nifi\n  - Amazon Relational Database Service (RDS)\n\n## Tested Environment\n\n### Versioning\n\n- Docker version\n\n  ```\n  docker -v\n  ---\n  Docker version 20.10.17-rd, build c2e4e01\n\n  docker version\n  ---\n  Client:\n    Version:           20.10.17-rd\n    API version:       1.41\n    Go version:        go1.17.11\n    Git commit:        c2e4e01\n    Built:             Fri Jul 22 18:31:17 2022\n    OS/Arch:           darwin/amd64\n    Context:           default\n    Experimental:      true\n\n  Server: Docker Desktop 4.14.1 (91661)\n  Engine:\n    Version:          20.10.21\n    API version:      1.41 (minimum version 1.12)\n    Go version:       go9.7\n    Git commit:       3056208\n    Built:            Tue Oct 25 18:00:19 2022\n    OS/Arch:          linux/amd64\n    Experimental:     false\n  containerd:\n    Version:          1.6.9\n    GitCommit:        1c90a442489720eec95342e1789ee8a5e1b9536f\n  runc:\n    Version:          1.1.4\n    GitCommit:        v1.1.4-0-g5fd4c4d\n  docker-init:\n    Version:          0.19.0\n    GitCommit:        de40ad0\n  ```\n\n- Docker-Compose version\n\n  ```\n  docker-compose -v\n  ---\n  Docker Compose version v2.11.1\n  ```\n\n- AWS Cli\n\n  ```\n  aws --version\n  ---\n  nifi/2.8.7 Python/3.9.11 Darwin/21.6.0 exe/x86_64 prompt/off\n  ```\n\n- Terraform Cli\n\n  ```\n  terraform version\n  ---\n  Terraform v1.3.5\n  on darwin_amd64\n  - provider registry.terraform.io/hashicorp/aws v3.74.3\n  - provider registry.terraform.io/hashicorp/local v2.1.0\n  - provider registry.terraform.io/hashicorp/null v3.1.0\n  - provider registry.terraform.io/hashicorp/random v3.1.0\n  - provider registry.terraform.io/hashicorp/time v0.7.2\n  ```\n\n- Terraform Environment Cli\n\n  ```\n  tfenv -v\n  ---\n  tfenv 2.2.2\n  ```\n\n## Security Check\n\nMake sure that you didn't push sensitive information in this repository\n\n- [ ] AWS Credentials (AWS_ACCESS_KEY, AWS_SECRET_KEY)\n- [ ] AWS Account ID\n- [ ] AWS Resources ARN\n- [ ] Username \u0026 Password\n- [ ] Private (id_rsa) \u0026 Public Key (id_rsa.pub)\n- [ ] DNS Zone ID\n- [ ] APP \u0026 API Key\n\n## Copyright\n\n- Author: **Dwi Fahni Denni (@zeroc0d3)**\n- Vendor: **DevOps Corner Indonesia (devopscorner.id)**\n- License: **Apache v2**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevopscorner%2Fnifi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevopscorner%2Fnifi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevopscorner%2Fnifi/lists"}