{"id":27939878,"url":"https://github.com/chaffelson/nipyapi","last_synced_at":"2026-01-17T14:55:30.468Z","repository":{"id":24357643,"uuid":"101291622","full_name":"Chaffelson/nipyapi","owner":"Chaffelson","description":"A convenient Python wrapper for Apache NiFi","archived":false,"fork":false,"pushed_at":"2025-04-01T16:12:56.000Z","size":4589,"stargazers_count":258,"open_issues_count":32,"forks_count":75,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-15T02:07:59.486Z","etag":null,"topics":["api-wrapper","automation","dataflow","nifi","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Chaffelson.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"docs/contributing.rst","funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-08-24T12:17:36.000Z","updated_at":"2025-05-03T14:54:47.000Z","dependencies_parsed_at":"2024-04-14T10:33:14.276Z","dependency_job_id":"65232592-8031-427d-ba6e-1b5d5243bbc8","html_url":"https://github.com/Chaffelson/nipyapi","commit_stats":{"total_commits":331,"total_committers":22,"mean_commits":"15.045454545454545","dds":0.5709969788519638,"last_synced_commit":"c687fb811486d7bcada099ac0785b55cfb30aea8"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chaffelson%2Fnipyapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chaffelson%2Fnipyapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chaffelson%2Fnipyapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Chaffelson%2Fnipyapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Chaffelson","download_url":"https://codeload.github.com/Chaffelson/nipyapi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254259384,"owners_count":22040820,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-wrapper","automation","dataflow","nifi","python"],"created_at":"2025-05-07T09:58:55.791Z","updated_at":"2026-01-17T14:55:30.457Z","avatar_url":"https://github.com/Chaffelson.png","language":"Python","readme":"==============\n|nipy| NiPyApi\n==============\n\n.. |nipy| image:: https://image.ibb.co/f0FRs0/nipy.png\n    :height: 28px\n\nNifi-Python-Api: A rich Apache NiFi Python Client SDK\n\n.. image:: https://img.shields.io/pypi/v/nipyapi.svg\n        :target: https://pypi.python.org/pypi/nipyapi\n        :alt: Release Status\n\n.. image:: https://readthedocs.org/projects/nipyapi/badge/?version=latest\n        :target: https://nipyapi.readthedocs.io/en/latest/?badge=latest\n        :alt: Documentation Status\n\n.. image:: https://codecov.io/gh/Chaffelson/nipyapi/branch/main/graph/badge.svg\n        :target: https://codecov.io/gh/Chaffelson/nipyapi\n        :alt: Coverage Status\n\n.. image:: https://pyup.io/repos/github/Chaffelson/nipyapi/shield.svg\n     :target: https://pyup.io/repos/github/Chaffelson/nipyapi/\n     :alt: Python Updates\n\n.. image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg\n    :target: https://opensource.org/licenses/Apache-2.0\n    :alt: License\n\n\nFeatures\n--------\n\n**Multiple interfaces for working with Apache NiFi:**\n - **Command-Line Interface**: Shell access for scripting and CI/CD pipelines (``pip install nipyapi[cli]``)\n - **Mid-level Client SDK**: Python functions for typical complex tasks\n - **Low-level Client SDKs**: Full API implementation of NiFi and NiFi Registry\n - **Examples**: Ready-to-use scripts in the `examples directory \u003cexamples/\u003e`_\n\n**Functionality Highlights:**\n - **CI/CD Operations**: Purpose-built commands for flow deployment pipelines (``nipyapi.ci``)\n - **Profiles System**: One-command environment switching with ``nipyapi.profiles.switch('single-user')`` (see `profiles documentation \u003cdocs/profiles.rst\u003e`_)\n - **Modern Authentication**: Built-in support for Basic Auth, mTLS, OIDC/OAuth2, and LDAP\n - **Environment Management**: YAML/JSON configuration with environment variable overrides\n - **CRUD wrappers** for common task areas like Processor Groups, Processors, Clients, Buckets, Flows, etc.\n - **Convenience functions** for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups\n - **Support for scheduling and purging** flows, controller services, and connections\n - **Support for import/export** of Versioned Flows from various sources\n - **Integrated Docker workflow** with Makefile automation and profile-based testing\n\n\nPlease see the `issue \u003chttps://github.com/Chaffelson/nipyapi/issues\u003e`_ register for more information on current development.\n\nQuick Start\n-----------\n\n| **Python SDK**: Mid-level functions in *nipyapi.canvas / nipyapi.security / nipyapi.parameters / nipyapi.versioning*\n| **CLI**: Shell commands via ``nipyapi \u003cmodule\u003e \u003cfunction\u003e`` (requires ``pip install nipyapi[cli]``)\n| **Low-level**: Full API access via *nipyapi.nifi / nipyapi.registry*\n\nYou need a running NiFi instance to connect to. Choose the approach that fits your situation:\n\n**Path A: Quick Start with Docker (Recommended for New Users)**\n\n.. Note:: You will need to have Docker Desktop installed and running to use the Docker profiles.\n\nUse our provided Docker environment for immediate testing::\n\n    # Clone the repository (includes Docker profiles and Makefile)\n    git clone https://github.com/Chaffelson/nipyapi.git\n    cd nipyapi\n\n    # Install NiPyAPI in development mode\n    pip install -e \".[dev]\"\n\n    # Start complete NiFi environment (this may take a few minutes)\n    make certs \u0026\u0026 make up NIPYAPI_PROFILE=single-user \u0026\u0026 make wait-ready NIPYAPI_PROFILE=single-user\n\n    # Test the connection\n    python3 -c \"\n    import nipyapi\n    nipyapi.profiles.switch('single-user')\n    version = nipyapi.system.get_nifi_version_info()\n    print(f'✓ Connected to NiFi {version}')\n    \"\n\n**Path B: Connect to Your Existing NiFi**\n\nIf you already have NiFi running, install and configure::\n\n    # Install NiPyAPI\n    pip install nipyapi\n\n    # Create your own profiles.yml\n    mkdir -p ~/.nipyapi\n    cat \u003e ~/.nipyapi/profiles.yml \u003c\u003c EOF\n    my-nifi:\n      nifi_url: https://your-nifi-host.com/nifi-api\n      registry_url: http://your-registry-host.com/nifi-registry-api\n      nifi_user: your_username\n      nifi_pass: your_password\n      nifi_verify_ssl: true\n    EOF\n\n    # Test your custom profile\n    python3 -c \"\n    import nipyapi\n    nipyapi.config.default_profiles_file = '~/.nipyapi/profiles.yml'\n    nipyapi.profiles.switch('my-nifi')\n    version = nipyapi.system.get_nifi_version_info()\n    print(f'✓ Connected to NiFi {version}')\n    \"\n\n**Path C: Manual Configuration (Advanced)**\n\nFor advanced use cases without profiles::\n\n    # Install NiPyAPI\n    pip install nipyapi\n\n    # Configure in Python code\n    import nipyapi\n    from nipyapi import config, utils\n\n    # Configure endpoints\n    config.nifi_config.host = 'https://your-nifi-host.com/nifi-api'\n    config.registry_config.host = 'http://your-registry-host.com/nifi-registry-api'\n\n    # Configure authentication\n    utils.set_endpoint(config.nifi_config.host, ssl=True, login=True,\n                       username='your_username', password='your_password')\n\n**Next Steps: Start Using NiPyAPI**\n\nOnce your environment is set up, you can start using NiPyAPI::\n\n    import nipyapi\n\n    # If using profiles (Paths A or B)\n    nipyapi.profiles.switch('single-user')  # or your custom profile name\n\n    # Basic operations\n    root_pg_id = nipyapi.canvas.get_root_pg_id()\n    version = nipyapi.system.get_nifi_version_info()\n    process_groups = nipyapi.canvas.list_all_process_groups()\n\n    print(f\"Connected to NiFi {version}\")\n    print(f\"Root Process Group: {root_pg_id}\")\n    print(f\"Found {len(process_groups)} process groups\")\n\n**Command-Line Interface:**\n\nFor shell scripting, CI/CD pipelines, and agentic workflows, install the CLI extras::\n\n    pip install \"nipyapi[cli]\"\n\n    # Set connection via environment variables\n    export NIFI_API_ENDPOINT=https://your-nifi-host.com/nifi-api\n    export NIFI_USERNAME=your_username\n    export NIFI_PASSWORD=your_password\n\n    # Or use a profile\n    nipyapi --profile single-user system get_nifi_version_info\n\n    # CI/CD operations\n    nipyapi ci deploy_flow --bucket flows --flow my-flow\n    nipyapi ci start_flow --process_group_id \u003cid\u003e\n    nipyapi ci get_status --process_group_id \u003cid\u003e\n\nSee ``docs/cli.rst`` for the complete CLI reference and ``docs/ci.rst`` for CI/CD pipeline examples.\nSee the companion repository `nipyapi-actions \u003chttps://github.com/Chaffelson/nipyapi-actions\u003e`_ for GitHub Actions and GitLab CI examples and support.\n\n**Built-in Docker Profiles:**\n\nWhen using Path A (Docker), these profiles are available:\n\n- ``single-user`` - HTTP Basic authentication (easiest to start with)\n- ``secure-ldap`` - LDAP authentication over TLS\n- ``secure-mtls`` - Mutual TLS certificate authentication\n- ``secure-oidc`` - OpenID Connect (OAuth2) authentication\n\nSee ``docs/profiles.rst`` for complete profiles documentation and ``docs/migration.rst`` for upgrading from 0.x.\n\n**Examples and Advanced Usage:**\n\n- **Flow Development Lifecycle**: See ``examples/fdlc.py`` for multi-environment workflow patterns\n- **Interactive Sandbox**: Run ``make sandbox NIPYAPI_PROFILE=single-user`` for experimentation (requires Docker setup)\n- **Custom Profiles**: Create your own ``profiles.yml`` for production environments\n- **Environment Variables**: Override any setting with ``NIFI_API_ENDPOINT``, ``NIFI_USERNAME``, etc.\n- **Testing Different Auth Methods**: Use ``make up NIPYAPI_PROFILE=secure-ldap`` to try LDAP authentication\n\nPlease check out the `Contribution Guide \u003chttps://github.com/Chaffelson/nipyapi/blob/main/docs/contributing.rst\u003e`_ if you are interested in contributing to the feature set.\n\nBackground and Documentation\n----------------------------\n\n| For more information on **Apache NiFi**, please visit `https://nifi.apache.org \u003chttps://nifi.apache.org\u003e`_\n| For **complete NiPyAPI documentation**, please visit `https://nipyapi.readthedocs.io \u003chttps://nipyapi.readthedocs.io/en/latest\u003e`_\n| For **CLI and CI/CD usage**, see ``docs/cli.rst`` and ``docs/ci.rst``\n| For **profiles and authentication**, see ``docs/profiles.rst`` and ``docs/security.rst``\n| For **migration from 0.x to 1.x**, see ``docs/migration.rst`` in the repository\n\n\nNiFi Version Support\n--------------------\n\n| **NiPyAPI 1.x targets Apache NiFi 2.x and NiFi Registry 2.x** (tested against 2.7.2).\n| **For NiFi 1.x compatibility**, please use NiPyAPI 0.x branch or pin nipyapi \u003c 1.0.0.\n| **Breaking changes** exist between 0.x and 1.x - see ``docs/migration.rst`` for upgrade guidance.\n| **Docker profiles require Docker Desktop** with sufficient memory (recommend 4GB+ for NiFi).\n| If you find a version compatibility problem please raise an `issue \u003chttps://github.com/Chaffelson/nipyapi/issues\u003e`_\n\nPython Support\n--------------\n\n| **Python 3.9-12 supported**, though other versions may work.\n| **Python2 is no longer supported** as of the NiPyAPI 1.0 release, please use the 0.x branch for Python2 projects.\n| OSX M1 chips have been known to have had various issues with Requests and Certificates, as did Python 3.10.\n\n| Tests are run against **upstream Apache NiFi and NiFi Registry Docker images** via integrated Makefile automation.\n| **Profile-driven testing** supports single-user, LDAP, mTLS, and OIDC authentication modes.\n| Developed on **macOS 14+ and Windows 10**.\n| **Runtime dependencies**: requests/urllib3, PyYAML, PySocks. CLI extras add google-fire.\n| **Development tools**: Comprehensive Makefile with ``make up``, ``make test``, ``make sandbox`` targets.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchaffelson%2Fnipyapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchaffelson%2Fnipyapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchaffelson%2Fnipyapi/lists"}