{"id":24208574,"url":"https://github.com/george0st/ff2cql","last_synced_at":"2025-09-22T06:32:02.714Z","repository":{"id":268858837,"uuid":"905672955","full_name":"george0st/Ff2Cql","owner":"george0st","description":"A simple transfer data from NiFi FileFlow to CQL (support Apache Cassandra, ScyllaDB, AstraDB). ","archived":false,"fork":false,"pushed_at":"2025-01-12T10:31:02.000Z","size":638,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-12T10:32:07.160Z","etag":null,"topics":["apache-cassandra","apache-nifi","astradb","cql","etl","fileflow","java","nifi","nifi-processor","processor","scylladb"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/george0st.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-19T09:50:28.000Z","updated_at":"2025-01-12T10:31:05.000Z","dependencies_parsed_at":"2025-01-12T10:32:16.117Z","dependency_job_id":null,"html_url":"https://github.com/george0st/Ff2Cql","commit_stats":null,"previous_names":["george0st/csv2cql","george0st/ffcsv2cql","george0st/ff2cql"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george0st%2FFf2Cql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george0st%2FFf2Cql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george0st%2FFf2Cql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george0st%2FFf2Cql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/george0st","download_url":"https://codeload.github.com/george0st/Ff2Cql/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233828201,"owners_count":18736585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-cassandra","apache-nifi","astradb","cql","etl","fileflow","java","nifi","nifi-processor","processor","scylladb"],"created_at":"2025-01-14T01:17:16.208Z","updated_at":"2025-09-22T06:32:02.701Z","avatar_url":"https://github.com/george0st.png","language":"Java","readme":"# Ff2Cql\n\n![NiFi + Cassandra](https://github.com/george0st/Csv2Cql/blob/main/docs/assets/nifi_cassandra.png?raw=true)\n\nA simple transfer data between NiFi and CQL (support Apache Cassandra, \nScyllaDB, AstraDB, YugabyteDB, etc.). The implementation details:\n - development NiFi v2 processor (with controller) and java application (support Java 17/21+)\n - support Apache Cassandra v4/v5, ScyllaDB, AstraDB, YugabyteDB based on CQL (Cassandra Query Language)\n\n## 1. The main motivation\n\n - the Apache NiFi v2 does not support Apache Cassandra v4/v5 (NiFi v2 removed \n   the Cassandra processor due to security vulnerabilities and unmaintained \n   code. The processor supported only Cassandra v3 not newer.)\n\n## 2. Usage in NiFi\n\nYou can use this preferred way (NiFi v2 processor with controller):\n - ✅ [**PutCQL**](./nifi/cql-processor/docs/README.md#3-putcql-nifi-processor), put data to CQL solution, where inputs are FlowFiles \n - ✅ [**GetCQL**](./nifi/cql-processor/docs/README.md#4-getcql-nifi-processor), get data from CQL solution, where outputs are FlowFiles\n - NOTE:\n   - You can download the [latest version](./nifi/cql-processor/output/), see 'nifi-cql-nar-*.nar' file \n   - Sample Flow definitions, [see](./nifi/cql-processor/docs/flow.md)\n\nor two other older alternative ways:\n - ✅ [**ExecuteProcess**](./console_app/Ff2Cql/docs/README.md#2-executeprocess-java-application) with java application (see 'Ff2Cql-*.jar'), where inputs are CSV files \n - ✅ [**ExecuteStreamCommand**](./console_app/Ff2Cql/docs/README.md#3-executestreamcommand-java-application) with java application (see 'Ff2Cql-*.jar'), where inputs are FlowFiles via stdin\n\n## 3. Connection setting\n\nYou can see a sample of relevant setting for CQL controller here\n(these connection settings are used directly in unit tests):\n\n### 3.1 Cassandra\n - [Cassandra setting](./nifi/cql-processor/nifi-cql/src/test/test-cassandra.json)\n - access with IP addresses (name/password)\n\n### 3.2 Scylla\n - [Scylla setting](./nifi/cql-processor/nifi-cql/src/test/test-scylla.json)\n - access with IP address (without name/password)\n\n### 3.3 AstraDB\n - [AstraDB setting](./nifi/cql-processor/nifi-cql/src/test/test-astra.json)\n - access with Security Connection Bundle (SCB) and Token\n - NOTE about SCB: see the file with **'.zip'** suffix\n   - How to get the SCB? Go to the https://astra.datastax.com/...,\n     Menu Databases/Your database/Region/[Download SCB](./docs/assets/astradb-download-SCB.png)\n - NOTE about token: see the file with **'token.json'** suffix\n   - How to get the token? Go to the https://astra.datastax.com/...,\n     Menu Tokens/Generate Token/Token Details/[Download Token Details](./docs/assets/astradb-download-token.png)\n\n### 3.4 YugabyteDB\n - [YugabyteDB setting](./nifi/cql-processor/nifi-cql/src/test/test-yugabyte.json)\n - access with IP address (without name/password)\n\n\n## 4. Apache NiFi v2 \u0026 Scylla/Cassandra v5\n\n[![Apache NiFi v2 \u0026 Scylla/Cassandra v5](./nifi/cql-processor/docs/assets/Apache_Nifi_Scylla_Cassandra_v5.png)](https://fivekg.onrender.com/images/ApacheNiFiv2\u0026Scylla-Cassandra.mp4)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge0st%2Fff2cql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeorge0st%2Fff2cql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge0st%2Fff2cql/lists"}