{"id":27974615,"url":"https://github.com/strapdata/cqlinject","last_synced_at":"2025-07-31T13:13:56.335Z","repository":{"id":42415124,"uuid":"84753887","full_name":"strapdata/cqlinject","owner":"strapdata","description":"Elassandra utiliy","archived":false,"fork":false,"pushed_at":"2017-03-12T20:13:07.000Z","size":29,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-08T00:24:57.072Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/strapdata.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.txt","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-12T20:08:45.000Z","updated_at":"2017-03-12T20:09:00.000Z","dependencies_parsed_at":"2022-09-13T11:31:30.097Z","dependency_job_id":null,"html_url":"https://github.com/strapdata/cqlinject","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/strapdata/cqlinject","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strapdata%2Fcqlinject","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strapdata%2Fcqlinject/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strapdata%2Fcqlinject/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strapdata%2Fcqlinject/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/strapdata","download_url":"https://codeload.github.com/strapdata/cqlinject/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strapdata%2Fcqlinject/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268045398,"owners_count":24186794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-31T02:00:08.723Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-08T00:19:03.019Z","updated_at":"2025-07-31T13:13:56.266Z","avatar_url":"https://github.com/strapdata.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CqlInject\n\n[![Build Status](https://travis-ci.org/strapdata/cqlinject.svg)](https://travis-ci.org/strapdata/cqlinject)\n\nCqlInject is a simple data loader for apache cassandra 2.2+ :\n* Insert the result of a JDBC query in a cassandra table (1).\n* Insert the result of a CQL query from a cassandra cluster (1).\n* Insert from a CSV file (1).\n* Insert an Elasticsearch bulk JSON file (see [Elasticsearch bulk API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html))(2)\n\n(1) automatically creates or updates the destination table schema.\n(2) destination table schema should match the JSON input record format.\n\nWARNING: CqlInject runs on a single JVM, it may not fit to move huge volume of data.\n\n# Installation\n\n## Building from source\n\n* CqlInject uses [Maven](http://maven.apache.org) for its build system. Simply run the `mvn clean package -Dmaven.test.skip=true` command in the cloned directory. \nThe distribution will be created under *target/releases*.\n\n## Tarball Installation\n\n* Install Java version 8 (check version with `java -version`). \n* Download CqlInject tarball from [CqlInject repository](https://github.com/vroyer/cqlinject/releases) and extract files in your installation directory.\n* Add CqlInject into your PATH environnement variable `PATH=\"$PATH:\u003cinstallation_dir\u003e/bin\"`.\n\n# Usage examples\n\n## CqlInject from a cassandra cluster\n\nExecute a CQL query and write the result in the destination cluster.keyspace.table. If the destination table does not exists, CqlInject create it.\n```\ncqlinject -h localhost -ks music -cf playlists -pk \"id,song_order\" -ptlen 1 cql \n-sh host2 \n\"SELECT * FROM playlists\"\n```\n\n## CqlInject from prestoDB \n\n[prestoDB](https://prestodb.io/) is a distributed SQL engine allowing to queries various systems including Hadoop, Cassandra, Mongodb, Redis, Kafka. \nWith to [prestodDb JDBC client](https://prestodb.io/docs/current/installation/jdbc.html) driver, you can write the result of a prestodDB query \nin a dynamically created (or updated) cassandra table. In the following exemple, it joins two tables and write the result in the first table, adding new columns.\n\n```\ncqlinject -h localhost -ks twitter -cf tweet -pk \"msg_id\" jdbc \n--jdbc.url \"jdbc:presto://localhost:8080\" \n--jdbc.user bob -jdbc.password bob \n\"SELECT t.msg_id, u.country FROM twitter t INNER JOIN users u ON (t.user = u.user)\"\n```\n\n## CqlInject from elasticsearch\n\nBound with the elasticsearch JDBC driver [sql4es ](https://github.com/Anchormen/sql4es), CqlInject can move data from elasticsearch to cassandra.\n\n```\ncqlinject -h localhost -ks twitter -cf tweet -pk \"_id\" jdbc \n--jdbc.url \"jdbc:sql4es://node1:9200/twitter?cluster_name=MyCluster\" \n\"SELECT * FROM twitter WHERE size \u003e 100\"\n```\n\n## CQL Inject a CSV file\n\nInsert a CSV File :\n* First line is column names separated by your separator.\n* Second line is MessageFormat pattern for each field separated by your separator.\n\n```\ncqlinject -ks lastfm -pk \"_id\" -cf profile csv --separator '\\t' userid-profile.tsv\n```\n\nCSV File content example :\n\n```\n_id     gender  age     country registred\n{0}     {0}     {0,number}      {0}     {0,date,MMM dd, yyyy}\nuser_000001     m       0       Japan   Feb 23, 2006\nuser_000002     f       0       Peru    Feb 24, 2006\nuser_000003     m       22      United States   Oct 30, 2005\nuser_000004     f                       Apr 26, 2006\nuser_000005     m               Bulgaria        Jun 29, 2006\nuser_000006             24      Russian Federation      May 18, 2006\nuser_000007     f               United States   Jan 22, 2006\nuser_000008     m       23      Slovakia        Sep 28, 2006\nuser_000009     f       19      United States   Jan 13, 2007\n```\n\n## CqlInject JSON bulk load\n\nElasticsearch provide a [bluk API](https://www.elastic.co/guide/en/elasticsearch/guide/current/bulk.html) to load JSON files into elasticsearch. \nCqlInject reads this JSON file format and build an INSERT INTO \\\u003cindex\\\u003e.\\\u003ctype\\\u003e JSON \\\u003cdoc\\\u003e' see [Cassandra JSON support](http://www.datastax.com/dev/blog/whats-new-in-cassandra-2-2-json-support). If your document _id is a string representation of a JSON array, \neach array element are mapped to primary key columns of the cassandra destination table (so the destination table must exists before injecting).\n```\ncqlinject -h localhost json shakespeare.json\n```\n\n# License\n\n```\nThis software is licensed under the Apache License, version 2 (\"ALv2\"), quoted below.\n\nCopyright 2017, Strapdata (contact@strapdata.com).\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not\nuse this file except in compliance with the License. You may obtain a copy of\nthe License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT\nWARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the\nLicense for the specific language governing permissions and limitations under\nthe License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstrapdata%2Fcqlinject","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstrapdata%2Fcqlinject","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstrapdata%2Fcqlinject/lists"}