Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bobbui/cassandra-cql-exporter
A highly configurable utility to export whole Apache Cassandra keyspace or table structure/data to CQL scripts.
https://github.com/bobbui/cassandra-cql-exporter
backup-database backup-script cassandra cql cql-scripts ddl java keyspace restore restore-database
Last synced: 9 days ago
JSON representation
A highly configurable utility to export whole Apache Cassandra keyspace or table structure/data to CQL scripts.
- Host: GitHub
- URL: https://github.com/bobbui/cassandra-cql-exporter
- Owner: bobbui
- License: apache-2.0
- Created: 2016-06-03T10:39:15.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2023-12-05T22:07:43.000Z (about 1 year ago)
- Last Synced: 2025-01-13T09:43:50.422Z (22 days ago)
- Topics: backup-database, backup-script, cassandra, cql, cql-scripts, ddl, java, keyspace, restore, restore-database
- Language: Java
- Homepage:
- Size: 4.74 MB
- Stars: 19
- Watchers: 1
- Forks: 8
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Summary
An alternative utility to **nodetool snapshot** to export whole [Apache Cassandra](http://cassandra.apache.org/) keyspace/table structure and data to Cassandra Query Language (CQL) scripts. CQL scripts is a lightweight,simple way to restore and backup databases.
Features:
- simple and configurable, see usage for detail.
- fast and highly scalable: Data is exported gradually, so memory usage is very low, e.g: for my keyspace with 1,5 million record took ~3m to generated.
- export process is tracked with detail information.
- CQL scripts is ready to import using [SOURCE command](http://docs.datastax.com/en/cql/3.3/cql/cql_reference/source_r.html).
- tested with Cassandra > 2.1, 2.2, 3.0 and tick-tock releases.
- require Java > 6., make sure Java is available in PATH variable.
Overcome nodetool snapshot caveats:
- snapshot can only be restored when table schema is there --> **cassandra-CQL-exporter** support both DDL and DML backup.
- snapshot can only run on a node, multiple node require parallel ssh to be setup --> **cassandra-CQL-exporter** dont need to care there is how many node.
- snapshot is stored the node itself, **cassandra-CQL-exporter** back up is stored on the backup client itself --> more isolated backup environment.
Generated script contains 2 component:
- DDL: include keyspace CREATE statement, all tables, indexs, materialized views, function, aggregate function, user defined type.
- DML: INSERT statement for tables data.
Be careful that script will be forward-compatible but not guarantee to be backward-compatible especially DDL statements. It's better that export and import using same Cassandra version.
I'm using this on a daily basis. But anyways, use this at **YOUR OWN RISK**!
## Usage
```
usage: cql-export [--drop] [-f ] [-fo] [-h ] [--help] [-k ] [-l] [-m]
[--noddl] [--nodml] [-p ] [-po ] [-s] [-t ] [--test] [--truncate] [-u
] [-v] [--secure]
--drop add DROP KEYSPACE statement. BE CAREFUL! THIS WILL WIPED OUT ENTIRE
KEYSPACE
-f,--file exported file path. default to ".CQL" or
"..CQL"
-fo,--force force overwrite of existing file
-h,--host server host name or IP of database server, default is "localhost"
--help print this help message
-k,--keyspace database keyspace to be exported.
or It allows to specify multiple keyspaces separated with comma. e.g. -k keyspace1,keyspace2
-kf,--keyspacesFile Allows to specify file which contains keyspaces you'd like to export, separated by new line
-l,--license Print this software license
-m,--merge merge table data, insert will be generated with "IF NOT EXISTS"
--noddl don't generate DDL statements (CREATE TABLE, INDEX, MATERIALIZED VIEW,
TYPE, TRIGGER, AGGREGATE), mutual exclusive with "nodml" option
--nodml don't generate DML statements (INSERT), mutual exclusive with "noddl"
option
-p,--pass database password
-po,--port database server port, default is 9042
-s,--separate seperated export by tables
-t,--table keyspace table to be exported
--test Enable test mode. for development testing only
--truncate add TRUNCATE TABLE statement. BE CAREFUL!
-u,--user database username
-v,--verbose print verbose message
--secure connect via SSL, -Djavax.net.ssl.trustStore=... -Djavax.net.ssl.trustStorePassword=...
must be added to JAVA_OPTS environment variable
```
##Sample usage
1. Simplest usages; only keyspace needed with **localhost** server and default port
$cql-export -k cycling
Trying connect to host "localhost"
Success!
Trying connect to port "9042"
Success!
Trying connect to keyspace "cycling"
Success!
All good!
Start exporting...
Write DDL to C:\cql-generator\cycling.CQL
Extract from cycling.cyclist_races
Total number of record: 117920
Start write "cyclist_teams" data DML to C:\cql-generator\cycling.CQL
Done 5.00%
Done 30.00%
Done 90.00%
Done exporting "cyclist_teams", total number of records exported: 117920
Export completed after 21.179 s!
Exited.
2. Simple usage:
```
$cql-export -h localhost-po 9043 -k cycling
```
3. Generate only DDL statement
```
$cql-export -h localhost-po 9043 -k keyspace_name -noddl
```
## TODO
TODO: optimized jar size.
## License
Apache 2.0 License
## Using gradle
### Run
```
./gradlew
```
or (Windows:)
```
gradlew.bat
```
### Build jar file
```
./gradlew jar
```
or (Windows:)
```
gradlew.bat jar
```
this places the jar file into the "dist" folder
## Change log
version 1.1 with gradle script and possibility to specify multiple keyspaces to export
contributed by matto3c https://www.codegravity.com