https://github.com/winvector/sqlscrewdriver

Iterate through database tables (by JDBC) and TSV(tab separated values)/CSV(comma separated values) and load/dump data.
https://github.com/winvector/sqlscrewdriver

Last synced: 2 months ago
JSON representation

Iterate through database tables (by JDBC) and TSV(tab separated values)/CSV(comma separated values) and load/dump data.

• Host: GitHub
• URL: https://github.com/winvector/sqlscrewdriver
• Owner: WinVector
• Created: 2012-03-28T02:51:46.000Z (almost 13 years ago)
• Default Branch: main
• Last Pushed: 2020-06-22T21:19:48.000Z (over 4 years ago)
• Last Synced: 2024-03-26T05:20:03.933Z (10 months ago)
• Language: Java
• Homepage: http://www.win-vector.com/blog/2011/01/sql-screwdriver/
• Size: 636 KB
• Stars: 8
• Watchers: 5
• Forks: 13
• Open Issues: 0
• Metadata Files:
  • Readme: README.txt

Awesome Lists containing this project

README

        

Update: 2-5-2016

This package is getting a bit old (though we do use it for some projects).

I just added a "colquote" key to the db connection properties/xml (see DBUtil.java line 19) to close an issue in using this tool with MySQL.

For our current advice on semi-generic methods for loading data (such as dbWriteTable() in R) please see here http://www.win-vector.com/blog/2016/02/using-postgresql-in-r/ .

3-27-2012

Crude SQL dump/restore tool (database agnostic, uses JDBC).

Described at: http://www.win-vector.com/blog/2011/01/sql-screwdriver/

Example use: dump from one PostgreSQL DB and restore to another (not all column types preserved).

(SQLScrewdriver.jar comes from https://github.com/WinVector/SQLScrewdriver/blob/master/SQLScrewdriver.jar or https://github.com/WinVector/SQLScrewdriver and postgresql.jar is the appropriate DB driver from http://jdbc.postgresql.org/download.html )

# dump the table to TSV from remote database

java -cp SQLScrewdriver.jar:postgresql.jar com.winvector.db.DBDump file:srvrdb.xml "SELECT * from model_res_summary" model_res_summary.tsv

# load the TSV into local database

java -cp SQLScrewdriver.jar:postgresql.jar com.winvector.db.LoadTable file:db.xml t file:model_res_summary.tsv model_res_summary

and srvdb.xml is:

 am

 jdbc:postgresql://192.168.1.9:5432/am

 

 org.postgresql.Driver

and db.xml is:

 miner_demo

 jdbc:postgresql://localhost:5432/miner_demo

 

 org.postgresql.Driver

with passwords filled in.

Additional property keys can set the SQL types used to populate integer types, double types and timestamp types.  The defaults are:

 DOUBLE PRECISION

 BIGINT

 TIMESTAMP

For Oracle databases we suggest adding the following line to your db properties files:

 NUMBER

The units tests in the test directory can be run if you include Junit and H2 database jars.