Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/neo4j-contrib/neo4j-etl

Data import from relational databases to Neo4j.
https://github.com/neo4j-contrib/neo4j-etl

csv cypher graph-database importer mysql neo4j postgresql rdbms relational-databases sql

Last synced: 2 months ago
JSON representation

Data import from relational databases to Neo4j.

Host: GitHub
URL: https://github.com/neo4j-contrib/neo4j-etl
Owner: neo4j-contrib
License: other
Created: 2017-08-30T15:02:46.000Z (over 7 years ago)
Default Branch: master
Last Pushed: 2022-11-03T13:01:34.000Z (about 2 years ago)
Last Synced: 2024-09-29T17:40:54.590Z (2 months ago)
Topics: csv, cypher, graph-database, importer, mysql, neo4j, postgresql, rdbms, relational-databases, sql
Language: HTML
Homepage: https://neo4j.com/developer/neo4j-etl/
Size: 1.16 MB
Stars: 214
Watchers: 38
Forks: 46
Open Issues: 63
Metadata Files:
- Readme: README.adoc
- License: LICENSE.txt

Awesome Lists containing this project

awesome-starred - neo4j-contrib/neo4j-etl - Data import from relational databases to Neo4j. (sql)

README

        = Neo4j ETL

:img: docs/img

Neo4j ETL allows importing data from relational databases into Neo4j.

== Features

* Neo4j-ETL UI in Neo4j Desktop

* Manage multiple RDBMS connections

* automatically extract database metadata from relational database

* derive graph model

* visually edit labels, relationship-types, property-names and types

* visualize current model as a graph

* persist mapping as json

* retrieve relevant CSV data from relational databases

* run import via neo4j-import, bolt-connector, cypher-shell, neo4j-shell

* bundles MySQL, PostgreSQL, allows custom JDBC driver with Neo4j Enterprise

== License

This tool is licensed under the link:LICENSE.txt[NEO4J PRE-RELEASE LICENSE AGREEMENT].

== Issues & Feedback & Contributions

* You can raise link:issues[GitHub issues] or send feedback to [email protected]

* Please also join our https://community.neo4j.com/c/neo4j-graph-platform/etl[Community Site] ask in the appropriate category

== Download & Run Command Line Tool

Download & unzip the latest https://github.com/neo4j-contrib/neo4j-etl/releases/latest[neo4j-etl.zip^].

== Examples of command usage:

.Minimal command line

----

./bin/neo4j-etl export \

 --rdbms:url  --rdbms:user  --rdbms:password  \

 --destination $NEO4J_HOME/data/databases/graph.db/ --import-tool $NEO4J_HOME/bin \

 --csv-directory $NEO4J_HOME/import

----

.Full set of command line options

----

./bin/neo4j-etl export \

 --rdbms:url  --rdbms:user  --rdbms:password  --rdbms:schema  \

 --using { bulk:neo4j-import | cypher:neo4j-shell | cypher:shell | cypher:direct | cypher:batch | cypher:fromSQL } \

 --neo4j:url  --neo4j:user  --neo4j:password  \

 --destination $NEO4J_HOME/data/databases/graph.db/ --import-tool $NEO4J_HOME/bin \

 --csv-directory $NEO4J_HOME/import --options-file import-tool-options.json --force --debug

----

.Additional command line options for `cypher:batch` and `cypher:fromSQL` import modes:

----

 --unwind-batch-size  (Batch size that will be used for unwind data) \

 --tx-batch-size  (Transaction Batch size that will be used for unwind commit) \

----

For detailed usage see also the: http://neo4j-contrib.github.io/neo4j-etl#neo4j-etl-cli[tool documentation].

== Neo4j-Desktop

Use the Application URL `https://r.neo4j.com/neo4j-etl-app` in the "Graph Apps" tab of Neo4j Desktop.

image:{img}/neo4j-etl-install.jpg[width=400]

image:{img}/graph-app-icon.jpg[width=400]

Then the next time you start Neo4j Desktop you'll see Neo4j ETL as a UI to be used interactively.

[cols="4*^.^",opts=header]

|===

| Configure Driver

| Load Mapping

| Edit Mapping

| Import Data

| image:{img}/driver.jpg[width=200]

| image:{img}/load-mapping.jpg[width=200]

| image:{img}/edit-mapping.jpg[width=200]

| image:{img}/import-data.jpg[width=200]

|===

////

.Location of $DESKTOP

|===

| macOS | ~/Library/Application Support/Neo4j Desktop |

| Windows | %APPDATA%/Neo4j Desktop |

| Linux | ~/.config/Neo4j Desktop |

|===

////

We put https://neo4j.com/developer/neo4j-etl[detailed usage instructions for the Neo4j ETL Tool] in the Neo4j Developer Pages.

////

If you want to test changes to the Neo4j Desktop UI, you can try:

* Make necessary changes

* npm install && yarn start to verify UI runs

* Install the production ETL tool in Neo4j Desktop 

* Replace the relevant JavaScript in $DESKTOP 

* Re-run the Neo4j Desktop App and veify UI and functionality are as expected

////

== JDBC Drivers

The drivers for MySQL and PostgreSQL are bundled with the Neo4j-ETL tool.

To use other JDBC drivers use these download links and JDBC URLs.

Provide the JDBC driver jar-file to the command line tool or Neo4j-ETL application.

And use the JDBC-URL with the `--rdbms:url` parameter or in the JDBC-URL input field.

[options="header",cols="a,3m,a"]

|===

|Database | JDBC-URL | Driver Source

|Oracle

|jdbc:oracle:thin:/@:/

|http://www.oracle.com/technetwork/database/features/jdbc/index.html[Oracle JDBC Driver^]

|MS SQLServer

|jdbc:sqlserver://;servername=;databaseName=;user=;password=

|https://www.microsoft.com/en-us/download/details.aspx?id=11774[SQLServer Driver^]

|IBM DB2

|jdbc:db2://:/:user=;password=;

|http://www-01.ibm.com/support/docview.wss?uid=swg21363866[DB2 Driver^]

|Derby

|jdbc:derby:derbyDB

|Included since JDK6

|Cassandra 

|jdbc:cassandra://:/

|link:https://github.com/adejanovski/cassandra-jdbc-wrapper#installing[Cassandra JDBC Wrapper^]

|SAP Hana

|jdbc:sap://:/?user=&password=

|https://www.sap.com/developer/tutorials/hxe-connect-hxe-using-jdbc.html[SAP Hana ngdbc Driver^]

|MySQL 

|jdbc:mysql://:/?user=&password=

|http://dev.mysql.com/downloads/connector/j/[MySQL Driver^]

|PostgreSQL

|jdbc:postgresql:///?user=&password=

|https://jdbc.postgresql.org/download.html[PostgreSQL JDBC Driver^]

|===