{"id":20513330,"url":"https://github.com/tessmore/neo4j-batch-index-stream","last_synced_at":"2026-04-15T20:32:41.755Z","repository":{"id":57309893,"uuid":"57082211","full_name":"Tessmore/neo4j-batch-index-stream","owner":"Tessmore","description":"A writable stream for batch indexing records into Neo4j","archived":false,"fork":false,"pushed_at":"2016-05-02T14:23:50.000Z","size":50,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-05T23:13:26.108Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tessmore.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-04-25T23:10:03.000Z","updated_at":"2016-04-25T23:15:23.000Z","dependencies_parsed_at":"2022-09-08T00:41:08.474Z","dependency_job_id":null,"html_url":"https://github.com/Tessmore/neo4j-batch-index-stream","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessmore%2Fneo4j-batch-index-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessmore%2Fneo4j-batch-index-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessmore%2Fneo4j-batch-index-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessmore%2Fneo4j-batch-index-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tessmore","download_url":"https://codeload.github.com/Tessmore/neo4j-batch-index-stream/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242117717,"owners_count":20074438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T21:10:12.075Z","updated_at":"2026-04-15T20:32:41.699Z","avatar_url":"https://github.com/Tessmore.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"Neo4j batch index stream\n==========================\n\nA writable stream, using `flushwritable`, to allow batch indexing labeled nodes and their relations into Neo4j. Useful if you have many specific nodes (\u003c 10 milion nodes) that have their own label, attributes, and relationship types. The `Neo4j batch import tool` is very good, but in this use case you have to build many seperate csv files for all node types and relations between them.\n\n\n### Usage\n\n`npm install neo4j-batch-index-stream`\n\n\nGiven the following `test.csv` file:\n\n```\nFabien|likes|Nick\nNick|dislikes|Fabien\nJochem|likes|Nick\nLydia|likes|Fabien\n```\n\n\nAnd the following example script, that uses `neo4j-batch-index-stream`. You can then run `less test.csv | node.js main.js` to pipe the contents of test.csv into main.js, which converts the csv into objects that can be batch inserted in neo4j. Before streaming, it will check if a connection to neo4j can be made, and optionally indexes can be set to speed up the inserting process.\n\n\n```javascript\n//// main.js\n\nvar through2 = require('through2');\nvar split2 = require('split2');\nvar Neo4jStream = require('neo4j-batch-index-stream');\n\nvar buildRecords = through2({ objectMode: true }, function(chunk, enc, callback) {\n    var line = chunk.toString().trim();\n\n    if (line \u0026\u0026 line.length) {\n        var parts = line.split(\"|\");\n\n        var Left = {\n            \"label\": \"Person\",\n            \"name\" : parts[0],\n        }\n\n        var Right = {\n            \"label\" : \"Person\",\n            \"name\" : parts[2],\n        };\n\n        var relation = {\n            \"relation\" : parts[1],\n            \"start\"    : Left,\n            \"end\"      : Right,\n        }\n\n        this.push(Left);\n        this.push(Right);\n        this.push(relation);\n    }\n\n    callback();\n});\n\n\nvar config = require(\"./secret-config.json\");\nvar username = config.username;\nvar password = config.password; // your own password ;\n\n\nvar stream = new Neo4jStream(username, password, {\n    \"index_key\"     : \"name\",\n    \"highWaterMark\" : 10000\n});\n\nstream.index([\n    [\"Person\", \"name\"]\n]);\n\n\nprocess.stdin\n.pipe(split2())\n.pipe(buildRecords)\n.pipe(stream)\n.on('error', function(error) {\n    console.log(error);\n})\n.on('finish', function() {\n    console.log(\"DONE\");\n})\n```\n\n#### Result\n\n![Neo4j database with people and relations](./test/neo4j_example.png)\n\n\n### Possible errors\n\n`Error: Error: connect ECONNREFUSED 127.0.0.1:7474`\n\u003e Neo4j is not running or not available.\n\n\u003chr\u003e\n\n```\nError: {\n  \"errors\" : [ {\n    \"message\" : \"Invalid username or password.\",\n    \"code\" : \"Neo.ClientError.Security.AuthorizationFailed\"\n  } ]\n}\n```\n\u003e You either have the default neo4j/neo4j password or gave incorrect credentials.\n\n\n### Contributing\n\nIf you have neo4j running, you can do a simple batch insert with `npm test`.\n\nIf you feel something is missing, you can open an issue stating the problem and desired result. If code is unclear give me a @mention. Pull requests are welcome.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftessmore%2Fneo4j-batch-index-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftessmore%2Fneo4j-batch-index-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftessmore%2Fneo4j-batch-index-stream/lists"}