https://github.com/jexp/cypher_remoting_experiments
zeromq msgpack client-server remoting protocol for cypher only
https://github.com/jexp/cypher_remoting_experiments
Last synced: about 1 year ago
JSON representation
zeromq msgpack client-server remoting protocol for cypher only
- Host: GitHub
- URL: https://github.com/jexp/cypher_remoting_experiments
- Owner: jexp
- Created: 2013-01-20T04:12:31.000Z (over 13 years ago)
- Default Branch: master
- Last Pushed: 2014-03-05T03:54:48.000Z (over 12 years ago)
- Last Synced: 2025-05-12T19:09:59.050Z (about 1 year ago)
- Language: Java
- Size: 233 KB
- Stars: 5
- Watchers: 3
- Forks: 3
- Open Issues: 2
-
Metadata Files:
- Readme: Readme.md
Awesome Lists containing this project
README
# Remoting Experiments for Neo4j's Cypher
## Usage
* right now queries are hard coded in clients
````
./server.sh [db-dir]
./client.sh
ruby client.rb
````
### Sample Session
````
Request
{"query","start n=({ids}) return n", "params": {"ids" : [1,2]},"stats": true,"result":true}
Response:
[n]
[{"id":1,"data":{"name":"foo"}}]
[{"id":2,"data":{"name":"bar"}}]
{"time": 0, "rows": 2, "bytes": 100}
````
## Ideas:
Write a Cypher only endpoint for Neo4j that uses a fast transport and serialization across multiple client languages.
Cypher Results are streamed from the server. Transaction support. Multithreaded clients and servers
Client examples in ruby, python, php, c#, c, javascript, java, scala, clojure, erlang
Public installation on Heroku for testing
### Serialization
Nodes, Relationships and Paths are converted into maps and lists for serialization recursively
Node : { id : id, [data : {foo:bar}]}
Relationship : { id : id, start: id, end: id, type : "FOO", [data : {foo:bar}]}
Path {start: node, nodes: [nodes], relationships [relationships], end: node, lenght: 1}
Header with Columns, optional Footer with time, bytes, tx-id, error, exception, rows, update-counts for nodes, relationships, properties.
### Compactness
* leave off footer, enable when needed
* ignore results (fire & forget)
### Transactions
* provide a "tx" parameter with `begin`, `commit`,`rollback`
* `tx-id` will be reported in footer
* provie a `tx-id` parameter with the transaction id
* transaction will be suspended, resumed per request (if a tx-id is provided) and finished and removed at rollback/commit
## Serialization
* fast, lightweight, portable
### MessagePack
* messagepack-lite (source, build, install in repo)
````
hg clone https://bitbucket.org/sirbrialliance/msgpack-java-lite
cd msgpack-java-lite
ant
mvn install:install-file -DgroupId=net.asdfa -DartifactId=msgpack -Dversion=0.0.1 -Dfile=dist/msgpack-java-lite.jar -Dpackaging=jar -DgeneratePom=true
````
## Transport
* fast, lightweight, portable
### ZeroMQ
brew install zeromq
#### Running as Neo4j Kernel Extension
The cypher server is now packaged as a Neo4j Kernel Extension. So if you build the jar with mvn package and drop the jar in your neo4j environment
either in /server/plugins or your classpath/build repository for embedded development, the extension will be started and listen on port 5555 by default.
The server starts by default with 1 Thread, the multiple threads don't work yet with transactions (wrong transactions resumed in the wrong thread) but for
non-transactional access the single server works fine. Threads and port can be configured in `neo4j.properties` or in the config-map passed to the database.
````
cypher_remoting_address=:5555 # a hostname and port
cypher_remoting_threads=1 # number of threads 1 to 10
````
#### Ruby
# didn't work:
sudo gem install zmq -- --with-zmq-dir=/usr/local/ --with-zmq-lib=/usr/local/lib/
# did work https://github.com/chuckremes/ffi-rzmq
sudo gem install ffi ffi-rzmq zmqmachine
### Websockets
## Alternatives, Resources
* BSON: http://bsonspec.org/
* Protocol Buffers http://code.google.com/p/protobuf/
### MessagePack
* http://msgpack.org/
* used msgpack implementation: https://bitbucket.org/sirbrialliance/msgpack-java-lite/overview
* https://github.com/msgpack/msgpack-java/blob/master/src/test/java/org/msgpack/TestSimpleArrays.java
* http://blog.andrewvc.com/why-arent-you-using-messagepack discussion: http://news.ycombinator.com/item?id=2571729
### ZeroMQ
* http://www.zeromq.org/intro:get-the-software
* http://www.zeromq.org/bindings:java
* https://github.com/imatix/zguide/blob/master/examples/Java/hwserver.java
* https://github.com/imatix/zguide
* zeromq-c: http://api.zeromq.org/2-1:zmq-recv
* ruby installation problems https://gist.github.com/2791766
* https://github.com/andrewvc/learn-ruby-zeromq/blob/master/001_Socket_Types/003_req_rep.rb
* http://www.zeromq.org/blog:multithreading-magic
* http://zguide.zeromq.org/page:all#Multithreading-with-MQ
* http://sysgears.com/articles/load-balancing-work-between-java-threads-using-zeromq/
* http://zguide.zeromq.org/java:mtserver
### Alternatives:
* Spread http://www.spread.org/ Spread, a asynchronous messaging protocol
* Netty: https://netty.io/ high performance NIO server/client, only for Java/JVM
* Storm (uses ZeroMQ): http://storm-project.net/ Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC
* Thrift: http://thrift.apache.org/ Code generator and RPC middleware for cross-language client-server applications