{"id":15538556,"url":"https://github.com/coreyauger/reactive-gremlin","last_synced_at":"2025-04-23T15:24:54.699Z","repository":{"id":69302598,"uuid":"52498385","full_name":"coreyauger/reactive-gremlin","owner":"coreyauger","description":"akka http gremlin 3 websocket connector","archived":false,"fork":false,"pushed_at":"2018-11-13T04:21:49.000Z","size":31,"stargazers_count":31,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-09T12:31:59.135Z","etag":null,"topics":["akka-streams","backpressure","bulk-loader","graph-database","scala","titan"],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coreyauger.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-02-25T05:06:53.000Z","updated_at":"2023-04-12T07:17:37.000Z","dependencies_parsed_at":"2023-03-06T02:00:24.204Z","dependency_job_id":null,"html_url":"https://github.com/coreyauger/reactive-gremlin","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreyauger%2Freactive-gremlin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreyauger%2Freactive-gremlin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreyauger%2Freactive-gremlin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreyauger%2Freactive-gremlin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coreyauger","download_url":"https://codeload.github.com/coreyauger/reactive-gremlin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250458575,"owners_count":21433899,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["akka-streams","backpressure","bulk-loader","graph-database","scala","titan"],"created_at":"2024-10-02T12:04:52.739Z","updated_at":"2025-04-23T15:24:54.693Z","avatar_url":"https://github.com/coreyauger.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# reactive-gremlin\nHigh throughput akka http gremlin 3 websocket connector with backpressure.\n\nHigh Flow Gremlin ....  Blows your hair back !\n\n![gremlin](https://cdn.obsev.com/wp-content/uploads/2018/04/Gremlins-Gizmo-1024x688.jpg.webp \"Blow your hair back!\")\n\n### What is reactive-gremlin\nreactive-gremlin is a `streaming` websocket client that connects to a gremlin server to execute gremlin scripts.  Current client including the Java client provided by datastax have the capability to crash your gremlin server by overflowing the server with asyn requests.  Your gremlin server will buffer incoming requests and service them when it has finished the ones in the queue.  For fast consumers (bulk loaders) it is possible to cause Out of memory Exceptions and crash the server.   \n\nreactive-gremlin provides a way around this by implementing backpressure through a side channel.  This will slow down the fast consumer while the server is overloaded and speed it up when the server catches up.  It does this throw a user controlled parameter on the maximum `in-flight` calls allowed to any one server.  The stream monitors responses from the server and controlls backpressure on the stream accordingly.\n\n### Build.sbt\nAdd the following dependency to your porject.\n\n`resolvers += \"Sonatype OSS Snapshots\" at \"https://oss.sonatype.org/content/repositories/snapshots\"`\n\n`\"io.surfkit\" %% \"reactive-gremlin\" % \"0.0.1\"`\n\n\n## Usage\n\n### scripts\nI have provided a simple builder to convert you gremlin scripts into the required types for transport.  Here are some examples of encoding a few gremlin `groovy` scripts\n\n```scala\nval simple = GremlinClient.buildRequest(\"g.V().has('email','donald@trumpdonald.org').has('is_douchebag','true').valueMap();\")\nval create = GremlinClient.buildRequest(\"graph.addVertex(label, 'entity','uri','https://en.wikipedia.org/wiki/Donald_Trump');\")\n```\n### client\nTo issue request to your server you must create a `GremlinClient`.  Parameters:\n* `host: String` is the websocket url to connect to your server (default localhost:8182).\n* `maxInFlight:Int` controls the max number of request that you will allow you server to process at one time (default 250)\n* `responder:Option[ActorRef]` an optional Akka actor that will receive `Gremlin.Response` messages.\n* `onResponse:Option[Gremline.Response =\u003e Unit]` an optional callback that will receive `Gremlin.Response` messages.\n```scala\nval gclient = new GremlinClient(host=\"ws://localhost:8182\",maxInFlight=100)\n```\n\n### pushing data\nThere are 3 ways to push your request data into the flow:\n* as a driver that will return a `Future`\n* via `ActorRef` publisher (usefull for standard interaction with your gremlin server)\n* via `Flow[GremlinRequest, _]` flow that you create from another `Source`.  This is extreamly usfull when bulk loading data from a file or other source.  More on this below.\n\n### driver returning a future example\n```scala\nval gclient = new GremlinClient(host=\"ws://localhost:8182\",maxInFlight=100)\ngclient.query(\"g.V().has('email','donald@trumpdonald.org').has('is_douchebag','true').valueMap();\")\n       .map{ x: Gremlin.Response =\u003e\n          val json: Option[List[JsValue]] = x.result.data\n          ...\n       }\n```\n\n### actor push example\nHere is a simple but full example of how to use the client\n```scala\ndef response(res:Gremlin.Response):Unit = {\n  println(s\"The response is ${res}\")\n}\n\nval gclient = new GremlinClient(host=\"ws://localhost:8182\",maxInFlight=100, onResponse = Some(response))\nval producer = gclient.connectActor\n\nval simple = GremlinClient.buildRequest(\"g.V().has('email','donald@trumpdonald.org').has('is_douchebag','true').valueMap();\")\nval create = GremlinClient.buildRequest(\"graph.addVertex(label, 'entity','uri','https://en.wikipedia.org/wiki/Donald_Trump');\")\n\nproducer ! simple\nproducer ! create\n\n````\n\n### flow example\nHere is a file streaming example (bulk loader)\n```scala\ndef response(res:Gremlin.Response):Unit = {\n  println(s\"The response is ${res}\")\n}\n\nval gclient = new GremlinClient(host=\"ws://localhost:8182\",maxInFlight=100, onResponse = Some(response))\n\nval csv = new File(\"/path/to/csv\")\nval flow = FileIO.fromFile(outFile)\n          .via(Framing.delimiter(\n            ByteString(\"\\n\"),\n            maximumFrameLength = 1000000,\n            allowTruncation = false))\n          .map(_.utf8String)\n          .mapConcat{ line =\u003e\n            // parse and form some gremlin groove insert script\n            // we call that `gscript`\n            GremlinClient.buildRequest(gscript)\n          }\n          \ngclient.connectFlow(flow)   // connect and run graph\n\n````\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoreyauger%2Freactive-gremlin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoreyauger%2Freactive-gremlin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoreyauger%2Freactive-gremlin/lists"}