{"id":13508779,"url":"https://github.com/novabyte/diver","last_synced_at":"2025-04-10T04:11:05.315Z","repository":{"id":22716465,"uuid":"26060757","full_name":"novabyte/diver","owner":"novabyte","description":"A HBase driver for Erlang/Elixir using Jinterface and the Asynchbase Java client to query the database.","archived":false,"fork":false,"pushed_at":"2016-09-30T10:37:57.000Z","size":26068,"stargazers_count":51,"open_issues_count":2,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-19T11:19:22.554Z","etag":null,"topics":["elixir","erlang","hbase","hbase-driver"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/novabyte.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-01T19:24:24.000Z","updated_at":"2025-02-12T19:01:29.000Z","dependencies_parsed_at":"2022-08-21T10:00:45.416Z","dependency_job_id":null,"html_url":"https://github.com/novabyte/diver","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/novabyte%2Fdiver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/novabyte%2Fdiver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/novabyte%2Fdiver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/novabyte%2Fdiver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/novabyte","download_url":"https://codeload.github.com/novabyte/diver/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154989,"owners_count":21056543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","erlang","hbase","hbase-driver"],"created_at":"2024-08-01T02:00:58.306Z","updated_at":"2025-04-10T04:11:05.291Z","avatar_url":"https://github.com/novabyte.png","language":"Java","funding_links":[],"categories":["ORM and Datamapping"],"sub_categories":[],"readme":"Diver\n=====\n\nA HBase driver for Erlang/Elixir using [Jinterface](http://www.erlang.org/doc/apps/jinterface/jinterface_users_guide.html) and the [Asynchbase Java client](https://github.com/OpenTSDB/asynchbase) to query the database.\n\nDiver creates a Java server as a [hidden Erlang node](http://www.erlang.org/doc/reference_manual/distributed.html#id85406) at startup and dispatches `GenServer` requests directly to the `HBaseClient` running on the Java server. These requests are executed asynchronously by the client on the HBase cluster and responses are returned directly to the calling process. The Java server is monitored by the `Diver.Supervisor` and is restarted as necessary.\n\nThis work is inspired by the [`lucene_server`](https://github.com/inaka/lucene_server) and [`yokozuna`](https://github.com/basho/yokozuna) projects.\n\nDiver was created and is maintained by Chris Molozian (@novabyte) and contributors.\n\u003cbr/\u003e\nCode licensed under the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).\n Documentation licensed under [CC BY 3.0](http://creativecommons.org/licenses/by/3.0/).\n\n## Download ##\n\nDiver is available on [Hex.pm](https://hex.pm/packages/diver).\n\nAdding Diver to your application takes two steps:\n\n1. Add `diver` to your `mix.exs` dependencies:\n\n    ```elixir\n    def deps do\n      [{:diver, \"~\u003e 0.1\"}]\n    end\n    ```\n\n2. Add `:diver` to your application dependencies:\n\n    ```elixir\n    def application do\n      [applications: [:diver]]\n    end\n    ```\n\nDiver requires the Java Runtime Environment (JRE) and the `java` executable accessible via [`os:find_executable/1`](http://www.erlang.org/doc/man/os.html#find_executable-1) to start. A standalone Jar package of the Java server (and its dependencies) is bundled with this codebase.\n\nYou will only need the Java Development Kit (JDK) if you want to build the Java code directly from source. See [Developer Notes](#developer-notes) for more information.\n\n## Usage ##\n\nDiver uses a registered name to send messages to the Java node which executes requests to HBase. You must set the Erlang [node name](http://www.erlang.org/doc/reference_manual/distributed.html#id85266) before you start the application.\n\n### Configuration ###\n\nThe driver can be configured in the usual way by adding these settings to your `config.exs` file:\n\n```elixir\nconfig :diver,\n  zk: [quorum_spec: \"localhost\",\n       base_path: \"/hbase\"],\n  jvm_args: [\"-Djava.awt.headless=true\", \"-Xms256m\", \"-Xmx1024m\", \"-XX:MaxPermSize=128m\"]\n```\n\nThe configuration above is the application's default settings.\n\n### Example ###\n\nIn these examples you'll need to have HBase setup and running and Diver configured. The default settings will work with a local HBase instance started with the `bin/start-hbase.sh` script in the HBase distribution. For more information on setup and configuration of HBase see [here](http://hbase.apache.org/book/quickstart.html).\n\nYou should start your application as a named node, you can do this with `iex`:\n\n```elixir\n$ iex --name \"myserver@127.0.0.1\" --cookie \"mycookie\" -S mix\niex(myserver@127.0.0.1)1\u003e Application.ensure_all_started(:diver)\n{:ok, []}\niex(myserver@127.0.0.1)2\u003e Diver.Client.ensure_table_exists(\"tbl\")\n{:ok}\niex(myserver@127.0.0.1)3\u003e Diver.Client.get(\"tbl\", \"key\", \"family\", \"qualifier\")\n{:ok, [...]}\n```\n\nFor more detailed examples on using Diver check out the [documentation](http://hexdocs.pm/diver/).\n\nTo learn more about HBase's data model, and how to build applications with HBase, check out this [book](http://hbase.apache.org/book/datamodel.html).\n\n__Note:__ This client does not yet have feature parity with the features in the Asynchbase client. It is under development, any feedback and bug reports are welcome.\n\n## Developer Notes ##\n\nThe codebase requires [Gradle](http://gradle.org/) and JDK 7 or greater, as well as the Elixir toolchain. The dependencies for the Java codebase will be downloaded from [Maven Central](http://search.maven.org/) and those for the Erlang/Elixir codebase from [Hex](https://hex.pm/).\n\nTo build the codebase:\n\n```bash\n$ gradle build \u0026\u0026 cp _java_build/libs/diver-0.3.0-dev.jar priv/\n$ mix do deps.get, deps.compile, compile\n```\n\n### Why Java? ###\n\nBuilding a client driver for HBase is a complex endeavour, the official [HTable](http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html) client and the Asynchbase client are \"smart clients\". They cache knowledge about the operational state of the HBase cluster by communicating directly with the Zookeeper ensemble.\n\nBuilding a pure Erlang/Elixir \"smart client\" is __a lot__ of work, it would need to replicate all the coordination logic with Zookeeper, handle the messaging protocol to retrieve data via the RegionServers, copy any other \"special\" (undocumented) client logic used by the official client, include work on performance tuning with benchmarks and would need a very large test suite to keep up with changes in the HBase project.\n\nAn alternative is to run a [Thrift server](http://hbase.apache.org/book/thrift.html) as a gateway to the cluster. There is an Erlang client for HBase that communicates over the Thrift gateway [here](https://github.com/zhentao/erlang-hbase-thrift2). This approach is great but introduces its own complexity including additional server components to run and manage.\n\nThere's also [Stargate](http://wiki.apache.org/hadoop/Hbase/Stargate), a REST gateway for HBase. This approach has similar problems to running a Thrift gateway server.\n\nWith this project I've chosen to build on the performance, maturity and stability of an existing HBase client, and focus on bridging the two languages (as transparently as possible) to provide a high performance, simple Erlang/Elixir driver API for HBase.\n\n### Special Thanks ###\n\n\u003e If I have seen further it is by standing on the shoulders of giants. - _Isaac Newton_\n\nA huge thank you to the [OpenTSDB](https://github.com/OpenTSDB) team for their fantastic work on the Asynchbase HBase client.\n\n### Contribute ###\n\nAll contributions to the documentation and the codebase are very welcome.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnovabyte%2Fdiver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnovabyte%2Fdiver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnovabyte%2Fdiver/lists"}