https://github.com/mpenet/spandex

Elasticsearch client for Clojure (built on new ES 8.x java client)
https://github.com/mpenet/spandex

clojure elasticsearch elasticsearch-client

Last synced: about 2 months ago
JSON representation

Elasticsearch client for Clojure (built on new ES 8.x java client)

• Host: GitHub
• URL: https://github.com/mpenet/spandex
• Owner: mpenet
• Created: 2016-12-15T10:50:11.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2023-11-20T07:18:08.000Z (7 months ago)
• Last Synced: 2024-01-31T05:49:10.602Z (5 months ago)
• Topics: clojure, elasticsearch, elasticsearch-client
• Language: Clojure
• Homepage:
• Size: 261 KB
• Stars: 254
• Watchers: 11
• Forks: 31
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Funding: FUNDING.yml

Lists

• awesome-clojure - Spandex
• awesome-stars - mpenet/spandex - Elasticsearch client for Clojure (built on new ES 8.x java client) (Clojure)

README

        
# spandex

[![Clojars Project](https://img.shields.io/clojars/v/cc.qbits/spandex.svg)](https://clojars.org/cc.qbits/spandex)

Elasticsearch new low level [rest-client](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html) wrapper

## Why?

To quote ["State of the official Elasticsearch Java clients"](https://www.elastic.co/blog/state-of-the-official-elasticsearch-java-clients)

> The Java REST client is the future for Java users of Elasticsearch.

Because the legacy native client is a bit of a nightmare to deal with

(for many reasons) and the new REST client is quite capable and fast

too, see ["Benchmarking REST client and transport client"](https://www.elastic.co/blog/benchmarking-rest-client-transport-client)

Not to mention it supports some interesting features:

* compatibility with any Elasticsearch version

* load balancing across all available nodes

* failover in case of node failures and upon specific response codes

* failed connection penalization

* persistent connections

* trace logging of requests and responses

* optional automatic discovery of cluster nodes (also known as sniffing)

## Goals

* Be minimal & performant

* RING inspired

* All "exotic" features should be optional

* Not a giant DSL over another DSL, just maps everywhere.

  Read ElasticSearch doc -> done, not another layer of indirection

* Provide minimal (and optional) utils to do the boring stuff

  (bulk, scroll queries, compose urls)

* Can do async via simple callbacks based api or `core.async`

* Provide [specs](https://github.com/mpenet/spandex/blob/master/src/clj/qbits/spandex/spec.clj)

### Setup

```clojure

(require '[qbits.spandex :as s])

(def c (s/client {:hosts ["http://127.0.0.1:9200" "https://foo2:3838"]}))

;; add optional sniffer

(def s (s/sniffer c {... options ...}))

```

#### Work with `https` via ssh tunneling

First setup and make sure that you have appropriate access to the host via tunneling.

e.g. Add/edit your `~/.ssh/config` to look something like

```

# Example of tunneling in ~/.ssh/config

# .. more config

Host my-aws-elasticsearch-host 

  HostName 10.123.345.456

  User ec2-user

  IdentitiesOnly yes

  IdentityFile ~/.ssh/my-aws-elasticsearch.pem

  LocalForward 9200 vpc-my-aws-elasticsearch-host-lb43i.us-east-1.es.amazonaws.com:443

  ServerAliveInterval 240

# .. more config

```

You can then start ssh tunneling with 

``` sh

# see manpage of `ssh` for more details

ssh -oStrictHostKeyChecking=no my-aws-elasticsearch-host -N 

```

Then you can create your client using the following `:http-client` options like

```clojure

;; if you are using tunnelling to host in AWS e.g.

(def client (s/client {:hosts ["https://localhost:9200"]

                       :http-client {:ssl-context (client/ssl-context-trust-all)

                                     :ssl-noop-hostname-verifier? true}}))

```

#### Constructing URLs

Most of spandex request functions take a request map as parameter. The

`:url` key differs a bit from the original RING spec, it allows to

pass a raw string but also a sequence (potentially 2d) of encodable

things, keywords, .toString'able objects that make sense or nil (which

could be caused by a missing :url key).

``` clojure

(s/request c {:url [:foo :bar :_search] ...})

(s/request c {:url [:foo [:bar :something "more"] :_search] ...})

(s/request c {:url :_search ...})

(s/request c {:url "/index/_search" ...})

(s/request c {:url (java.util.UUID/randomUUID) ...})

(s/request c {...}) ;; defaults to "/"

```

### Blocking requests

```clojure

(s/request c {:url [:entries :entry :_search]

              :method :get

              :body {:query {:match_all {}}}})

>> {:body {:_index "entries", :_type "entry", :_id "AVkDDJvdkd2OsNWu4oYk", :_version 1, :_shards {:total 2, :successful 1, :failed 0}, :created true}, :status 201, :headers {"Content-Type" "application/json; charset=UTF-8", "Content-Length" "141"}, :host #object[org.apache.http.HttpHost 0x62b90fad "http://127.0.0.1:9200"]}

```

### Async requests (callbacks)

```clojure

(s/request-async c {:url "/urls/url/"

                    :method :get

                    :body {:query {:match {:message "this is a test"}}}

                    :success (fn [response-as-clj] ... )

                    :error (fn [ex] :boom)})

```

### Async requests: `core.async/promise-chan`

``` clojure

(async/