Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/fabric8io/elasticsearch-cloud-kubernetes

Last synced: 3 months ago
JSON representation

Host: GitHub
URL: https://github.com/fabric8io/elasticsearch-cloud-kubernetes
Owner: fabric8io
License: other
Archived: true
Created: 2014-10-15T08:58:06.000Z (over 9 years ago)
Default Branch: master
Last Pushed: 2018-09-18T17:45:46.000Z (almost 6 years ago)
Last Synced: 2024-01-21T10:38:17.320Z (5 months ago)
Language: Java
Size: 424 KB
Stars: 169
Watchers: 34
Forks: 90
Open Issues: 5
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt

Lists

awesome-es - elasticsearch-cloud-kubernetes
awesome-stars - elasticsearch-cloud-kubernetes - (Java)

README

This plugin is no longer under active development and can be replaced by utilizing a headless service as noted below
======

Kubernetes Cloud Plugin for Elasticsearch:
=========================================

[![Build Status](https://travis-ci.org/fabric8io/elasticsearch-cloud-kubernetes.svg?branch=master)](https://travis-ci.org/fabric8io/elasticsearch-cloud-kubernetes)

The Kubernetes Cloud plugin allows to use Kubernetes API for the unicast discovery mechanism.
Alternatively, you can utilize [headless services](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services):

apiVersion: v1
kind: Service
metadata:
annotations:
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
name: elasticsearch-cluster
spec:
clusterIP: None
ports:
- port: 9300
protocol: TCP
targetPort: 9300
publishNotReadyAddresses: true
selector:
component: es
sessionAffinity: None
type: ClusterIP

elasticsearch.yml:

...
discovery.zen:
minimum_master_nodes: ${NODE_QUORUM}
ping.unicast.hosts: elasticsearch-cluster
...

**NOTE**: You are highly encouraged to use headless services as the they obsolete this plugin.

Installation
============
```
elasticsearch-plugin install io.fabric8:elasticsearch-cloud-kubernetes:6.2.3.2

```

Versions available
==================

As of `2.0.0`, each released version of this plugin is compatible with the equal version of Elasticsearch,
e.g. `2.2.0` of this plugin is compatible with `2.2.0` of Elasticsearch.

**Older versions**:
* `1.3.x` for Elasticsearch `1.6` and `1.7`

Kubernetes Pod Discovery
===============================

Kubernetes Pod discovery allows to use the kubernetes APIs to perform automatic discovery.
Here is a simple sample configuration:

```yaml
cloud:
kubernetes:
service: ${SERVICE}
namespace: ${NAMESPACE}
discovery.zen.hosts_provider: kubernetes

path:
data: /data/data
logs: /data/log
```

## Kubernetes auth

The preferred way to authenticate to Kubernetes is to use [service accounts](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/service_accounts.md).
This is fully supported by this Elasticsearch Kubernetes plugin as it uses the [Fabric8](http://fabric8.io) Kubernetes API client.

As an example, to create a service account do something like:

```
cat <
spec:
serviceAccountName: elasticsearch

```

This will mount the service account token at `/var/run/secrets/kubernetes.io/servicaccount/token` & will be automatically
read by the fabric8 kubernetes client when the request is made to the API server.

# Kubernetes example

In this example, we're going to use a headless service to look up the Elasticsearch cluster nodes to join.

The following manifest uses 3 replication controllers to created Elasticsearch pods in 3 different modes:

* master
* data
* coordinating-only

We use 2 services as well:

* One to target the coordinating-only pods so that all requests to the cluster go through the coordinating-only nodes
* A headless service to list all cluster endpoints managed by all 3 RCs.

```yaml
kind: "List"
apiVersion: "v1"
items:
- apiVersion: "v1"
kind: "Service"
metadata:
name: "elasticsearch"
spec:
ports:
- port: 9200
targetPort: "http"
selector:
component: "elasticsearch"
type: "coordinating-only"
provider: "fabric8"
type: "LoadBalancer"
- apiVersion: "v1"
kind: "Service"
metadata:
name: "elasticsearch-cluster"
spec:
clusterIP: "None"
ports:
- port: 9300
targetPort: 9300
selector:
provider: "fabric8"
component: "elasticsearch"
- apiVersion: "v1"
kind: "ReplicationController"
metadata:
name: "elasticsearch-data"
spec:
replicas: 1
selector:
component: "elasticsearch"
type: "data"
provider: "fabric8"
template:
metadata:
labels:
component: "elasticsearch"
type: "data"
provider: "fabric8"
spec:
serviceAccount: elasticsearch
serviceAccountName: elasticsearch
containers:
- env:
- name: "SERVICE"
value: "elasticsearch-cluster"
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
- name: "NODE_MASTER"
value: "false"

image: "fabric8/elasticsearch-k8s:6.2.3"
name: "elasticsearch"
ports:
- containerPort: 9300
name: "transport"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: "elasticsearch-data"
readOnly: false
volumes:
- emptyDir:
medium: ""
name: "elasticsearch-data"
- apiVersion: "v1"
kind: "ReplicationController"
metadata:
name: "elasticsearch-master"
spec:
replicas: 1
selector:
component: "elasticsearch"
type: "master"
provider: "fabric8"
template:
metadata:
labels:
component: "elasticsearch"
type: "master"
provider: "fabric8"
spec:
serviceAccount: elasticsearch
serviceAccountName: elasticsearch
containers:
- env:
- name: "SERVICE"
value: "elasticsearch-cluster"
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
- name: "NODE_DATA"
value: "false"

image: "fabric8/elasticsearch-k8s:6.2.3"
name: "elasticsearch"
ports:
- containerPort: 9300
name: "transport"
- apiVersion: "v1"
kind: "ReplicationController"
metadata:
name: "elasticsearch-coordinating-only"
spec:
replicas: 1
selector:
component: "elasticsearch"
type: "coordinating-only"
provider: "fabric8"
template:
metadata:
labels:
component: "elasticsearch"
type: "coordinating-only"
provider: "fabric8"
spec:
serviceAccount: elasticsearch
serviceAccountName: elasticsearch
containers:
- env:
- name: "SERVICE"
value: "elasticsearch-cluster"
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
- name: "NODE_DATA"
value: "false"
- name: "NODE_MASTER"
value: "false"
image: "fabric8/elasticsearch-k8s:6.2.3"
name: "elasticsearch"
ports:
- containerPort: 9200
name: "http"
- containerPort: 9300
name: "transport"
```