https://github.com/binwiederhier/elastictl

Simple tool to import/export Elasticsearch indices into a file, and/or reshard an index
https://github.com/binwiederhier/elastictl

Last synced: about 15 hours ago
JSON representation

Simple tool to import/export Elasticsearch indices into a file, and/or reshard an index

• Host: GitHub
• URL: https://github.com/binwiederhier/elastictl
• Owner: binwiederhier
• License: apache-2.0
• Created: 2020-03-11T01:58:27.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-01-25T15:34:56.000Z (almost 3 years ago)
• Last Synced: 2024-10-17T15:54:36.306Z (22 days ago)
• Language: Go
• Homepage:
• Size: 62.5 KB
• Stars: 17
• Watchers: 3
• Forks: 8
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# elastictl

Simple tool to import/export Elasticsearch indices into a file, and/or reshard an index. The tool can be used for:

* Backup/restore of an Elasticsearch index 

* Performance test an Elasticsearch cluster (import with high concurrency, see `--workers`)

* Change the shard/replica count of an index (see `reshard` subcomment)

In my local cluster, I was able to import ~10k documents per second.

## Build

```

$ go build

```

Or via goreleaser:

```

$ make [build | build-snapshot]

```

## Installation

For Debian/Ubuntu

```

wget https://github.com/binwiederhier/elastictl/releases/download/v0.0.5/elastictl_0.0.5_amd64.deb

dpkg -i *.deb

```

All others:

```

wget https://github.com/binwiederhier/elastictl/releases/download/v0.0.5/elastictl_0.0.5_linux_x86_64.tar.gz

tar zxvf elastictl_0.0.5_linux_x86_64.tar.gz

./elastictl

```

## Usage:

### Export/dump an index to a file

The first line of the output format is the mapping, the rest are the documents.

```

# Entire index

elastictl export dummy > dummy.json

# Only a subset of documents

elastictl export \

  --search '{"query":{"bool":{"must_not":{"match":{"eventType":"Success"}}}}}' \

  dummy > dummy.json

```

### Import to new index

```

# With high concurrency

cat dummy.json | elastictl import --workers 100 dummy-copy

```

### Reshard (import/export) an index

This commands export the index `dummy` to `dummy.json` and re-imports it as `dummy` using a different number of shards.

This command does `DELETE` the index after exporting it!

```

elastictl reshard \

  --search '{"query":{"bool":{"must_not":{"match":{"eventType":"Success"}}}}}' \

  --shards 1 \

  --replicas 1 \

  dummy

```