Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ywangd/peek

Peek into Elasticsearch clusters
https://github.com/ywangd/peek

cli elasticsearch python

Last synced: 3 months ago
JSON representation

Peek into Elasticsearch clusters

Awesome Lists containing this project

README 

        
====

Peek

====



.. image:: https://github.com/ywangd/peek/workflows/Peek/badge.svg

        :target: https://github.com/ywangd/peek



Peek is an interactive CLI tool for working with Elasticsearch clusters.

It is like `Kibana Console `_

running in terminal with additional features for tinkers.



Installation

------------



Please note the project requires Python 3.8+. It is recommended to install from PyPI with:



.. code-block:: bash



  pip install es-peek



The tool is now available as the ``peek`` command. Peek will ask permission to access system keyring

for storing credentials.

This can be disabled  temporarily by invoking the command as ``peek -e use_keyring=False``

or permanently by setting ``use_keyring = False`` in `peekrc `_ file



Besides HTTP calls to Elasticsearch clusters, type ``help`` to see a list of builtin functions.

To **enable auto-completions** for APIs, run the ``_download_api_specs`` builtin function

(note the leading underscore) to download API spec file from the

`Elasticsearch Specification project `_.



Alternatively, peek can also be installed from source with:



.. code-block:: bash



    python setup.py install



Features

--------



Peek supports most editing features offered by

`Kibana Console `_,

e.g. auto-completion, syntax highlighting, auto-formatting, auto-indent,

par-editing, triple-quotes, etc. It also offers following additional features:



* Lightweight CLI tool

* Multiplex a single terminal session to multiple Elasticsearch clusters or multiple connections to a single cluster

* Flexible usages of quotes, comma, comments for the JSON payload, case-insensitive http method names

* Multiple authentication schemes, including UserPass, API key, Token, SAML, OIDC, Kerberos, PKI

* Support run-as, x-opaque-id and arbitrary request headers

* Load JSON payload from external files

* Run file input in batch mode

* Readline editing features, e.g. ``Ctrl-_`` for undo, ``Ctrl-r`` for reverse search, etc.

* History management

* Capture terminal input and output into file

* Connect to `Elastic Cloud `_ with Cloud ID

* Shell out for system commands

* Minimal scripting support

* Extensible via external scripts



Sample Usages

-------------



Assuming a locally running Elasticsearch cluster, start a Peek session with:



.. code-block:: bash



  peek --hosts localhost:9200 --username elastic



The following sample is a quick guide on Peek usages:



.. code-block:: javascript



  // Download the API schema from the elasticsearch-specification project to enable auto-completion

  // If the output says the schema file already exists, you can safely proceed to the next command.

  _download_api_specs



  // NOTE a blank line is needed to trigger API execution, or type "ESC + Enter" to execute regardlessly

  // Exit the interactive session any time by pressing Ctrl-d or type exit

  GET /_cluster/health  // comment is allowed almost anywhere



  // Index a single document

  POST /my-index/_doc

  {'foo': "bar"}  // both single and double quotes are acceptable



  // Bulk indexing

  // Press  to switch between pretty and compact formatting for the JSON payload

  PUT _bulk

  {"index":{"_index":"test","_id":"1"}}

  {"value":"1","category":"click"}

  {"index":{"_index":"test","_id":"2"}}

  {"value":"2","category":"click"}



  // Auto encoding for date math expression

  // The following is encoded automatically into "PUT /%3Cmy-index-%7Bnow%2Fd%7D%3E" on the background

  PUT /



  // Shell out to download the EQL threat hunting demo file

  !curl -o normalized-T1117-AtomicRed-regsvr32.json https://raw.githubusercontent.com/elastic/elasticsearch/master/docs/src/test/resources/normalized-T1117-AtomicRed-regsvr32.json



  // Bulk indexing the above downloaded file

  POST my-index-000001/_bulk

  @normalized-T1117-AtomicRed-regsvr32.json



  // Execute an EQL query (triple quotes can be either ''' or """)

  GET /my-index-000001/_eql/search?filter_path=-hits.events

  {

    "query": """

      any where process.name == "regsvr32.exe"

    """,

    "size": 200

  }



  // Create an API key

  PUT _security/api_key

  {

    "name": "key-1",  // extra comma is OK, and this comment is ok as well

  }



  // Connect using the above generated API key

  // The dot notation is used to index into JSON object and array

  connect api_key=_.'id' + ":" + _.'api_key' hosts='localhost:9200'  // Quotes are necessary for strings



  // Connect to Elastic Cloud with Cloud ID

  connect cloud_id='YOUR_CLOUD_ID' username='elastic'



  // Issue a call to the cloud cluster

  get /  // HTTP method is case-insensitive

  get / conn=0  // send the request to the first connection (zero-based index) with the conn option



  // Check configuration location and values

  config



  // List available connections

  connection

  connection @info  // check details

  connection rename='my-cloud-cluster'  // give a friendly name to the current connection

  connection 0  // switch to the first connection

  connection remove=0  // remove the first connection

  connection 'my-cloud-cluster'  // switch to the cloud cluster connection



  // Save the connections we have so far. Session is also auto-saved on exit.

  session @save  // it can be loaded later with "session @load"



  // Session auto-load on start up can be enabled by set "auto_load_session = True" in peekrc file.

  // This helps preserving connections across restart.



  // Builtin help

  help  // list available functions

  help session  // a bit more detailed info about the "session" builtin function



  // Capture the terminal I/O

  capture @start

  capture  // show capture status



  // Run-AS and other headers

  GET _security/_authenticate runas='foo' xoid='my-x-opaque-id' headers={'some-other-header': 'blah'}



  // Show only the first role from previous response

  echo _."roles".0



  // If the cluster has SAML integration configured, authenticate with saml

  // Note this opens a web browser to perform the front-channel flow

  saml_authenticate



  // Load and run an external script

  run 'my-setup.es'



  // Stop the capture

  capture @stop



  // Minimal scripting for populating an index

  let tags = range(0, 100)

  for i in tags {

    PUT ("my-index/_doc/" + i)

    { 'tag': i, "value": i * i }

  }



  // Or with bulk index

  for i in range(1, 100) {  // first prepare the payload file

    echo {"index":{"_index":"test","_id":"" + i}} file='payload.json'

    echo {"value":i,"category":"click"} file='payload.json'

  }

  // Now bulk indexing with the above generated file

  PUT _bulk

  @payload.json



The tool can also run in batch mode. Assuming above commands are saved in a file called ``script.es``,

it can be executed as:



.. code-block:: bash



  # Positional argument

  peek script.es



  # Pipe

  cat script.es | peek



  # Or with heredoc

  peek << EOF

  GET /_cluster/health

  // etc

  EOF



External scripts can used to provide extra functions. They are simple Python scripts that define

and expose callabes under a variable named ``EXPORTS``. Please refer `natives.py `_

for examples.



Please also read `Usages `_ for more details.



Credits

-------

`Pgcli `_ has been a great reference of learning how to use

`prompt-toolkit `_, which is a critical

dependency of this project.