elasticsearch-collection
An ongoing & curated collection of awesome software, libraries, learning tutorials, and most important tools, esources and cool stuff about Elasticsearch
https://github.com/exajobs/elasticsearch-collection
Last synced: 4 days ago
JSON representation
Articles
- Elasticsearch Intro - Elasticsearch: What it is, How it works, and what it’s used for.
Books
- Deep Learning for Search - teaches you how to leverage neural networks, NLP, and deep learning techniques to improve search performance. (2019)
- Relevant Search: with applications for Solr and Elasticsearch - demystifies relevance work. Using Elasticsearch, it teaches you how to return engaging search results to your users, helping you understand and leverage the internals of Lucene-based search engines. (2016)
- Elasticsearch in Action - teaches you how to build scalable search applications using Elasticsearch (2015)
- Elasticsearch in Action, Second edition - hands-on guide to developing fully functional search engines with Elasticsearch and Kibana. (2021)
Certifications
Elastic Certified Engineer
- Elastic Certified Engineer notes - notes and exercises to prepare the certification exam
Discussions and social media
- /r/elasticsearch
- Elasticsearch forum
- Books on Amazon - does not fit well into this category, but worth checking out!
Elastic Architecture
Indices
- Best Practices for Managing Elasticsearch Indices - Understanding indices
Elasticsearch developer tools and utilities
Development and debugging
- ES-mode - An Emacs major mode for interacting with Elasticsearch (similar to Sense)
- Elasticstat - CLI tool displaying monitoring informations like htop
- Elastic for Visual Studio Code - An extension for developing Elasticsearch queries like Kibana and Sense extention in Visual Studio Code
- Elastic Builder - A Node.js implementation of the Elasticsearch DSL
- Bodybuilder - A Node.js elasticsearch query body builder
- enju - A Node.js elasticsearch ORM
- Peek - An interactive CLI in Python that works like Kibana Console with additional features
Import and Export
- Knapsack plugin - is an "swiss knife" export/import plugin for Elasticsearch
- Elasticsearch-Exporter - is a command line script to import/export data from Elasticsearch to various other storage systems
- esbulk - Parallel elasticsearch bulk indexing utility for the command line.
- elasticsearch-loader - Tool for loading common file types to elasticsearch including csv, json, and parquet
- elasticdump - tools for moving and saving indices
Management
- Esctl - High-level command line interface to manage Elasticsearch clusters.
- Vulcanizer - Github's open sourced cluster management library based on Elasticsearch's REST API. Comes with a high level CLI tool
Elasticsearch plugins
Cluster
- sscarduzio/elasticsearch-readonlyrest-plugin - Safely expose Elasticsearch REST API directly to the public
- mobz/elasticsearch-head - is a powerful and essential plugin for managing your cluster, indices and mapping
- Cerebro - is an open source(MIT License) elasticsearch web admin tool. Supports ES 5.x
- Kopf - Another management plugin that have REST console and manual shard allocation
- Search Guard - Elasticsearch and elastic stack security and alerting for free
- Elasticsearch Comrade - Elasticsearch admin panel built for ops and monitoring
- elasticsearch-admin - Web administration for Elasticsearch
- Elastic HQ - Elasticsearch cluster management console with live monitoring and beautiful UI
- Bigdesk - Live charts and statistics for elasticsearch cluster
Integrations and SQL support
- elastic/elasticsearch-hadoop - Elasticsearch real-time search and analytics natively integrated with Hadoop (and Hive)
- jprante/elasticsearch-jdbc - JDBC importer for Elasticsearch
- pandasticsearch - An Elasticsearch client exposing DataFrame API
- monstache - Go daemon that syncs MongoDB to Elasticsearch in near realtime
- NLPchina/elasticsearch-sql - Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL.
Other
- SIREn Join Plugin for Elasticsearch - This plugin extends Elasticsearch with new search actions and a filter query parser that enables to perform a "Filter Join" between two set of documents (in the same index or in different indexes).
You know, for search
- jprante/elasticsearch-plugin-bundle - A plugin that consists of a compilation of useful Elasticsearch plugins related to indexing and searching documents
Elastic Stack
- Elasticsearch - official website
- Logstash - is a data pipeline that helps you process logs and other event data from a variety of systems
- Kibana - is a data analysis tool that helps to visualize your data; Kibana Manual docs
- beats - is the platform for building lightweight, open source data shippers for many types of data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana.
Java tuning
- Tuning Garbage Collection for Mission-Critical Java Applications
- Garbage Collection Settings for Elasticsearch Master Nodes - Fine tunine your garbage collector
- Understanding G1 GC Log Format - To tune and troubleshoot G1 GC enabled JVMs, one must have a proper understanding of G1 GC log format. This article walks through key things that one should know about the G1 GC log format.
- JVM Garbage Collector settingsinvestigation - PDF Comparison of JVM GC
Kibana plugins and applications
Kibana Visualization plugins
- nbs-system/mapster - a visualization which allows to create live event 3d maps in Kibana
- Kibana Tag Cloud Plugin - tag cloud visualization plugin based on d3-cloud plugin
- LogTrail - a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail
-
Analyze API
- Kibana 6 application to manipulate the
_analyzeAPI graphically - kbn_network - This is a plugin developed for Kibana that displays a network node that link two fields that have been previously selected.
- elastic/timelion - time-series analyses application. Overview and installation guide: Timelion: The time series composer for Kibana
- VulnWhisperer - VulnWhisperer is a vulnerability data and report aggregator.
- Wazuh Kibana App - A Kibana app for working with data generated by Wazuh.
- Kibana Alert App for Elasticsearch - Kibana plugin with monitoring, alerting and reporting capabilities
Open-source and free products, based on Elasticsearch
- Fess - is an open source full featured Enterprise Search, with a web-crawler
- logagent - a log shipper to parse and ship logs to Elasticsearch including bulk indexing, disk buffers and log format detection.
- Yelp/elastalert - is a modular flexible rules based alerting system written in Python
- etsy/411 - an Alert Management Web Application https://demo.fouroneone.io (credentials: user/user)
- appbaseio/mirage - is a 🔎 GUI for composing Elasticsearch queries
- exceptionless/Exceptionless - is an error (exceptions) collecting and reporting server with client bindings for a various programming languages
- searchkit/searchkit - is a UI framework based on React to build awesome search experiences with Elasticsearch
- Simple File Server - is an Openstack Swift compatible distributed object store that can serve and securely store billions of large and small files using minimal resources.
- ItemsAPI - simplified search API for web and mobile (based on Elasticsearch and Express.js)
- Kuzzle - An open-source backend with advanced real-time features for Web, Mobile and IoT that uses ElasticSearch as a database. (Website)
- SIAC - SIAC is an enterprise SIEM built on the ELK stack and other open-source components.
- DataStation - Easily query, script, and visualize data from every database, file, and API.
- appbaseio/dejavu - The missing UI for Elasticsearch; landing page
- Sentinl - Sentinl is a Kibana alerting and reporting app.
Other
- Structured logging with ELK - part 1
- Graylog2/graylog2-server - Free and open source log management (based on ES)
- logiq - Simple WebUI Monitoring Tool for Logstash ver. 5.0 and up
- Elasticsearch Engineer Interview Questions
Related (awesome) lists
- frutik/awesome-search - I am building e-commerce search now. Below are listed some of my build blocks
Scalable Infrastructure and performance
Alerts
- Elastalert: implementing rich monitoring with Elasticsearch
- ElastAlert: Alerting At Scale With Elasticsearch, Part 1 - by engineeringblog.yelp.com
- ElastAlert: Alerting At Scale With Elasticsearch, Part 2 - by engineeringblog.yelp.com
Integrations
- Connecting Tableau to Elasticsearch (READ: How to query Elasticsearch with Hive SQL and Hadoop)
- mradamlacey/elasticsearch-tableau-connector
Machine Learning
- Elasticsearch with Machine Learning - (English translation) by Kunihiko Kido
- Recommender System with Mahout and Elasticsearch
Time series
- Elasticsearch as a Time Series Data Store - by Felix Barnsteiner
- Running derivatives on Voyager velocity data - By Colin Goodheart-Smithe
- Part 1 - Part 2 by Zachary Tong
- Part 1 - Part 2 - Part 3 by Zachary Tong
Use cases for Elasticsearch
- OFAC compliance with Elasticsearch - using AWS
- Data Infrastructure at IFTTT - Elasticsearch, Kafka, Apache Spark, Redhsift, other AWS services
- Building a Streaming Search Platform - -Streaming Search on Tweets: Storm, Elasticsearch, and Redis
- The Authoritative Guide to Elasticsearch Performance Tuning (Part 1) - Part 2 Part 3
- Understanding Elasticsearch Performance
- How to monitor Elasticsearch performance
- How to collect Elasticsearch metrics
- How to monitor Elasticsearch with Datadog
- How to solve 5 Elasticsearch performance and scaling problems
- Scaling Elasticsearch to Hundreds of Developers - by Joseph Lynch @yelp
- Operator for running Elasticsearch in Kubernetes
- Elasticsearch for Logging - Elasticsearch configuration tips and tricks from Sanity
System configuration
- Elasticsearch pre-flight checklist
- 9 Tips on Elasticsearch Configuration for High Performance
- How to Secure Elasticsearch and Kibana - with NGINX, LDAP and SSL 🔒
- Elasticsearch Guides - Useful Elasticsearch guides with best practices, troubleshooting instructions for errors, tips, examples of code snippets and more.
- The definitive guide for Elasticsearch on Windows Azure
- Elasticsearch server on Webfaction using NGINX with basic authorization and HTTPS protocol
Tutorials
- Make Sense of your Logs: From Zero to Hero in less than an Hour! by Britta Weber - demonstrates how you can build Elasticsearch + Logstash + Kibana stack to collect and discover your data
- $$ Elasticsearch 7 and Elastic Stack - liveVideo course that teaches you to search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more.
- dwyl/learn-elasticsearch - a getting started tutorial with a pack of valuable references
Videos
Advanced
- #bbuzz 2015: Adrien Grand – Algorithms and data-structures that power Lucene and Elasticsearch
- Rafał Kuć - Running High Performance Fault-tolerant Elasticsearch Clusters on Docker - and slides
- End-to-end Recommender System with Spark and Elasticsearch - by Nick Pentreath & Jean-François Puget. Slide deck
- Working with Elasticsearch - Search, Aggregate, Analyze, and Scale Large Volume Datastores - O'Reilly Media
Code, configuration file samples and other gists
- Elasticsearch config for a write-heavy cluster - reyjrar/elasticsearch.yml
- chenryn/ESPL - Elastic Search Processing Language - PEG parser sample for SPL to Elasticsearch DSL
- thomaspatzke/EQUEL - an Elasticsearch QUEry Language, based on G4 grammar parser
Overviews
- Elasticsearch for logs and metrics: A deep dive – Velocity 2016 - by Sematext Developers
- Elasticsearch in action Thijs Feryn - a beginner overview
- Getting Down and Dirty with ElasticSearch by Clinton Gormley
- Speed is a Key: Elasticsearch under the Hood - introduction + basic performance optimization
- $$ Complete Guide to Elasticsearch - Comprehensive guide to Elasticsearch, the popular search engine built on Apache Lucene
- Elasticsearch Query Editor in Grafana
- Scale Your Metrics with Elasticsearch - 2019 by Philipp Krenn (Elastic) optimization tips and tricks
- Getting started with Elasticsearch
What is ElasticSearch?
- Elasticsearch - is a distributed, open-source search and analytics engine built on Apache Lucene and developed in Java. . It was developed in Java and is designed to operate in real time. It can search and index document files in diverse formats. It was designed to be used in distributed environments by providing flexibility and scalability. Now, Elasticsearch is a widely popular enterprise search engine. Elasticsearch allows you to store, search, and analyze huge volumes of data quickly and in near real-time and give back answers in milliseconds.