{"id":22486261,"url":"https://github.com/nearform/leaistic","last_synced_at":"2025-08-02T19:31:19.152Z","repository":{"id":139000338,"uuid":"134390958","full_name":"nearform/leaistic","owner":"nearform","description":"An ElasticSearch manager","archived":true,"fork":false,"pushed_at":"2020-04-16T15:23:21.000Z","size":566,"stargazers_count":11,"open_issues_count":14,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-16T19:08:12.222Z","etag":null,"topics":["elasticsearch","javascript","library","microservice","node","node-js","nodejs"],"latest_commit_sha":null,"homepage":"https://www.nearform.com/blog/leaistic-a-library-and-microservice-for-managing-elasticsearch-content/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nearform.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-22T09:20:31.000Z","updated_at":"2024-01-17T15:50:00.000Z","dependencies_parsed_at":"2023-05-28T17:00:35.513Z","dependency_job_id":null,"html_url":"https://github.com/nearform/leaistic","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/nearform/leaistic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nearform%2Fleaistic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nearform%2Fleaistic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nearform%2Fleaistic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nearform%2Fleaistic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nearform","download_url":"https://codeload.github.com/nearform/leaistic/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nearform%2Fleaistic/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268440311,"owners_count":24250771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","javascript","library","microservice","node","node-js","nodejs"],"created_at":"2024-12-06T17:14:13.934Z","updated_at":"2025-08-02T19:31:18.842Z","avatar_url":"https://github.com/nearform.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"[![CircleCI](https://circleci.com/gh/nearform/leaistic.svg?style=svg\u0026circle-token=4b2a232c7e549a0ef8df33ad69929077cb15acb4)](https://circleci.com/gh/nearform/leaistic)\n[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release)\n\n[Changelog](https://github.com/nearform/leaistic/releases/)\n\n# Leaistic\nLeaistic is an opinionated ElasticSearch manager micro-service and embeddable library. It allows to manage index creation, its mapping, its settings and update in ElasticSearch with no downtime and high availability.\n\n## TLDR; Usage\nInstall it and use it as a library in your project:\n```console\n$\u003e npm i --save leaistic\n```\n\nUse it instantly using npx, e.g.:\n```\n$\u003e env ES_URL=http://myhost:9200 npx leaistic | npx pino-colada\n```\nthen open [http://localhost:3000/documentation]() in your browser.\n\n\n... or just clone the repository, then read the rest of the README to have more control over it 😉\n\n## Why Leaistic ?\n\nBrief, [ElasticSearch is smart, but not enough, with your data](./WhyLeaistic.md)\n\n## The Leaistic way\n\nIn order to provide high availability and simplicity of usage, it uses a set of simple rules:\n\n1.  Every `index` in usage should have an `alias`\n2.  Every programs should use the previously mentionned `alias` to read and write data\n3.  It is highly advised to deploy an `index template` to setup the `settings` and the `mappings` of the indices\n\n## How Leaistic creates an index\n\nIn order to follow te above rules, to simplify the developer work, Leaistic has a simple convention for \"creating an index\":\n\n1.  When you want to use `myIndex` for an index, Leaistic will first, if you provide it, create an index template `myIndex` matching `myIndex-*`, so any index using this convention will use the latest mapping\n2.  Then, Leaistic actually creates an index with the convention `{alias}-{date}`, e.g. `myIndex-1970-01-01t00:00:00.000z` (with the end part being the current ISO-like UTC date), matching the index template\n3.  Once the index created and refreshed, it creates an alias `myIndex` pointing to the index previously created, `myIndex-1970-01-01t00:00:00.000z`\n4.  Then you can use `myIndex` like if it was an index, and start to work !\n\n## How Leaistic updates an index\n\nUpdating an index is a bit more complicated: ElasticSearch does not manage breaking changes on mappings and settings, but thanks to the aliases and the reindexing API, it provides a good way to make the change.\n\n1.  For updating `myIndex`, Leaistic will first check the alias `myIndex` is existing, and what is the index it points to\n2.  Then, it will update the index template if one is provided (it is likely)\n3.  After that, it will create a new index using the same convention, `{alias}-{date}` , e.g. `myIndex-1970-01-02t00:00:00.000z`\n4.  Then it will ask ElasticSearch to reindex the source index that was pointed by `myIndex` alias and await for the reindexation to complete\n5.  After some checks, it will then switch the alias `myIndex` to the new index `myIndex-1970-01-02t00:00:00.000z` when it will be ready\n6.  It will finally both check everything is alright and delete the old index that is no more useful\n\n## How Leaistic deletes an index\n\nDeleting an index is pretty simple:\n\n1.  Leaistic first finds out what is the first index pointed by the alias\n2.  Then, it will delete both the alias and the index in parallel\n\n## And if something goes wrong ?\n\nGiven ElasticSearch does not have a transaction system, the best we can do is trying to manage rollbacks as well as possible when things goes wrong\n\nFor now, during rollbacks, the last `index template` will be deleted if we deployed a new one in the query we would need to back it up somewhere to be able to rollback to the original one.\n\nEvery created resource will be deleted, and alias switched back to their original indices\n\n# Run as a microservice\n\nFirst you need an ElasticSearch server accessible at [http://127.0.0.1:9200](http://127.0.0.1:9200) if you don't want any configuration. We provide a `docker-compose` file to allow spawning a cluster and a Cerebro interface easily.\n\nTo spawn an ElasticSearch cluster, and Cerebro, run:\n``` console\n$\u003e npm run es:local \u0026\n```\n\nStart the server (using nodemon to monitor the process, and with human oriented logs)\n```console\n$\u003e npm start\n```\n\nThen go to:\n-   [http://localhost:3000/documentation](http://localhost:3000/documentation) to use the Swagger interface\n-   [http://localhost:9000](http://localhost:9000) with `http://elasticsearch:9200` as a connection Url to use the Cerebro interface\n\n**Note**: *you'll find more commands you can use to run the service, using:*\n```console\n$\u003e npm run\n```\n\n# Usage as a library\n\nYou can use `Leaistic` as a library, allowing you, notably, to create scripts, for migrations, etc.\n\nYou can have a look at the [full fledge example](./examples.js) in order to see how you can use it in a script.\n\nSee more explanations and context about the API below\n\n## Index and its Alias creation, update and deletion\n\n### Creation\n```javascript\nconst {create} = require('leaistic')\n\nconst indexTemplate = {\n  index_patterns: ['myindex-*'],\n  settings: {\n    number_of_shards: 1\n  }\n}\n\n// create an index, optionally with an indexTemplate that should match it:\nawait create('an-index', indexTemplate)\n```\n\n### Update\n\n#### With automatic reindexation ( e.g. for index template change, if compatible )\n\n```javascript\nconst {update} = require('leaistic')\n\nconst indexTemplate = {\n  index_patterns: ['myindex-*'],\n  settings: {\n    number_of_shards: 2\n  }\n}\n\n// create an index, optionally with an indexTemplate that should match it:\nawait update('an-index', indexTemplate)\n```\n\n#### With manual reindexation ( e.g. for creating a whole new version of the data )\n\n```javascript\nconst {update} = require('leaistic')\n\nconst indexTemplate = {\n  index_patterns: ['myindex-*'],\n  settings: {\n    number_of_shards: 2\n  }\n}\n\nconst data = [\n  { hello: 'world'},\n  { hello: 'foo'},\n  { hello: 'bar'},\n  { hello: 'baz'}\n]\n\nconst reindexer = async (indexName, client) =\u003e client.bulk({\n    refresh: true,\n    body: data.reduce((acc, current) =\u003e {\n      return acc.concat([\n        {\n          index: {\n            _index: indexName,\n            _id: current.hello,\n          },\n        },\n        current\n      ])\n    }, [])\n  })\n}\n\n// create an index, optionally with an indexTemplate that should match it:\nawait update('an-index', indexTemplate, reindexer)\n```\n\n\n### Deletion\n\n```javascript\nconst {delete: del} = require('leaistic')\n\nawait del('an-index')\n```\n\n### Build a new Index name for a given alias (useful to manage some updates on your own)\n```javascript\nconst { newIndexName } = require('leaistic')\n\nnewIndexName('an-index')\n// 'an-index-1234-12-12t12:34:56.789z'\n```\n\n\n## ElasticSearch Connection\n\nBy default, Leaistic will connect to `http://127.0.0.1:9200` or the value provided by `ES_URL` environment variable.\n\nYou can provide your own `url` to connect to:\n\n```javascript\nconst {connect} = require('leaistic')\n\nconnect({url: 'http://myhost:9200'})\n// ... use Leaistic with this url\n```\n\nor you can define your own client, providing your own options, including the logger, instead of the default `pino` based one reserved for elasticsearch logging\n\nFor example, using ES default logger is simple to setup:\n```javascript\nconst {connect} = require('leaistic')\nconst elasticsearch = require('elasticsearch')\n\nconst client = new elasticsearch.Client({\n  host: 'http://myhost:9200',\n  log: 'trace' // note that using the default ES logger is not advised for production\n})\n\nconnect({client})\n```\n\n`connect` will also always return the ElasticSearch client currently used\n\n```javascript\nconst {connect} = require('leaistic')\n\nconst es = connect()\nawait es.bulk({\n  body: [\n    { index:  { _index: 'myindex', _type: 'mytype', _id: 1 } }, { title: 'foo' },\n    { update: { _index: 'myindex', _type: 'mytype', _id: 2 } }, { doc: { title: 'foo' } },\n    { delete: { _index: 'myindex', _type: 'mytype', _id: 3 } },\n  ]\n})\n```\n\n## Logger\n\nLeaistic is using [pino](https://getpino.io) loggers for providing fast and useful logs, however you may want to overrided them to use your own. There is one for the http service when used using `start`, one for the main code, and one dedicated to exchanges with ElasticSearch.\n\n### Change main Logger\n\nYou can override it using a [`pino`-compatible]() logger syntax (just the log levels functions are needed).\n\nFor example, this is enough:\n\n```javascript\nconst {logger} = require('leaistic')\n\n// change the main logger, with a pino-like interface ( https://getpino.io/#/docs/API)\nconst log = {\n  trace: (...args) =\u003e console.log(...args),\n  debug: (...args) =\u003e console.debug(...args),\n  info: (...args) =\u003e console.info(...args),\n  warn: (...args) =\u003e console.warn(...args),\n  error: (...args) =\u003e console.error(...args),\n  fatal: (...args) =\u003e console.error('💀', ...args)\n}\n\nlogger(log)\n```\n\n### ElasticSearch logger\n\nElasticSearch uses its own logger.\n\nYou can override it by setting the `ElasticSearch` client by yourself, as described [here](https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/logging.html):\n\n```javascript\nconst {connect} = require('leaistic')\nconst elasticsearch = require('elasticsearch')\n\n// change the ElasticSearch logger ( https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/logging.html )\nconst esLog = function () {\n  this.trace = (...args) =\u003e console.log(...args)\n  this.debug = (...args) =\u003e console.debug(...args)\n  this.info = (...args) =\u003e console.info(...args)\n  this.warn = (...args) =\u003e console.warn(...args)\n  this.error = (...args) =\u003e console.error(...args)\n  this.fatal = (...args) =\u003e console.error('💀', ...args)\n}\n\nconst client = new elasticsearch.Client({\n  host: 'http://127.0.0.1:9200',\n  log: esLog\n})\n\nconst es = connect({client})\n```\n\n## Example usage\n\nSee also a more [in depth example](./example.js), but you should get the idea with this :\n\n```javascript\nconst {connect, create, update, delete: del} = require('leaistic')\n\n// only needed if you want to obtain the ES client ot set up its\nconst es = connect()\n\n// create an index and an alias\nconst {name, index} = await create('myindex')\n\n// load some data\nawait es.bulk({\n  body: [\n    { index:  { _index: 'myindex', _type: 'mytype', _id: 1 } }, { title: 'foo', createdAt: Date.now() },\n    { index:  { _index: 'myindex', _type: 'mytype', _id: 2 } }, { title: 'bar', createdAt: Date.now() },\n    { index:  { _index: 'myindex', _type: 'mytype', _id: 3 } }, { title: 'baz', createdAt: Date.now() }\n  ]\n})\n\n// oh, wait, createdAt was considered as a number by ES, not a date,\n// and we actually need an exact match on 'title'? Let's fix that:\nconst indexTemplate = {\n  index_patterns: ['myindex-*'],\n  settings: {\n    number_of_shards: 1\n  },\n  mappings: {\n    mytype: {\n      properties: {\n        title: {\n          type: 'keyword'\n        },\n        createdAt: {\n          type: 'date',\n          format: 'epoch_millis'\n        }\n      }\n    }\n  }\n}\n\n// update the settings to add a index template (you could have done it during creation as well)\nawait update(name, { indexTemplate })\n\n// now 'createdAt' will be actually considered like a date\nconst res = await es.search({\n  index: 'myindex',\n  body: {\n  \"query\": {\n    \"bool\": {\n      \"must\": {\n        \"match\": {\n          \"title\": \"foo\"\n        }\n      },\n      \"filter\": {\n        \"range\": {\n          \"createdAt\": {\n            \"gte\": \"01/01/2012\",\n            \"lte\": \"2019\",\n            \"format\": \"dd/MM/yyyy||yyyy\"\n          }\n        }\n      }\n    }\n  }\n})\n\n// let's say this index is now deprecated, delete it\nawait del(name)\n```\n\nNote: if anything goes wrong during one of these steps, the promise will be rejected. When using `async`/`await` like above, it means an exception will be thrown\n\n# Development\n\n## Running the tests\n\nTo run the local ElasticSearch cluster using docker for running the tests :\n\n```console\n$\u003e npm run es:local \u0026\n$\u003e npm test\n```\n*Note*: alternatively, feel free to run `npm run es:local` in an alternative tab, or use `npm run es:local:start -- -d`\n\nYou can also run the tests in watch mode:\n```\n$\u003e npm run test:watch\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnearform%2Fleaistic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnearform%2Fleaistic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnearform%2Fleaistic/lists"}