{"id":16732353,"url":"https://github.com/danielberkompas/elasticsearch-elixir","last_synced_at":"2025-05-15T14:07:28.906Z","repository":{"id":39583240,"uuid":"109027767","full_name":"danielberkompas/elasticsearch-elixir","owner":"danielberkompas","description":"No-nonsense Elasticsearch library for Elixir","archived":false,"fork":false,"pushed_at":"2023-10-17T08:49:53.000Z","size":286,"stargazers_count":417,"open_issues_count":21,"forks_count":70,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-05-21T19:19:19.760Z","etag":null,"topics":["elasticsearch","elixir"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danielberkompas.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-10-31T17:07:59.000Z","updated_at":"2024-05-16T03:10:21.000Z","dependencies_parsed_at":"2024-01-26T05:07:10.968Z","dependency_job_id":"d185cd4d-c943-4bb3-a033-866d22563238","html_url":"https://github.com/danielberkompas/elasticsearch-elixir","commit_stats":{"total_commits":193,"total_committers":19,"mean_commits":"10.157894736842104","dds":"0.31088082901554404","last_synced_commit":"91cb81251b76e0ea98df6c1c7f7ba86aad8fe5c7"},"previous_names":["infinitered/elasticsearch-elixir"],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielberkompas%2Felasticsearch-elixir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielberkompas%2Felasticsearch-elixir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielberkompas%2Felasticsearch-elixir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danielberkompas%2Felasticsearch-elixir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danielberkompas","download_url":"https://codeload.github.com/danielberkompas/elasticsearch-elixir/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247704569,"owners_count":20982298,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","elixir"],"created_at":"2024-10-12T23:44:01.611Z","updated_at":"2025-04-07T18:10:11.487Z","avatar_url":"https://github.com/danielberkompas.png","language":"Elixir","funding_links":[],"categories":["Elixir"],"sub_categories":[],"readme":"# Elasticsearch\n\n[![Hex.pm](https://img.shields.io/hexpm/v/elasticsearch.svg)](https://hex.pm/packages/elasticsearch)\n[![Build Status](https://danielberkompas.semaphoreci.com/badges/elasticsearch-elixir/branches/master.svg?style=shields)](https://danielberkompas.semaphoreci.com/projects/elasticsearch-elixir)\n[![Coverage Status](https://coveralls.io/repos/github/danielberkompas/elasticsearch-elixir/badge.svg?branch=master)](https://coveralls.io/github/danielberkompas/elasticsearch-elixir?branch=master)\n\nA simple, no-nonsense Elasticsearch library for Elixir. Highlights include:\n\n- **No DSLs.** Interact directly with the `Elasticsearch` JSON API.\n- **Zero-downtime index (re)building.** Via `Mix.Tasks.Elasticsearch.Build` task.\n- **Dev Tools**. Helpers for running Elasticsearch as part of your supervision\n  tree during development.\n\n## Installation\n\nAdd `elasticsearch` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:elasticsearch, \"~\u003e 1.0.0\"}\n  ]\nend\n```\n\nThen, create an `Elasticsearch.Cluster` in your application:\n\n```elixir\ndefmodule MyApp.ElasticsearchCluster do\n  use Elasticsearch.Cluster, otp_app: :my_app\nend\n```\n\nOnce you have created your cluster, add it to your application's supervision tree:\n\n```elixir\nchildren = [\n  MyApp.ElasticsearchCluster\n]\n```\n\nFinally, you can issue requests to Elasticsearch using it.\n\n```elixir\nElasticsearch.get(MyApp.ElasticsearchCluster, \"/_cat/health\")\n```\n\n## Configuration\n\nSee the annotated example configuration below.\n\n```elixir\nconfig :my_app, MyApp.ElasticsearchCluster,\n  # The URL where Elasticsearch is hosted on your system\n  url: \"http://localhost:9200\",\n\n  # If your Elasticsearch cluster uses HTTP basic authentication,\n  # specify the username and password here:\n  username: \"username\",\n  password: \"password\",\n\n  # If you want to mock the responses of the Elasticsearch JSON API\n  # for testing or other purposes, you can inject a different module\n  # here. It must implement the Elasticsearch.API behaviour.\n  api: Elasticsearch.API.HTTP,\n\n  # Customize the library used for JSON encoding/decoding.\n  json_library: Poison, # or Jason\n\n  # You should configure each index which you maintain in Elasticsearch here.\n  # This configuration will be read by the `mix elasticsearch.build` task,\n  # described below.\n  indexes: %{\n    # This is the base name of the Elasticsearch index. Each index will be\n    # built with a timestamp included in the name, like \"posts-5902341238\".\n    # It will then be aliased to \"posts\" for easy querying.\n    posts: %{\n      # This file describes the mappings and settings for your index. It will\n      # be posted as-is to Elasticsearch when you create your index, and\n      # therefore allows all the settings you could post directly.\n      settings: \"priv/elasticsearch/posts.json\",\n\n      # This store module must implement a store behaviour. It will be used to\n      # fetch data for each source in each indexes' `sources` list, below:\n      store: MyApp.ElasticsearchStore,\n\n      # This is the list of data sources that should be used to populate this\n      # index. The `:store` module above will be passed each one of these\n      # sources for fetching.\n      #\n      # Each piece of data that is returned by the store must implement the\n      # Elasticsearch.Document protocol.\n      sources: [MyApp.Post],\n\n      # When indexing data using the `mix elasticsearch.build` task,\n      # control the data ingestion rate by raising or lowering the number\n      # of items to send in each bulk request.\n      bulk_page_size: 5000,\n\n      # Likewise, wait a given period between posting pages to give\n      # Elasticsearch time to catch up.\n      bulk_wait_interval: 15_000, # 15 seconds\n\n      # By default bulk indexing uses the \"create\" action. To allow existing\n      # documents to be replaced, use the \"index\" action instead.\n      bulk_action: \"create\"\n    }\n  }\n```\n\n#### Specifying HTTPoison Options\n\n```elixir\nconfig :my_app, MyApp.ElasticsearchCluster,\n  default_options: [\n    timeout: 5_000,\n    recv_timeout: 5_000,\n    hackney: [pool: :pool_name]\n  ]\n```\n\n## Protocols and Behaviours\n\n#### Elasticsearch.Store\n\nYour app must provide a `Store` module, which will fetch data to upload to\nElasticsearch. This module must implement the `Elasticsearch.Store`\nbehaviour.\n\nThe example below uses `Ecto`, but you can implement the behaviour on top\nof any persistence layer.\n\n```elixir\ndefmodule MyApp.ElasticsearchStore do\n  @behaviour Elasticsearch.Store\n\n  import Ecto.Query\n\n  alias MyApp.Repo\n\n  @impl true\n  def stream(schema) do\n    Repo.stream(schema)\n  end\n\n  @impl true\n  def transaction(fun) do\n    {:ok, result} = Repo.transaction(fun, timeout: :infinity)\n    result\n  end\nend\n```\n\n#### Elasticsearch.Document\n\nEach result returned by your store must implement the `Elasticsearch.Document`\nprotocol.\n\n```elixir\ndefimpl Elasticsearch.Document, for: MyApp.Post do\n  def id(post), do: post.id\n  def routing(_), do: false\n  def encode(post) do\n    %{\n      title: post.title,\n      author: post.author\n    }\n  end\nend\n```\n\n#### Elasticsearch.API\n\nYou can plug in a different module to make API requests, as long as it\nimplements the `Elasticsearch.API` behaviour.\n\nThis can be used in test mode, for example:\n\n```elixir\n# config/test.exs\nconfig :my_app, MyApp.ElasticsearchCluster,\n  api: MyApp.ElasticsearchMock\n```\n\nYour mock can then stub requests and responses from Elasticsearch.\n\n```elixir\ndefmodule MyApp.ElasticsearchMock do\n  @behaviour Elasticsearch.API\n\n  @impl true\n  def request(_config, :get, \"/posts/1\", _data, _opts) do\n    {:ok, %HTTPoison.Response{\n      status_code: 404,\n      body: %{\n        \"status\" =\u003e \"not_found\"\n      }\n    }}\n  end\nend\n```\n\n#### Elasticsearch.API.AWS\n\nAs AWS does not provide credentials' based http authentication, you can use the `Elasticsearch.API.AWS` module if you want to use AWS Elasticsearch Service with AWS Signature V4 signed HTTP connections.\n\nTo use this, just add `sigaws` to your dependencies and add this to your configuration:\n\n```elixir\n# Add to deps \ndef deps do\n  [          \n    # ...\n    {:sigaws, \"\u003e= 0.0.0\"}\n  ]\nend\n\n# config/prod.exs\nconfig :my_app, MyApp.ElasticsearchCluster,\n  api: Elasticsearch.API.AWS,\n  default_options: [\n    aws: [\n      region: \"us-east-1\",\n      service: \"es\",\n      access_key: \"aws_access_key_id\",\n      secret: \"aws_secret_access_key\"\n    ]\n  ]\n```\n\n## Indexing\n\n#### Bulk\n\nUse the `mix elasticsearch.build` task to build indexes using a zero-downtime,\nhot-swap technique with Elasticsearch aliases.\n\n```bash\n# This will read the `indexes[posts]` configuration seen above, to build\n# an index, `posts-123123123`, which will then be aliased to `posts`.\n$ mix elasticsearch.build posts --cluster MyApp.ElasticsearchCluster\n```\n\nSee the docs on `Mix.Tasks.Elasticsearch.Build` and `Elasticsearch.Index`\nfor more details.\n\n#### Individual Documents\n\nUse `Elasticsearch.put_document/3` to upload a document to a particular index.\n\n```elixir\n# MyApp.Post must implement Elasticsearch.Document\nElasticsearch.put_document(MyApp.ElasticsearchCluster, %MyApp.Post{}, \"index-name\")\n```\n\nTo remove documents, use `Elasticsearch.delete_document/3`:\n\n```elixir\nElasticsearch.delete_document(MyApp.ElasticsearchCluster, %MyApp.Post{}, \"index-name\")\n```\n\n## Querying\n\nYou can query Elasticsearch the `post/3` function:\n\n```elixir\n# Raw query\nElasticsearch.post(MyApp.ElasticsearchCluster, \"/posts/_doc/_search\", '{\"query\": {\"match_all\": {}}}')\n\n# Using a map\nElasticsearch.post(MyApp.ElasticsearchCluster, \"/posts/_doc/_search\", %{\"query\" =\u003e %{\"match_all\" =\u003e %{}}})\n```\n\nSee the official Elasticsearch [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/index.html)\nfor how to write queries.\n\n## Dev Tools\n\nThis package provides two utilities for developing with Elasticsearch:\n\n- `mix elasticsearch.install`: A mix task to install Elasticsearch and Kibana\n  to a folder of your choosing.\n\n- `Elasticsearch.Executable`. Use this to start and stop Elasticsearch as part\n  of your supervision tree.\n\n  ```elixir\n  children = [\n    worker(Elasticsearch.Executable, [\n      \"Elasticsearch\",\n      \"./vendor/elasticsearch/bin/elasticsearch\", # assuming elasticsearch is in your vendor/ dir\n      9200\n    ], id: :elasticsearch),\n    worker(Elasticsearch.Executable, [\n      \"Kibana\",\n      \"./vendor/kibana/bin/kibana\", # assuming kibana is in your vendor/ dir\n      5601\n    ], id: :kibana)\n  ]\n  ```\n\n## Elasticsearch 5.x Support\n\nAs of version `0.3.0` of this client library, multiple document types are not\nsupported, because support for these was removed in Elasticsearch 6.x. You\ncan still use this library with Elasticsearch 5.x, but you must design your\nindexes in the Elasticsearch 6.x style.\n\nRead more about this in Elasticsearch's guide, [\"Removal of Mapping\nTypes\"](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/removal-of-types.html).\n\n## Documentation\n\n[Hex Documentation](https://hexdocs.pm/elasticsearch)\n\nRun `mix docs` to generate local documentation.\n\n## Contributing\n\nTo contribute code to this project, you'll need to:\n\n1. Fork the repo\n2. Clone your fork\n3. Run `bin/setup`\n4. Create a branch\n5. Commit your changes\n6. Open a PR\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielberkompas%2Felasticsearch-elixir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanielberkompas%2Felasticsearch-elixir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanielberkompas%2Felasticsearch-elixir/lists"}