{"id":24761660,"url":"https://github.com/driebit/mod_elasticsearch2","last_synced_at":"2026-05-01T10:32:47.863Z","repository":{"id":41386260,"uuid":"504449219","full_name":"driebit/mod_elasticsearch2","owner":"driebit","description":null,"archived":false,"fork":false,"pushed_at":"2023-10-03T09:52:13.000Z","size":120,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-23T13:15:55.989Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/driebit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-06-17T08:14:29.000Z","updated_at":"2023-01-31T19:57:31.000Z","dependencies_parsed_at":"2023-10-03T17:42:20.667Z","dependency_job_id":null,"html_url":"https://github.com/driebit/mod_elasticsearch2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/driebit/mod_elasticsearch2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/driebit%2Fmod_elasticsearch2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/driebit%2Fmod_elasticsearch2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/driebit%2Fmod_elasticsearch2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/driebit%2Fmod_elasticsearch2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/driebit","download_url":"https://codeload.github.com/driebit/mod_elasticsearch2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/driebit%2Fmod_elasticsearch2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32494270,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-28T19:18:39.786Z","updated_at":"2026-05-01T10:32:47.846Z","avatar_url":"https://github.com/driebit.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"mod_elasticsearch2\n==================\n\nThis [Zotonic](https://github.com/zotonic/zotonic) module gives you more relevant search results\nby making resources searchable through  \n[Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html).\n\n\nConfiguration\n-------------\n\nTo configure the Elasticsearch host and port, edit your \n[zotonic.config](http://docs.zotonic.com/en/latest/ref/configuration/zotonic-configuration.html)\nfile:\n\n```erlang\n[\n    %% ...\n    {elasticsearch2_host, \u003c\u003c\"elasticsearch\"\u003e\u003e},  %% Defaults to 127.0.0.1\n    {elasticsearch2_port, 9200},                 %% Defaults to 9200\n    %% ...\n].\n```\n\nOr in your site config:\n\n * `mod_elasticsearch2.host`\n * `mod_elasticsearch2.port`\n\nConfig keys:\n\n * `mod_elasticsearch2.track_total_hits` Elastic search normally doesn't count hits beyond 10K.\n   To enable or disable counting the real total number of hits set this config.\n   Defaults to `true` (track total hits).\n\n * `mod_elasticsearch2.default_operator`  The default text operator for words in a query string.\n   This is either AND or OR. Defaults to 'OR'.\n\n * `mod_elasticsearch2.add_search_wildcards` Automatically add wildcards to the words\n   in a text search string. This rewrites `\"The quick fox\"` to \n   `((the | the*) (quick | quick*) (fox | fox*)) | \"the quick fox\"`\n   Defaults to `true` (do add wildcards).\n\n * `mod_elasticsearch2.log_scores` Set to `true` to add an _info_ level log messages with the \n   document scores after each search. Useful for understanding search results. Defaults to `false`.\n\n### Elasticsearch security config\n\nOut of the box Elastic is configured to use TLS and passwords.\nBoth are not (yet) supported by this library, and also don't have any purpose on localhost.\n\nThe following change must be made to `config/elasticsearch.yml`:\n\n```yaml\n# Disable security features\nxpack.security.enabled: false\nxpack.security.enrollment.enabled: false\n\n# Disable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents\nxpack.security.http.ssl:\n  enabled: false\n  keystore.path: certs/http.p12\n\n```\n\n\nSearch queries\n--------------\n\nWhen mod_elasticsearch is enabled, it will direct all search operations of the \n‘query’ type to elasticsearch2:\n\n```erlang\nz_search:search({query, Args}, Context).\n```\n\nFor `Args`, you can pass all regular Zotonic [query arguments](http://docs.zotonic.com/en/latest/developer-guide/search.html#query-arguments),\nsuch as:\n\n```erlang\nz_search:search({query, [{hasobject, 507}]}, Context).\n````\n\n### Query context filters\n\nThe `filter` search argument that you know from Zotonic will be used in\nElasticsearch’s [filter context](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html).\nTo add filters that influence score (ranking), use the `query_context_filter`\ninstead. The syntax is identical to that of `filter`:\n\n```erlang\nz_search:search({query, [{query_context_filter, [[\"some_field\", \"value\"]]}]}, Context).\n```\n\n### Extra query arguments\n\nThis module adds some extra query arguments on top of Zotonic’s default ones.\n\nTo find documents that have a field, whatever its value (make sure to pass \n`exists` as atom): \n\n```erlang\n{filter, [\u003c\u003c\"some_field\"\u003e\u003e, exists]}\n```\n\nTo find documents that do not have a field (make sure to pass `missing` as \natom): \n\n```erlang\n{filter, [\u003c\u003c\"some_field\"\u003e\u003e, missing]}\n````\n\nFor a [match phrase prefix query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query-phrase-prefix.html),\nuse the `prefix` argument:\n\n```erlang\nz_search:search({query, [{prefix, \u003c\u003c\"Match this pref\"\u003e\u003e}]}, Context).\n```\n\nTo exclude a document:\n\n```erlang\n{exclude_document, [Type, Id]}\n```\n\nTo supply a custom [`function_score`](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html) \nclause, supply one or more `score_function`s. For instance, to rank recent\narticles above older ones:\n\n```erlang\nz_search:search(\n    {query, [\n        {text, \"Search this\"},\n        {score_function, #{\n            \u003c\u003c\"filter\"\u003e\u003e =\u003e [{cat, \"article\"}],\n            \u003c\u003c\"exp\"\u003e\u003e =\u003e #{\n                \u003c\u003c\"publication_start\"\u003e\u003e =\u003e #{\n                    \u003c\u003c\"scale\"\u003e\u003e =\u003e \u003c\u003c\"30d\"\u003e\u003e\n                }\n            }\n        }}\n    ]},\n    Context\n).\n```\n\n### Wildcards\n\nThe `text` query term is modified to search for prefix strings by appending `*` operators to the words in the query string.\n\nThis is _not_ done if either:\n\n * the search text contains simple query string operators, especially the `\"`; or\n * the config `mod_elasticsearch2.no_automatic_wildcard` is set to a true-ish value.\n\nSee https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html for the query string operators.\n\n\nBuffered put/delete\n-------------------\n\nUse the `mod_elasticsearch2:put_doc` and `mod_elasticsearch2:delete_doc` routines to perform bulk put and delete requests.\n\nThe requests are buffered for one second, or 500 commands, whatever comes first.\n\nAfter a command has been executed the following notification is emitted:\n\n```erlang\n-record(elasticsearch_bulk_result, {\n    action :: put | delete,\n    doc_id :: binary(),\n    index :: binary(),\n    result :: binary(),\n    status :: 100..500,\n    error :: map() | undefined\n}).\n```\n\nNotifications\n-------------\n\n### elasticsearch_fields\n\nObserve this foldr notification to change the document fields that are queried.\nYou can use Elasticsearch [multi_match syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html)\nfor boosting fields:\n\n```erlang\n%% your_site.erl\n\n-export([\n    % ...\n    observe_elasticsearch_fields/3\n]).\n\nobserve_elasticsearch_fields({elasticsearch_fields, QueryText}, Fields, Context) -\u003e\n    %% QueryText is the search query text\n\n    %% Add or remove fields: \n    [\u003c\u003c\"some_field\"\u003e\u003e, \u003c\u003c\"boosted_field^2\"\u003e\u003e|Fields].   \n```\n\n### elasticsearch_put\n\nObserve this notification to change the resource properties before they are\nstored in Elasticsearch. For instance, to store their zodiac sign alongside \nperson resources:\n\n```erlang\n%% your_site.erl\n\n-include_lib(\"mod_elasticsearch2/include/elasticsearch.hrl\").\n\n-export([\n    % ...\n    observe_elasticsearch_put/3\n]).\n\n-spec observe_elasticsearch_put(#elasticsearch_put{}, map(), z:context()) -\u003e map().\nobserve_elasticsearch_put(#elasticsearch_put{ index = _, type = \u003c\u003c\"resource\"\u003e\u003e, id = Id }, Data, Context) -\u003e\n    case m_rsc:is_a(Id, person, Context) of\n        true -\u003e\n            Data#{ zodiac =\u003e calculate_zodiac(Id, Context) };\n        false -\u003e\n            Data\n    end;\nobserve_elasticsearch_put(#elasticsearch_put{}, Data, Context) -\u003e\n    Data.\n```\n\nType\n----\n\nIn Elastic 5.x a document was associated with a _type_.\n\nThis _type_ has been removed in Elastic 7+.\n\nIn this library we still use the _type_, it is stored as `es_type` and it set to `resource` for\nall Zotonic resources.\n\nOn fetch of a document record the `_source.es_type` is copied to `_type`. This for compatibility with software\nwritten for Elastic 5.x.\n\nLikewise references to `_type` in queries are mapped to `es_type`.\n\nAs the document types are often used to distinguish ids between sources (for example Adlib databases) there are routines to combine the type and id:\n\n```erlang\nDocId = mod_elasticsearch2:typed_id(Id, Type),\n{Id, Type} = mod_elasticsearch2:type_id_split(DocId)\n```\n\nThe empty type and the `resource` type are not appended to the document id.\n\n\nLogging\n-------\n\nBy default, mod_elasticsearch2 logs outgoing queries at the debug log level. To\nsee them in your Zotonic console, change the minimum log level to debug:\n\n```erlang\nlager:set_loglevel(lager_console_backend, debug).\n```\n\nHow resources are indexed\n-------------------------\n\nContent in all languages is stored in the index, following the \n[one language per field](https://www.elastic.co/guide/en/elasticsearch/guide/current/one-lang-fields.html)\nstrategy: \n\n\u003e Each translation is stored in a separate field, which is analyzed according to\n\u003e the language it contains. At query time, the user’s language is used to boost\n\u003e fields in that particular language.\n\nDevelopment and low disk space\n------------------------------\n\nIf you happen to be low on disk space then Elastic will become read only.\n\nTo disable this, especially on development machines, perform the following two commands:\n\n```shell\ncurl -XPUT -H \"Content-Type: application/json\" http://localhost:9200/_cluster/settings -d '{ \"transient\": { \"cluster.routing.allocation.disk.threshold_enabled\": false } }'\ncurl -XPUT -H \"Content-Type: application/json\" http://localhost:9200/_all/_settings -d '{\"index.blocks.read_only_allow_delete\": null}'\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdriebit%2Fmod_elasticsearch2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdriebit%2Fmod_elasticsearch2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdriebit%2Fmod_elasticsearch2/lists"}