{"id":15151936,"url":"https://github.com/elastic/elasticsearch-mapper-attachments","last_synced_at":"2025-12-18T00:09:57.622Z","repository":{"id":1984302,"uuid":"2916197","full_name":"elastic/elasticsearch-mapper-attachments","owner":"elastic","description":"Mapper Attachments Type plugin for Elasticsearch","archived":true,"fork":false,"pushed_at":"2023-06-20T23:11:30.000Z","size":6866,"stargazers_count":503,"open_issues_count":6,"forks_count":94,"subscribers_count":386,"default_branch":"master","last_synced_at":"2024-09-22T23:03:43.327Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.elastic.co","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-12-05T12:04:04.000Z","updated_at":"2024-08-23T11:20:47.000Z","dependencies_parsed_at":"2022-09-09T15:00:53.534Z","dependency_job_id":null,"html_url":"https://github.com/elastic/elasticsearch-mapper-attachments","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Felasticsearch-mapper-attachments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Felasticsearch-mapper-attachments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Felasticsearch-mapper-attachments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Felasticsearch-mapper-attachments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastic","download_url":"https://codeload.github.com/elastic/elasticsearch-mapper-attachments/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219874707,"owners_count":16554610,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T15:40:42.727Z","updated_at":"2025-09-29T22:31:16.037Z","avatar_url":"https://github.com/elastic.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"**IMPORTANT**: this project now Read-Only and moved to elasticsearch repository (from elasticsearch 2.2.0): \n\n* [mapper-attachments](https://github.com/elastic/elasticsearch/tree/5.6/plugins/mapper-attachments) ([documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/mapper-attachments.html))\n\nIf you have a question about the plugin, please use [discuss.elastic.co](https://discuss.elastic.co/c/elasticsearch). \nIf you want to report a bug, please use [elasticsearch repository](https://github.com/elastic/elasticsearch/issues/new).\n\n---\n\nMapper Attachments Type for Elasticsearch\n=========================================\n\nThe mapper attachments plugin lets Elasticsearch index file attachments in over a thousand formats (such as PPT, XLS, PDF) using the Apache text extraction library [Tika](http://lucene.apache.org/tika/).\n\nIn practice, the plugin adds the `attachment` type when mapping properties so that documents can be populated with file attachment contents (encoded as `base64`).\n\nInstallation\n------------\n\nIn order to install the plugin, run:\n\n```sh\nbin/plugin install elasticsearch/elasticsearch-mapper-attachments/3.1.2\n```\n\nYou need to install a version matching your Elasticsearch version:\n\n|       Elasticsearch    | Attachments Plugin|                                                             Docs                                                                   |\n|------------------------|-------------------|------------------------------------------------------------------------------------------------------------------------------------|\n|    \u003e 2.1 \u003c 6.0         | elasticsearch repository | [github](https://github.com/elastic/elasticsearch/tree/5.x/plugins/mapper-attachments)                  |\n|    es-2.1              | Build from source | [es-2.1](https://github.com/elastic/elasticsearch-mapper-attachments/tree/es-2.1/)                  |\n|    2.1.2               |     3.1.2         | [3.1.2](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.1.2/#version-312-for-elasticsearch-21)                  |\n|    2.1.1               |     3.1.1         | [3.1.1](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.1.1/#version-311-for-elasticsearch-21)                  |\n|    2.1.0               |     3.1.0         | [3.1.0](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.1.0/#version-310-for-elasticsearch-21)                  |\n|    es-2.0              | Build from source | [es-2.0](https://github.com/elastic/elasticsearch-mapper-attachments/tree/es-2.0/)                  |\n|    2.0.2               |     3.0.4         | [3.0.4](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.0.4/#version-304-for-elasticsearch-20)                  |\n|    2.0.1               |     3.0.3         | [3.0.3](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.0.3/#version-303-for-elasticsearch-20)                  |\n|    2.0.0               |     3.0.2         | [3.0.2](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.0.2/#version-302-for-elasticsearch-20)                  |\n|    2.0.0-beta2         |     3.0.1         | [3.0.1](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.0.1/#version-301-for-elasticsearch-20)                  |\n|    2.0.0-beta1         |     3.0.0         | [3.0.0](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v3.0.0/#version-300-for-elasticsearch-20)                  |\n|    es-1.7              |     2.7.1         | [2.7.1](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v2.7.1/#version-271-for-elasticsearch-17)                  |\n|    es-1.6              |     2.6.0         | [2.6.0](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v2.6.0/#version-260-for-elasticsearch-16)                  |\n|    es-1.5              |     2.5.0         | [2.5.0](https://github.com/elastic/elasticsearch-mapper-attachments/tree/v2.5.0/#version-250-for-elasticsearch-15)                  |\n|    es-1.4              |     2.4.3         | [2.4.3](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v2.4.3/#version-243-for-elasticsearch-14)                  |\n|    es-1.3              |     2.3.2         | [2.3.2](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v2.3.2/#version-232-for-elasticsearch-13)                  |\n|    es-1.2              |     2.2.1         | [2.2.1](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v2.2.1/#version-221-for-elasticsearch-12)                  |\n|    es-1.1              |     2.0.0         | [2.0.0](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v2.0.0/#mapper-attachments-type-for-elasticsearch)  |\n|    es-1.0              |     2.0.0         | [2.0.0](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v2.0.0/#mapper-attachments-type-for-elasticsearch)  |\n|    es-0.90             |     1.9.0         | [1.9.0](https://github.com/elasticsearch/elasticsearch-mapper-attachments/tree/v1.9.0/#mapper-attachments-type-for-elasticsearch)  |\n\nTo build a `SNAPSHOT` version, you need to build it with Maven:\n\n```bash\ngradle clean assemble\nplugin --install mapper-attachments \\\n       --url file:target/releases/elasticsearch-mapper-attachments-X.X.X-SNAPSHOT.zip\n```\n\nHello, world\n------------\n\nCreate a property mapping using the new type `attachment`:\n\n```javascript\nPOST /trying-out-mapper-attachments\n{\n  \"mappings\": {\n    \"person\": {\n      \"properties\": {\n        \"cv\": { \"type\": \"attachment\" }\n}}}}\n```\n\nIndex a new document populated with a `base64`-encoded attachment:\n\n```javascript\nPOST /trying-out-mapper-attachments/person/1\n{\n  \"cv\": \"e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=\"\n}\n```\n\nSearch for the document using words in the attachment:\n\n```javascript\nPOST /trying-out-mapper-attachments/person/_search\n{\n  \"query\": {\n    \"query_string\": {\n      \"query\": \"ipsum\"\n}}}\n```\n\nIf you get a hit for your indexed document, the plugin should be installed and working.\n\nUsage\n------------------------\n\nUsing the attachment type is simple, in your mapping JSON, simply set a certain JSON element as attachment, for example:\n\n```javascript\nPUT /test\nPUT /test/person/_mapping\n{\n    \"person\" : {\n        \"properties\" : {\n            \"my_attachment\" : { \"type\" : \"attachment\" }\n        }\n    }\n}\n```\n\nIn this case, the JSON to index can be:\n\n```javascript\nPUT /test/person/1\n{\n    \"my_attachment\" : \"... base64 encoded attachment ...\"\n}\n```\n\nOr it is possible to use more elaborated JSON if content type, resource name or language need to be set explicitly:\n\n```\nPUT /test/person/1\n{\n    \"my_attachment\" : {\n        \"_content_type\" : \"application/pdf\",\n        \"_name\" : \"resource/name/of/my.pdf\",\n        \"_language\" : \"en\",\n        \"_content\" : \"... base64 encoded attachment ...\"\n    }\n}\n```\n\nThe `attachment` type not only indexes the content of the doc in `content` sub field, but also automatically adds meta \ndata on the attachment as well (when available).\n\nThe metadata supported are:\n\n* `date`\n* `title`\n* `name` only available if you set `_name` see above\n* `author`\n* `keywords`\n* `content_type`\n* `content_length` is the original content_length before text extraction (aka file size)\n* `language`\n\nThey can be queried using the \"dot notation\", for example: `my_attachment.author`.\n\nBoth the meta data and the actual content are simple core type mappers (string, date, ...), thus, they can be controlled\nin the mappings. For example:\n\n```javascript\nPUT /test/person/_mapping\n{\n    \"person\" : {\n        \"properties\" : {\n            \"file\" : {\n                \"type\" : \"attachment\",\n                \"fields\" : {\n                    \"content\" : {\"index\" : \"no\"},\n                    \"title\" : {\"store\" : \"yes\"},\n                    \"date\" : {\"store\" : \"yes\"},\n                    \"author\" : {\"analyzer\" : \"myAnalyzer\"},\n                    \"keywords\" : {\"store\" : \"yes\"},\n                    \"content_type\" : {\"store\" : \"yes\"},\n                    \"content_length\" : {\"store\" : \"yes\"},\n                    \"language\" : {\"store\" : \"yes\"}\n                }\n            }\n        }\n    }\n}\n```\n\nIn the above example, the actual content indexed is mapped under `fields` name `content`, and we decide not to index it, so\nit will only be available in the `_all` field. The other fields map to their respective metadata names, but there is no\nneed to specify the `type` (like `string` or `date`) since it is already known.\n\nCopy To feature\n---------------\n\nIf you want to use [copy_to](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#copy-to)\nfeature, you need to define it on each sub-field you want to copy to another field:\n\n```javascript\nPUT /test/person/_mapping\n{\n  \"person\": {\n    \"properties\": {\n      \"file\": {\n        \"type\": \"attachment\",\n        \"fields\": {\n          \"content\": {\n            \"type\": \"string\",\n            \"copy_to\": \"copy\"\n          }\n        }\n      },\n      \"copy\": {\n        \"type\": \"string\"\n      }\n    }\n  }\n}\n```\n\nIn this example, the extracted content will be copy as well to `copy` field.\n\nQuerying or accessing metadata\n------------------------------\n\nIf you need to query on metadata fields, use the attachment field name dot the metadata field. For example:\n\n```\nDELETE /test\nPUT /test\nPUT /test/person/_mapping\n{\n  \"person\": {\n    \"properties\": {\n      \"file\": {\n        \"type\": \"attachment\",\n        \"fields\": {\n          \"content_type\": {\n            \"type\": \"string\",\n            \"store\": true\n          }\n        }\n      }\n    }\n  }\n}\nPUT /test/person/1?refresh=true\n{\n  \"file\": \"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==\"\n}\nGET /test/person/_search\n{\n  \"fields\": [ \"file.content_type\" ],\n  \"query\": {\n    \"match\": {\n      \"file.content_type\": \"text plain\"\n    }\n  }\n}\n```\n\nWill give you:\n\n```\n{\n   \"took\": 2,\n   \"timed_out\": false,\n   \"_shards\": {\n      \"total\": 5,\n      \"successful\": 5,\n      \"failed\": 0\n   },\n   \"hits\": {\n      \"total\": 1,\n      \"max_score\": 0.16273327,\n      \"hits\": [\n         {\n            \"_index\": \"test\",\n            \"_type\": \"person\",\n            \"_id\": \"1\",\n            \"_score\": 0.16273327,\n            \"fields\": {\n               \"file.content_type\": [\n                  \"text/plain; charset=ISO-8859-1\"\n               ]\n            }\n         }\n      ]\n   }\n}\n```\n\nIndexed Characters\n------------------\n\nBy default, `100000` characters are extracted when indexing the content. This default value can be changed by setting\nthe `index.mapping.attachment.indexed_chars` setting. It can also be provided on a per document indexed using the\n`_indexed_chars` parameter. `-1` can be set to extract all text, but note that all the text needs to be allowed to be\nrepresented in memory:\n\n```\nPUT /test/person/1\n{\n    \"my_attachment\" : {\n        \"_indexed_chars\" : -1,\n        \"_content\" : \"... base64 encoded attachment ...\"\n    }\n}\n```\n\nMetadata parsing error handling\n-------------------------------\n\nWhile extracting metadata content, errors could happen for example when parsing dates.\nParsing errors are ignored so your document is indexed.\n\nYou can disable this feature by setting the `index.mapping.attachment.ignore_errors` setting to `false`.\n\nLanguage Detection\n------------------\n\nBy default, language detection is disabled (`false`) as it could come with a cost.\nThis default value can be changed by setting the `index.mapping.attachment.detect_language` setting.\nIt can also be provided on a per document indexed using the `_detect_language` parameter.\n\nNote that you can force language using `_language` field when sending your actual document:\n\n```javascript\n{\n    \"my_attachment\" : {\n        \"_language\" : \"en\",\n        \"_content\" : \"... base64 encoded attachment ...\"\n    }\n}\n```\n\nHighlighting attachments\n------------------------\n\nIf you want to highlight your attachment content, you will need to set `\"store\": true` and `\"term_vector\":\"with_positions_offsets\"`\nfor your attachment field. Here is a full script which does it:\n\n```\nDELETE /test\nPUT /test\nPUT /test/person/_mapping\n{\n  \"person\": {\n    \"properties\": {\n      \"file\": {\n        \"type\": \"attachment\",\n        \"fields\": {\n          \"content\": {\n            \"type\": \"string\",\n            \"term_vector\":\"with_positions_offsets\",\n            \"store\": true\n          }\n        }\n      }\n    }\n  }\n}\nPUT /test/person/1?refresh=true\n{\n  \"file\": \"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==\"\n}\nGET /test/person/_search\n{\n  \"fields\": [],\n  \"query\": {\n    \"match\": {\n      \"file.content\": \"king queen\"\n    }\n  },\n  \"highlight\": {\n    \"fields\": {\n      \"file.content\": {\n      }\n    }\n  }\n}\n```\n\nIt gives back:\n\n```js\n{\n   \"took\": 9,\n   \"timed_out\": false,\n   \"_shards\": {\n      \"total\": 1,\n      \"successful\": 1,\n      \"failed\": 0\n   },\n   \"hits\": {\n      \"total\": 1,\n      \"max_score\": 0.13561106,\n      \"hits\": [\n         {\n            \"_index\": \"test\",\n            \"_type\": \"person\",\n            \"_id\": \"1\",\n            \"_score\": 0.13561106,\n            \"highlight\": {\n               \"file.content\": [\n                  \"\\\"God Save the \u003cem\u003eQueen\u003c/em\u003e\\\" (alternatively \\\"God Save the \u003cem\u003eKing\u003c/em\u003e\\\"\\n\"\n               ]\n            }\n         }\n      ]\n   }\n}\n```\n\nStand alone runner\n------------------\n\nIf you want to run some tests within your IDE, you can use `StandaloneRunner` class.\nIt accepts arguments:\n\n*  `-u file://URL/TO/YOUR/DOC`\n*  `--size` set extracted size (default to mapper attachment size)\n*  `BASE64` encoded binary\n\nExample:\n\n```sh\nStandaloneRunner BASE64Text\nStandaloneRunner -u /tmp/mydoc.pdf\nStandaloneRunner -u /tmp/mydoc.pdf --size 1000000\n```\n\nIt produces something like:\n\n```\n## Extracted text\n--------------------- BEGIN -----------------------\nThis is the extracted text\n---------------------- END ------------------------\n## Metadata\n- author: null\n- content_length: null\n- content_type: application/pdf\n- date: null\n- keywords: null\n- language: null\n- name: null\n- title: null\n```\n\nLicense\n-------\n\n    This software is licensed under the Apache 2 license, quoted below.\n\n    Copyright 2009-2014 Elasticsearch \u003chttp://www.elasticsearch.org\u003e\n\n    Licensed under the Apache License, Version 2.0 (the \"License\"); you may not\n    use this file except in compliance with the License. You may obtain a copy of\n    the License at\n\n        http://www.apache.org/licenses/LICENSE-2.0\n\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT\n    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the\n    License for the specific language governing permissions and limitations under\n    the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Felasticsearch-mapper-attachments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastic%2Felasticsearch-mapper-attachments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Felasticsearch-mapper-attachments/lists"}