https://github.com/ev2900/opensearch_deleteddocuments
Examples explaining how deletes work in OpenSearch
https://github.com/ev2900/opensearch_deleteddocuments
aws opensearch opensearch-examples performance-tuning
Last synced: about 1 month ago
JSON representation
Examples explaining how deletes work in OpenSearch
- Host: GitHub
- URL: https://github.com/ev2900/opensearch_deleteddocuments
- Owner: ev2900
- Created: 2022-11-29T14:44:48.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2026-04-02T14:25:18.000Z (2 months ago)
- Last Synced: 2026-04-03T03:03:42.168Z (2 months ago)
- Topics: aws, opensearch, opensearch-examples, performance-tuning
- Language: Python
- Homepage:
- Size: 493 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# OpenSearch DeletedDocuments
When a document is deleted in OpenSearch it is marked for deletion. It is not physically removed from storage until a merge removes the document(s) from the segments underlying the index shards.
This only applies when documents are being deleted from an index. If an entire index is deleted the delete is immediate. Deleting an index doesn't create any delete markers.
To determine the number of documents marked for deletion in indices you can either
* Run the ```GET _cat/indices?v``` API looking at the *docs.deleted* field
OR
* Look at the **DeletedDocuments** metric in CloudWatch
If you have indices with document marked for deletion you can expunge the deleted documents using the force merge API with the *only_expunge_deletes* parameter.
Run the ```POST //_forcemerge?only_expunge_deletes=true``` API to expunge the deleted documents.
*Note* the force merge operation triggers an **I/O intensive process** and **blocks all new requests** to your cluster until the merge is complete. Only call the force merge operation against read-only indices, **when no additional data is being written to the index**.
## Example
Go to the [Example](https://github.com/ev2900/OpenSearch_DeletedDocuments/tree/main/Example) section of this repository for instructions on how to run an example that will demonstrate these concepts