{"id":29193173,"url":"https://github.com/ironcorelabs/try-cloaked-search","last_synced_at":"2025-07-02T02:07:48.637Z","repository":{"id":37940333,"uuid":"383620788","full_name":"IronCoreLabs/try-cloaked-search","owner":"IronCoreLabs","description":"Quick start for IronCore's encrypted Elasticsearch proxy","archived":false,"fork":false,"pushed_at":"2024-07-29T19:36:55.000Z","size":528,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-07-30T22:15:11.315Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IronCoreLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-06T23:24:35.000Z","updated_at":"2024-07-29T19:36:57.000Z","dependencies_parsed_at":"2024-07-26T23:44:59.897Z","dependency_job_id":null,"html_url":"https://github.com/IronCoreLabs/try-cloaked-search","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/IronCoreLabs/try-cloaked-search","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCoreLabs%2Ftry-cloaked-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCoreLabs%2Ftry-cloaked-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCoreLabs%2Ftry-cloaked-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCoreLabs%2Ftry-cloaked-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IronCoreLabs","download_url":"https://codeload.github.com/IronCoreLabs/try-cloaked-search/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IronCoreLabs%2Ftry-cloaked-search/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263061404,"owners_count":23407606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-02T02:07:48.167Z","updated_at":"2025-07-02T02:07:48.627Z","avatar_url":"https://github.com/IronCoreLabs.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Try Cloaked Search (in ~5 Minutes)\n\nCloaked Search is a proxy for Elasticsearch/OpenSearch that protects the indexed data from prying eyes. Cloaked Search's API is the same as the underlying search service API.\n\nIn about 5 minutes, you will have:\n\n- Elasticsearch/OpenSearch running on your local machine\n- Cloaked Search running on your local machine\n- sample data indexed with `summary` and `body` as protected fields\n- query results from sample queries using the protected `summary` and `body` fields\n\nAll stored data is on a temporary volume inside of docker. No changes will be made to your machine outside the directory where you cloned the `try-cloaked-search` repo.\n\n## Dependencies\n\nTo try Cloaked Search you just need a basic \\*nix installation and `docker` + `docker-compose`. Some of the commands below also use `jq` for JSON formatting. If you don't have `jq`, you can safely remove those portions of the command.\n\n## Get Cloaked Search Running\n\nClone the [try-cloaked-search](https://github.com/IronCoreLabs/try-cloaked-search) git repo.\n\n```bash\ngit clone https://github.com/IronCoreLabs/try-cloaked-search.git\n```\n\nAll other commands are assumed be run from within directory where you cloned the repo.\n\n### Start Cloaked Search and a search service\n\n_Note: This example install uses ports 9200 (Elasticsearch/OpenSearch) and 8675 (Cloaked Search). Be sure you don't have an existing search service running on port 9200 before beginning._\n\n```bash\ndocker-compose -f elasticsearch/docker-compose.yml up # for Elasticsearch\n```\n\nor\n\n```bash\ndocker-compose -f open-search/docker-compose.yml up # for Open Search\n```\n\n**Note: Future commands will be targeting Cloaked Search on port 8675**\n\n## Indexing\n\n`try-cloaked-search` includes some test data. Since Cloaked Search uses a different key per (tenant, index, field),\ndocuments with protected fields must be tagged with the tenant to which they belong.\nHalf of the articles in the test dataset are associated with `tenant-1`, and the other half are associated with `tenant-2`.\nTo better understand Cloaked Search's key management, refer to the [configuration documentation](https://ironcorelabs.com/docs/cloaked-search/configuration/).\n\n```bash\n./populate_index.sh\n```\n\n### (Optional) Look at an encrypted index\n\n_Note that all queries made with `./query-search-service.sh` are being made directly to your search service. We are using a script to detect if you're running OpenSearch or Elasticsearch, but there is no involvement from Cloaked Search. Since these requests go directly to the search service (port 9200), we can see what's actually stored._\n\nLet's get all the documents belonging to `tenant-1` and see what's in the index!\n\n```bash\n./query-search-service.sh '+tenant_id.keyword:\"tenant-1\"' | jq\n```\n\nWe are protecting the `body` and `summary` fields from the original document. These fields are no longer attached to the document;\ninstead, the blind index tokens for `body` are stored in the `_icl_p_body`. Because we enabled `index_prefixes` for the `summary` field,\nit has been translated to `_icl_p_summary._value` and `_icl_p_summary._index_prefix`.\nYou will also notice an `_icl_encrypted_source` field; this contains an encrypted version of the fields that have been protected, which allows Cloaked Search\nto return them to their original versions.\n\n```json\n\"_icl_p_summary\": \"7b76c95a 616544a2 b41fa81e 85933317 e30236d5 ...\",\n```\n\n## Querying Protected Fields\n\n### Sample Queries\n\nThese are a couple examples of simple term queries. They are still querying on the `title` field, which is unprotected.\n\n```bash\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND title:Japan' | jq\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND title:cup' | jq\n```\n\nCompare these results to the ones returned by querying the search service directly. The same documents are returned, but the contents are very different.\nYou can see how Cloaked Search transparently handles the decryption of the document to allow you to see the data in the fields that were protected, `summary` and `body`.\n\n```bash\n./query-search-service.sh '+tenant_id.keyword:\"tenant-1\" AND title:Japan' | jq\n./query-search-service.sh '+tenant_id.keyword:\"tenant-1\" AND title:cup' | jq\n```\n\nNow try querying on a protected field:\n\n```bash\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND summary:glasgow' | jq\n```\n\nQueries can also be combined with ORs or ANDs, and you can mix protected and unprotected fields. For example,\n\n```bash\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND (title:cup OR title:Japan)' | jq\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND (summary:cup OR title:Japan)' | jq\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND (summary:cup OR body:Japan)' | jq\n```\n\nPhrases can also be searched using quoted queries like this:\n\n```bash\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND summary:\"Cheerleading in Japan\"' | jq\n```\n\nFinally, here is an example of a prefix query on the `summary` field (the field with `index_prefixes` enabled):\n\n```bash\n./query-cloaked-search.sh '+tenant_id.keyword:\"tenant-1\" AND summary:pro*' | jq\n```\n\nYou can replace the query with anything you like. Make sure you have the `+tenant_id.keyword` in the query. `populate_index.sh` loaded 1000 documents, half are tagged with `tenant-1` and the others are tagged with `tenant-2`.\n\n### JSON queries\n\nCloaked Search also supports requests other than Query String queries. For example,\n\n```bash\n./query-cloaked-search-json.sh 'japan' 'tenant-1' | jq\n```\n\nThis searches for `Japan` in the protected `summary` field over all the documents belonging to `tenant-1`.\n\n## Next Steps\n\nYou will want to try out Cloaked Search on some of your own data in a more real environment.\n\nUse the [Kubernetes template](kubernetes) in this repository to make a simple Kubernetes deployment.\n\nSee the [configuration docs](https://ironcorelabs.com/docs/saas-shield/cloaked-search/configuration/) for info on how to configure and deploy Cloaked Search.\n\nIf you are interested in the underlying technology or in the security of the underlying search service index, see the [What Is Encrypted Search](https://ironcorelabs.com/docs/saas-shield/cloaked-search/what-is-encrypted-search/) page.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fironcorelabs%2Ftry-cloaked-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fironcorelabs%2Ftry-cloaked-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fironcorelabs%2Ftry-cloaked-search/lists"}