{"id":37100908,"url":"https://github.com/onsdigital/dp-elasticsearch","last_synced_at":"2026-01-14T12:16:44.758Z","repository":{"id":37897779,"uuid":"232834110","full_name":"ONSdigital/dp-elasticsearch","owner":"ONSdigital","description":"elasticsearch client","archived":false,"fork":false,"pushed_at":"2025-12-23T10:00:42.000Z","size":317,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-12-24T22:55:44.859Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ONSdigital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-01-09T14:59:30.000Z","updated_at":"2025-12-23T08:52:00.000Z","dependencies_parsed_at":"2024-06-19T06:18:25.330Z","dependency_job_id":"51bb17fc-d34b-449b-b0b5-33704107d639","html_url":"https://github.com/ONSdigital/dp-elasticsearch","commit_stats":null,"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"purl":"pkg:github/ONSdigital/dp-elasticsearch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ONSdigital%2Fdp-elasticsearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ONSdigital%2Fdp-elasticsearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ONSdigital%2Fdp-elasticsearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ONSdigital%2Fdp-elasticsearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ONSdigital","download_url":"https://codeload.github.com/ONSdigital/dp-elasticsearch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ONSdigital%2Fdp-elasticsearch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28419864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T12:16:44.092Z","updated_at":"2026-01-14T12:16:44.744Z","avatar_url":"https://github.com/ONSdigital.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dp-elasticsearch\n\nElasticsearch library to create an elasticsearch client to be able to make requests to elasticsearch. Currently the library supports elasticsearch 7.10. The version 7.10 uses go-elasticsearch library behind the scenes and this library can be viewed as a wrapper around go-elasticsearch library.  Please follow readme on how to create a client and how to consume this.\n\n## elasticsearch package\n\nIncludes implementation of a health checker, that reuses the elasticsearch client to check requests can be made against elasticsearch cluster and known indexes.\n\n### setup elasticsearch client\n\n#### setup ES 7.10 client\n\nSetting up ES7.10 client is similar as setting up es2.2, just that we need to specify client library as ```GoElasticV710```, as follows:\n\n```golang\nimport (\n    dpEs \"github.com/ONSdigital/dp-elasticsearch/v4\"\n)\n\n...\n    esClient, esClientErr := dpEs.NewClient(dpEsClient.Config{\n        ClientLib: dpEsClient.GoElasticV710,\n        Address:   cfg.esURL,\n    })\n    if esClientErr != nil {\n        log.Fatal(ctx, \"Failed to create dp-elasticsearch client\", esClientErr)\n    }\n...\n```\n\n##### Embedding custom http roundtripper with es7.10\n\nYou could create custom roundtripper (say if you have to sign requests if you are using es7.10), as follows:\n\n```golang\nimport (\n    dpEs \"github.com/ONSdigital/dp-elasticsearch/v4\"\n    \"github.com/ONSdigital/dp-net/v3/awsauth\"\n)\n\n...\nif cfg.signRequests {\n        fmt.Println(\"Use a signing roundtripper client\")\n        awsSignerRT, err := awsauth.NewAWSSignerRoundTripper(awsauth.NewAWSSignerRoundTripper(\u003caws_filename_placeholder\u003e, \u003caws_profile_placeholder\u003e, \u003caws_region_placeholder\u003e, \"es\",\n            awsauth.Options{TlsInsecureSkipVerify: cfg.aws.tlsInsecureSkipVerify})\n        if err != nil {\n            log.Fatal(ctx, \"Failed to create http signer\", err)\n        }\n\n        esHTTPClient = dphttp.NewClientWithTransport(awsSignerRT)\n    }\n    \n    esClient, esClientErr := dpEs.NewClient(dpEsClient.Config{\n        ClientLib: dpEsClient.GoElasticV710,\n        Address:   cfg.esURL,\n        Transport: esHTTPClient,\n    })\n    if esClientErr != nil {\n        log.Fatal(ctx, \"Failed to create dp-elasticsearch client\", esClientErr)\n    }\n...\n```\n\nMore details on [aws signer roundtripper](https://github.com/ONSdigital/dp-net/tree/main/awsauth#round-tripper)\n\n##### setting up bulk indexer with es7.10\n\n```golang\nimport (\n    dpEs \"github.com/ONSdigital/dp-elasticsearch/v4\"\n)\n\n...\n    esClient, esClientErr := dpEs.NewClient(dpEsClient.Config{\n        ClientLib: dpEsClient.GoElasticV710,\n        Address:   cfg.esURL,\n    })\n    if esClientErr != nil {\n        log.Fatal(ctx, \"Failed to create dp-elasticsearch client\", esClientErr)\n    }\n    \n    if err := esClient.NewBulkIndexer(ctx); err != nil {\n        log.Fatal(ctx, \"Failed to create new bulk indexer\")\n    }\n    \n    // Adding docs to bulk indexer\n    err := esClient.BulkIndexAdd(ctx, v710.Create, indexName, documentID, documentBody)\n        if err != nil {\n            log.Fatal(ctx, \"Failed to add documents to bulk indexer\")\n        }\n        \n    // Close bulk indexer\n    err := esClient.BulkIndexClose(ctx)\n    if err != nil {\n        log.Fatal(ctx, \"Failed to close bulk indexer\")\n    }\n...\n```\n\n#### health checker\n\nUsing elasticsearch checker function currently performs a GET request against elasticsearch 'cluster health' API (`/_cluster/health\"`)\n\nThe healthcheck will only succeed if the request can be performend and the cluster is in `green` state.\nIf the cluster is in `yellow` state, a Checker in WARNING status will be returned. In any other case, a CRITICAL Checker will be returned.\n\nRead the [Health Check Specification](https://github.com/ONSdigital/dp/blob/master/standards/HEALTH_CHECK_SPECIFICATION.md) for details.\n\nMore information about elasticsearch [cluster health API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html)\n\nInstantiate an elasticsearch client\n\n```golang\nimport \"github.com/ONSdigital/dp-elasticsearch/elasticsearch/v3\"\n\n...\n    cli := elasticsearch.NewClient(\u003curl\u003e, \u003cmaxRetries\u003e, \u003coptional array of indexes\u003e)\n...\n```\n\nCall elasticsearch health checker with `cli.Checker(context.Background())` and this will return a check object like so:\n\n```json\n{\n    \"name\": \"string\",\n    \"status\": \"string\",\n    \"message\": \"string\",\n    \"status_code\": \"int\",\n    \"last_checked\": \"ISO8601 - UTC date time\",\n    \"last_success\": \"ISO8601 - UTC date time\",\n    \"last_failure\": \"ISO8601 - UTC date time\"\n}\n```\n\n## Contributing\n\nSee [CONTRIBUTING](CONTRIBUTING.md) for details.\n\n### License\n\nCopyright © 2019-2025, Office for National Statistics \u003chttps://www.ons.gov.uk\u003e\n\nReleased under MIT license, see [LICENSE](LICENSE.md) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonsdigital%2Fdp-elasticsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fonsdigital%2Fdp-elasticsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonsdigital%2Fdp-elasticsearch/lists"}