{"id":19651316,"url":"https://github.com/embulk/embulk-base-restclient","last_synced_at":"2025-07-23T17:36:02.257Z","repository":{"id":51099375,"uuid":"77013049","full_name":"embulk/embulk-base-restclient","owner":"embulk","description":"Base class library for Embulk plugins to access RESTful services","archived":false,"fork":false,"pushed_at":"2021-05-24T04:21:11.000Z","size":642,"stargazers_count":6,"open_issues_count":12,"forks_count":7,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-28T16:49:32.619Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.embulk.org/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/embulk.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-21T03:37:32.000Z","updated_at":"2021-05-24T03:36:55.000Z","dependencies_parsed_at":"2022-09-04T04:01:40.594Z","dependency_job_id":null,"html_url":"https://github.com/embulk/embulk-base-restclient","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/embulk/embulk-base-restclient","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-base-restclient","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-base-restclient/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-base-restclient/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-base-restclient/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/embulk","download_url":"https://codeload.github.com/embulk/embulk-base-restclient/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-base-restclient/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266720815,"owners_count":23974036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:06:01.789Z","updated_at":"2025-07-23T17:36:02.234Z","avatar_url":"https://github.com/embulk.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"embulk-base-restclient\n=======================\n\nBase class library to access RESTful services. See [an example](embulk-input-example/).\n\nVersions and compatibility\n---------------------------\n\nPlease remember to specify a version of this library when you use it. This is just a helper library to build a plugin. The interfaces may change per library version for improvements, optimization, and catch-up.\n\nUsage: input plugins\n---------------------\n\n### Overview\n\nThe easiest case has two classes: `-InputPlugin` and `-InputPluginDelegate` as follows.\n\n```\npublic class -InputPlugin\n        extends RestClientInputPluginBase\u003c-InputPluginDelegate.PluginTask\u003e\n{\n    public -InputPlugin()\n    {\n        super(-InputPluginDelegate.PluginTask.class, new -InputPluginDelegate());\n    }\n}\n```\n\n```\npublic class -InputPluginDelegate\n        implements RestClientInputPluginDelegate\u003c-InputPluginDelegate.PluginTask\u003e\n{\n    public interface PluginTask\n            extends RestClientInputTaskBase\n    {\n    }\n\n    .........\n}\n```\n\n`-InputPlugin` is just to delegate to `-InputPluginDelegate`. `-InputPluginDelegate` is to implement actual behaviors. `-InputPluginDelegate` is a set of some `abstract` methods which are called implicitly from `-InputPlugin`.\n\n\n### Key method 1: `buildServiceResponseMapper`\n\nIt defines 1) which values to pick up from the response, and 2) which Embulk columns to import the picked-up values. Return an instance of `org.embulk.base.restclient.jackson.JacksonServiceResponseMapper` if the target RESTful service responds with JSON. Implement another type of `ServiceResponseMapper` if the response is not in JSON (e.g. XML).\n\n`JacksonServiceResponseMapper` is usually built as follows:\n\n```\nreturn JacksonServiceResponseMapper.builder()\n    .add(...)\n    .add(...)\n    ...\n    .build();\n```\n\nOne `.add(...)` represents one value imported into one Embulk column. There are a few overloaded `add` methods.\n\n* To pick up a top-level JSON value into an Embulk column with the same name, use the simplest `add(String, org.embulk.spi.type.Type)`.\n* To pick up into an `Timestamp` Embulk column, use `add` with `embulkColumnTimestampFormat`.\n* To pick up a non-top-level JSON value, or a value into an Embulk column with a different name, use `add` with `JacksonValueLocator`. `JacksonValueLocator` is to specify a location in JSON. For example, `JacksonJsonPointerValueLocator` points a location in JSON with [JSON Pointer](https://tools.ietf.org/html/rfc6901).\n\nFrom the following JSON for example:\n\n```\n{\n    \"fullname\": \"example_name\",\n    \"timestamps\": {\n        \"start\": 1487056476,\n        \"end\": 1487056536\n    }\n}\n```\n\n* `add(\"fullname\", Types.STRING)` picks up a `String` value `\"example_name\"` into an Embulk column `\"fullname\"` .\n* `add(new JacksonJsonPointerValueLocator(\"/timestamps/start\"), \"starting_time\", Types.TIMESTAMP, \"%s\")` picks up a `Timestamp` value `2017-02-14T07:14:36Z` (`1487056476`) into an Embulk column `\"starting_time\"`.\n\n### Key method 2: `ingestServiceData`\n\nIt actually retrieves data from the target RESTful service, and imports the data as Embulk records. Given `RecordImporter` imports one record into Embulk based on the `ServiceResponseMapper` generated above.\n\nGiven `RetryHelper` encapsulates complicated retrying implementation. Its usage is in three steps as follows:\n\n```\nString responseBody = retryHelper.requestWithRetry(\n    new org.embulk.base.restclient.request.StringResponseEntityReader(),\n    new org.embulk.base.restclient.request.SingleRequester() {\n        @Override\n        public javax.ws.rs.core.Response requestOnce(javax.ws.rs.client.Client client)\n        {\n            // Returns a JAX-RS Response retrieve with the given JAX-RS Client.\n            return client...get();\n        }\n\n        @Override\n        public boolean isResponseStatusToRetry(javax.ws.rs.core.Response response)\n        {\n            return response.getStatus() / 100 != 4;\n        }\n    });\n```\n\n1. Request to the target RESTful service by implementing `requestOnce`. It returns JAX-RS Response `javax.ws.rs.core.Response`.\n2. Decice whether the request should be retried or not by implementing `isResponseStatusToRetry`. It decies with the `Response` (usually based on the HTTP response status).\n3. Achieve the response body entity from the JAX-RS Response by specifying a `ResponseReadable` instance to `requestWithRetry`. A couple of `ResponseReadable` classes are pre-defined:\n  * `StringResponseEntityReader` to read the entity as `String`.\n  * `InputStreamResponseEntityReader` to read the entity as `InputStream`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-base-restclient","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembulk%2Fembulk-base-restclient","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-base-restclient/lists"}