{"id":30283214,"url":"https://github.com/scrapegraphai/scrapegraphai-java","last_synced_at":"2025-08-16T17:33:52.238Z","repository":{"id":309648241,"uuid":"1036617912","full_name":"ScrapeGraphAI/scrapegraphai-java","owner":"ScrapeGraphAI","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-13T02:11:15.000Z","size":265,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-13T04:11:24.334Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ScrapeGraphAI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-12T10:39:57.000Z","updated_at":"2025-08-12T12:25:18.000Z","dependencies_parsed_at":"2025-08-13T04:11:31.897Z","dependency_job_id":"bd6c05c3-0205-481e-88ab-86b224b08825","html_url":"https://github.com/ScrapeGraphAI/scrapegraphai-java","commit_stats":null,"previous_names":["scrapegraphai/scrapegraphai-java"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ScrapeGraphAI/scrapegraphai-java","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapeGraphAI%2Fscrapegraphai-java","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapeGraphAI%2Fscrapegraphai-java/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapeGraphAI%2Fscrapegraphai-java/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapeGraphAI%2Fscrapegraphai-java/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ScrapeGraphAI","download_url":"https://codeload.github.com/ScrapeGraphAI/scrapegraphai-java/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapeGraphAI%2Fscrapegraphai-java/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270746900,"owners_count":24638445,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-16T17:33:51.533Z","updated_at":"2025-08-16T17:33:52.195Z","avatar_url":"https://github.com/ScrapeGraphAI.png","language":"Kotlin","readme":"# Scrapegraphai Java API Library\n\n\u003c!-- x-release-please-start-version --\u003e\n\n[![Maven Central](https://img.shields.io/maven-central/v/com.scrapegraphai.api/scrapegraphai-java)](https://central.sonatype.com/artifact/com.scrapegraphai.api/scrapegraphai-java/0.0.2)\n[![javadoc](https://javadoc.io/badge2/com.scrapegraphai.api/scrapegraphai-java/0.0.2/javadoc.svg)](https://javadoc.io/doc/com.scrapegraphai.api/scrapegraphai-java/0.0.2)\n\n\u003c!-- x-release-please-end --\u003e\n\nThe Scrapegraphai Java SDK provides convenient access to the [Scrapegraphai REST API](https://scrapegraphai.com) from applications written in Java.\n\nIt is generated with [Stainless](https://www.stainless.com/).\n\n\u003c!-- x-release-please-start-version --\u003e\n\nThe REST API documentation can be found on [scrapegraphai.com](https://scrapegraphai.com). Javadocs are available on [javadoc.io](https://javadoc.io/doc/com.scrapegraphai.api/scrapegraphai-java/0.0.2).\n\n\u003c!-- x-release-please-end --\u003e\n\n## Installation\n\n\u003c!-- x-release-please-start-version --\u003e\n\n### Gradle\n\n```kotlin\nimplementation(\"com.scrapegraphai.api:scrapegraphai-java:0.0.2\")\n```\n\n### Maven\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.scrapegraphai.api\u003c/groupId\u003e\n  \u003cartifactId\u003escrapegraphai-java\u003c/artifactId\u003e\n  \u003cversion\u003e0.0.2\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n\u003c!-- x-release-please-end --\u003e\n\n## Requirements\n\nThis library requires Java 8 or later.\n\n## Usage\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\n// Configures using the `scrapegraphai.apiKey` and `scrapegraphai.baseUrl` system properties\n// Or configures using the `SCRAPEGRAPHAI_API_KEY` and `SCRAPEGRAPHAI_BASE_URL` environment variables\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.fromEnv();\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(\"Extract the product name, price, and description\")\n    .build();\nCompletedSmartscraper completedSmartscraper = client.smartscraper().create(params);\n```\n\n## Client configuration\n\nConfigure the client using system properties or environment variables:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\n// Configures using the `scrapegraphai.apiKey` and `scrapegraphai.baseUrl` system properties\n// Or configures using the `SCRAPEGRAPHAI_API_KEY` and `SCRAPEGRAPHAI_BASE_URL` environment variables\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.fromEnv();\n```\n\nOr manually:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .apiKey(\"My API Key\")\n    .build();\n```\n\nOr using a combination of the two approaches:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    // Configures using the `scrapegraphai.apiKey` and `scrapegraphai.baseUrl` system properties\n    // Or configures using the `SCRAPEGRAPHAI_API_KEY` and `SCRAPEGRAPHAI_BASE_URL` environment variables\n    .fromEnv()\n    .apiKey(\"My API Key\")\n    .build();\n```\n\nSee this table for the available options:\n\n| Setter    | System property         | Environment variable     | Required | Default value                        |\n| --------- | ----------------------- | ------------------------ | -------- | ------------------------------------ |\n| `apiKey`  | `scrapegraphai.apiKey`  | `SCRAPEGRAPHAI_API_KEY`  | true     | -                                    |\n| `baseUrl` | `scrapegraphai.baseUrl` | `SCRAPEGRAPHAI_BASE_URL` | true     | `\"https://api.scrapegraphai.com/v1\"` |\n\nSystem properties take precedence over environment variables.\n\n\u003e [!TIP]\n\u003e Don't create more than one client in the same application. Each client has a connection pool and\n\u003e thread pools, which are more efficient to share between requests.\n\n### Modifying configuration\n\nTo temporarily use a modified client configuration, while reusing the same connection and thread pools, call `withOptions()` on any client or service:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\n\nScrapegraphaiClient clientWithOptions = client.withOptions(optionsBuilder -\u003e {\n    optionsBuilder.baseUrl(\"https://example.com\");\n    optionsBuilder.maxRetries(42);\n});\n```\n\nThe `withOptions()` method does not affect the original client or service.\n\n## Requests and responses\n\nTo send a request to the Scrapegraphai API, build an instance of some `Params` class and pass it to the corresponding client method. When the response is received, it will be deserialized into an instance of a Java class.\n\nFor example, `client.smartscraper().create(...)` should be called with an instance of `SmartscraperCreateParams`, and it will return an instance of `CompletedSmartscraper`.\n\n## Immutability\n\nEach class in the SDK has an associated [builder](https://blogs.oracle.com/javamagazine/post/exploring-joshua-blochs-builder-design-pattern-in-java) or factory method for constructing it.\n\nEach class is [immutable](https://docs.oracle.com/javase/tutorial/essential/concurrency/immutable.html) once constructed. If the class has an associated builder, then it has a `toBuilder()` method, which can be used to convert it back to a builder for making a modified copy.\n\nBecause each class is immutable, builder modification will _never_ affect already built class instances.\n\n## Asynchronous execution\n\nThe default client is synchronous. To switch to asynchronous execution, call the `async()` method:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\nimport java.util.concurrent.CompletableFuture;\n\n// Configures using the `scrapegraphai.apiKey` and `scrapegraphai.baseUrl` system properties\n// Or configures using the `SCRAPEGRAPHAI_API_KEY` and `SCRAPEGRAPHAI_BASE_URL` environment variables\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.fromEnv();\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(\"Extract the product name, price, and description\")\n    .build();\nCompletableFuture\u003cCompletedSmartscraper\u003e completedSmartscraper = client.async().smartscraper().create(params);\n```\n\nOr create an asynchronous client from the beginning:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClientAsync;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClientAsync;\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\nimport java.util.concurrent.CompletableFuture;\n\n// Configures using the `scrapegraphai.apiKey` and `scrapegraphai.baseUrl` system properties\n// Or configures using the `SCRAPEGRAPHAI_API_KEY` and `SCRAPEGRAPHAI_BASE_URL` environment variables\nScrapegraphaiClientAsync client = ScrapegraphaiOkHttpClientAsync.fromEnv();\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(\"Extract the product name, price, and description\")\n    .build();\nCompletableFuture\u003cCompletedSmartscraper\u003e completedSmartscraper = client.smartscraper().create(params);\n```\n\nThe asynchronous client supports the same options as the synchronous one, except most methods return `CompletableFuture`s.\n\n## Examples\n\nThe `scrapegraphai-java-example` module contains comprehensive examples demonstrating all SDK features:\n\n### Running Examples\n\nRun the basic example:\n```bash\n./gradlew :scrapegraphai-java-example:run\n```\n\nRun specific examples:\n```bash\n# SmartScraper examples\n./gradlew :scrapegraphai-java-example:run -Pexample=Smartscraper\n\n# SearchScraper examples  \n./gradlew :scrapegraphai-java-example:run -Pexample=Searchscraper\n\n# Crawl examples\n./gradlew :scrapegraphai-java-example:run -Pexample=Crawl\n\n# Markdownify examples\n./gradlew :scrapegraphai-java-example:run -Pexample=Markdownify\n\n# Async examples\n./gradlew :scrapegraphai-java-example:run -Pexample=Async\n\n# Schema Generation examples\n./gradlew :scrapegraphai-java-example:run -Pexample=SchemaGeneration\n\n# Utility examples (validation, health, credits)\n./gradlew :scrapegraphai-java-example:run -Pexample=Utility\n```\n\n### Example Categories\n\n| Example | Description | Features Demonstrated |\n|---------|-------------|----------------------|\n| **Main** | Basic SmartScraper usage | Simple scraping, error handling |\n| **SmartscraperExample** | Comprehensive scraping scenarios | Custom schemas, pagination, JavaScript rendering, headers/cookies |\n| **SearchscraperExample** | Web search + scraping | Multi-source data aggregation, product comparison, news aggregation |\n| **CrawlExample** | Website crawling | Site exploration, path filtering, progress monitoring |\n| **MarkdownifyExample** | HTML to Markdown conversion | Content formatting, documentation generation |\n| **AsyncExample** | Asynchronous operations | Non-blocking requests, parallel processing, error handling |\n| **SchemaGenerationExample** | Automatic schema creation | Schema analysis, structured data extraction |\n| **UtilityExample** | Service utilities | API validation, health checks, credit monitoring, feedback |\n\n### Quick Start Example\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\n// Initialize client (reads from SCRAPEGRAPHAI_API_KEY environment variable)\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.fromEnv();\n\n// Create scraping request\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(\"Extract the main heading and description\")\n    .websiteUrl(\"https://example.com\")\n    .build();\n\n// Execute scraping\nCompletedSmartscraper result = client.smartscraper().create(params);\nSystem.out.println(\"Extracted data: \" + result.result());\n```\n\n## Raw responses\n\nThe SDK defines methods that deserialize responses into instances of Java classes. However, these methods don't provide access to the response headers, status code, or the raw response body.\n\nTo access this data, prefix any HTTP method call on a client or service with `withRawResponse()`:\n\n```java\nimport com.scrapegraphai.api.core.http.Headers;\nimport com.scrapegraphai.api.core.http.HttpResponseFor;\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(\"Extract the product name, price, and description\")\n    .build();\nHttpResponseFor\u003cCompletedSmartscraper\u003e completedSmartscraper = client.smartscraper().withRawResponse().create(params);\n\nint statusCode = completedSmartscraper.statusCode();\nHeaders headers = completedSmartscraper.headers();\n```\n\nYou can still deserialize the response into an instance of a Java class if needed:\n\n```java\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\n\nCompletedSmartscraper parsedCompletedSmartscraper = completedSmartscraper.parse();\n```\n\n## Error handling\n\nThe SDK throws custom unchecked exception types:\n\n- [`ScrapegraphaiServiceException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiServiceException.kt): Base class for HTTP errors. See this table for which exception subclass is thrown for each HTTP status code:\n\n  | Status | Exception                                                                                                                                |\n  | ------ | ---------------------------------------------------------------------------------------------------------------------------------------- |\n  | 400    | [`BadRequestException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/BadRequestException.kt)                     |\n  | 401    | [`UnauthorizedException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/UnauthorizedException.kt)                 |\n  | 403    | [`PermissionDeniedException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/PermissionDeniedException.kt)         |\n  | 404    | [`NotFoundException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/NotFoundException.kt)                         |\n  | 422    | [`UnprocessableEntityException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/UnprocessableEntityException.kt)   |\n  | 429    | [`RateLimitException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/RateLimitException.kt)                       |\n  | 5xx    | [`InternalServerException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/InternalServerException.kt)             |\n  | others | [`UnexpectedStatusCodeException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/UnexpectedStatusCodeException.kt) |\n\n- [`ScrapegraphaiIoException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiIoException.kt): I/O networking errors.\n\n- [`ScrapegraphaiRetryableException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiRetryableException.kt): Generic error indicating a failure that could be retried by the client.\n\n- [`ScrapegraphaiInvalidDataException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiInvalidDataException.kt): Failure to interpret successfully parsed data. For example, when accessing a property that's supposed to be required, but the API unexpectedly omitted it from the response.\n\n- [`ScrapegraphaiException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiException.kt): Base class for all exceptions. Most errors will result in one of the previously mentioned ones, but completely generic errors may be thrown using the base class.\n\n## Logging\n\nThe SDK uses the standard [OkHttp logging interceptor](https://github.com/square/okhttp/tree/master/okhttp-logging-interceptor).\n\nEnable logging by setting the `SCRAPEGRAPHAI_LOG` environment variable to `info`:\n\n```sh\n$ export SCRAPEGRAPHAI_LOG=info\n```\n\nOr to `debug` for more verbose logging:\n\n```sh\n$ export SCRAPEGRAPHAI_LOG=debug\n```\n\n## ProGuard and R8\n\nAlthough the SDK uses reflection, it is still usable with [ProGuard](https://github.com/Guardsquare/proguard) and [R8](https://developer.android.com/topic/performance/app-optimization/enable-app-optimization) because `scrapegraphai-java-core` is published with a [configuration file](scrapegraphai-java-core/src/main/resources/META-INF/proguard/scrapegraphai-java-core.pro) containing [keep rules](https://www.guardsquare.com/manual/configuration/usage).\n\nProGuard and R8 should automatically detect and use the published rules, but you can also manually copy the keep rules if necessary.\n\n## Jackson\n\nThe SDK depends on [Jackson](https://github.com/FasterXML/jackson) for JSON serialization/deserialization. It is compatible with version 2.13.4 or higher, but depends on version 2.18.2 by default.\n\nThe SDK throws an exception if it detects an incompatible Jackson version at runtime (e.g. if the default version was overridden in your Maven or Gradle config).\n\nIf the SDK threw an exception, but you're _certain_ the version is compatible, then disable the version check using the `checkJacksonVersionCompatibility` on [`ScrapegraphaiOkHttpClient`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClient.kt) or [`ScrapegraphaiOkHttpClientAsync`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClientAsync.kt).\n\n\u003e [!CAUTION]\n\u003e We make no guarantee that the SDK works correctly when the Jackson version check is disabled.\n\n## Network options\n\n### Retries\n\nThe SDK automatically retries 2 times by default, with a short exponential backoff between requests.\n\nOnly the following error types are retried:\n\n- Connection errors (for example, due to a network connectivity problem)\n- 408 Request Timeout\n- 409 Conflict\n- 429 Rate Limit\n- 5xx Internal\n\nThe API may also explicitly instruct the SDK to retry or not retry a request.\n\nTo set a custom number of retries, configure the client using the `maxRetries` method:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    .maxRetries(4)\n    .build();\n```\n\n### Timeouts\n\nRequests time out after 1 minute by default.\n\nTo set a custom timeout, configure the method call using the `timeout` method:\n\n```java\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\n\nCompletedSmartscraper completedSmartscraper = client.smartscraper().create(\n  params, RequestOptions.builder().timeout(Duration.ofSeconds(30)).build()\n);\n```\n\nOr configure the default for all method calls at the client level:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\nimport java.time.Duration;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    .timeout(Duration.ofSeconds(30))\n    .build();\n```\n\n### Proxies\n\nTo route requests through a proxy, configure the client using the `proxy` method:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\nimport java.net.InetSocketAddress;\nimport java.net.Proxy;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    .proxy(new Proxy(\n      Proxy.Type.HTTP, new InetSocketAddress(\n        \"https://example.com\", 8080\n      )\n    ))\n    .build();\n```\n\n### HTTPS\n\n\u003e [!NOTE]\n\u003e Most applications should not call these methods, and instead use the system defaults. The defaults include\n\u003e special optimizations that can be lost if the implementations are modified.\n\nTo configure how HTTPS connections are secured, configure the client using the `sslSocketFactory`, `trustManager`, and `hostnameVerifier` methods:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    // If `sslSocketFactory` is set, then `trustManager` must be set, and vice versa.\n    .sslSocketFactory(yourSSLSocketFactory)\n    .trustManager(yourTrustManager)\n    .hostnameVerifier(yourHostnameVerifier)\n    .build();\n```\n\n### Environments\n\nThe SDK sends requests to the production by default. To send requests to a different environment, configure the client like so:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    .environment1()\n    .build();\n```\n\n### Custom HTTP client\n\nThe SDK consists of three artifacts:\n\n- `scrapegraphai-java-core`\n  - Contains core SDK logic\n  - Does not depend on [OkHttp](https://square.github.io/okhttp)\n  - Exposes [`ScrapegraphaiClient`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClient.kt), [`ScrapegraphaiClientAsync`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientAsync.kt), [`ScrapegraphaiClientImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientImpl.kt), and [`ScrapegraphaiClientAsyncImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientAsyncImpl.kt), all of which can work with any HTTP client\n- `scrapegraphai-java-client-okhttp`\n  - Depends on [OkHttp](https://square.github.io/okhttp)\n  - Exposes [`ScrapegraphaiOkHttpClient`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClient.kt) and [`ScrapegraphaiOkHttpClientAsync`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClientAsync.kt), which provide a way to construct [`ScrapegraphaiClientImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientImpl.kt) and [`ScrapegraphaiClientAsyncImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientAsyncImpl.kt), respectively, using OkHttp\n- `scrapegraphai-java`\n  - Depends on and exposes the APIs of both `scrapegraphai-java-core` and `scrapegraphai-java-client-okhttp`\n  - Does not have its own logic\n\nThis structure allows replacing the SDK's default HTTP client without pulling in unnecessary dependencies.\n\n#### Customized [`OkHttpClient`](https://square.github.io/okhttp/3.x/okhttp/okhttp3/OkHttpClient.html)\n\n\u003e [!TIP]\n\u003e Try the available [network options](#network-options) before replacing the default client.\n\nTo use a customized `OkHttpClient`:\n\n1. Replace your [`scrapegraphai-java` dependency](#installation) with `scrapegraphai-java-core`\n2. Copy `scrapegraphai-java-client-okhttp`'s [`OkHttpClient`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/OkHttpClient.kt) class into your code and customize it\n3. Construct [`ScrapegraphaiClientImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientImpl.kt) or [`ScrapegraphaiClientAsyncImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientAsyncImpl.kt), similarly to [`ScrapegraphaiOkHttpClient`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClient.kt) or [`ScrapegraphaiOkHttpClientAsync`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClientAsync.kt), using your customized client\n\n### Completely custom HTTP client\n\nTo use a completely custom HTTP client:\n\n1. Replace your [`scrapegraphai-java` dependency](#installation) with `scrapegraphai-java-core`\n2. Write a class that implements the [`HttpClient`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/core/http/HttpClient.kt) interface\n3. Construct [`ScrapegraphaiClientImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientImpl.kt) or [`ScrapegraphaiClientAsyncImpl`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/client/ScrapegraphaiClientAsyncImpl.kt), similarly to [`ScrapegraphaiOkHttpClient`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClient.kt) or [`ScrapegraphaiOkHttpClientAsync`](scrapegraphai-java-client-okhttp/src/main/kotlin/com/scrapegraphai/api/client/okhttp/ScrapegraphaiOkHttpClientAsync.kt), using your new client class\n\n## Undocumented API functionality\n\nThe SDK is typed for convenient usage of the documented API. However, it also supports working with undocumented or not yet supported parts of the API.\n\n### Parameters\n\nTo set undocumented parameters, call the `putAdditionalHeader`, `putAdditionalQueryParam`, or `putAdditionalBodyProperty` methods on any `Params` class:\n\n```java\nimport com.scrapegraphai.api.core.JsonValue;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .putAdditionalHeader(\"Secret-Header\", \"42\")\n    .putAdditionalQueryParam(\"secret_query_param\", \"42\")\n    .putAdditionalBodyProperty(\"secretProperty\", JsonValue.from(\"42\"))\n    .build();\n```\n\nThese can be accessed on the built object later using the `_additionalHeaders()`, `_additionalQueryParams()`, and `_additionalBodyProperties()` methods.\n\nTo set undocumented parameters on _nested_ headers, query params, or body classes, call the `putAdditionalProperty` method on the nested class:\n\n```java\nimport com.scrapegraphai.api.core.JsonValue;\nimport com.scrapegraphai.api.models.crawl.CrawlStartParams;\n\nCrawlStartParams params = CrawlStartParams.builder()\n    .rules(CrawlStartParams.Rules.builder()\n        .putAdditionalProperty(\"secretProperty\", JsonValue.from(\"42\"))\n        .build())\n    .build();\n```\n\nThese properties can be accessed on the nested built object later using the `_additionalProperties()` method.\n\nTo set a documented parameter or property to an undocumented or not yet supported _value_, pass a [`JsonValue`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/core/Values.kt) object to its setter:\n\n```java\nimport com.scrapegraphai.api.core.JsonValue;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(JsonValue.from(42))\n    .build();\n```\n\nThe most straightforward way to create a [`JsonValue`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/core/Values.kt) is using its `from(...)` method:\n\n```java\nimport com.scrapegraphai.api.core.JsonValue;\nimport java.util.List;\nimport java.util.Map;\n\n// Create primitive JSON values\nJsonValue nullValue = JsonValue.from(null);\nJsonValue booleanValue = JsonValue.from(true);\nJsonValue numberValue = JsonValue.from(42);\nJsonValue stringValue = JsonValue.from(\"Hello World!\");\n\n// Create a JSON array value equivalent to `[\"Hello\", \"World\"]`\nJsonValue arrayValue = JsonValue.from(List.of(\n  \"Hello\", \"World\"\n));\n\n// Create a JSON object value equivalent to `{ \"a\": 1, \"b\": 2 }`\nJsonValue objectValue = JsonValue.from(Map.of(\n  \"a\", 1,\n  \"b\", 2\n));\n\n// Create an arbitrarily nested JSON equivalent to:\n// {\n//   \"a\": [1, 2],\n//   \"b\": [3, 4]\n// }\nJsonValue complexValue = JsonValue.from(Map.of(\n  \"a\", List.of(\n    1, 2\n  ),\n  \"b\", List.of(\n    3, 4\n  )\n));\n```\n\nNormally a `Builder` class's `build` method will throw [`IllegalStateException`](https://docs.oracle.com/javase/8/docs/api/java/lang/IllegalStateException.html) if any required parameter or property is unset.\n\nTo forcibly omit a required parameter or property, pass [`JsonMissing`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/core/Values.kt):\n\n```java\nimport com.scrapegraphai.api.core.JsonMissing;\nimport com.scrapegraphai.api.models.smartscraper.SmartscraperCreateParams;\n\nSmartscraperCreateParams params = SmartscraperCreateParams.builder()\n    .userPrompt(JsonMissing.of())\n    .build();\n```\n\n### Response properties\n\nTo access undocumented response properties, call the `_additionalProperties()` method:\n\n```java\nimport com.scrapegraphai.api.core.JsonValue;\nimport java.util.Map;\n\nMap\u003cString, JsonValue\u003e additionalProperties = client.smartscraper().create(params)._additionalProperties();\nJsonValue secretPropertyValue = additionalProperties.get(\"secretProperty\");\n\nString result = secretPropertyValue.accept(new JsonValue.Visitor\u003c\u003e() {\n    @Override\n    public String visitNull() {\n        return \"It's null!\";\n    }\n\n    @Override\n    public String visitBoolean(boolean value) {\n        return \"It's a boolean!\";\n    }\n\n    @Override\n    public String visitNumber(Number value) {\n        return \"It's a number!\";\n    }\n\n    // Other methods include `visitMissing`, `visitString`, `visitArray`, and `visitObject`\n    // The default implementation of each unimplemented method delegates to `visitDefault`, which throws by default, but can also be overridden\n});\n```\n\nTo access a property's raw JSON value, which may be undocumented, call its `_` prefixed method:\n\n```java\nimport com.scrapegraphai.api.core.JsonField;\nimport java.util.Optional;\n\nJsonField\u003cString\u003e userPrompt = client.smartscraper().create(params)._userPrompt();\n\nif (userPrompt.isMissing()) {\n  // The property is absent from the JSON response\n} else if (userPrompt.isNull()) {\n  // The property was set to literal null\n} else {\n  // Check if value was provided as a string\n  // Other methods include `asNumber()`, `asBoolean()`, etc.\n  Optional\u003cString\u003e jsonString = userPrompt.asString();\n\n  // Try to deserialize into a custom type\n  MyClass myObject = userPrompt.asUnknown().orElseThrow().convert(MyClass.class);\n}\n```\n\n### Response validation\n\nIn rare cases, the API may return a response that doesn't match the expected type. For example, the SDK may expect a property to contain a `String`, but the API could return something else.\n\nBy default, the SDK will not throw an exception in this case. It will throw [`ScrapegraphaiInvalidDataException`](scrapegraphai-java-core/src/main/kotlin/com/scrapegraphai/api/errors/ScrapegraphaiInvalidDataException.kt) only if you directly access the property.\n\nIf you would prefer to check that the response is completely well-typed upfront, then either call `validate()`:\n\n```java\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\n\nCompletedSmartscraper completedSmartscraper = client.smartscraper().create(params).validate();\n```\n\nOr configure the method call to validate the response using the `responseValidation` method:\n\n```java\nimport com.scrapegraphai.api.models.smartscraper.CompletedSmartscraper;\n\nCompletedSmartscraper completedSmartscraper = client.smartscraper().create(\n  params, RequestOptions.builder().responseValidation(true).build()\n);\n```\n\nOr configure the default for all method calls at the client level:\n\n```java\nimport com.scrapegraphai.api.client.ScrapegraphaiClient;\nimport com.scrapegraphai.api.client.okhttp.ScrapegraphaiOkHttpClient;\n\nScrapegraphaiClient client = ScrapegraphaiOkHttpClient.builder()\n    .fromEnv()\n    .responseValidation(true)\n    .build();\n```\n\n## FAQ\n\n### Why don't you use plain `enum` classes?\n\nJava `enum` classes are not trivially [forwards compatible](https://www.stainless.com/blog/making-java-enums-forwards-compatible). Using them in the SDK could cause runtime exceptions if the API is updated to respond with a new enum value.\n\n### Why do you represent fields using `JsonField\u003cT\u003e` instead of just plain `T`?\n\nUsing `JsonField\u003cT\u003e` enables a few features:\n\n- Allowing usage of [undocumented API functionality](#undocumented-api-functionality)\n- Lazily [validating the API response against the expected shape](#response-validation)\n- Representing absent vs explicitly null values\n\n### Why don't you use [`data` classes](https://kotlinlang.org/docs/data-classes.html)?\n\nIt is not [backwards compatible to add new fields to a data class](https://kotlinlang.org/docs/api-guidelines-backward-compatibility.html#avoid-using-data-classes-in-your-api) and we don't want to introduce a breaking change every time we add a field to a class.\n\n### Why don't you use checked exceptions?\n\nChecked exceptions are widely considered a mistake in the Java programming language. In fact, they were omitted from Kotlin for this reason.\n\nChecked exceptions:\n\n- Are verbose to handle\n- Encourage error handling at the wrong level of abstraction, where nothing can be done about the error\n- Are tedious to propagate due to the [function coloring problem](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function)\n- Don't play well with lambdas (also due to the function coloring problem)\n\n## Semantic versioning\n\nThis package generally follows [SemVer](https://semver.org/spec/v2.0.0.html) conventions, though certain backwards-incompatible changes may be released as minor versions:\n\n1. Changes to library internals which are technically public but not intended or documented for external use. _(Please open a GitHub issue to let us know if you are relying on such internals.)_\n2. Changes that we do not expect to impact the vast majority of users in practice.\n\nWe take backwards-compatibility seriously and work hard to ensure you can rely on a smooth upgrade experience.\n\nWe are keen for your feedback; please open an [issue](https://www.github.com/ScrapeGraphAI/scrapegraphai-java/issues) with questions, bugs, or suggestions.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapegraphai%2Fscrapegraphai-java","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrapegraphai%2Fscrapegraphai-java","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapegraphai%2Fscrapegraphai-java/lists"}