{"id":20709927,"url":"https://github.com/oxylabs/oxylabs-sdk-go","last_synced_at":"2025-08-31T22:07:50.806Z","repository":{"id":220941045,"uuid":"720991190","full_name":"oxylabs/oxylabs-sdk-go","owner":"oxylabs","description":"Go SDK for the Oxylabs Scraper APIs.","archived":false,"fork":false,"pushed_at":"2025-07-09T14:06:34.000Z","size":180,"stargazers_count":31,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-08-31T22:07:49.090Z","etag":null,"topics":["api","mslm","oxylabs","oxylabs-library","proxy","scrape","scraper","sdk","sdk-oxylabs","serp-api","serp-api-go"],"latest_commit_sha":null,"homepage":"https://developers.oxylabs.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxylabs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-20T06:04:01.000Z","updated_at":"2025-08-16T00:28:59.000Z","dependencies_parsed_at":"2024-02-05T10:26:39.505Z","dependency_job_id":"a0b1484f-ee41-49ff-8738-6e889c2e5aa0","html_url":"https://github.com/oxylabs/oxylabs-sdk-go","commit_stats":null,"previous_names":["mslmio/oxylabs-sdk-go","oxylabs/oxylabs-sdk-go"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/oxylabs/oxylabs-sdk-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Foxylabs-sdk-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Foxylabs-sdk-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Foxylabs-sdk-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Foxylabs-sdk-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxylabs","download_url":"https://codeload.github.com/oxylabs/oxylabs-sdk-go/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Foxylabs-sdk-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273046500,"owners_count":25036181,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","mslm","oxylabs","oxylabs-library","proxy","scrape","scraper","sdk","sdk-oxylabs","serp-api","serp-api-go"],"created_at":"2024-11-17T02:09:04.535Z","updated_at":"2025-08-31T22:07:50.793Z","avatar_url":"https://github.com/oxylabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Oxylabs Go SDK\n\n[![Oxylabs promo code](https://raw.githubusercontent.com/oxylabs/product-integrations/refs/heads/master/Affiliate-Universal-1090x275.png)](https://oxylabs.io/pages/gitoxy?utm_source=877\u0026utm_medium=affiliate\u0026groupid=877\u0026utm_content=oxylabs-sdk-go-github\u0026transaction_id=102f49063ab94276ae8f116d224b67)\n\n[![](https://dcbadge.vercel.app/api/server/eWsVUJrnG5)](https://discord.gg/Pds3gBmKMH)\n\nThis is a Go SDK for the [Oxylabs](https://oxylabs.io) [Scraper APIs](https://developers.oxylabs.io/scraper-apis/serp-scraper-api#getting-started).\n\nThis will help simplify integrating with Oxylabs's APIs, which can help you with retrieving search engine results (SERP), eCommerce data, real estate data, and more.\n\nThe Go SDK provides you with several benefits over using the raw APIs directly:\n\n- **Simplified Interface**: abstracts away complexities, offering a straightforward user interface for interacting with the Oxylabs SERP API.\n- **Automated Request Management**: streamlines the handling of API requests and responses for enhanced efficiency and reliability.\n- **Error Handling**: provides meaningful error messages and handles common API errors, simplifying troubleshooting.\n- **Result Parsing**: streamlines the process of extracting relevant data from SERP results, allowing developers to focus on application logic.\n\n## Requirements\n\n- Go 1.21.0 or above.\n\nYou can check your go version by running the following command in your preferred terminal:\n\n```sh\ngo version\n```\n\nIf you need to install or update go you can do so by following the steps mentioned [here](https://go.dev/doc/install).\n\n## Authentication\n\nYou will need an Oxylabs API username and password which you can get by signing up at https://oxylabs.io. You can check things out with a free trial at https://oxylabs.io/products/scraper-api/serp.\n\n## Installation\n\n```bash\ngo get github.com/oxylabs/oxylabs-sdk-go\n```\n\n## Usage\n\nStart a local Go project if you don't have one:\n\n```bash\ngo mod init\n```\n\nInstall the package:\n\n```bash\ngo get github.com/oxylabs/oxylabs-sdk-go\n```\n\n### Quick Start\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/oxylabs/oxylabs-sdk-go/serp\"\n)\n\nfunc main() {\n\t// Set your Oxylabs API Credentials.\n\tconst username = \"username\"\n\tconst password = \"password\"\n\n\t// Initialize the SERP realtime client with your credentials.\n\tc := serp.Init(username, password)\n\n\t// Use `google_search` as a source to scrape Google with adidas as a query.\n\tres, err := c.ScrapeGoogleSearch(\n\t\t\"adidas\",\n\t)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tfmt.Printf(\"Results: %+v\\n\", res)\n}\n```\n\n### Integration Methods\n\nThere are three integration method for the Oxylabs SERP API, each exposed via different packages:\n\n- Realtime (Sync) - `serp.Init(username, password)`\n- Push-Pull (Async) - `serp.InitAsync(username, password)`\n- Proxy Endpoint - `proxy.Init(username, password)`\n\nLearn more about integration methods [on the official documentation](https://developers.oxylabs.io/scraper-apis/serp-scraper-api/integration-methods) and how this SDK uses them [here](#integration-methods-1).\n\n### Sources\n\nThe Oxylabs SERP API scrapes according to the source provided via the API.\n\nThere are currently four search engines you can scrape with the Oxylabs SERP API, each with different sources.\n\n| Search Engine | Sources\n| ------------- | --------------\n| **Google**    | `google`, `google_search`, `google_ads`, `google_hotels`, `google_travel_hotels`, `google_suggest`, `google_trends_explore`, `google_lens`\n| **Bing**      | `bing`, `bing_search`\n\nIn the SDK you'll just need to call the relevant function name from the client.\n\nFor example if you wish to scrape Google with `google_search` as a source:\n\n```go\nres, err := c.ScrapeGoogleSearch(\"football\")\n```\n\n### Query Parameters\n\nEach source has different accepted query parameters. For a detailed list of accepted parameters by each source you can head over to https://developers.oxylabs.io/scraper-apis/serp-scraper-api#request-parameter-values.\n\nBy default, scrape functions will use default parameters. If you need to send specific query parameters, here is an example of how to do it:\n\n```go\nres, err := c.ScrapeGoogleSearch(\n\t\"football\",\n\t\u0026serp.GoogleSearchOpts{\n\t\tStartPage: 1,\n\t\tPages:     3,\n\t\tLimit:     4,\n\t\tDomain:    \"com\",\n\t\tLocale:    \"en\",\n\t},\n)\n```\n\n### Configurable Options\n\nFor consistency and ease of use, this SDK provides a list of pre-defined commonly used parameter values as constants in our library. You can use them by importing the oxylabs package.\n\n```go\nimport (\n\t\"github.com/oxylabs/oxylabs-sdk-go/oxylabs\"\n)\n```\n\nCurrently these are available for the `Render` and`UserAgent` parameters. For the full list you can check `oxylabs/types.go`. You can send in these values as strings too.\n\nThese can be used as follows:\n\n```go\nres, err := c.ScrapeGoogleSearch(\n\t\"adidas\",\n\t\u0026serp.GoogleSearchOpts{\n\t\tUserAgent: oxylabs.UA_DESKTOP_CHROME, // desktop_chrome\n\t\tRender:    oxylabs.HTML,              // html\n\t\tDomain:    oxylabs.DOMAIN_COM,        // com\n\t},\n)\n```\n\n### Context Options for Google sources\n\nYou can send in context options relevant to `google` sources. Here's an example for Google Search scraping:\n\n```go\nres, err := c.ScrapeGoogleSearch(\n\t\"adidas\",\n\t\u0026serp.GoogleSearchOpts{\n\t\tParse: true,\n\t\tContext: []func(oxylabs.ContextOption){\n\t\t\toxylabs.ResultsLanguage(\"en\"),\n\t\t\toxylabs.Filter(1),\n\t\t\toxylabs.Tbm(\"isch\"),\n\t\t\toxylabs.LimitPerPage([]serp.PageLimit{{Page: 1, Limit: 1}, {Page: 2, Limit: 6}}),\n\t\t},\n\t},\n)\n```\n\n### Parse instructions\n\nSDK supports [custom parsing](https://developers.oxylabs.io/scraper-apis/custom-parser).\nThere are 2 ways to provide `parsing_instructions` `_fns`:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/oxylabs/oxylabs-sdk-go/ecommerce\"\n\t\"github.com/oxylabs/oxylabs-sdk-go/oxylabs\"\n)\n\nfunc main() {\n\tconst username = \"username\"\n\tconst password = \"password\"\n\n\t// Initialize the SERP push-pull client with your credentials.\n\tc := ecommerce.InitAsync(username, password)\n\n\tch, err := c.ScrapeUniversalUrl(\n\t\t\"https://example.com\",\n\t\t\u0026ecommerce.UniversalUrlOpts{\n\t\t\tParse: true,\n\t\t\tParseInstructions: \u0026map[string]interface{}{\n\t\t\t\t\"title\": map[string]interface{}{\n\t\t\t\t\t// Providing `_fns` as a map[string]interface{}.\n\t\t\t\t\t\"_fns\": []map[string]interface{}{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\t\"_fn\":   oxylabs.Xpath,\n\t\t\t\t\t\t\t\"_args\": []string{\"//h1/text()\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t\"second_paragraph\": map[string]interface{}{\n\t\t\t\t\t// Providing `_fns` as a `[]oxylabs.Fn`.\n\t\t\t\t\t\"_fns\": []oxylabs.Fn{\n\t\t\t\t\t\t{\n\t\t\t\t\t\t\tName: oxylabs.Xpath,\n\t\t\t\t\t\t\tArgs: []string{\"/html/body/div/p[2]\"},\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\tres := \u003c-ch\n\tfmt.Println(res)\n}\n```\n\n## Integration Methods\n\n### Realtime Integration\n\nRealtime is a synchronous integration method. This means that upon sending your job submission request, **you will have to keep the connection open** until we successfully finish your job or return an error.\n\nThe **TTL** of Realtime connections is **150 seconds**. There may be rare cases where your connection times out before you receive a response from us, for example, if our system is under heavier-than-usual load or the job you submitted was extremely hard to complete:\n\n### Push Pull(Polling) Integration \u003ca id=\"push-pull\"\u003e\u003c/a\u003e\n\nPush-Pull is an asynchronous integration method. This SDK implements this integration with a polling technique to poll the endpoint for results after a set interval of time.\n\nUsing it as straightforward as using the realtime integration. The only difference is that it will return a channel with the Response. Below is an example of this integration method:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/oxylabs/oxylabs-sdk-go/oxylabs\"\n\t\"github.com/oxylabs/oxylabs-sdk-go/serp\"\n)\n\nfunc main() {\n\tconst username = \"username\"\n\tconst password = \"password\"\n\n\t// Initialize the SERP push-pull client with your credentials.\n\tc := serp.InitAsync(username, password)\n\n\tch, err := c.ScrapeGoogleAds(\n\t\t\"adidas shoes\",\n\t\t\u0026serp.GoogleAdsOpts{\n\t\t\tUserAgent: oxylabs.UA_DESKTOP,\n\t\t\tParse:     true,\n\t\t},\n\t)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tres := \u003c-ch\n\tfmt.Printf(\"Results: %+v\\n\", res)\n}\n```\n\n### Proxy Endpoint\n\nThis method is also synchronous (like Realtime), but instead of using our service via a RESTful interface, you **can use our endpoint like a proxy**. Use Proxy Endpoint if you've used proxies before and would just like to get unblocked content from us.\n\nSince the parameters in this method are sent as as headers there are only a few parameters which this integration method accepts. You can find those parameters at\nhttps://developers.oxylabs.io/scraper-apis/serp-scraper-api/integration-methods/proxy-endpoint#accepted-parameters.\n\nThe proxy endpoint integration is very open ended allowing many different use cases. To cater this, the user is provided a pre-configured `http.Client` and they can use it as they deem fit:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\n\t\"github.com/oxylabs/oxylabs-sdk-go/oxylabs\"\n\t\"github.com/oxylabs/oxylabs-sdk-go/proxy\"\n)\n\nfunc main() {\n\tconst username = \"username\"\n\tconst password = \"password\"\n\n\t// Init returns an http client pre configured with the proxy settings.\n\tc, _ := proxy.Init(username, password)\n\n\trequest, _ := proxy.NewRequest(\n\t\t\"GET\",\n\t\t\"https://www.example.com\",\n\t\tnil,\n\t)\n\n\t// Add relevant Headers.\n\tproxy.AddUserAgentHeader(request, oxylabs.UA_DESKTOP)\n\tproxy.AddRenderHeader(request, oxylabs.HTML)\n\n\trequest.SetBasicAuth(username, password)\n\tresponse, _ := c.Do(request)\n\n\tresp, _ := io.ReadAll(response.Body)\n\tfmt.Println(string(resp))\n}\n```\n\n## Additional Resources\n\nSee the official [API Documentation](https://developers.oxylabs.io/) for\ndetails on each API's actual interface, which is implemented by this SDK.\n\n## Contributing\n\nSee [CONTRIBUTING](CONTRIBUTING.md) for more information.\n\n## Security\n\nSee [Security Issue\nNotifications](CONTRIBUTING.md#security-issue-notifications) for more\ninformation.\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n\n## About Oxylabs\n\nEstablished in 2015, Oxylabs are a market-leading web intelligence collection platform, driven by the highest business, ethics, and compliance standards, enabling companies worldwide to unlock data-driven insights.\n\n[![image](https://oxylabs.io/images/og-image.png)](https://oxylabs.io/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Foxylabs-sdk-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxylabs%2Foxylabs-sdk-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Foxylabs-sdk-go/lists"}