Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/oxylabs/oxylabs-readme

Oxylabs repository collections' guide.
https://github.com/oxylabs/oxylabs-readme

Last synced: 3 months ago
JSON representation

Oxylabs repository collections' guide.

Awesome Lists containing this project

README 

          
# Welcome to Oxylabs on GitHub



[![Oxylabs promo code](https://raw.githubusercontent.com/oxylabs/product-integrations/refs/heads/master/Affiliate-Universal-1090x275.png)](https://oxylabs.io/pages/gitoxy?utm_source=877&utm_medium=affiliate&groupid=877&utm_content=oxylabs-readme-github&transaction_id=102f49063ab94276ae8f116d224b67)



[![](https://dcbadge.limes.pink/api/server/Pds3gBmKMH?style=for-the-badge&theme=discord)](https://discord.gg/Pds3gBmKMH) [![YouTube](https://img.shields.io/badge/YouTube-Oxylabs-red?style=for-the-badge&logo=youtube&logoColor=white)](https://www.youtube.com/@oxylabs)



We provide premium proxy services and web scraping solutions, trusted by developers worldwide. Our GitHub organisation offers tools, SDKs, and tutorials. Explore our repositories' collections:



## 🔧 Open-Source projects



Open-source frameworks designed to facilitate your web scraping projecs.



- **[OxyParser](https://github.com/oxylabs/OxyParser):** Oxy®Parser simplifies HTML parsing by leveraging LLMs and Pydantic models. Define your HTML structure with Pydantic, and let OxyParser automatically transform HTML into structured data models.

- **[OxyMouse](https://github.com/oxylabs/OxyMouse):** Generates realistic mouse movement paths using advanced algorithms.

- **[Web Scraper API Scheduler](https://github.com/oxylabs/Oxylabs-Web-Scraper-API-Scheduler/):** Quickly start and test Scraper API payloads with basic programming knowledge. 

  



## 📦 SDKs and integrations



SDKs and code samples to help you integrate Oxylabs' solutions into your projects.



- **[oxylabs-sdk-go](https://github.com/oxylabs/oxylabs-sdk-go):** Go SDK for Oxylabs' Scraper API.

- **[oxylabs-sdk-python](https://github.com/oxylabs/oxylabs-sdk-python):** Python SDK for working with Oxylabs' API.

- **[product-integrations](https://github.com/oxylabs/product-integrations):** Oxylabs product integration tutorials.

- **[proxy-integrations](https://github.com/oxylabs/proxy-integrations):** Proxy integration tutorials.

- **[selenium-proxy-integration-python](https://github.com/oxylabs/selenium-proxy-integration-python):** Residential Proxies' integration with Selenium in Python.

- **[selenium-proxy-integration-java](https://github.com/oxylabs/selenium-proxy-integration-java):** Residential Proxies' integration with Selenium using Java.

- **[puppeteer-proxy-integration-js](https://github.com/oxylabs/puppeteer-proxy-integration-js):** Proxies' integration with Puppeteer using JavaScript.

- **[playwright-proxy-integration-js](https://github.com/oxylabs/playwright-proxy-integration-js):** Residential and Datacenter proxies' integration with Playwright using JavaScript.

- **[mechanicalsoup-proxy-integration](https://github.com/oxylabs/mechanicalsoup-proxy-integration):** Python tutorial for integrating Residential Proxies with MechanicalSoup.

- **[aiohttp-proxy-integration](https://github.com/oxylabs/aiohttp-proxy-integration):** Python tutorial for integrating Residential Proxies with AIOHTTP.

- **[proxy-integration-with-parsehub](https://github.com/oxylabs/proxy-integration-with-parsehub):** Proxies' integration with ParseHub.

- **[integration-python-requests](https://github.com/oxylabs/integration-python-requests):** Residential and Datacenter Proxies' integration with Python Requests.

- **[integration-with-node-fetch](https://github.com/oxylabs/proxies-with-node-fetch):** Node-Fetch integration with proxies.



## 📚 Tutorials and guides



Tutorials and guides to help you get started with web scraping (organized by language).



### Python



- **[web-scraping-selenium-python](https://github.com/oxylabs/web-scraping-selenium-python):** Web Scraping with Python Selenium for beginners.

- **[seo-monitoring](https://github.com/oxylabs/seo-monitoring):** SEO Monitoring System with Python, Celery, and a SERP Scraper API (now a part of Web Scraper API).

- **[scrapy-web-scraping](https://github.com/oxylabs/scrapy-web-scraping):** Tutorial for web scraping with Scrapy.

- **[python-cache-tutorial](https://github.com/oxylabs/python-cache-tutorial):** Tutorial to caching web scraping scripts in Python.

- **[playwright-web-scraping](https://github.com/oxylabs/playwright-web-scraping):** Web scraping using Playwright headless browser.

- **[how-to-make-web-scraping-faster](https://github.com/oxylabs/how-to-make-web-scraping-faster):** Tips to make web scraping faster with Python.

- **[automate-competitors-benchmark-analysis](https://github.com/oxylabs/automate-competitors-benchmark-analysis):** Tutorial to automating competitors’ & benchmark analysis.

- **[asynchronous-web-scraping-python](https://github.com/oxylabs/asynchronous-web-scraping-python):** A comparison of asynchronous and synchronous web scraping methods.

- **[Python-Web-Scraping-Tutorial](https://github.com/oxylabs/Python-Web-Scraping-Tutorial):** Guide to Python web scraping.

- **[oxycon2021-active-fingerprinting-pyppeteer](https://github.com/oxylabs/oxycon2021-active-fingerprinting-pyppeteer):** Tutorial for active fingerprinting with Pyppeteer.

- **[best-buy-price-tracker](https://github.com/oxylabs/best-buy-price-tracker):** Scalable Best Buy price tracker tutorial.

- **[walmart-price-tracker](https://github.com/oxylabs/walmart-price-tracker):** Walmart price tracker tutorial.



### JavaScript



- **[puppeteer-tutorial](https://github.com/oxylabs/puppeteer-tutorial):** Web scraping with Puppeteer tutorial.

- **[crawlee-web-scraping-tutorial](https://github.com/oxylabs/crawlee-web-scraping-tutorial):** Web scraping using Crawlee tutorial.



### Ruby



- **[webscraping-with-ruby](https://github.com/oxylabs/webscraping-with-ruby):** Web scraping with Ruby.



### R



- **[web-scraping-r](https://github.com/oxylabs/web-scraping-r):** Web scraping with R.



### PHP



- **[web-scraping-php](https://github.com/oxylabs/web-scraping-php):** Web scraping with PHP.

- **[guzzle-proxy](https://github.com/oxylabs/guzzle-proxy):** Proxies' setup in Guzzle for PHP.



### PowerShell



- **[web-scraping-powershell](https://github.com/oxylabs/web-scraping-powershell):** Tutorial on using PowerShell and PowerHTML for web scraping.



## 🛠️ Google scraping solutions



Solutions and tools for scraping Google pages.



- **[google-scraper](https://github.com/oxylabs/scrape-google-python):** Scrape Google data with Python.

- **[google-images-scraper](https://github.com/oxylabs/how-to-scrape-google-images):** Scrape Google images in real time.

- **[google-shopping-scraper](https://github.com/oxylabs/scrape-google-shopping):** Get real-time data from Google Shopping.

- **[google-maps-scraper](https://github.com/oxylabs/google-maps-scraper):** Scrape Google Maps for data from various Google Maps listings.

- **[google-trends-scraper](https://github.com/oxylabs/how-to-scrape-google-trends):** Scrape Google Trends data and make a result comparison using Python.

- **[google-flights-scraper](https://github.com/oxylabs/how-to-scrape-google-flights):** Scrape flight data from Google Flights and analyze flight prices with Python.

- **[google-news-scraper](https://github.com/oxylabs/google-news-scraper):** Use Google News API to get the latest global news.

- **[google-maps-scraper](https://github.com/oxylabs/google-maps-scraper):** Scrape Google Maps for data from various Google Maps listings.

- **[google-play-scraper](https://github.com/oxylabs/google-play-scraper):** Scrape Google Play and collect public details about applications, and more.

- **[google-scholar-scraper](https://github.com/oxylabs/how-to-scrape-google-scholar):** Extract titles, authors, and citations from Google Scholar using Python.

- **[google-jobs-scraper](https://github.com/oxylabs/how-to-scrape-google-jobs):** Build your own Google Jobs scraper with Python.



## 🛍️ E-commerce scraping tutorials



Specialized scrapers for popular e-commerce platforms.



- **[quick-start-guide](https://github.com/oxylabs/quick-start-guide/blob/main/Guides/E-Commerce%20Websites.md):** A tutorial on using Oxylabs' Web Scraper API for E-Commerce targets.

- **[amazon-asin-scraper](https://github.com/oxylabs/amazon-asin-scraper):** Extract product ASINs from Amazon.

- **[bypass-amazon-captcha](https://github.com/oxylabs/how-to-bypass-amazon-captcha):** Bypass CAPTCHA when extracting public data from Amazon.

- **[amazon-review-scraper](https://github.com/oxylabs/amazon-review-scraper):** Extract Amazonreview data.

- **[ebay-scraper](https://github.com/oxylabs/ebay-scraper):** Extract product details from eBay.

- **[alibaba-scraper](https://github.com/oxylabs/alibaba-scraper):** Extract data from Alibaba for e-commerce analysis.

- **[walmart-scraper](https://github.com/oxylabs/walmart-scraper):** Scrape product information from Walmart.

- **[wayfair-scraper](https://github.com/oxylabs/wayfair-scraper):** Extract titles, prices, reviews Wayfair.

- **[zara-scraper](https://github.com/oxylabs/zara-scraper):** Extract data from Zara's online store.

- **[redbubble-scraper](https://github.com/oxylabs/redbubble-scraper):** Extract data from the Redbubble platform.

- **[aliexpress-scraper](https://github.com/oxylabs/aliexpress-scraper):** Extract prices, product details, and ratings.

- **[priceline-scraper](https://github.com/oxylabs/priceline-scraper):** Get Priceline data.

- **[shein-scraper](https://github.com/oxylabs/shein-scraper):** Extract product data from Shein.

- **[macys-scraper](https://github.com/oxylabs/macys-scraper):** Extract data from the Macy's ecommerce platform.

- **[opensea-scraper](https://github.com/oxylabs/opensea-scraper):** Get NFT's data. 

- **[azada-scraper](https://github.com/oxylabs/lazada-scraper):** Get product data from the Lazada e-commerce platform.

- **[etsy-scraper](https://github.com/oxylabs/etsy-scraper):** Extract data from Etsy.

- **[adidas-scraper](https://github.com/oxylabs/adidas-scraper):** Extract data from Adidas' online platforms.

- **[1688-scraper](https://github.com/oxylabs/1688-scraper):** Extract data from the 1688 e-commerce platform.

- **[xbox-scraper](https://github.com/oxylabs/xbox-scraper):** Extract game data from Xbox.

- **[biglots-scraper](https://github.com/oxylabs/biglots-scraper):** Get data from Biglots's retailers' websites.

- **[depop-scraper](https://github.com/oxylabs/depop-scraper):** Extract Depop's online shopping data.

- **[gamestop-scraper](https://github.com/oxylabs/gamestop-scraper):** Extract data from Gamestop's site.

- **[worten-scraper](https://github.com/oxylabs/worten-scraper):** Extract product data from Worten.

- **[digikey-scraper](https://github.com/oxylabs/digikey-scraper):** Extract electronic component data from the Digikey.

- **[bunnings-scraper](https://github.com/oxylabs/bunnings-scraper):** Extract data from Bunnings' website.



## 🌐 Other websites



Specialized scrapers for various websites. 



- **[indeed-scraper](https://github.com/oxylabs/how-to-scrape-indeed):** Extract job postings from Indeed.

- **[redfin-scraper](https://github.com/oxylabs/scraping-real-estate-data-with-python):** Extract real estate data from Redfin.

- **[youtube-scraper](https://github.com/oxylabs/youtube-scraper):** Extract public YouTube data.

- **[expedia-scraper](https://github.com/oxylabs/expedia-scraper):** Extract data from the Expedia website.

- **[trip-scraper](https://github.com/oxylabs/trip-scraper):** Extract travel data from various sources globally.

- **[trivago-scraper](https://github.com/oxylabs/trivago-scraper):** Get real-time data from Trivago.

- **[craigslist-scraper](https://github.com/oxylabs/craigslist-scraper):** Gather public Craigslist data.



## 📝 Data parsing and processing



Guides and tools for parsing and processing scraped data.



- **[parse-html-pyquery](https://github.com/oxylabs/parse-html-pyquery):** Tutorial for parsing HTML with PyQuery in Python.

- **[beautiful-soup-parsing-tutorial](https://github.com/oxylabs/beautiful-soup-parsing-tutorial):** Tutorial for using Beautiful Soup to parse data in Python.

- **[pandas-read-html](https://github.com/oxylabs/pandas-read-html):** Tutorial for reading HTML tables with pandas.

- **[date-parser-tutorial](https://github.com/oxylabs/Date-Parser-Tutorial):** Tutorial for parsing dates in Python.

- **[parse-xml-in-python](https://github.com/oxylabs/how-to-parse-xml-in-python):** Tutorial for parsing XML data in Python.

- **[price-parsing-tutorial](https://github.com/oxylabs/Price-Parsing-Tutorial):** Tutorial for parsing prices in Python.

- **[python-parse-json](https://github.com/oxylabs/python-parse-json):** Tutorial for parsing JSON data with Python.

- **[custom-parser-instructions](https://github.com/oxylabs/custom-parser-instructions):** Tutorial for writing parsing instructions with Oxylabs' Custom Parser.



## 🔒 Proxy solutions



Guides and tools for setting up and using proxies.



- **[proxy-manager-app](https://github.com/oxylabs/proxy-manager-app):** Tutorial for proxy manager app.

- **[proxy-chrome-extension](https://github.com/oxylabs/proxy-chrome-extension):** Tutorial for Proxy Chrome extension.

- **[curl-with-proxy](https://github.com/oxylabs/curl-with-proxy):** A tutorial on using cURL command with proxies.

- **[http-headers-with-curl](https://github.com/oxylabs/http-headers-with-curl):** Guide to sending and receiving HTTP headers using cURL.

- **[rotating-proxies-javascript](https://github.com/oxylabs/rotating-proxies-javascript):** Tutorial for rotating proxies with JavaScript.

- **[rotating-proxies-python](https://github.com/oxylabs/Rotating-Proxies-With-Python):** Tutorial for rotating proxies with Python.

- **[wget-proxy](https://github.com/oxylabs/wget-proxy):** Tutorial for using wget command with proxies.

- **[tamilyogi-proxy](https://github.com/oxylabs/tamilyogi-proxy):** Tutorial for TamilYogi proxies.

- **[datacenter-proxies](https://github.com/oxylabs/datacenter-proxies):** Tutorial for Oxylabs' Datacenter proxies.



## 🗨️ Community contributions



We welcome contributions from the community. If you would like to contribute to our projects, please check out our [Contribution Guidelines](https://github.com/oxylabs/oxylabs-readme/blob/main/CONTRIBUTING.md) for more information on how to get involved.



Additionally, feel free to join our [Discord server](https://discord.gg/GbxmdGhZjq) to connect with other developers, share ideas, and stay updated on the latest developments.



## 💬 Get in Touch



If you have any questions or need further assistance, feel free to [contact us](mailto:hello@oxylabs.io) or visit our [website](https://www.oxylabs.io) for more information.