https://github.com/oxylabs/oxylabs-mcp
https://github.com/oxylabs/oxylabs-mcp
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/oxylabs/oxylabs-mcp
- Owner: oxylabs
- License: mit
- Created: 2025-01-17T13:47:05.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-02-28T14:33:21.000Z (8 months ago)
- Last Synced: 2025-02-28T19:09:22.254Z (8 months ago)
- Language: Python
- Size: 46.9 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-mcp-zh - Oxylabs
- awesome-mcp-servers - **oxylabs-mcp** - Official Oxylabs MCP integration `python` `ai-tools` `data-collection` `e-comerce-scraping` `llm` `pip install git+https://github.com/oxylabs/oxylabs-mcp` (🤖 AI/ML)
- awesome-mcp-servers - **oxylabs-mcp** - Official Oxylabs MCP integration `python` `ai-tools` `data-collection` `e-comerce-scraping` `llm` `pip install git+https://github.com/oxylabs/oxylabs-mcp` (AI/ML)
- Awesome-MCP-Servers-directory - Oxylabs - Scrape websites with Oxylabs Web API, supporting dynamic rendering and parsing for structured data extraction (Browser Automation)
- Awesome-Official-MCP-Servers - Oxylabs - mcp?style=social) | Scrape websites with Oxylabs Web API, supporting dynamic rendering and parsing for structured data extraction. | | (官方 MCP 服务器列表)
- awesome-mcp-servers - Oxylabs - Provides web scraping capabilities for AI assistants, bypassing anti-scraping measures and geo-restrictions (Table of Contents / Browser Automation)
- awesome-mcp-servers - Oxylabs - Provides web scraping capabilities for AI assistants, bypassing anti-scraping measures and geo-restrictions (Table of Contents / Browser Automation)
- awesome-mcp-servers - Oxylabs - Scrape websites with Oxylabs Web API, supporting dynamic rendering and parsing for structured data extraction. (Official Servers)
- Awesome-Official-MCP-Servers - Oxylabs - mcp?style=social) | Scrape websites with Oxylabs Web API, supporting dynamic rendering and parsing for structured data extraction. | | (官方 MCP 服务器列表)
- metorial-index - Oxylabs MCP - Fetch and process content from specified URLs using the Oxylabs Web Scraper API, enabling easy access to web data. (Cloud Services)
README
Oxylabs MCP Server
The missing link between AI models and the real‑world web: one API that delivers clean, structured data from any site.
[](https://smithery.ai/server/@oxylabs/oxylabs-mcp)
[](https://pypi.org/project/oxylabs-mcp/)
[](https://discord.gg/Pds3gBmKMH)
[](LICENSE)
[](https://mseep.ai/app/f6a9c0bc-83a6-4f78-89d9-f2cec4ece98d)
---
## 📖 Overview
The Oxylabs MCP server provides a bridge between AI models and the web. It enables them to scrape any URL, render JavaScript-heavy pages, extract and format content for AI use, bypass anti-scraping measures, and access geo-restricted web data from 195+ countries.
This implementation leverages the Model Context Protocol (MCP) to create a secure, standardized way for AI assistants to interact with web content.
---
## Why Oxylabs MCP? 🕸️ ➜ 📦 ➜ 🤖
Imagine telling your LLM *"Summarise the latest Hacker News discussion about GPT‑7"* – and it simply answers.
MCP (Multi‑Client Proxy) makes that happen by doing the boring parts for you:
| What Oxylabs MCP does | Why it matters to you |
|-------------------------------------------------------------------|------------------------------------------|
| **Bypasses anti‑bot walls** with the Oxylabs global proxy network | Keeps you unblocked and anonymous |
| **Renders JavaScript** in headless Chrome | Single‑page apps, sorted |
| **Cleans HTML → JSON** | Drop straight into vector DBs or prompts |
| **Optional structured parsers** (Google, Amazon, etc.) | One‑line access to popular targets |
## ✨ Key Features
Scrape content from any site
- Extract data from any URL, including complex single-page applications
- Fully render dynamic websites using headless browser support
- Choose full JavaScript rendering, HTML-only, or none
- Emulate Mobile and Desktop viewports for realistic rendering
Automatically get AI-ready data
- Automatically clean and convert HTML to Markdown for improved readability
- Use automated parsers for popular targets like Google, Amazon, and etc.
Bypass blocks & geo-restrictions
- Bypass sophisticated bot protection systems with high success rate
- Reliably scrape even the most complex websites
- Get automatically rotating IPs from a proxy pool covering 195+ countries
Flexible setup & cross-platform support
- Set rendering and parsing options if needed
- Feed data directly into AI models or analytics tools
- Works on macOS, Windows, and Linux
Built-in error handling and request management
- Comprehensive error handling and reporting
- Smart rate limiting and request management
---
## 🛠️ MCP Tools
Oxylabs MCP provides the following tools:
1. **universal_scraper**: Uses Oxylabs Web Scraper API for general website scraping.
2. **google_search_scraper**: Uses Oxylabs Web Scraper API to extract results from Google Search.
3. **amazon_search_scraper**: Uses Oxylabs Web Scraper API to scrape Amazon search result pages.
4. **amazon_product_scraper**: Uses Oxylabs Web Scraper API to extract data from individual Amazon product pages.
## 💡 Example Queries
When you've set up the MCP server with **Claude**, you can make requests like:
- Could you scrape `https://www.google.com/search?q=ai` page?
- Scrape `https://www.amazon.de/-/en/Smartphone-Contract-Function-Manufacturer-Exclusive/dp/B0CNKD651V` with **parse** enabled
- Scrape `https://www.amazon.de/-/en/gp/bestsellers/beauty/ref=zg_bs_nav_beauty_0` with **parse** and **render** enabled
- Use web unblocker with **render** to scrape `https://www.bestbuy.com/site/top-deals/all-electronics-on-sale/pcmcat1674241939957.c`
---
## ✅ Prerequisites
Before you begin, make sure you have:
- **Oxylabs Account**: Obtain your username and password from [Oxylabs](https://dashboard.oxylabs.io/) (1-week free trial available)
### Basic Usage
Via Smithery CLI:
- **Node.js** (v16+)
- `npx` command-line tool
Via uv:
- `uv` package manager – install it using [this guide](https://docs.astral.sh/uv/getting-started/installation/)
### Local/Dev Setup
- **Python 3.12+**
- `uv` package manager – install it using [this guide](https://docs.astral.sh/uv/getting-started/installation/)
---
## 🧩 API Parameters
The Oxylabs MCP Universal Scraper accepts these parameters:
| Parameter | Description | Values |
|-------------------|-------------------------------------------------|---------------------------|
| `url` | The URL to scrape | Any valid URL |
| `render` | Use headless browser rendering | `html` or `None` |
| `geo_location` | Sets the proxy's geo location to retrieve data. | `Brasil`, `Canada`, etc. |
| `user_agent_type` | Device type and browser | `desktop`, `tablet`, etc. |
| `output_format` | The format of the output | `links`, `md`, `html` |
---
## 🔧 Configuration
[](https://cursor.com/install-mcp?name=oxylabs&config=eyJjb21tYW5kIjoidXZ4IG94eWxhYnMtbWNwIiwiZW52Ijp7Ik9YWUxBQlNfVVNFUk5BTUUiOiJPWFlMQUJTX1VTRVJOQU1FIiwiT1hZTEFCU19QQVNTV09SRCI6Ik9YWUxBQlNfUEFTU1dPUkQifX0%3D)
smithery
1. Go to https://smithery.ai/server/@oxylabs/oxylabs-mcp
2. Login with GitHub
3. Find the _Install_ section
4. Follow the instructions to generate the config
Auto install with Smithery CLI
```bash
# example for Claude Desktop
npx -y @smithery/cli@latest install @upstash/context7-mcp --client claude --key
```
uvx
1. Install the uv
```bash
# macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. Use the following config
```json
{
"mcpServers": {
"oxylabs": {
"command": "uvx",
"args": ["oxylabs-mcp"],
"env": {
"OXYLABS_USERNAME": "OXYLABS_USERNAME",
"OXYLABS_PASSWORD": "OXYLABS_PASSWORD"
}
}
}
}
```
uv
1. Install the uvx
```bash
# macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. Use the following config
```json
{
"mcpServers": {
"oxylabs": {
"command": "uv",
"args": [
"--directory",
"//oxylabs-mcp",
"run",
"oxylabs-mcp"
],
"env": {
"OXYLABS_USERNAME": "OXYLABS_USERNAME",
"OXYLABS_PASSWORD": "OXYLABS_PASSWORD"
}
}
}
}
```
### Manual Setup with Claude Desktop
Navigate to **Claude → Settings → Developer → Edit Config** and add one of the configurations above to the `claude_desktop_config.json` file.
### Manual Setup with Cursor AI
Navigate to **Cursor → Settings → Cursor Settings → MCP**. Click **Add new global MCP server** and add one of the configurations above.
---
## ⚙️ Environment variables
Oxylabs MCP server supports the following environment variables
| Name | Description | Mandatory | Default |
|--------------------|-----------------------------------------------|---------------------|---------|
| `OXYLABS_USERNAME` | Your Oxylabs username | true | |
| `OXYLABS_PASSWORD` | Your Oxylabs password | true | |
| `LOG_LEVEL` | Log level for the logs returned to the client | false | `INFO` |
---
## 📝 Logging
Server provides additional information about the tool calls in `notification/message` events
```json
{
"method": "notifications/message",
"params": {
"level": "info",
"data": "Create job with params: {\"url\": \"https://ip.oxylabs.io\"}"
}
}
```
```json
{
"method": "notifications/message",
"params": {
"level": "info",
"data": "Job info: job_id=7333113830223918081 job_status=done"
}
}
```
```json
{
"method": "notifications/message",
"params": {
"level": "error",
"data": "Error: request to Oxylabs API failed"
}
}
```
---
## 🛡️ License
Distributed under the MIT License – see [LICENSE](LICENSE) for details.
---
## About Oxylabs
Established in 2015, Oxylabs is a market-leading web intelligence collection
platform, driven by the highest business, ethics, and compliance standards,
enabling companies worldwide to unlock data-driven insights.
[](https://oxylabs.io/)