https://github.com/ritvij14/crawl4ai-mcp
https://github.com/ritvij14/crawl4ai-mcp
Last synced: 6 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/ritvij14/crawl4ai-mcp
- Owner: ritvij14
- Created: 2025-03-06T06:18:30.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2025-04-02T14:45:02.000Z (6 months ago)
- Last Synced: 2025-04-02T15:36:03.785Z (6 months ago)
- Language: Python
- Size: 50.8 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Crawl4AI MCP Server
A Model Context Protocol (MCP) server implementation that integrates Crawl4AI with Cursor AI, providing web scraping and crawling capabilities as tools for LLMs in Cursor Composer's agent mode.
## System Requirements
Python 3.10 or higher installed.
## Current Features
- Single page scraping
## Installation
Basic setup instructions also available in the [Official Docs for MCP Server QuickStart](https://modelcontextprotocol.io/quickstart/server#why-claude-for-desktop-and-not-claude-ai).
### Set up your environment
First, let's install `uv` and set up our Python project and environment:
MacOS/Linux:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Windows:
```bash
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
Make sure to restart your terminal afterwards to ensure that the uv command gets picked up.
After that:
1. Clone the repository
2. Install dependencies using UV:
```bash
# Navigate to the crawl4ai-mcp directory
cd crawl4ai-mcp
# Install dependencies (Only first time)
uv venv
uv sync
# Activate the venv
source .venv/bin/activate
# Run the server
python main.py
```
3. Add to Cursor's MCP Servers or Claude's MCP Servers
You may need to put the full path to the uv executable in the command field. You can get this by running `which uv` on MacOS/Linux or `where uv` on Windows.
```json
{
"mcpServers": {
"Crawl4AI": {
"command": "uv",
"args": [
"--directory",
"/ABSOLUTE/PATH/TO/PARENT/FOLDER/crawl4ai-mcp",
"run",
"main.py"
]
}
}
}
```