https://github.com/msparihar/mcp-server-firecrawl

Last synced: 7 months ago
JSON representation

Host: GitHub
URL: https://github.com/msparihar/mcp-server-firecrawl
Owner: Msparihar
License: mit
Created: 2025-02-07T17:51:32.000Z (8 months ago)
Default Branch: main
Last Pushed: 2025-02-07T18:01:14.000Z (8 months ago)
Last Synced: 2025-02-07T19:19:06.627Z (8 months ago)
Language: JavaScript
Size: 14.6 KB
Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

# Firecrawl MCP Server

A Model Context Protocol (MCP) server for web scraping, content searching, site crawling, and data extraction using the Firecrawl API.

## Features

- **Web Scraping**: Extract content from any webpage with customizable options
- Mobile device emulation
- Ad and popup blocking
- Content filtering
- Structured data extraction
- Multiple output formats

- **Content Search**: Intelligent search capabilities
- Multi-language support
- Location-based results
- Customizable result limits
- Structured output formats

- **Site Crawling**: Advanced web crawling functionality
- Depth control
- Path filtering
- Rate limiting
- Progress tracking
- Sitemap integration

- **Site Mapping**: Generate site structure maps
- Subdomain support
- Search filtering
- Link analysis
- Visual hierarchy

- **Data Extraction**: Extract structured data from multiple URLs
- Schema validation
- Batch processing
- Web search enrichment
- Custom extraction prompts

## Installation

```bash
# Global installation
npm install -g @modelcontextprotocol/mcp-server-firecrawl

# Local project installation
npm install @modelcontextprotocol/mcp-server-firecrawl
```

## Quick Start

1. Get your Firecrawl API key from the [developer portal](https://firecrawl.dev/dashboard)

2. Set your API key:

**Unix/Linux/macOS (bash/zsh):**

```bash
export FIRECRAWL_API_KEY=your-api-key
```

**Windows (Command Prompt):**

```cmd
set FIRECRAWL_API_KEY=your-api-key
```

**Windows (PowerShell):**

```powershell
$env:FIRECRAWL_API_KEY = "your-api-key"
```

**Alternative: Using .env file (recommended for development):**

```bash
# Install dotenv
npm install dotenv

# Create .env file
echo "FIRECRAWL_API_KEY=your-api-key" > .env
```

Then in your code:

```javascript
import dotenv from 'dotenv';
dotenv.config();
```

3. Run the server:

```bash
mcp-server-firecrawl
```

## Integration

### Claude Desktop App

Add to your MCP settings:

```json
{
"firecrawl": {
"command": "mcp-server-firecrawl",
"env": {
"FIRECRAWL_API_KEY": "your-api-key"
}
}
}
```

### Claude VSCode Extension

Add to your MCP configuration:

```json
{
"mcpServers": {
"firecrawl": {
"command": "mcp-server-firecrawl",
"env": {
"FIRECRAWL_API_KEY": "your-api-key"
}
}
}
}
```

## Usage Examples

### Web Scraping

```typescript
// Basic scraping
{
name: "scrape_url",
arguments: {
url: "https://example.com",
formats: ["markdown"],
onlyMainContent: true
}
}

// Advanced extraction
{
name: "scrape_url",
arguments: {
url: "https://example.com/blog",
jsonOptions: {
prompt: "Extract article content",
schema: {
title: "string",
content: "string"
}
},
mobile: true,
blockAds: true
}
}
```

### Site Crawling

```typescript
// Basic crawling
{
name: "crawl",
arguments: {
url: "https://example.com",
maxDepth: 2,
limit: 100
}
}

// Advanced crawling
{
name: "crawl",
arguments: {
url: "https://example.com",
maxDepth: 3,
includePaths: ["/blog", "/products"],
excludePaths: ["/admin"],
ignoreQueryParameters: true
}
}
```

### Site Mapping

```typescript
// Generate site map
{
name: "map",
arguments: {
url: "https://example.com",
includeSubdomains: true,
limit: 1000
}
}
```

### Data Extraction

```typescript
// Extract structured data
{
name: "extract",
arguments: {
urls: ["https://example.com/product1", "https://example.com/product2"],
prompt: "Extract product details",
schema: {
name: "string",
price: "number",
description: "string"
}
}
}
```

## Configuration

See [configuration guide](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/configuration.md) for detailed setup options.

## API Documentation

See [API documentation](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/api.md) for detailed endpoint specifications.

## Development

```bash
# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Start in development mode
npm run dev
```

## Examples

Check the [examples](https://github.com/Msparihar/mcp-server-firecrawl/tree/main/examples) directory for more usage examples:

- Basic scraping: [scrape.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/scrape.ts)
- Crawling and mapping: [crawl-and-map.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/crawl-and-map.ts)

## Error Handling

The server implements robust error handling:

- Rate limiting with exponential backoff
- Automatic retries
- Detailed error messages
- Debug logging

## Security

- API key protection
- Request validation
- Domain allowlisting
- Rate limiting
- Safe error messages

## Contributing

See [CONTRIBUTING.md](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/CONTRIBUTING.md) for contribution guidelines.

## License

MIT License - see [LICENSE](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/LICENSE) for details.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/msparihar/mcp-server-firecrawl

Awesome Lists containing this project

README