{"id":26369854,"url":"https://github.com/msparihar/mcp-server-firecrawl","last_synced_at":"2025-03-16T23:16:57.654Z","repository":{"id":276358340,"uuid":"929044887","full_name":"Msparihar/mcp-server-firecrawl","owner":"Msparihar","description":null,"archived":false,"fork":false,"pushed_at":"2025-02-07T18:01:14.000Z","size":15,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-07T19:19:06.627Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Msparihar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-07T17:51:32.000Z","updated_at":"2025-02-07T18:01:20.000Z","dependencies_parsed_at":"2025-02-07T19:19:12.448Z","dependency_job_id":null,"html_url":"https://github.com/Msparihar/mcp-server-firecrawl","commit_stats":null,"previous_names":["msparihar/mcp-server-firecrawl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Msparihar%2Fmcp-server-firecrawl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Msparihar%2Fmcp-server-firecrawl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Msparihar%2Fmcp-server-firecrawl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Msparihar%2Fmcp-server-firecrawl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Msparihar","download_url":"https://codeload.github.com/Msparihar/mcp-server-firecrawl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243945592,"owners_count":20372897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-16T23:16:56.989Z","updated_at":"2025-03-16T23:16:57.637Z","avatar_url":"https://github.com/Msparihar.png","language":"JavaScript","funding_links":[],"categories":["Browser Automation"],"sub_categories":["How to Submit"],"readme":"# Firecrawl MCP Server\n\nA Model Context Protocol (MCP) server for web scraping, content searching, site crawling, and data extraction using the Firecrawl API.\n\n## Features\n\n- **Web Scraping**: Extract content from any webpage with customizable options\n  - Mobile device emulation\n  - Ad and popup blocking\n  - Content filtering\n  - Structured data extraction\n  - Multiple output formats\n\n- **Content Search**: Intelligent search capabilities\n  - Multi-language support\n  - Location-based results\n  - Customizable result limits\n  - Structured output formats\n\n- **Site Crawling**: Advanced web crawling functionality\n  - Depth control\n  - Path filtering\n  - Rate limiting\n  - Progress tracking\n  - Sitemap integration\n\n- **Site Mapping**: Generate site structure maps\n  - Subdomain support\n  - Search filtering\n  - Link analysis\n  - Visual hierarchy\n\n- **Data Extraction**: Extract structured data from multiple URLs\n  - Schema validation\n  - Batch processing\n  - Web search enrichment\n  - Custom extraction prompts\n\n## Installation\n\n```bash\n# Global installation\nnpm install -g @modelcontextprotocol/mcp-server-firecrawl\n\n# Local project installation\nnpm install @modelcontextprotocol/mcp-server-firecrawl\n```\n\n## Quick Start\n\n1. Get your Firecrawl API key from the [developer portal](https://firecrawl.dev/dashboard)\n\n2. Set your API key:\n\n   **Unix/Linux/macOS (bash/zsh):**\n\n   ```bash\n   export FIRECRAWL_API_KEY=your-api-key\n   ```\n\n   **Windows (Command Prompt):**\n\n   ```cmd\n   set FIRECRAWL_API_KEY=your-api-key\n   ```\n\n   **Windows (PowerShell):**\n\n   ```powershell\n   $env:FIRECRAWL_API_KEY = \"your-api-key\"\n   ```\n\n   **Alternative: Using .env file (recommended for development):**\n\n   ```bash\n   # Install dotenv\n   npm install dotenv\n\n   # Create .env file\n   echo \"FIRECRAWL_API_KEY=your-api-key\" \u003e .env\n   ```\n\n   Then in your code:\n\n   ```javascript\n   import dotenv from 'dotenv';\n   dotenv.config();\n   ```\n\n3. Run the server:\n\n   ```bash\n   mcp-server-firecrawl\n   ```\n\n## Integration\n\n### Claude Desktop App\n\nAdd to your MCP settings:\n\n```json\n{\n  \"firecrawl\": {\n    \"command\": \"mcp-server-firecrawl\",\n    \"env\": {\n      \"FIRECRAWL_API_KEY\": \"your-api-key\"\n    }\n  }\n}\n```\n\n### Claude VSCode Extension\n\nAdd to your MCP configuration:\n\n```json\n{\n  \"mcpServers\": {\n    \"firecrawl\": {\n      \"command\": \"mcp-server-firecrawl\",\n      \"env\": {\n        \"FIRECRAWL_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n## Usage Examples\n\n### Web Scraping\n\n```typescript\n// Basic scraping\n{\n  name: \"scrape_url\",\n  arguments: {\n    url: \"https://example.com\",\n    formats: [\"markdown\"],\n    onlyMainContent: true\n  }\n}\n\n// Advanced extraction\n{\n  name: \"scrape_url\",\n  arguments: {\n    url: \"https://example.com/blog\",\n    jsonOptions: {\n      prompt: \"Extract article content\",\n      schema: {\n        title: \"string\",\n        content: \"string\"\n      }\n    },\n    mobile: true,\n    blockAds: true\n  }\n}\n```\n\n### Site Crawling\n\n```typescript\n// Basic crawling\n{\n  name: \"crawl\",\n  arguments: {\n    url: \"https://example.com\",\n    maxDepth: 2,\n    limit: 100\n  }\n}\n\n// Advanced crawling\n{\n  name: \"crawl\",\n  arguments: {\n    url: \"https://example.com\",\n    maxDepth: 3,\n    includePaths: [\"/blog\", \"/products\"],\n    excludePaths: [\"/admin\"],\n    ignoreQueryParameters: true\n  }\n}\n```\n\n### Site Mapping\n\n```typescript\n// Generate site map\n{\n  name: \"map\",\n  arguments: {\n    url: \"https://example.com\",\n    includeSubdomains: true,\n    limit: 1000\n  }\n}\n```\n\n### Data Extraction\n\n```typescript\n// Extract structured data\n{\n  name: \"extract\",\n  arguments: {\n    urls: [\"https://example.com/product1\", \"https://example.com/product2\"],\n    prompt: \"Extract product details\",\n    schema: {\n      name: \"string\",\n      price: \"number\",\n      description: \"string\"\n    }\n  }\n}\n```\n\n## Configuration\n\nSee [configuration guide](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/configuration.md) for detailed setup options.\n\n## API Documentation\n\nSee [API documentation](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/api.md) for detailed endpoint specifications.\n\n## Development\n\n```bash\n# Install dependencies\nnpm install\n\n# Build\nnpm run build\n\n# Run tests\nnpm test\n\n# Start in development mode\nnpm run dev\n```\n\n## Examples\n\nCheck the [examples](https://github.com/Msparihar/mcp-server-firecrawl/tree/main/examples) directory for more usage examples:\n\n- Basic scraping: [scrape.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/scrape.ts)\n- Crawling and mapping: [crawl-and-map.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/crawl-and-map.ts)\n\n## Error Handling\n\nThe server implements robust error handling:\n\n- Rate limiting with exponential backoff\n- Automatic retries\n- Detailed error messages\n- Debug logging\n\n## Security\n\n- API key protection\n- Request validation\n- Domain allowlisting\n- Rate limiting\n- Safe error messages\n\n## Contributing\n\nSee [CONTRIBUTING.md](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/CONTRIBUTING.md) for contribution guidelines.\n\n## License\n\nMIT License - see [LICENSE](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsparihar%2Fmcp-server-firecrawl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmsparihar%2Fmcp-server-firecrawl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsparihar%2Fmcp-server-firecrawl/lists"}