{"id":26570940,"url":"https://github.com/jae-jae/fetcher-mcp","last_synced_at":"2026-08-27T17:05:33.235Z","repository":{"id":283295697,"uuid":"951224571","full_name":"jae-jae/fetcher-mcp","owner":"jae-jae","description":"MCP server for fetch web page content using Playwright headless browser.","archived":false,"fork":false,"pushed_at":"2026-01-14T07:07:21.000Z","size":128,"stargazers_count":1073,"open_issues_count":10,"forks_count":101,"subscribers_count":6,"default_branch":"main","last_synced_at":"2026-08-16T00:35:24.760Z","etag":null,"topics":["ai","mcp","playwright"],"latest_commit_sha":null,"homepage":"https://1team.top","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jae-jae.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"claude":null,"gemini":null,"cursor":null,"copilot":null,"dco":null,"cla":null,"disclosure":null}},"created_at":"2025-03-19T10:54:49.000Z","updated_at":"2026-08-14T07:24:58.000Z","dependencies_parsed_at":"2026-08-16T00:37:44.940Z","dependency_job_id":null,"html_url":"https://github.com/jae-jae/fetcher-mcp","commit_stats":null,"previous_names":["jae-jae/fetch-mcp","jae-jae/fetcher-mcp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jae-jae/fetcher-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jae-jae%2Ffetcher-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jae-jae%2Ffetcher-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jae-jae%2Ffetcher-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jae-jae%2Ffetcher-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jae-jae","download_url":"https://codeload.github.com/jae-jae/fetcher-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jae-jae%2Ffetcher-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36944935,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-08-22T15:14:58.755Z","status":"online","status_checked_at":"2026-08-27T02:00:07.166Z","response_time":96,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","mcp","playwright"],"created_at":"2025-03-22T22:19:23.441Z","updated_at":"2026-08-27T17:05:33.230Z","avatar_url":"https://github.com/jae-jae.png","language":"TypeScript","funding_links":[],"categories":["Web Scraping","Data Access \u0026 Integration MCP Servers","MCP 服务器精选列表","📚 Projects (1974 total)","Legend","پیاده‌سازی‌های سرور","🤖 AI/ML","TypeScript","Browser Automation","Servers","カテゴリ","Table of Contents","MCP Servers","Search \u0026 Data Extraction","🔎 Search \u0026 Data Extraction (83 servers)","Server Implementations"],"sub_categories":["SIEM \u0026 SecOps","🌐 浏览器自动化与网页交互","MCP Servers","🔎 \u003ca name=\"search\"\u003e\u003c/a\u003eSearch","🔎 \u003ca name=\"search\"\u003e\u003c/a\u003eجستجو و استخراج داده","How to Submit","Web Browsing \u0026 Scraping","🕸️ \u003ca name=\"web-scraping--collection\"\u003e\u003c/a\u003eWebスクレイピング・収集","Browser Automation","🔎 \u003ca name=\"search\"\u003e\u003c/a\u003eSearch \u0026 Data Extraction"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/jae-jae/fetcher-mcp/refs/heads/main/icon.svg\" width=\"100\" height=\"100\" alt=\"Fetcher MCP Icon\" /\u003e\n\u003c/div\u003e\n\n[中文](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=zh) |\n[Deutsch](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=de) |\n[Español](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=es) |\n[français](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=fr) |\n[日本語](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ja) |\n[한국어](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ko) |\n[Português](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=pt) |\n[Русский](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ru)\n\n# Fetcher MCP\n\nMCP server for fetch web page content using Playwright headless browser.\n\n\u003e 🌟 **Recommended**: [OllaMan](https://ollaman.com/) - Powerful Ollama AI Model Manager.\n\n## Advantages\n\n- **JavaScript Support**: Unlike traditional web scrapers, Fetcher MCP uses Playwright to execute JavaScript, making it capable of handling dynamic web content and modern web applications.\n\n- **Intelligent Content Extraction**: Built-in Readability algorithm automatically extracts the main content from web pages, removing ads, navigation, and other non-essential elements.\n\n- **Flexible Output Format**: Supports both HTML and Markdown output formats, making it easy to integrate with various downstream applications.\n\n- **Parallel Processing**: The `fetch_urls` tool enables concurrent fetching of multiple URLs, significantly improving efficiency for batch operations.\n\n- **Resource Optimization**: Automatically blocks unnecessary resources (images, stylesheets, fonts, media) to reduce bandwidth usage and improve performance.\n\n- **Robust Error Handling**: Comprehensive error handling and logging ensure reliable operation even when dealing with problematic web pages.\n\n- **Configurable Parameters**: Fine-grained control over timeouts, content extraction, and output formatting to suit different use cases.\n\n## Quick Start\n\nRun directly with npx:\n\n```bash\nnpx -y fetcher-mcp\n```\n\nFirst time setup - install the required browser by running the following command in your terminal:\n\n```bash\nnpx playwright install chromium\n```\n\n### HTTP and SSE Transport\n\nUse the `--transport=http` parameter to start both Streamable HTTP endpoint and SSE endpoint services simultaneously:\n\n```bash\nnpx -y fetcher-mcp --log --transport=http --host=0.0.0.0 --port=3000\n```\n\nAfter startup, the server provides the following endpoints:\n\n- `/mcp` - Streamable HTTP endpoint (modern MCP protocol)\n- `/sse` - SSE endpoint (legacy MCP protocol)\n\nClients can choose which method to connect based on their needs.\n\n### Debug Mode\n\nRun with the `--debug` option to show the browser window for debugging:\n\n```bash\nnpx -y fetcher-mcp --debug\n```\n\n## Configuration MCP\n\nConfigure this MCP server in Claude Desktop:\n\nOn MacOS: `~/Library/Application Support/Claude/claude_desktop_config.json`\n\nOn Windows: `%APPDATA%/Claude/claude_desktop_config.json`\n\n```json\n{\n  \"mcpServers\": {\n    \"fetcher\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"fetcher-mcp\"]\n    }\n  }\n}\n```\n\n## Docker Deployment\n\n### Running with Docker\n\n```bash\ndocker run -p 3000:3000 ghcr.io/jae-jae/fetcher-mcp:latest\n```\n\n### Deploying with Docker Compose\n\nCreate a `docker-compose.yml` file:\n\n```yaml\nversion: \"3.8\"\n\nservices:\n  fetcher-mcp:\n    image: ghcr.io/jae-jae/fetcher-mcp:latest\n    container_name: fetcher-mcp\n    restart: unless-stopped\n    ports:\n      - \"3000:3000\"\n    environment:\n      - NODE_ENV=production\n    # Using host network mode on Linux hosts can improve browser access efficiency\n    # network_mode: \"host\"\n    volumes:\n      # For Playwright, may need to share certain system paths\n      - /tmp:/tmp\n    # Health check\n    healthcheck:\n      test: [\"CMD\", \"wget\", \"--spider\", \"-q\", \"http://localhost:3000\"]\n      interval: 30s\n      timeout: 10s\n      retries: 3\n```\n\nThen run:\n\n```bash\ndocker-compose up -d\n```\n\n## Features\n\n- `fetch_url` - Retrieve web page content from a specified URL\n\n  - Uses Playwright headless browser to parse JavaScript\n  - Supports intelligent extraction of main content and conversion to Markdown\n  - Supports the following parameters:\n    - `url`: The URL of the web page to fetch (required parameter)\n    - `timeout`: Page loading timeout in milliseconds, default is 30000 (30 seconds)\n    - `waitUntil`: Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load'\n    - `extractContent`: Whether to intelligently extract the main content, default is true\n    - `maxLength`: Maximum length of returned content (in characters), default is no limit\n    - `returnHtml`: Whether to return HTML content instead of Markdown, default is false\n    - `waitForNavigation`: Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false\n    - `navigationTimeout`: Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds)\n    - `disableMedia`: Whether to disable media resources (images, stylesheets, fonts, media), default is true\n    - `debug`: Whether to enable debug mode (showing browser window), overrides the --debug command line flag if specified\n\n- `fetch_urls` - Batch retrieve web page content from multiple URLs in parallel\n  - Uses multi-tab parallel fetching for improved performance\n  - Returns combined results with clear separation between webpages\n  - Supports the following parameters:\n    - `urls`: Array of URLs to fetch (required parameter)\n    - Other parameters are the same as `fetch_url`\n\n- `browser_install` - Install Playwright Chromium browser binary automatically\n\n  - Installs required Chromium browser binary when not available\n  - Automatically suggested when browser installation errors occur\n  - Supports the following parameters:\n    - `withDeps`: Install system dependencies required by Chromium browser, default is false\n    - `force`: Force installation even if Chromium is already installed, default is false\n\n## Tips\n\n### Handling Special Website Scenarios\n\n#### Dealing with Anti-Crawler Mechanisms\n\n- **Wait for Complete Loading**: For websites using CAPTCHA, redirects, or other verification mechanisms, include in your prompt:\n\n  ```\n  Please wait for the page to fully load\n  ```\n\n  This will use the `waitForNavigation: true` parameter.\n\n- **Increase Timeout Duration**: For websites that load slowly:\n  ```\n  Please set the page loading timeout to 60 seconds\n  ```\n  This adjusts both `timeout` and `navigationTimeout` parameters accordingly.\n\n#### Content Retrieval Adjustments\n\n- **Preserve Original HTML Structure**: When content extraction might fail:\n\n  ```\n  Please preserve the original HTML content\n  ```\n\n  Sets `extractContent: false` and `returnHtml: true`.\n\n- **Fetch Complete Page Content**: When extracted content is too limited:\n\n  ```\n  Please fetch the complete webpage content instead of just the main content\n  ```\n\n  Sets `extractContent: false`.\n\n- **Return Content as HTML**: When HTML format is needed instead of default Markdown:\n  ```\n  Please return the content in HTML format\n  ```\n  Sets `returnHtml: true`.\n\n### Debugging and Authentication\n\n#### Enabling Debug Mode\n\n- **Dynamic Debug Activation**: To display the browser window during a specific fetch operation:\n  ```\n  Please enable debug mode for this fetch operation\n  ```\n  This sets `debug: true` even if the server was started without the `--debug` flag.\n\n#### Using Custom Cookies for Authentication\n\n- **Manual Login**: To login using your own credentials:\n\n  ```\n  Please run in debug mode so I can manually log in to the website\n  ```\n\n  Sets `debug: true` or uses the `--debug` flag, keeping the browser window open for manual login.\n\n- **Interacting with Debug Browser**: When debug mode is enabled:\n\n  1. The browser window remains open\n  2. You can manually log into the website using your credentials\n  3. After login is complete, content will be fetched with your authenticated session\n\n- **Enable Debug for Specific Requests**: Even if the server is already running, you can enable debug mode for a specific request:\n  ```\n  Please enable debug mode for this authentication step\n  ```\n  Sets `debug: true` for this specific request only, opening the browser window for manual login.\n\n## Development\n\n### Install Dependencies\n\n```bash\nnpm install\n```\n\n### Install Playwright Browser\n\nInstall the browsers needed for Playwright:\n\n```bash\nnpm run install-browser\n```\n\n### Build the Server\n\n```bash\nnpm run build\n```\n\n## Debugging\n\nUse MCP Inspector for debugging:\n\n```bash\nnpm run inspector\n```\n\nYou can also enable visible browser mode for debugging:\n\n```bash\nnode build/index.js --debug\n```\n\n## Related Projects\n\n- [g-search-mcp](https://github.com/jae-jae/g-search-mcp): A powerful MCP server for Google search that enables parallel searching with multiple keywords simultaneously. Perfect for batch search operations and data collection.\n\n## License\n\nLicensed under the [MIT License](https://choosealicense.com/licenses/mit/)\n\n[![Powered by DartNode](https://dartnode.com/branding/DN-Open-Source-sm.png)](https://dartnode.com \"Powered by DartNode - Free VPS for Open Source\")\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjae-jae%2Ffetcher-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjae-jae%2Ffetcher-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjae-jae%2Ffetcher-mcp/lists"}