{"id":28316165,"url":"https://github.com/pinkpixel-dev/web-scout-mcp","last_synced_at":"2026-03-06T10:32:10.298Z","repository":{"id":292597518,"uuid":"981277142","full_name":"pinkpixel-dev/web-scout-mcp","owner":"pinkpixel-dev","description":"A powerful MCP server extension providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content programmatically.","archived":false,"fork":false,"pushed_at":"2026-02-13T22:18:28.000Z","size":8961,"stargazers_count":116,"open_issues_count":1,"forks_count":11,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-14T04:44:06.408Z","etag":null,"topics":["ai-assistant","ai-tools","cheerio","content-extraction","crawler","duckduckgo","duckduckgo-search","google-search","mcp","mcp-server","web-content","web-crawler","web-scraper","web-scraping","web-search","web-search-agent"],"latest_commit_sha":null,"homepage":"https://pinkpixel.dev","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pinkpixel-dev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-10T18:29:09.000Z","updated_at":"2026-02-13T22:17:23.000Z","dependencies_parsed_at":"2025-06-17T12:26:11.149Z","dependency_job_id":"57f7d76a-0967-4703-9735-b5b7615467ed","html_url":"https://github.com/pinkpixel-dev/web-scout-mcp","commit_stats":null,"previous_names":["pinkpixel-dev/web-scout-mcp"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/pinkpixel-dev/web-scout-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinkpixel-dev%2Fweb-scout-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinkpixel-dev%2Fweb-scout-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinkpixel-dev%2Fweb-scout-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinkpixel-dev%2Fweb-scout-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pinkpixel-dev","download_url":"https://codeload.github.com/pinkpixel-dev/web-scout-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinkpixel-dev%2Fweb-scout-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30171869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T07:56:45.623Z","status":"ssl_error","status_checked_at":"2026-03-06T07:55:55.621Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-assistant","ai-tools","cheerio","content-extraction","crawler","duckduckgo","duckduckgo-search","google-search","mcp","mcp-server","web-content","web-crawler","web-scraper","web-scraping","web-search","web-search-agent"],"created_at":"2025-05-25T01:13:56.169Z","updated_at":"2026-03-06T10:32:10.275Z","avatar_url":"https://github.com/pinkpixel-dev.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![MseeP.ai Security Assessment Badge](https://mseep.net/pr/pinkpixel-dev-web-scout-mcp-badge.png)](https://mseep.ai/app/pinkpixel-dev-web-scout-mcp)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo.png\" alt=\"Web Scout MCP Logo\" width=\"300\"/\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eWeb Scout MCP Server\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@pinkpixel/web-scout-mcp\"\u003e\u003cimg src=\"https://img.shields.io/npm/v/@pinkpixel/web-scout-mcp.svg\" alt=\"npm version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/pinkpixel-dev/web-scout-mcp/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://nodejs.org/en/\"\u003e\u003cimg src=\"https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg\" alt=\"Node.js Version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://smithery.ai/badge/@pinkpixel-dev/web-scout-mcp\"\u003e\u003cimg alt=\"Smithery Badge\" src=\"https://smithery.ai/badge/@pinkpixel-dev/web-scout-mcp\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://mseep.ai/app/f19a6453-c635-4bc8-b26a-3e9e36428a98\"\u003e\u003cimg src=\"https://mseep.ai/badge.svg\" alt=\"Verified on MseeP\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  An MCP server for web search using DuckDuckGo and content extraction, with support for multiple URLs and memory optimizations.\n\u003c/p\u003e\n\n## ✨ Features\n\n- 🔍 **DuckDuckGo Search**: Fast and privacy-focused web search capability\n- 📄 **Content Extraction**: Clean, readable text extraction from web pages\n- 🚀 **Parallel Processing**: Support for extracting content from multiple URLs simultaneously\n- 💾 **Memory Optimization**: Smart memory management to prevent application crashes\n- ⏱️ **Rate Limiting**: Intelligent request throttling to avoid API blocks\n- 🛡️ **Error Handling**: Robust error handling for reliable operation\n\n## 📦 Installation\n\n### Installing via Smithery\n\nTo install Web Scout for Claude Desktop automatically via [Smithery](https://smithery.ai/server/@pinkpixel-dev/web-scout-mcp):\n\n```bash\nnpx -y @smithery/cli install @pinkpixel-dev/web-scout-mcp --client claude\n```\n\n### Global Installation\n\n```bash\nnpm install -g @pinkpixel/web-scout-mcp\n```\n\n### Local Installation\n\n```bash\nnpm install @pinkpixel/web-scout-mcp\n```\n\n## 🚀 Usage\n\n### Command Line\n\nAfter installing globally, run:\n\n```bash\nweb-scout-mcp\n```\n\n### With MCP Clients\n\nAdd this to your MCP client's `config.json` (Claude Desktop, Cursor, etc.):\n\n```json\n{\n  \"mcpServers\": {\n    \"web-scout\": {\n      \"command\": \"npx\",\n      \"args\": [\n        \"-y\",\n        \"@pinkpixel/web-scout-mcp@latest\"\n      ]\n    }\n  }\n}\n```\n\n### Environment Variables\n\nSet the `WEB_SCOUT_DISABLE_AUTOSTART=1` environment variable when embedding the package and calling `createServer()` yourself. By default running the published entrypoint (for example `node dist/index.js` or `npx @pinkpixel/web-scout-mcp`) automatically bootstraps the stdio transport.\n\n## 🧰 Tools\n\nThe server provides the following MCP tools:\n\n### 🔍 DuckDuckGoWebSearch\n\nInitiates a web search query using the DuckDuckGo search engine and returns a well-structured list of findings.\n\n**Input:**\n- `query` (string): The search query string\n- `maxResults` (number, optional): Maximum number of results to return (default: 10)\n\n**Example:**\n```json\n{\n  \"query\": \"latest advancements in AI\",\n  \"maxResults\": 5\n}\n```\n\n**Output:**\nA formatted list of search results with titles, URLs, and snippets.\n\n### 📄 UrlContentExtractor\n\nFetches and extracts clean, readable content from web pages by removing unnecessary elements like scripts, styles, and navigation.\n\n**Input:**\n- `url`: Either a single URL string or an array of URL strings\n\n**Example (single URL):**\n```json\n{\n  \"url\": \"https://example.com/article\"\n}\n```\n\n**Example (multiple URLs):**\n```json\n{\n  \"url\": [\n    \"https://example.com/article1\",\n    \"https://example.com/article2\"\n  ]\n}\n```\n\n**Output:**\nExtracted text content from the specified URL(s).\n\n## 🛠️ Development\n\n```bash\n# Clone the repository\ngit clone https://github.com/pinkpixel-dev/web-scout-mcp.git\ncd web-scout-mcp\n\n# Install dependencies\nnpm install\n\n# Build\nnpm run build\n\n# Run\nnpm start\n```\n\n## 📚 Documentation\n\nFor more detailed information about the project, check out these resources:\n\n- [OVERVIEW.md](OVERVIEW.md) - Technical overview and architecture\n- [CONTRIBUTING.md](CONTRIBUTING.md) - Guidelines for contributors\n- [CHANGELOG.md](CHANGELOG.md) - Version history and changes\n\n## 📋 Requirements\n\n- Node.js \u003e= 18.0.0\n- npm or yarn\n\n## 📄 License\n\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n\n\u003cp align=\"center\"\u003e\n  \u003csub\u003eMade with ❤️ by \u003ca href=\"https://pinkpixel.dev\"\u003ePink Pixel\u003c/a\u003e\u003c/sub\u003e\n  \u003cbr\u003e\n  \u003csub\u003e✨ Dream it, Pixel it ✨\u003c/sub\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinkpixel-dev%2Fweb-scout-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpinkpixel-dev%2Fweb-scout-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinkpixel-dev%2Fweb-scout-mcp/lists"}