{"id":29273459,"url":"https://github.com/mseri/snf-mcp","last_synced_at":"2025-07-05T02:36:09.022Z","repository":{"id":301353482,"uuid":"1008982198","full_name":"mseri/snf-mcp","owner":"mseri","description":"A Model Context Protocol server for DuckDuckGo search and web content fetching (using ocaml-mcp by avsm)","archived":false,"fork":false,"pushed_at":"2025-06-29T09:30:59.000Z","size":135,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-29T09:37:02.191Z","etag":null,"topics":["fetch-tools","mcp-server","ocaml","search"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mseri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-26T11:47:39.000Z","updated_at":"2025-06-29T09:31:01.000Z","dependencies_parsed_at":"2025-06-29T09:37:04.852Z","dependency_job_id":null,"html_url":"https://github.com/mseri/snf-mcp","commit_stats":null,"previous_names":["mseri/ddg_mcp","mseri/snf-mcp"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mseri/snf-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mseri%2Fsnf-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mseri%2Fsnf-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mseri%2Fsnf-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mseri%2Fsnf-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mseri","download_url":"https://codeload.github.com/mseri/snf-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mseri%2Fsnf-mcp/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263671856,"owners_count":23494053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fetch-tools","mcp-server","ocaml","search"],"created_at":"2025-07-05T02:36:07.412Z","updated_at":"2025-07-05T02:36:08.974Z","avatar_url":"https://github.com/mseri.png","language":"OCaml","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web fetch and search MCP Server\n\nA Model Context Protocol (MCP) server that provides web search, Wikipedia search, and web content fetching capabilities, written in OCaml using the eio asynchronous runtime.\n\n## Features\n\n- **DuckDuckGo Search**: Search the web using DuckDuckGo's search engine\n- **Wikipedia Search**: Search Wikipedia for articles and content\n- **Web Content Fetching**: Fetch and parse content from web pages, with support for both cleaned-up HTML and Markdown formats\n- **Rate Limiting**: Built-in rate limiting to respect service limits\n- **MCP Protocol**: Fully compatible with the Model Context Protocol specification (vendoring \u003chttps://tangled.sh/@anil.recoil.org/ocaml-mcp/\u003e)\n- **Asynchronous**: Built on Eio for efficient concurrent operations\n\n## Tools Provided\n\n### `search`\nSearch DuckDuckGo and return formatted results.\n\n**Parameters:**\n- `query` (string, required): The search query string\n- `max_results` (integer, optional): Maximum number of results to return (default: 10)\n\n**Example:**\n```json\n{\n  \"query\": \"OCaml programming language\",\n  \"max_results\": 5\n}\n```\n\n### `search_wikipedia`\nSearch Wikipedia and return formatted results.\n\n**Parameters:**\n- `query` (string, required): The search query string\n- `max_results` (integer, optional): Maximum number of results to return (default: 10)\n\n**Example:**\n```json\n{\n  \"query\": \"OCaml programming language\",\n  \"max_results\": 5\n}\n```\n\n### `fetch_content`\nFetch and parse content from a webpage URL.\n\n**Parameters:**\n- `url` (string, required): The webpage URL to fetch content from\n- `max_length` (integer, optional): Maximum length (in bytes) of content to return (default: 8192). Set `-1` to disable length limit.\n\n**Example:**\n```json\n{\n  \"url\": \"https://example.com/article\",\n  \"max_length\": 16384\n}\n```\n\n### `fetch_markdown`\nFetch and parse content from a webpage URL as Markdown.\n\n**Parameters:**\n- `url` (string, required): The webpage URL to fetch content from\n- `max_length` (integer, optional): Maximum length (in bytes) of content to return (default: 8192). Set `-1` to disable length limit.\n\n**Example:**\n```json\n{\n  \"url\": \"https://example.com/article\",\n  \"max_length\": 16384\n}\n```\n\n## Usage\n\n### Running the Server\n\nThe `snf-mcp` binary supports two modes of operation:\n\n1. **HTTP Server Mode** (default): Listens on a network port\n2. **Standard I/O Mode**: Communicates through stdin/stdout\n\nCaveat: the installed binary is called `snf-mcp`\n\nStart the MCP server in HTTP mode on port 3000:\n```bash\ndune exec snf-mcp -- --serve 3000\n```\n\nUse Standard I/O mode (useful for integrating with LLM clients):\n```bash\ndune exec snf-mcp\n```\n\nWhen installed via OPAM, you can run it directly:\n```bash\nsnf_mcp [--serve PORT | --stdio]\n  --serve  Run http server, listening on PORT\n  --stdio  Use stdio for communication instead of port (default)\n  --debug  Enable debug logging\n  --verbose  Enable verbose logging\n  --quiet  Suppress non-error logs (default)\n  -help  Display this list of options\n  --help  Display this list of options\n```\n\n### Testing the Server\n\n#### HTTP Mode\n\nWhen running in HTTP mode, you can test if the server is working by sending MCP protocol messages using curl.\n\nFirst start the server with:\n```bash\ndune exec snf-mcp --serve 8080\n```\nThen, on a different terminal, you can use `curl` to interact with the server. Here are some example requests:\n\n**List available tools:**\n```bash\ncurl -X POST http://localhost:8080 -H \"Content-Type: application/json\" -d '{\n  \"jsonrpc\": \"2.0\",\n  \"id\": 1,\n  \"method\": \"tools/list\"\n}'\n```\n\n**Perform a search:**\n```bash\ncurl -X POST http://localhost:8080 -H \"Content-Type: application/json\" -d '{\n  \"jsonrpc\": \"2.0\",\n  \"id\": 2,\n  \"method\": \"tools/call\",\n  \"params\": {\n    \"name\": \"search\",\n    \"arguments\": {\n      \"query\": \"OCaml programming language\",\n      \"max_results\": 3\n    }\n  }\n}'\n```\n\n**Fetch webpage content:**\n```bash\ncurl -X POST http://localhost:8080 -H \"Content-Type: application/json\" -d '{\n  \"jsonrpc\": \"2.0\",\n  \"id\": 3,\n  \"method\": \"tools/call\",\n  \"params\": {\n    \"name\": \"fetch_content\",\n    \"arguments\": {\n      \"url\": \"https://ocaml.org\"\n    }\n  }\n}'\n```\n\n**Search Wikipedia:**\n```bash\ncurl -X POST http://localhost:8080 -H \"Content-Type: application/json\" -d '{\n  \"jsonrpc\": \"2.0\",\n  \"id\": 4,\n  \"method\": \"tools/call\",\n  \"params\": {\n    \"name\": \"search_wikipedia\",\n    \"arguments\": {\n      \"query\": \"OCaml programming language\",\n      \"max_results\": 3\n    }\n  }\n}'\n```\n\n**Fetch webpage content as Markdown:**\n```bash\ncurl -X POST http://localhost:8080 -H \"Content-Type: application/json\" -d '{\n  \"jsonrpc\": \"2.0\",\n  \"id\": 5,\n  \"method\": \"tools/call\",\n  \"params\": {\n    \"name\": \"fetch_markdown\",\n    \"arguments\": {\n      \"url\": \"https://ocaml.org\"\n    }\n  }\n}'\n```\n\n#### Standard I/O Mode\n\nWhen using stdio mode, you can pipe JSON-RPC requests to the binary:\n\n```bash\necho '{\"jsonrpc\":\"2.0\",\"method\":\"tools/list\",\"id\":1}' | dune exec snf-mcp | jq\n```\n\n```bash\necho '{\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"search\",\"arguments\":{\"query\":\"OCaml programming language\"}},\"id\":2}' | dune exec snf-mcp | jq\n```\n\n```bash\necho '{\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"search_wikipedia\",\"arguments\":{\"query\":\"OCaml programming language\"}},\"id\":3}' | dune exec snf-mcp | jq\n```\n\n```bash\necho '{\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"fetch_content\",\"arguments\":{\"url\":\"https://ocaml.org\"}},\"id\":4}' | dune exec snf-mcp | jq\n```\n\n```bash\necho '{\"jsonrpc\":\"2.0\",\"method\":\"tools/call\",\"params\":{\"name\":\"fetch_markdown\",\"arguments\":{\"url\":\"https://ocaml.org\"}},\"id\":5}' | dune exec snf-mcp | jq\n```\n\nThis mode is particularly useful when integrating with LLM clients that communicate over stdin/stdout.\n\n## Installation\n\n### Build from Source\n\n1. Clone the repository\n2. Install dependencies and build:\n```bash\n$ cd snf_mcp\n$ opam install . --deps-only\n$ dune build\n$ dune install\n```\n\nThis will make the `snf-mcp` binary available in your PATH.\n\n### Integration with MCP Clients\n\nThis server can be integrated with any MCP-compatible client. Configure your client to connect to this server using the appropriate transport method.\nBelow we show how to configure the stdio version, the remote version is very similar.\nKeep in mind that this is early software and not recommended for production or to be exposed on unprotected networks.\n\n### LLM CLI\n\nInstall the [`llm-tools-mcp` plugin](https://github.com/VirtusLab/llm-tools-mcp) with\n```\nllm install llm-tools-mcp\n```\nthen edit (or create) `~/.llm-tools-mcp/mcp.json` with\n```\n{\n  \"mcpServers\": {\n    \"snf_mcp\": {\n      \"command\": \"/path/to/snf-mcp\",\n      \"args\": [\n        \"--stdio\"\n      ]\n    }\n  }\n}\n```\n\n#### LMStudio\n\nEdit the json file from the interface adding the same json entry as in the LLM CLI example above.\nSee also the [official documentation](https://lmstudio.ai/docs/app/plugins/mcp).\n\n#### Jan\n\nUse the _full path_ to snf_mcp as command, and `--stdio` as the only argument.\nSee also the [official documentation](https://jan.ai/docs/mcp).\n\nNote, _I was only able to configure stdio-based mcp servers_ with Jan.\n\n### Rate Limiting\n\nThe server implements rate limiting to be respectful to external services:\n- **Search requests (DuckDuckGo and Wikipedia)**: Limited to 30 requests per minute\n- **Content fetching**: Limited to 20 requests per minute\n\n## Troubleshooting\n\n### Rate Limiting Issues\n\nIf you encounter errors or timeout messages, you might be hitting the rate limits. The server will automatically wait when rate limits are reached, but external services might still block requests if they detect automated usage.\n\n### Search Quality\n\nDuckDuckGo's search results are parsed from the HTML response. If search results appear incorrect or incomplete, it might be due to:\n1. DuckDuckGo changing their HTML structure\n2. Bot detection preventing proper results\n3. Issues with the search query format\n\nTry rephrasing your query or checking if DuckDuckGo's service is functioning normally.\n\n### Content Extraction Quality\n\nThe `fetch_markdown` tool tries to use the `trafilatura` Python library if it's available on your system, as it produces higher quality text extraction. If `trafilatura` is not found, it falls back to [jina reader](https://jina.ai/reader/).\n\n\nFor best results, consider installing `trafilatura`, for example in one of the following 3 ways:\n\n```bash\nuv tool install trafilatura # Method 1: Using `uv` tool\npipx install trafilatura # Method 2: Using `pipx`\npip install trafilatura # Method 3: Using `pip`\n```\n\n# TODO\n- Use pagination in the fetch\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmseri%2Fsnf-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmseri%2Fsnf-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmseri%2Fsnf-mcp/lists"}