{"id":44346477,"url":"https://github.com/j0hanz/fetch-url-mcp","last_synced_at":"2026-04-01T20:11:11.724Z","repository":{"id":328015091,"uuid":"1113312608","full_name":"j0hanz/fetch-url-mcp","owner":"j0hanz","description":"A web content fetcher MCP server that converts HTML to clean, AI and human readable markdown.","archived":false,"fork":false,"pushed_at":"2026-03-28T20:35:10.000Z","size":5805,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-03-28T22:07:12.112Z","etag":null,"topics":["content-extraction","fetch","fetch-api","html-to-markdown","llm-context","mcp","mcp-server","model-context-protocol","readable-format","typescript","web-fetch","web-fetching","webscraping"],"latest_commit_sha":null,"homepage":"https://github.com/j0hanz/fetch-url-mcp#readme","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/j0hanz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-12-09T20:02:57.000Z","updated_at":"2026-03-28T20:35:13.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/j0hanz/fetch-url-mcp","commit_stats":null,"previous_names":["j0hanz/super-fetch-mcp-server","j0hanz/fetch-url-mcp"],"tags_count":71,"template":false,"template_full_name":null,"purl":"pkg:github/j0hanz/fetch-url-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j0hanz%2Ffetch-url-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j0hanz%2Ffetch-url-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j0hanz%2Ffetch-url-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j0hanz%2Ffetch-url-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/j0hanz","download_url":"https://codeload.github.com/j0hanz/fetch-url-mcp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/j0hanz%2Ffetch-url-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31168414,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-29T21:28:10.185Z","status":"ssl_error","status_checked_at":"2026-03-29T21:23:32.226Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["content-extraction","fetch","fetch-api","html-to-markdown","llm-context","mcp","mcp-server","model-context-protocol","readable-format","typescript","web-fetch","web-fetching","webscraping"],"created_at":"2026-02-11T14:04:53.388Z","updated_at":"2026-04-01T20:11:11.712Z","avatar_url":"https://github.com/j0hanz.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fetch URL MCP Server\n\n[![npm version](https://img.shields.io/npm/v/%40j0hanz%2Ffetch-url-mcp?style=flat-square\u0026logo=npm)](https://www.npmjs.com/package/%40j0hanz%2Ffetch-url-mcp) [![License](https://img.shields.io/badge/license-MIT-blue?style=flat-square)](#contributing-and-license)\n\n[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square\u0026logo=visualstudiocode\u0026logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url\u0026config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D) [![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square\u0026logo=visualstudiocode\u0026logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url\u0026config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D\u0026quality=insiders) [![Install in Visual Studio](https://img.shields.io/badge/Visual_Studio-Install_Server-C16FDE?logo=visualstudio\u0026logoColor=white)](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)\n\n[![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-light.svg)](https://lmstudio.ai/install-mcp?name=fetch-url\u0026config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D) [![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=fetch-url\u0026config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D) [![Install in Goose](https://block.github.io/goose/img/extension-install-dark.svg)](https://block.github.io/goose/extension?cmd=npx\u0026arg=-y\u0026arg=%40j0hanz%2Ffetch-url-mcp%40latest\u0026id=%40j0hanz%2Ffetch-url-mcp\u0026name=fetch-url\u0026description=fetch-url%20MCP%20server)\n\nAn MCP server that fetches web pages and converts them to clean, readable Markdown.\n\n## Overview\n\nThis server takes a URL, fetches the page, and strips away everything you don't need — navigation, sidebars, banners, scripts — leaving just the main content as Markdown. It's perfect for feeding into LLMs, giving them the distilled essence of a page without the noise. It also recognizes GitHub, GitLab, Bitbucket, and Gist URLs and rewrites them to fetch the raw content directly.\n\nBy default it runs over stdio. Pass `--http` if you need a proper HTTP endpoint with auth, rate limiting, TLS, and session support.\n\n## Key Features\n\n- **HTML to Markdown** — Turns any public web page into clean, readable Markdown with metadata like `title`, `url`, `contentSize`, and `truncated`.\n- **Smart URL handling** — Recognizes GitHub, GitLab, Bitbucket, and Gist page URLs and rewrites them to raw-content endpoints before fetching.\n- **Task mode** — Big or slow pages can run as async MCP tasks with progress updates, instead of blocking. In HTTP mode, tasks are bound to the authenticated caller rather than a single MCP session, so they can be resumed after reconnecting with the same credentials. Polling task state also exposes `progress` and `total` when available.\n- **Self-documenting** — Includes an `internal://instructions` resource and a `get-help` prompt so clients know how to use it.\n- **HTTP mode** — Optionally serves over Streamable HTTP with host/origin validation, bearer or OAuth auth, rate limiting, health checks, and TLS.\n\n## Web Client\n\nA browser-based client is available if you want to use the server without any MCP setup.\n\n**[Live app](https://fetch-url-client.vercel.app)** · [Source code](https://github.com/j0hanz/fetch-url)\n\n## Requirements\n\n- **Node.js** \u003e= 24\n- **Docker** (optional) — only needed if you want to run the container image\n\n## Quick Start\n\nAdd this to your MCP client config:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\n## Client Configuration\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in VS Code\u003c/b\u003e\u003c/summary\u003e\n\n[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square\u0026logo=visualstudiocode\u0026logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url\u0026config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D)\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n  \"servers\": {\n    \"fetch-url-mcp\": {\n      \"type\": \"stdio\",\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nOr install via CLI:\n\n```sh\ncode --add-mcp '{\"name\":\"fetch-url-mcp\",\"command\":\"npx\",\"args\":[\"-y\",\"@j0hanz/fetch-url-mcp@latest\"]}'\n```\n\nFor more info, see [VS Code MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in VS Code Insiders\u003c/b\u003e\u003c/summary\u003e\n\n[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square\u0026logo=visualstudiocode\u0026logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url\u0026config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D\u0026quality=insiders)\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n  \"servers\": {\n    \"fetch-url-mcp\": {\n      \"type\": \"stdio\",\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nOr install via CLI:\n\n```sh\ncode-insiders --add-mcp '{\"name\":\"fetch-url-mcp\",\"command\":\"npx\",\"args\":[\"-y\",\"@j0hanz/fetch-url-mcp@latest\"]}'\n```\n\nFor more info, see [VS Code Insiders MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Cursor\u003c/b\u003e\u003c/summary\u003e\n\n[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=fetch-url\u0026config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)\n\nAdd to `~/.cursor/mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Cursor MCP docs](https://docs.cursor.com/context/model-context-protocol).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Visual Studio\u003c/b\u003e\u003c/summary\u003e\n\n[![Install in Visual Studio](https://img.shields.io/badge/Visual_Studio-Install_Server-C16FDE?logo=visualstudio\u0026logoColor=white)](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)\n\nFor solution-scoped setup, add this to `.mcp.json` at the solution root:\n\n```json\n{\n  \"servers\": {\n    \"fetch-url-mcp\": {\n      \"type\": \"stdio\",\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Visual Studio MCP docs](https://learn.microsoft.com/en-us/visualstudio/ide/mcp-servers).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Goose\u003c/b\u003e\u003c/summary\u003e\n\n[![Install in Goose](https://block.github.io/goose/img/extension-install-dark.svg)](https://block.github.io/goose/extension?cmd=npx\u0026arg=-y\u0026arg=%40j0hanz%2Ffetch-url-mcp%40latest\u0026id=%40j0hanz%2Ffetch-url-mcp\u0026name=fetch-url\u0026description=A%20web%20content%20fetcher%20MCP%20server%20that%20converts%20HTML%20to%20clean%2C%20AI%20and%20human%20readable%20markdown.)\n\nAdd to `~/.config/goose/config.yaml` on macOS/Linux or `%APPDATA%\\Block\\goose\\config\\config.yaml` on Windows:\n\n```yaml\nextensions:\n  fetch-url-mcp:\n    name: fetch-url-mcp\n    cmd: npx\n    args: ['-y', '@j0hanz/fetch-url-mcp@latest']\n    enabled: true\n    type: stdio\n    timeout: 300\n```\n\nFor more info, see [Goose extension docs](https://block.github.io/goose/docs/getting-started/using-extensions/).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in LM Studio\u003c/b\u003e\u003c/summary\u003e\n\n[![Add to LM Studio](https://files.lmstudio.ai/deeplink/mcp-install-light.svg)](https://lmstudio.ai/install-mcp?name=fetch-url\u0026config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)\n\nAdd to `~/.lmstudio/mcp.json` on macOS/Linux or `%USERPROFILE%/.lmstudio/mcp.json` on Windows:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [LM Studio MCP docs](https://lmstudio.ai/docs/basics/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Claude Desktop\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Claude Desktop MCP docs](https://modelcontextprotocol.io/quickstart/user).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Claude Code\u003c/b\u003e\u003c/summary\u003e\n\nUse the CLI:\n\n```sh\nclaude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nFor project-scoped config, Claude Code writes `.mcp.json` with:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"],\n      \"env\": {}\n    }\n  }\n}\n```\n\nFor more info, see [Claude Code MCP docs](https://docs.anthropic.com/en/docs/claude-code/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Windsurf\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `~/.codeium/windsurf/mcp_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Windsurf MCP docs](https://docs.windsurf.com/windsurf/cascade/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Amp\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `~/.config/amp/settings.json` on macOS/Linux, `%USERPROFILE%\\.config\\amp\\settings.json` on Windows, or `.amp/settings.json` for workspace-scoped config:\n\n```json\n{\n  \"amp.mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nOr install via CLI:\n\n```sh\namp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nFor more info, see [Amp docs](https://ampcode.com/manual).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Cline\u003c/b\u003e\u003c/summary\u003e\n\nOpen the MCP Servers panel, choose `Configure MCP Servers`, and add this to `cline_mcp_settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Cline MCP docs](https://docs.cline.bot/mcp/configuring-mcp-servers).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Codex CLI\u003c/b\u003e\u003c/summary\u003e\n\nUse the CLI:\n\n```sh\ncodex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nOr add this to `~/.codex/config.toml` or project-scoped `.codex/config.toml`:\n\n```toml\n[mcp_servers.fetch-url-mcp]\ncommand = \"npx\"\nargs = [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n```\n\nFor more info, see [Codex MCP docs](https://developers.openai.com/codex/mcp/).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in GitHub Copilot\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n  \"servers\": {\n    \"fetch-url-mcp\": {\n      \"type\": \"stdio\",\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [GitHub Copilot MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Warp\u003c/b\u003e\u003c/summary\u003e\n\nOpen `Personal \u003e MCP Servers` in Warp, choose `+ Add`, and either add a CLI server with:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nOr paste this JSON snippet when using Warp's multi-server import flow:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Warp MCP docs](https://docs.warp.dev/features/warp-ai/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Kiro\u003c/b\u003e\u003c/summary\u003e\n\nUse Kiro's MCP Servers panel or the `Add to Kiro` install flow. Kiro stores workspace-scoped MCP config in `.kiro/settings/mcp.json` and user-scoped config in `~/.kiro/settings/mcp.json`.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Kiro MCP docs](https://kiro.dev/blog/unlock-your-development-productivity-with-kiro-and-mcp/).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Gemini CLI\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `~/.gemini/settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Gemini CLI MCP docs](https://google-gemini.github.io/gemini-cli/docs/tools/mcp-server.html).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Zed\u003c/b\u003e\u003c/summary\u003e\n\nAdd to `~/.config/zed/settings.json`:\n\n```json\n{\n  \"context_servers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"],\n      \"env\": {}\n    }\n  }\n}\n```\n\nFor more info, see [Zed MCP docs](https://zed.dev/docs/ai/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Augment\u003c/b\u003e\u003c/summary\u003e\n\nUse the Augment Settings panel and either add the server manually or choose `Import from JSON`:\n\n```json\n{\n  \"mcpServers\": {\n    \"fetch-url-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n    }\n  }\n}\n```\n\nFor more info, see [Augment MCP docs](https://docs.augmentcode.com/setup-augment/mcp).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Roo Code\u003c/b\u003e\u003c/summary\u003e\n\nUse Roo Code's MCP Servers UI or marketplace flow.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Roo Code docs](https://docs.roocode.com/).\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eInstall in Kilo Code\u003c/b\u003e\u003c/summary\u003e\n\nUse Kilo Code's MCP Servers UI or marketplace flow.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Kilo Code docs](https://kilocode.ai/docs).\n\n\u003c/details\u003e\n\n## Use Cases\n\n- **Documentation for LLMs** — Grab a docs page, blog post, or reference article as Markdown and pass it straight into a context window.\n- **Repository content** — Hand it a GitHub, GitLab, or Bitbucket URL and it resolves the raw content endpoint. Works with Gists too.\n- **Slow or large pages** — Task mode lets big fetches run in the background while sending monotonic progress updates back to the client, while `tasks/get` exposes the latest `statusMessage`, `progress`, and `total`.\n\n## Architecture\n\n```text\n[MCP Client]\n  ├─ stdio -\u003e `src/index.ts` -\u003e `startStdioServer()` -\u003e `createMcpServer()`\n  └─ HTTP (`--http`) -\u003e `src/index.ts` -\u003e `startHttpServer()` -\u003e HTTP dispatcher\n       ├─ `GET /health`\n       ├─ `GET /.well-known/oauth-protected-resource`\n       ├─ `GET /.well-known/oauth-protected-resource/mcp`\n       └─ `POST|GET|DELETE /mcp`\n\n`createMcpServer()`\n  ├─ registers tool: `fetch-url`\n  ├─ registers prompt: `get-help`\n  ├─ registers resources:\n  │    - `internal://instructions`\n  ├─ enables capabilities: logging, resources, prompts, tasks\n  └─ installs task handlers, log-level handling, and shutdown cleanup\n\n`fetch-url` execution\n  ├─ validate input with `fetchUrlInputSchema`\n  ├─ normalize URL and block local/private targets unless allowed\n  ├─ rewrite supported code-host URLs to raw endpoints when possible\n  ├─ fetch content via the shared pipeline\n  ├─ transform HTML into Markdown in the transform worker path\n  └─ validate `structuredContent` with `fetchUrlOutputSchema`\n```\n\n### Request Lifecycle\n\n```text\n[Client] -- initialize {protocolVersion, capabilities} --\u003e [Server]\n[Server] -- {protocolVersion, capabilities, serverInfo} --\u003e [Client]\n[Client] -- notifications/initialized --\u003e [Server]\n[Client] -- tools/call {name, arguments} --\u003e [Server]\n[Server] -- {content: [{type, text}], structuredContent?, isError?} --\u003e [Client]\n```\n\n## MCP Surface\n\n### Tools\n\n#### `fetch-url`\n\nTakes a URL and returns Markdown. Read-only — no JavaScript execution. Supports running as a background MCP task for large or slow pages. When task mode is used, `tasks/get` and `tasks/list` include `statusMessage`, `progress`, and `total` whenever progress has been reported.\n\n| Parameter | Type     | Required | Description                 |\n| --------- | -------- | -------- | --------------------------- |\n| `url`     | `string` | yes      | Target URL. Max 2048 chars. |\n\nYou get text content back by default. If output validation passes, the response also includes `structuredContent` with typed fields: `url`, `resolvedUrl`, `finalUrl`, `title`, `metadata`, `markdown`, `fetchedAt`, `contentSize`, and `truncated`. A `true` value for `truncated` means the content hit a server-side size limit.\n\nTo opt into progress updates, include `_meta.progressToken` in the tool call. The token may be a string or number. The server may then emit monotonic `notifications/progress` updates, and task mode reuses the same token until the task reaches a terminal state.\n\nTo run the tool in task mode, include `_meta[\"modelcontextprotocol.io/task\"] = { \"taskId\": \"\u003cclient-id\u003e\", \"keepAlive\": \u003cms\u003e }`. `tasks/result` returns output only after the task reaches `completed`. Task-linked progress notifications, task summaries, and final results include `_meta[\"modelcontextprotocol.io/related-task\"] = { \"taskId\": \"\u003cclient-id\u003e\" }`.\n\n```json\n{\n  \"method\": \"tools/call\",\n  \"params\": {\n    \"name\": \"fetch-url\",\n    \"arguments\": {\n      \"url\": \"https://example.com/docs\"\n    },\n    \"_meta\": {\n      \"progressToken\": 7\n    }\n  }\n}\n```\n\n```text\n1. [Client] -- tools/call {name: \"fetch-url\", arguments} --\u003e [Server]\n2. [Server] -- dispatch(\"fetch-url\") --\u003e [src/tools/fetch-url.ts]\n3. [Handler] -- validate(fetchUrlInputSchema) --\u003e normalize / fetch / transform\n4. [Handler] -- validate(fetchUrlOutputSchema) --\u003e assemble content + structuredContent\n5. [Server] -- result or tool error --\u003e [Client]\n```\n\n### Resources\n\n| Resource                     | URI                       | MIME Type       | Description                                  |\n| ---------------------------- | ------------------------- | --------------- | -------------------------------------------- |\n| `fetch-url-mcp-instructions` | `internal://instructions` | `text/markdown` | Guidance for using the Fetch URL MCP server. |\n\n### Prompts\n\n| Prompt     | Arguments | Description                                                                     |\n| ---------- | --------- | ------------------------------------------------------------------------------- |\n| `get-help` | none      | Return Fetch URL server instructions: workflows, task mode, and error handling. |\n\n## MCP Capabilities\n\n| Capability                      | Status    | Notes                                                                                                                                                                                                          |\n| ------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| completions                     | confirmed | Advertised in `createServerCapabilities()`.                                                                                                                                                                    |\n| logging                         | confirmed | Advertised in `createServerCapabilities()` and handled through `SetLevelRequestSchema`.                                                                                                                        |\n| resources subscribe/listChanged | confirmed | Advertised in `createServerCapabilities()`.                                                                                                                                                                    |\n| prompts                         | confirmed | `get-help` is registered during server startup.                                                                                                                                                                |\n| tasks                           | confirmed | Advertised in `createServerCapabilities()` and backed by registered task handlers plus optional tool task support.                                                                                             |\n| progress notifications          | confirmed | Opt-in via `_meta.progressToken`. Tool execution reports monotonic `notifications/progress` updates during fetch and transform stages, and task-mode progress reuses the caller's token for the task lifetime. |\n\n### Tool Annotations\n\n| Annotation        | Value   |\n| ----------------- | ------- |\n| `readOnlyHint`    | `true`  |\n| `destructiveHint` | `false` |\n| `idempotentHint`  | `true`  |\n| `openWorldHint`   | `true`  |\n\n### Structured Output\n\nThe tool declares an `outputSchema` and includes `structuredContent` in the response when validation passes. Clients that support structured output get typed data directly; the rest use the text fallback.\n\n## Configuration\n\nAll configuration is through environment variables. For basic stdio usage, nothing needs to be set.\n\n### HTTP Server\n\n| Variable                              | Default     | Notes                                                                 |\n| ------------------------------------- | ----------- | --------------------------------------------------------------------- |\n| `HOST`                                | `127.0.0.1` | Bind address. Non-loopback bindings also require `ALLOW_REMOTE=true`. |\n| `PORT`                                | `3000`      | Listening port for `--http`.                                          |\n| `ALLOW_REMOTE`                        | `false`     | Must be enabled to bind to a non-loopback interface.                  |\n| `ALLOWED_HOSTS`                       | empty       | Additional allowed `Host` and `Origin` values.                        |\n| `SERVER_MAX_CONNECTIONS`              | `0`         | Optional connection cap.                                              |\n| `SERVER_HEADERS_TIMEOUT_MS`           | unset       | Optional Node server tuning.                                          |\n| `SERVER_REQUEST_TIMEOUT_MS`           | unset       | Optional Node server tuning.                                          |\n| `SERVER_KEEP_ALIVE_TIMEOUT_MS`        | unset       | Optional keep-alive tuning.                                           |\n| `SERVER_KEEP_ALIVE_TIMEOUT_BUFFER_MS` | unset       | Optional keep-alive tuning buffer.                                    |\n| `SERVER_MAX_HEADERS_COUNT`            | unset       | Optional header count limit.                                          |\n| `SERVER_BLOCK_PRIVATE_CONNECTIONS`    | `false`     | Enables inbound private-network protections.                          |\n\n### Authentication \u0026 OAuth\n\n| Variable                  | Default | Notes                                                       |\n| ------------------------- | ------- | ----------------------------------------------------------- |\n| `ACCESS_TOKENS`           | unset   | Comma- or space-separated static bearer tokens.             |\n| `API_KEY`                 | unset   | Alternate static token source for header auth.              |\n| `OAUTH_ISSUER_URL`        | unset   | Enables OAuth mode when combined with the other OAuth URLs. |\n| `OAUTH_AUTHORIZATION_URL` | unset   | Optional explicit authorization endpoint.                   |\n| `OAUTH_TOKEN_URL`         | unset   | Optional explicit token endpoint.                           |\n| `OAUTH_REVOCATION_URL`    | unset   | Optional OAuth revocation endpoint.                         |\n| `OAUTH_REGISTRATION_URL`  | unset   | Optional OAuth dynamic client registration endpoint.        |\n| `OAUTH_INTROSPECTION_URL` | unset   | Required for OAuth token introspection.                     |\n| `OAUTH_REQUIRED_SCOPES`   | empty   | Required scopes enforced after auth.                        |\n| `OAUTH_CLIENT_ID`         | unset   | Optional introspection client ID.                           |\n| `OAUTH_CLIENT_SECRET`     | unset   | Optional introspection client secret.                       |\n\n### TLS\n\n| Variable               | Default | Notes                                                       |\n| ---------------------- | ------- | ----------------------------------------------------------- |\n| `SERVER_TLS_KEY_FILE`  | unset   | Enable HTTPS when set together with `SERVER_TLS_CERT_FILE`. |\n| `SERVER_TLS_CERT_FILE` | unset   | TLS certificate path.                                       |\n| `SERVER_TLS_CA_FILE`   | unset   | Optional custom CA bundle.                                  |\n\n### Fetching\n\n| Variable            | Default                   | Notes                                              |\n| ------------------- | ------------------------- | -------------------------------------------------- |\n| `ALLOW_LOCAL_FETCH` | `false`                   | Allows loopback and private-network fetch targets. |\n| `FETCH_TIMEOUT_MS`  | `15000`                   | Network fetch timeout in milliseconds.             |\n| `USER_AGENT`        | `fetch-url-mcp/\u003cversion\u003e` | Override the outbound user agent string.           |\n\n### Tool Output\n\n| Variable                   | Default | Notes                                          |\n| -------------------------- | ------- | ---------------------------------------------- |\n| `MAX_INLINE_CONTENT_CHARS` | `0`     | `0` means no explicit inline truncation limit. |\n\n### Tasks\n\n| Variable                     | Default | Notes                                                                                |\n| ---------------------------- | ------- | ------------------------------------------------------------------------------------ |\n| `TASKS_MAX_TOTAL`            | `5000`  | Total retained task capacity, including completed/cancelled tasks until they expire. |\n| `TASKS_MAX_PER_OWNER`        | `1000`  | Per-owner retained task cap, clamped to the total cap.                               |\n| `TASKS_STATUS_NOTIFICATIONS` | `false` | Enables status notifications for tasks.                                              |\n| `TASKS_REQUIRE_INTERCEPTION` | `true`  | Requires interception for task-capable tool execution.                               |\n\n### Transform Workers\n\n| Variable                                   | Default   | Notes                                 |\n| ------------------------------------------ | --------- | ------------------------------------- |\n| `TRANSFORM_CANCEL_ACK_TIMEOUT_MS`          | `200`     | Cancellation acknowledgement timeout. |\n| `TRANSFORM_WORKER_MODE`                    | `threads` | Worker execution mode.                |\n| `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB`   | unset     | Optional worker memory limit.         |\n| `TRANSFORM_WORKER_MAX_YOUNG_GENERATION_MB` | unset     | Optional worker memory limit.         |\n| `TRANSFORM_WORKER_CODE_RANGE_MB`           | unset     | Optional worker memory limit.         |\n| `TRANSFORM_WORKER_STACK_MB`                | unset     | Optional worker stack size.           |\n\n### Content Cleanup\n\n| Variable                              | Default        | Notes                                      |\n| ------------------------------------- | -------------- | ------------------------------------------ |\n| `FETCH_URL_MCP_EXTRA_NOISE_TOKENS`    | empty          | Extra noise-removal tokens.                |\n| `FETCH_URL_MCP_EXTRA_NOISE_SELECTORS` | empty          | Extra DOM selectors for noise removal.     |\n| `FETCH_URL_MCP_LOCALE`                | system default | Locale override for extraction heuristics. |\n| `MARKDOWN_HEADING_KEYWORDS`           | built-in list  | Override heading keywords used by cleanup. |\n\n### Logging\n\n| Variable     | Default | Notes                                |\n| ------------ | ------- | ------------------------------------ |\n| `LOG_LEVEL`  | `info`  | `debug`, `info`, `warn`, or `error`. |\n| `LOG_FORMAT` | `text`  | Set to `json` for structured logs.   |\n\n## HTTP Endpoints\n\n| Method   | Path                                        | Auth                                       | Purpose                                                 |\n| -------- | ------------------------------------------- | ------------------------------------------ | ------------------------------------------------------- |\n| `GET`    | `/health`                                   | no, unless `?verbose=1` on a remote server | Basic health response, with optional diagnostics.       |\n| `GET`    | `/.well-known/oauth-protected-resource`     | no                                         | OAuth protected-resource metadata.                      |\n| `GET`    | `/.well-known/oauth-protected-resource/mcp` | no                                         | OAuth protected-resource metadata for the MCP endpoint. |\n| `POST`   | `/mcp`                                      | yes                                        | Session initialization and JSON-RPC requests.           |\n| `GET`    | `/mcp`                                      | yes                                        | Session-bound server-to-client stream handling.         |\n| `DELETE` | `/mcp`                                      | yes                                        | Session shutdown.                                       |\n\n## Security\n\n| Control                    | Status      | Notes                                                                                                                                    |\n| -------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------- |\n| Host and origin validation | implemented | HTTP requests are rejected unless `Host` and `Origin` match the allowlist built from loopback, the configured host, and `ALLOWED_HOSTS`. |\n| Authentication             | implemented | HTTP mode supports static bearer tokens locally or OAuth token introspection; remote bindings require OAuth.                             |\n| Protocol version checks    | implemented | Session-bound MCP HTTP requests validate `MCP-Protocol-Version` and pin it to the negotiated session version.                            |\n| Rate limiting              | implemented | Requests pass through the HTTP rate limiter before route dispatch.                                                                       |\n| Outbound SSRF protections  | implemented | Local/private IPs, metadata endpoints, and `.local`/`.internal` hosts are blocked unless `ALLOW_LOCAL_FETCH=true`.                       |\n| TLS                        | optional    | HTTPS is enabled when both TLS key and certificate files are configured.                                                                 |\n| Stdio logging safety       | implemented | Server logs are written to stderr, not stdout, so stdio MCP traffic stays clean.                                                         |\n\n## Development\n\n### Essential Commands\n\n| Command              | Description                                       |\n| -------------------- | ------------------------------------------------- |\n| `npm run build`      | Clean, compile TypeScript, copy assets.           |\n| `npm run dev`        | Watch mode TypeScript compilation.                |\n| `npm run dev:run`    | Run the server with `--watch` and `.env` support. |\n| `npm start`          | Start the compiled server.                        |\n| `npm test`           | Run the full test suite.                          |\n| `npm run lint`       | Lint with ESLint.                                 |\n| `npm run lint:fix`   | Auto-fix lint issues.                             |\n| `npm run type-check` | Type-check source and tests.                      |\n| `npm run format`     | Format with Prettier.                             |\n| `npm run inspector`  | Build and launch MCP Inspector.                   |\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eAll npm scripts\u003c/b\u003e\u003c/summary\u003e\n\n| Script                   | Command                                                                                                             |\n| ------------------------ | ------------------------------------------------------------------------------------------------------------------- |\n| `clean`                  | `node scripts/tasks.mjs clean`                                                                                      |\n| `build`                  | `node scripts/tasks.mjs build`                                                                                      |\n| `copy:assets`            | `node scripts/tasks.mjs copy:assets`                                                                                |\n| `prepare`                | `npm run build`                                                                                                     |\n| `dev`                    | `tsc --watch --preserveWatchOutput`                                                                                 |\n| `dev:run`                | `node --env-file=.env --watch dist/index.js`                                                                        |\n| `start`                  | `node dist/index.js`                                                                                                |\n| `format`                 | `prettier --write .`                                                                                                |\n| `type-check`             | `node scripts/tasks.mjs type-check`                                                                                 |\n| `type-check:src`         | `node node_modules/typescript/bin/tsc -p tsconfig.json --noEmit`                                                    |\n| `type-check:tests`       | `node node_modules/typescript/bin/tsc -p tsconfig.test.json --noEmit`                                               |\n| `type-check:diagnostics` | `tsc --noEmit --extendedDiagnostics`                                                                                |\n| `type-check:trace`       | `node -e \"require('fs').rmSync('.ts-trace',{recursive:true,force:true})\" \u0026\u0026 tsc --noEmit --generateTrace .ts-trace` |\n| `lint`                   | `eslint .`                                                                                                          |\n| `lint:tests`             | `eslint src/__tests__`                                                                                              |\n| `lint:fix`               | `eslint . --fix`                                                                                                    |\n| `test`                   | `node scripts/tasks.mjs test`                                                                                       |\n| `test:fast`              | `node --test --import tsx/esm src/__tests__/**/*.test.ts node-tests/**/*.test.ts`                                   |\n| `test:coverage`          | `node scripts/tasks.mjs test --coverage`                                                                            |\n| `knip`                   | `knip`                                                                                                              |\n| `knip:fix`               | `knip --fix`                                                                                                        |\n| `inspector`              | `npm run build \u0026\u0026 npx -y @modelcontextprotocol/inspector node dist/index.js --stdio`                                |\n| `prepublishOnly`         | `npm run lint \u0026\u0026 npm run type-check \u0026\u0026 npm run build`                                                               |\n\n\u003c/details\u003e\n\n## Build and Release\n\n- `npm run prepublishOnly` runs lint, type-check, and build as a single release gate.\n- CI workflows are under `.github/workflows/`.\n- `Dockerfile` and `docker-compose.yml` are included for containerized runs.\n- Published on npm as [`@j0hanz/fetch-url-mcp`](https://www.npmjs.com/package/@j0hanz/fetch-url-mcp).\n\n## Troubleshooting\n\n| Symptom                                       | Likely Cause                        | Fix                                                                           |\n| --------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------- |\n| Server output mixes with MCP traffic on stdio | Logs going to stdout                | Ensure all logging writes to stderr; the server does this by default.         |\n| HTTP mode returns `403`                       | Host/origin mismatch                | Add the domain to `ALLOWED_HOSTS` or verify loopback bindings.                |\n| HTTP mode returns `401`                       | Missing or invalid token            | Set `ACCESS_TOKENS` or configure OAuth env vars for remote bindings.          |\n| Fetch returns private-IP error                | SSRF protections blocked the target | Set `ALLOW_LOCAL_FETCH=true` if the target is intentionally local.            |\n| `truncated: true` in response                 | Content exceeded inline limits      | Increase `MAX_INLINE_CONTENT_CHARS` or accept truncated output.               |\n| Transform timeout or worker crash             | Large or complex HTML               | Tune `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB` or increase `FETCH_TIMEOUT_MS`. |\n| Client config not working                     | Wrong config format for the client  | Check the matching `\u003cdetails\u003e` block above — config keys vary by client.      |\n\n## Credits\n\n| Dependency                                                                           | Registry |\n| ------------------------------------------------------------------------------------ | -------- |\n| [@modelcontextprotocol/sdk](https://www.npmjs.com/package/@modelcontextprotocol/sdk) | npm      |\n| [@mozilla/readability](https://www.npmjs.com/package/@mozilla/readability)           | npm      |\n| [linkedom](https://www.npmjs.com/package/linkedom)                                   | npm      |\n| [node-html-markdown](https://www.npmjs.com/package/node-html-markdown)               | npm      |\n| [undici](https://www.npmjs.com/package/undici)                                       | npm      |\n| [zod](https://www.npmjs.com/package/zod)                                             | npm      |\n\n## Contributing and License\n\nPull requests welcome. Please make sure these pass before submitting:\n\n1. `npm run lint` and `npm run type-check`\n2. `npm test`\n3. `npm run format`\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj0hanz%2Ffetch-url-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fj0hanz%2Ffetch-url-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fj0hanz%2Ffetch-url-mcp/lists"}