{"id":47770814,"url":"https://github.com/ek-mc/brender","last_synced_at":"2026-04-03T09:18:37.255Z","repository":{"id":344035551,"uuid":"1179131443","full_name":"ek-mc/brender","owner":"ek-mc","description":"🕷️ A sleek wrapper for Cloudflare's Browser Rendering Crawl API. Crawl entire websites with a single command.","archived":false,"fork":false,"pushed_at":"2026-03-27T23:02:42.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-28T05:14:36.972Z","etag":null,"topics":["browser-rendering","cli","cloudflare","crawl-api","html","json","markdown","scraper","typescript","web-crawler"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/brender","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ek-mc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-11T18:08:40.000Z","updated_at":"2026-03-27T22:58:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ek-mc/brender","commit_stats":null,"previous_names":["ek-mc/brender"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/ek-mc/brender","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ek-mc%2Fbrender","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ek-mc%2Fbrender/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ek-mc%2Fbrender/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ek-mc%2Fbrender/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ek-mc","download_url":"https://codeload.github.com/ek-mc/brender/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ek-mc%2Fbrender/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31344863,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T08:03:20.796Z","status":"ssl_error","status_checked_at":"2026-04-03T08:00:37.834Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["browser-rendering","cli","cloudflare","crawl-api","html","json","markdown","scraper","typescript","web-crawler"],"created_at":"2026-04-03T09:18:35.301Z","updated_at":"2026-04-03T09:18:37.250Z","avatar_url":"https://github.com/ek-mc.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# brender 🕷️\n\n[![npm version](https://img.shields.io/npm/v/brender.svg)](https://www.npmjs.com/package/brender)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Node.js Version](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg)](https://nodejs.org/)\n[![CI](https://github.com/ek-mc/brender/actions/workflows/ci.yml/badge.svg)](https://github.com/ek-mc/brender/actions/workflows/ci.yml)\n\n\u003e **B**rowser **Render**ing - A sleek wrapper for Cloudflare's Browser Rendering Crawl API\n\nCrawl entire websites with a single command. Convert any site to Markdown, HTML, or structured JSON for RAG pipelines, AI training, or content migration.\n\n---\n\n## ✨ Features\n\n- 🚀 **Simple CLI** - One command to crawl any site\n- 📦 **Zero Dependencies** - Lightweight and fast\n- 📝 **Multiple Formats** - HTML, Markdown, or JSON output\n- 🔍 **Smart Discovery** - Automatic URL discovery from sitemaps \u0026 links\n- ⏱️ **Async Jobs** - Start crawl, check back later\n- 🎯 **Path Filtering** - Include/exclude specific URL patterns\n- 🔒 **Type-Safe** - Full TypeScript support\n\n---\n\n## 📦 Installation\n\n```bash\n# Install globally\nnpm install -g brender\n\n# Or use without installing\nnpx brender \u003curl\u003e\n```\n\n---\n\n## 🚀 Quick Start\n\n```bash\n# Set your Cloudflare credentials\nexport CF_ACCOUNT_ID=\"your-account-id\"\nexport CF_API_TOKEN=\"your-api-token\"\n\n# Crawl a website\nbrender https://blog.cloudflare.com\n\n# Crawl and wait for completion\nbrender https://example.com --wait\n\n# Get markdown output, limit to 50 pages\nbrender https://example.com --format markdown --limit 50 --output ./content.json\n```\n\n---\n\n## 📚 Usage\n\n### CLI Options\n\n```\nUsage: brender \u003curl\u003e [options]\n\nOptions:\n  --format \u003cformat\u003e      Output format: html, markdown, json (default: markdown)\n  --limit \u003cnumber\u003e       Max pages to crawl (default: 100)\n  --depth \u003cnumber\u003e       Crawl depth (default: 3)\n  --no-render            Don't use headless browser (faster for static sites)\n  --include \u003cpatterns\u003e   Comma-separated URL patterns to include\n  --exclude \u003cpatterns\u003e   Comma-separated URL patterns to exclude\n  --output \u003cfile\u003e        Save results to file\n  --wait                 Wait for job completion\n  --poll \u003cms\u003e            Poll interval in ms (default: 5000)\n  --timeout \u003cms\u003e         Timeout in ms (default: 300000)\n  --status \u003cjob-id\u003e      Check existing job status\n  --help                 Show help\n```\n\n### Examples\n\n```bash\n# Basic crawl\nbrender https://example.com\n\n# Crawl specific paths only\nbrender https://docs.example.com --include \"/api/*,/guides/*\"\n\n# Skip certain paths\nbrender https://example.com --exclude \"/admin/*,/private/*\"\n\n# Fast static crawl (no browser rendering)\nbrender https://example.com --no-render --format json\n\n# Check job status later\nbrender --status abc-123-def\n```\n\n---\n\n## 🛠️ Library Usage\n\n```typescript\nimport { BrenderClient } from 'brender';\n\nconst client = new BrenderClient({\n  accountId: process.env.CF_ACCOUNT_ID!,\n  apiToken: process.env.CF_API_TOKEN!,\n});\n\n// Start a crawl job\nconst job = await client.crawl({\n  url: 'https://example.com',\n  format: 'markdown',\n  limit: 100,\n});\n\nconsole.log('Job started:', job.id);\n\n// Wait for completion\nconst result = await client.waitForCompletion(job.id);\nconsole.log(`Crawled ${result.pages?.length} pages`);\n\n// Process results\nfor (const page of result.pages || []) {\n  console.log(`${page.url}: ${page.content.substring(0, 100)}...`);\n}\n```\n\n---\n\n## 🔑 Getting Credentials\n\n1. **Account ID**: Find in [Cloudflare Dashboard](https://dash.cloudflare.com/) sidebar\n2. **API Token**: Create at [API Tokens](https://dash.cloudflare.com/profile/api-tokens)\n   - Use the \"Browser Rendering\" template\n   - Or manually grant: `Account \u003e Browser Rendering \u003e Edit`\n\n---\n\n## 📖 API Reference\n\n### `crawl(options)`\n\nStart a new crawl job.\n\n| Option | Type | Default | Description |\n|--------|------|---------|-------------|\n| `url` | `string` | required | Starting URL |\n| `format` | `'html' \\| 'markdown' \\| 'json'` | `'markdown'` | Output format |\n| `limit` | `number` | `100` | Max pages to crawl |\n| `depth` | `number` | `3` | Crawl depth |\n| `render` | `boolean` | `true` | Use headless browser |\n| `includePaths` | `string[]` | - | URL patterns to include |\n| `excludePaths` | `string[]` | - | URL patterns to exclude |\n| `modifiedSince` | `string` | - | ISO date for incremental crawl |\n| `maxAge` | `number` | - | Skip recently crawled pages |\n\n### `getJob(jobId)`\n\nGet crawl job status and results.\n\n### `waitForCompletion(jobId, pollInterval?, timeout?)`\n\nPoll until job completes.\n\n### `crawlAndWait(options, pollInterval?, timeout?)`\n\nStart crawl and wait for completion.\n\n---\n\n## 🧪 Development \u0026 CI\n\nThis repository uses GitHub Actions for continuous integration.\n\n- Workflow: `.github/workflows/ci.yml`\n- Triggers: pushes to `main` and pull requests\n- Checks: `npm ci`, `npm run build`, `npm test`\n\nYou can view all runs here: https://github.com/ek-mc/brender/actions/workflows/ci.yml\n\n---\n\n## 🔗 Links\n\n- [📦 NPM Package Page](https://www.npmjs.com/package/brender)\n- [👤 NPM Publisher](https://www.npmjs.com/~ek-mc)\n- [🐙 GitHub Repository](https://github.com/ek-mc/brender)\n- [📘 Cloudflare Browser Rendering Docs](https://developers.cloudflare.com/browser-rendering/)\n- [📗 Crawl Endpoint Docs](https://developers.cloudflare.com/browser-rendering/rest-api/crawl-endpoint/)\n- [🐛 Report Issues](https://github.com/ek-mc/brender/issues)\n\n---\n\n## 📝 Changelog\n\nSee [CHANGELOG.md](./CHANGELOG.md) for version history.\n\n---\n\n## 📄 License\n\nSee [LICENSE](LICENSE).\n\nLicense: MIT\n\n---\n\n\u003cp align=\"center\"\u003e\n  Made with ❤️ for the Cloudflare community\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fek-mc%2Fbrender","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fek-mc%2Fbrender","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fek-mc%2Fbrender/lists"}