{"id":48939272,"url":"https://github.com/kreuzberg-dev/kreuzcrawl","last_synced_at":"2026-06-07T12:01:00.032Z","repository":{"id":351300191,"uuid":"1176688502","full_name":"kreuzberg-dev/kreuzcrawl","owner":"kreuzberg-dev","description":"High-performance web crawling engine with bindings for 11 languages","archived":false,"fork":false,"pushed_at":"2026-06-04T15:08:38.000Z","size":107924,"stargazers_count":102,"open_issues_count":8,"forks_count":13,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-04T16:11:35.474Z","etag":null,"topics":["crawling","csharp","elixir","ffi","golang","java","mcp","php","python","ruby","rust","typescript","wasm","web-crawler","web-scraping"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kreuzberg-dev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-09T09:21:16.000Z","updated_at":"2026-06-04T15:12:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"d792d7eb-e175-4328-9c40-0f8e2af12142","html_url":"https://github.com/kreuzberg-dev/kreuzcrawl","commit_stats":null,"previous_names":["kreuzberg-dev/kreuzcrawl"],"tags_count":110,"template":false,"template_full_name":null,"purl":"pkg:github/kreuzberg-dev/kreuzcrawl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreuzberg-dev%2Fkreuzcrawl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreuzberg-dev%2Fkreuzcrawl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreuzberg-dev%2Fkreuzcrawl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreuzberg-dev%2Fkreuzcrawl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kreuzberg-dev","download_url":"https://codeload.github.com/kreuzberg-dev/kreuzcrawl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kreuzberg-dev%2Fkreuzcrawl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33949212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawling","csharp","elixir","ffi","golang","java","mcp","php","python","ruby","rust","typescript","wasm","web-crawler","web-scraping"],"created_at":"2026-04-17T13:06:57.093Z","updated_at":"2026-06-07T12:00:59.992Z","avatar_url":"https://github.com/kreuzberg-dev.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kreuzcrawl\n\n\u003cdiv align=\"center\" style=\"display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0;\"\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/alef\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Bindings-alef%20%D7%90-007ec6\" alt=\"Bindings\"\u003e\n  \u003c/a\u003e\n  \u003c!-- Language Bindings --\u003e\n  \u003ca href=\"https://crates.io/crates/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/crates/v/kreuzcrawl?label=Rust\u0026color=007ec6\" alt=\"Rust\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/kreuzcrawl/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/kreuzcrawl?label=Python\u0026color=007ec6\" alt=\"Python\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@kreuzberg/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/npm/v/@kreuzberg/kreuzcrawl?label=Node.js\u0026color=007ec6\" alt=\"Node.js\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@kreuzberg/kreuzcrawl-wasm\"\u003e\n    \u003cimg src=\"https://img.shields.io/npm/v/@kreuzberg/kreuzcrawl-wasm?label=WASM\u0026color=007ec6\" alt=\"WASM\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://central.sonatype.com/artifact/dev.kreuzberg.kreuzcrawl/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/maven-central/v/dev.kreuzberg.kreuzcrawl/kreuzcrawl?label=Java\u0026color=007ec6\" alt=\"Java\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pkg.go.dev/github.com/kreuzberg-dev/kreuzcrawl/packages/go\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/v/tag/kreuzberg-dev/kreuzcrawl?label=Go\u0026color=007ec6\" alt=\"Go\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.nuget.org/packages/Kreuzcrawl/\"\u003e\n    \u003cimg src=\"https://img.shields.io/nuget/v/Kreuzcrawl?label=C%23\u0026color=007ec6\" alt=\"C#\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://packagist.org/packages/kreuzberg-dev/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/packagist/v/kreuzberg-dev/kreuzcrawl?label=PHP\u0026color=007ec6\" alt=\"PHP\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://rubygems.org/gems/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/gem/v/kreuzcrawl?label=Ruby\u0026color=007ec6\" alt=\"Ruby\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://hex.pm/packages/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/hexpm/v/kreuzcrawl?label=Elixir\u0026color=007ec6\" alt=\"Elixir\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pub.dev/packages/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/pub/v/kreuzcrawl?label=Dart\u0026color=007ec6\" alt=\"Dart\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://central.sonatype.com/artifact/dev.kreuzberg.kreuzcrawl.android/kreuzcrawl-android\"\u003e\n    \u003cimg src=\"https://img.shields.io/maven-central/v/dev.kreuzberg.kreuzcrawl.android/kreuzcrawl-android?label=Kotlin\u0026color=007ec6\" alt=\"Kotlin\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/swift\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Swift-SPM-007ec6\" alt=\"Swift\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/zig\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Zig-package-007ec6\" alt=\"Zig\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/releases\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/C-FFI-007ec6\" alt=\"C FFI\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/pkgs/container/kreuzcrawl\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Docker-ghcr.io-007ec6?logo=docker\u0026logoColor=white\" alt=\"Docker\"\u003e\n  \u003c/a\u003e\n\n  \u003c!-- Project Info --\u003e\n  \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/blob/main/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-Elastic--2.0-007ec6\" alt=\"License\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://docs.kreuzcrawl.kreuzberg.dev\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Docs-kreuzcrawl-007ec6\" alt=\"Documentation\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\" style=\"margin: 24px 0 0;\"\u003e\n  \u003ca href=\"https://kreuzberg.dev\"\u003e\n    \u003cimg alt=\"Kreuzcrawl\" src=\"https://raw.githubusercontent.com/kreuzberg-dev/kreuzcrawl/main/docs/assets/docs_top_banner.svg\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\" style=\"display: flex; flex-wrap: wrap; gap: 12px; justify-content: center; margin: 28px 0 24px;\"\u003e\n  \u003ca href=\"https://discord.gg/xt9WY3GnKR\"\u003e\n    \u003cimg height=\"22\" src=\"https://img.shields.io/badge/Discord-Chat-007ec6?logo=discord\u0026logoColor=white\" alt=\"Join Discord\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\nHigh-performance Rust web crawling engine for structured data extraction. Scrape, crawl, and map websites with native bindings for 14 languages — same engine, identical results across every runtime.\n\n## Key Features\n\n- **Structured extraction** — Text, metadata, links, images, assets, JSON-LD, Open Graph, hreflang, favicons, headings, and response headers\n- **Markdown conversion** — Clean Markdown output with citations, document structure, and fit-content mode\n- **Concurrent crawling** — Depth-first, breadth-first, or best-first traversal with configurable depth, page limits, and concurrency\n- **14 language bindings** — Rust, Python, Node.js, TypeScript, Ruby, Go, Java, Kotlin (Android), C#, PHP, Elixir, Dart, Swift, Zig, and WebAssembly\n- **Smart filtering** — BM25 relevance scoring, URL include/exclude patterns, robots.txt compliance, and sitemap discovery\n- **Browser rendering** — Optional headless browser for JavaScript-heavy SPAs with WAF detection and bypass\n- **Batch operations** — Scrape or crawl hundreds of URLs concurrently with partial failure handling\n- **Streaming** — Real-time crawl events via async streams for progress tracking\n- **Authentication** — HTTP Basic, Bearer token, and custom header auth with persistent cookie jars\n- **Rate limiting** — Per-domain request throttling with configurable delays\n- **Asset download** — Download, deduplicate, and filter images, documents, and other linked assets\n- **MCP server** — Model Context Protocol integration for AI agents\n- **REST API** — HTTP server with OpenAPI spec\n\n**[Documentation](https://docs.kreuzcrawl.kreuzberg.dev)** | **[API Reference](https://docs.kreuzcrawl.kreuzberg.dev/reference/api-rust/)**\n\n## Installation\n\n| Language                                                                                    | Package                                                                                    | Install                                                                           |\n| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- |\n| **[Python](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/python)**         | [kreuzcrawl](https://pypi.org/project/kreuzcrawl/)                                         | `pip install kreuzcrawl`                                                          |\n| **[Node.js](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl-node)** | [@kreuzberg/kreuzcrawl](https://www.npmjs.com/package/@kreuzberg/kreuzcrawl)               | `npm install @kreuzberg/kreuzcrawl`                                               |\n| **[Rust](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl)**         | [kreuzcrawl](https://crates.io/crates/kreuzcrawl)                                          | `cargo add kreuzcrawl`                                                            |\n| **[Go](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/go)**                 | [pkg.go.dev](https://pkg.go.dev/github.com/kreuzberg-dev/kreuzcrawl/packages/go)           | `go get github.com/kreuzberg-dev/kreuzcrawl/packages/go`                          |\n| **[Java](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/java)**             | [Maven Central](https://central.sonatype.com/artifact/dev.kreuzberg.kreuzcrawl/kreuzcrawl) | See [README](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/java) |\n| **[C#](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/csharp)**             | [NuGet](https://www.nuget.org/packages/Kreuzcrawl/)                                        | `dotnet add package Kreuzcrawl`                                                   |\n| **[Ruby](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/ruby)**             | [kreuzcrawl](https://rubygems.org/gems/kreuzcrawl)                                         | `gem install kreuzcrawl`                                                          |\n| **[PHP](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/php)**               | [kreuzberg-dev/kreuzcrawl](https://packagist.org/packages/kreuzberg-dev/kreuzcrawl)        | `composer require kreuzberg-dev/kreuzcrawl`                                       |\n| **[Elixir](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/elixir)**         | [kreuzcrawl](https://hex.pm/packages/kreuzcrawl)                                           | `{:kreuzcrawl, \"~\u003e 0.2\"}`                                                         |\n| **[WASM](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl-wasm)**    | [@kreuzberg/kreuzcrawl-wasm](https://www.npmjs.com/package/@kreuzberg/kreuzcrawl-wasm)     | `npm install @kreuzberg/kreuzcrawl-wasm`                                          |\n| **[C FFI](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl-ffi)**    | [GitHub Releases](https://github.com/kreuzberg-dev/kreuzcrawl/releases)                    | C header + shared library                                                         |\n| **[CLI](https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl-cli)**      | [crates.io](https://crates.io/crates/kreuzcrawl-cli)                                       | `cargo install kreuzcrawl-cli`                                                    |\n| **CLI (Homebrew)**                                                                          | [kreuzberg-dev/tap](https://github.com/kreuzberg-dev/homebrew-tap)                         | `brew install kreuzberg-dev/tap/kreuzcrawl`                                       |\n\n## Quick Start\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003ePython\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/python\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```python\nfrom kreuzcrawl import create_engine, scrape\n\nengine = create_engine()\nresult = scrape(engine, \"https://example.com\")\n\nprint(result.metadata.title)\nprint(result.markdown.content)\nprint(len(result.links))\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eNode.js / TypeScript\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl-node\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```typescript\nimport { createEngine, scrape } from \"@kreuzberg/kreuzcrawl\";\n\nconst engine = createEngine();\nconst result = await scrape(engine, \"https://example.com\");\n\nconsole.log(result.metadata.title);\nconsole.log(result.markdown.content);\nconsole.log(result.links.length);\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRust\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/crates/kreuzcrawl\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```rust\nlet engine = kreuzcrawl::create_engine(None)?;\nlet result = kreuzcrawl::scrape(\u0026engine, \"https://example.com\").await?;\n\nprintln!(\"{}\", result.metadata.title);\nprintln!(\"{}\", result.markdown.content);\nprintln!(\"{}\", result.links.len());\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGo\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/go\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```go\nengine, _ := kcrawl.CreateEngine()\nresult, _ := kcrawl.Scrape(engine, \"https://example.com\")\n\nfmt.Println(result.Metadata.Title)\nfmt.Println(result.Markdown.Content)\nfmt.Println(len(result.Links))\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eJava\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/java\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```java\nvar engine = Kreuzcrawl.createEngine(null);\nvar result = Kreuzcrawl.scrape(engine, \"https://example.com\");\n\nSystem.out.println(result.metadata().title());\nSystem.out.println(result.markdown().content());\nSystem.out.println(result.links().size());\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eC#\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/csharp\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```csharp\nvar engine = KreuzcrawlLib.CreateEngine(null);\nvar result = await KreuzcrawlLib.Scrape(engine, \"https://example.com\");\n\nConsole.WriteLine(result.Metadata.Title);\nConsole.WriteLine(result.Markdown.Content);\nConsole.WriteLine(result.Links.Count);\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eRuby\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/ruby\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```ruby\nengine = Kreuzcrawl.create_engine(nil)\nresult = Kreuzcrawl.scrape(engine, \"https://example.com\")\n\nputs result.metadata.title\nputs result.markdown.content\nputs result.links.length\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003ePHP\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/php\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```php\n$engine = Kreuzcrawl::createEngine(null);\n$result = Kreuzcrawl::scrape($engine, \"https://example.com\");\n\necho $result-\u003emetadata-\u003etitle;\necho $result-\u003emarkdown-\u003econtent;\necho count($result-\u003elinks);\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eElixir\u003c/strong\u003e — \u003ca href=\"https://github.com/kreuzberg-dev/kreuzcrawl/tree/main/packages/elixir\"\u003eFull docs\u003c/a\u003e\u003c/summary\u003e\n\n```elixir\n{:ok, engine} = Kreuzcrawl.create_engine(nil)\n{:ok, result} = Kreuzcrawl.scrape(engine, \"https://example.com\")\n\nIO.puts(result.metadata.title)\nIO.puts(result.markdown.content)\nIO.puts(length(result.links))\n```\n\n\u003c/details\u003e\n\n## Platform Support\n\n| Language | Linux x86_64 | Linux aarch64 | macOS ARM64 | Windows x64 |\n| -------- | :----------: | :-----------: | :---------: | :---------: |\n| Python   |      ✅      |      ✅       |     ✅      |     ✅      |\n| Node.js  |      ✅      |      ✅       |     ✅      |     ✅      |\n| WASM     |      ✅      |      ✅       |     ✅      |     ✅      |\n| Ruby     |      ✅      |      ✅       |     ✅      |      —      |\n| Elixir   |      ✅      |      ✅       |     ✅      |     ✅      |\n| Go       |      ✅      |      ✅       |     ✅      |     ✅      |\n| Java     |      ✅      |      ✅       |     ✅      |     ✅      |\n| C#       |      ✅      |      ✅       |     ✅      |     ✅      |\n| PHP      |      ✅      |      ✅       |     ✅      |     ✅      |\n| Rust     |      ✅      |      ✅       |     ✅      |     ✅      |\n| C (FFI)  |      ✅      |      ✅       |     ✅      |     ✅      |\n| CLI      |      ✅      |      ✅       |     ✅      |     ✅      |\n\n## Architecture\n\n```text\nYour Application (Python, Node.js, Ruby, Java, Go, C#, PHP, Elixir, ...)\n    │\nLanguage Bindings (PyO3, NAPI-RS, Magnus, ext-php-rs, Rustler, cgo, Panama, P/Invoke)\n    │\nRust Core Engine (async, concurrent, SIMD-optimized)\n    │\n    ├── HTTP Client (reqwest + tower middleware stack)\n    ├── HTML Parser (html5ever + lol_html)\n    ├── Markdown Converter (html-to-markdown-rs)\n    ├── Content Extraction (metadata, JSON-LD, Open Graph, readability)\n    ├── Link Discovery (robots.txt, sitemaps, anchor analysis)\n    └── Browser Rendering (optional headless Chrome/Firefox)\n```\n\n## Contributing\n\nContributions are welcome! See our [Contributing Guide](https://github.com/kreuzberg-dev/kreuzcrawl/blob/main/CONTRIBUTING.md).\n\n## Part of Kreuzberg.dev\n\n- [Kreuzberg](https://github.com/kreuzberg-dev/kreuzberg) — document intelligence: text, tables, metadata from 90+ formats with optional OCR.\n- [Kreuzberg Cloud](https://github.com/kreuzberg-dev/kreuzberg-cloud) — managed extraction API with SDKs, dashboards, and observability.\n- [html-to-markdown](https://github.com/kreuzberg-dev/html-to-markdown) — fast, lossless HTML→Markdown engine.\n- [liter-llm](https://github.com/kreuzberg-dev/liter-llm) — universal LLM API client with native bindings for 14 languages and 143 providers.\n- [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) — tree-sitter grammars and code-intelligence primitives.\n- [alef](https://github.com/kreuzberg-dev/alef) — the polyglot binding generator that produces all per-language bindings.\n- [Discord](https://discord.gg/xt9WY3GnKR) — community, roadmap, announcements.\n\n## License\n\n[Elastic License 2.0](https://github.com/kreuzberg-dev/kreuzcrawl/blob/main/LICENSE)\n\n## Links\n\n- [Documentation](https://docs.kreuzcrawl.kreuzberg.dev)\n- [API Reference](https://docs.kreuzcrawl.kreuzberg.dev/reference/api-rust/)\n- [GitHub](https://github.com/kreuzberg-dev/kreuzcrawl)\n- [Issues](https://github.com/kreuzberg-dev/kreuzcrawl/issues)\n- [Issues](https://github.com/kreuzberg-dev/kreuzcrawl/issues)\n- [Discord](https://discord.gg/xt9WY3GnKR)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkreuzberg-dev%2Fkreuzcrawl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkreuzberg-dev%2Fkreuzcrawl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkreuzberg-dev%2Fkreuzcrawl/lists"}