https://github.com/joe-stifler/crawler

Crawler is a Python package that crawls web pages and converts their content into Markdown format, making it easy to create documentation, notes, or other text-based representations. It features domain restrictions, flexible output options, and graph visualization.

context-extraction conversion file-system-crawling github-repository-crawling large-language-models llm markdown python web-crawling

Last synced: 24 Jul 2025