https://github.com/fauzaanu/markdown-crawler
Python tool that crawls websites and neatly saves their text content into markdown files, providing a convenient way to archive the text content of the web locally
https://github.com/fauzaanu/markdown-crawler
crawler llm markdown rag scraper
Last synced: about 2 months ago
JSON representation
Python tool that crawls websites and neatly saves their text content into markdown files, providing a convenient way to archive the text content of the web locally
- Host: GitHub
- URL: https://github.com/fauzaanu/markdown-crawler
- Owner: fauzaanu
- Created: 2025-03-08T00:58:01.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-08T01:58:53.000Z (over 1 year ago)
- Last Synced: 2025-03-08T02:27:03.049Z (over 1 year ago)
- Topics: crawler, llm, markdown, rag, scraper
- Language: Python
- Homepage: https://t.me/fauzaanu
- Size: 28.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0