Projects in Awesome Lists tagged with crawl4ai

https://github.com/cyberagiinc/devdocs

Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App

cline crawl4ai cursor documentation llm playwright python3 scraper security typescript windsurf

Last synced: 11 Apr 2025

https://github.com/cyberagiinc/DevDocs

Completely free, private, intuitive UI based Web Scraping MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App

cline crawl4ai cursor documentation llm playwright python3 scraper security typescript windsurf

Last synced: 06 Mar 2025

https://github.com/kaymen99/ai-web-scraper

AI web scraper built with Crawl4AI for extracting structured leads data from websites.

ai-agents ai-scraping crawl4ai data-scraper lead-generation llms scraper web-scraper web-scraping

Last synced: 14 Mar 2025

https://github.com/tecno-consultores/llm-lab

LLM laboratory

chromadb crawl4ai docker flowise-ai kafka laboratory llm-agents n8n ollama open-webui openhands postgresql qdrant-vector-database redis whisper-ai

Last synced: 11 Apr 2025

https://github.com/sitamgithub-msit/crawl4ai-txtfy

Easily retrieve the full-text content from web pages using Crawl4AI.

crawl4ai flask-api html-css-javascript pytest python tailwindcss

Last synced: 23 Mar 2025

https://github.com/baitiche34/yt-shorts-generator

Generate YouTube Shorts Video From an Article Link using Pydantic AI

agents ai ai-generated aiagent aiagents api automation crawl4ai deepgram-stt elevenlabs-api ffmpeg python tts-api youtube

Last synced: 29 Mar 2025

https://github.com/oussemabenhassena5/crawl4deepseek

Crawl4DeepSeek = Crawl4AI + DeepSeek 🚀 Smart, efficient, and built for deep web exploration! 🌐🤖

crawl4ai deepseek python webcrawling webscraping

Last synced: 09 Apr 2025

https://github.com/e-d-i-n-i/ai-data-extraction

AI-driven system for structured data extraction, storage, and vector search, leveraging Crawl4AI, PydanticAI, and Supabase to enable efficient retrieval and RAG-based AI applications.

api crawl4ai nextjs pdf-data-extraction pydantic-ai python sitemap-scraping supabase vector-embeddings web-scraping

Last synced: 09 Apr 2025

https://github.com/svantevith/talk-to-your-website

RAG agent enhanced with LLM-optimized website crawler built using Crawl4AI, Langchain, ChromaDB and Ollama.

ai artificial-intelligence chatbot chromadb crawl4ai genai generative-ai langchain large-language-models llm machine-learning ml ollama python rag

Last synced: 15 Apr 2025

https://github.com/greathayat/yt-shorts-generator

Generate YouTube Shorts Video From an Article Link using Pydantic AI

agents ai aiagent aiagents crawl4ai dalle-3 deepgram-api deepgram-stt ffmpeg pydantic-ai python

Last synced: 18 Mar 2025

https://github.com/yashpinjarkar10/pinescript-agent

PineScript Agentic RAG system that provides an interactive chat interface to answer questions about PineScript. It combines a Streamlit-based UI with an asynchronous AI agent to retrieve and summarize documentation stored in a Supabase database.

crawl4ai gemini pydantic python streamlit supabase-db

Last synced: 19 Feb 2025

https://github.com/shsharkar/crawl4ai-web-crawler

Crawl4AI Web Crawler

ai crawl4ai scraper scraper-api scraper-python scraper-tools web-crawler web-crawler-python

Last synced: 22 Mar 2025

https://github.com/laurentvv/crawl4ai-mcp

Web crawling tool that integrates with AI assistants via the MCP

ai-tools crawl4ai mcp python3 web-crawler

Last synced: 12 Mar 2025

https://github.com/balaji1233/web_master

AI tool to transforms any URL into a structured knowledge source by: extracting content using Crawl4AI ,vectorizing and summarizing data , running Retrieval-Augmented Generation (RAG) for deep information discovery, enabling a smart chatbot for interactive Q&A.

crawl4ai deepseek-r1 docker faiss-vector-database ollama rag streamlit