{"id":28640370,"url":"https://github.com/mendableai/firesearch","last_synced_at":"2025-06-12T20:06:16.782Z","repository":{"id":296449440,"uuid":"993392548","full_name":"mendableai/firesearch","owner":"mendableai","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-02T14:21:33.000Z","size":263,"stargazers_count":256,"open_issues_count":2,"forks_count":44,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-09T09:11:47.138Z","etag":null,"topics":["firecrawl","langchain","langgraph","llm","research"],"latest_commit_sha":null,"homepage":"https://tools.firecrawl.dev/firesearch","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mendableai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-30T17:58:19.000Z","updated_at":"2025-06-09T08:19:38.000Z","dependencies_parsed_at":"2025-05-31T04:37:24.451Z","dependency_job_id":"10e39ebc-64be-4243-9ac3-71837daa0890","html_url":"https://github.com/mendableai/firesearch","commit_stats":null,"previous_names":["mendableai/firesearch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mendableai/firesearch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Ffiresearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Ffiresearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Ffiresearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Ffiresearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mendableai","download_url":"https://codeload.github.com/mendableai/firesearch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Ffiresearch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259522095,"owners_count":22870449,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["firecrawl","langchain","langgraph","llm","research"],"created_at":"2025-06-12T20:06:16.150Z","updated_at":"2025-06-12T20:06:16.765Z","avatar_url":"https://github.com/mendableai.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# Firesearch - AI-Powered Deep Research Tool\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExd2F2YWo4amdieGVnOXR3aGM5ZnBlcDZvbnRjNW1vNmtpeWNhc3VtbSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/Jw7Q08ll8Vh0BoApI8/giphy.gif\" alt=\"Firesearch Demo\" width=\"100%\" /\u003e\n\u003c/div\u003e\n\nComprehensive web research powered by [Firecrawl](https://www.firecrawl.dev/) and [LangGraph](https://www.langchain.com/langgraph)\n\n## Technologies\n\n- **Firecrawl**: Multi-source web content extraction\n- **OpenAI GPT-4o**: Search planning and follow-up generation\n- **Next.js 15**: Modern React framework with App Router\n\n[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fmendableai%2Ffiresearch\u0026env=FIRECRAWL_API_KEY,OPENAI_API_KEY\u0026envDescription=API%20keys%20required%20for%20Firesearch\u0026envLink=https%3A%2F%2Fgithub.com%2Fmendableai%2Ffiresearch%23required-api-keys)\n\n## Setup\n\n### Required API Keys\n\n| Service | Purpose | Get Key |\n|---------|---------|---------|\n| Firecrawl | Web scraping and content extraction | [firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys) |\n| OpenAI | Search planning and summarization | [platform.openai.com/api-keys](https://platform.openai.com/api-keys) |\n\n### Quick Start\n\n1. Clone this repository\n2. Create a `.env.local` file with your API keys:\n   ```\n   FIRECRAWL_API_KEY=your_firecrawl_key\n   OPENAI_API_KEY=your_openai_key\n   ```\n3. Install dependencies: `npm install` or `yarn install`\n4. Run the development server: `npm run dev` or `yarn dev`\n\n## How It Works\n\n### Architecture Overview\n\n```mermaid\nflowchart TB\n    Query[\"'Compare Samsung Galaxy S25\u003cbr/\u003eand iPhone 16'\"]:::query\n    \n    Query --\u003e Break\n    \n    Break[\"🔍 Break into Sub-Questions\"]:::primary\n    \n    subgraph SubQ[\"🌐 Search Queries\"]\n        S1[\"iPhone 16 Pro specs features\"]:::search\n        S2[\"Samsung Galaxy S25 Ultra specs\"]:::search\n        S3[\"iPhone 16 vs Galaxy S25 comparison\"]:::search\n    end\n    \n    Break --\u003e SubQ\n    \n    subgraph FC[\"🔥 Firecrawl API Calls\"]\n        FC1[\"Firecrawl /search API\u003cbr/\u003eQuery 1\"]:::firecrawl\n        FC2[\"Firecrawl /search API\u003cbr/\u003eQuery 2\"]:::firecrawl\n        FC3[\"Firecrawl /search API\u003cbr/\u003eQuery 3\"]:::firecrawl\n    end\n    \n    S1 --\u003e FC1\n    S2 --\u003e FC2\n    S3 --\u003e FC3\n    \n    subgraph Sources[\"📄 Sources Found\"]\n        R1[\"Apple.com ✓\u003cbr/\u003eThe Verge ✓\u003cbr/\u003eCNET ✓\"]:::source\n        R2[\"GSMArena ✓\u003cbr/\u003eTechRadar ✓\u003cbr/\u003eSamsung.com ✓\"]:::source\n        R3[\"AndroidAuth ✓\u003cbr/\u003eTomsGuide ✓\"]:::source\n    end\n    \n    FC1 --\u003e R1\n    FC2 --\u003e R2\n    FC3 --\u003e R3\n    \n    subgraph Valid[\"✅ Answer Validation\"]\n        V1[\"iPhone 16 specs ✓ (0.95)\"]:::good\n        V2[\"S25 specs ✓ (0.9)\"]:::good\n        V3[\"S25 price ❌ (0.3)\"]:::bad\n    end\n    \n    Sources --\u003e Valid\n    \n    Valid --\u003e Retry\n    \n    Retry{\"Need info:\u003cbr/\u003eS25 pricing?\"}:::check\n    \n    subgraph Strat[\"🧠 Alternative Strategy\"]\n        Original[\"Original: 'Galaxy S25 price'\u003cbr/\u003e❌ No specific pricing found\"]:::bad\n        NewTerms[\"Try: 'Galaxy S25 MSRP cost'\u003cbr/\u003e'Samsung S25 pricing leak'\u003cbr/\u003e'S25 vs S24 price comparison'\"]:::strategy\n    end\n    \n    Retry --\u003e|Yes| Strat\n    \n    subgraph Retry2[\"🔄 Retry Searches\"]\n        Alt1[\"Galaxy S25 MSRP retail\"]:::search\n        Alt2[\"Samsung S25 pricing leak\"]:::search\n        Alt3[\"S25 vs S24 price comparison\"]:::search\n    end\n    \n    Strat --\u003e Retry2\n    \n    subgraph FC2G[\"🔥 Retry API Calls\"]\n        FC4[\"Firecrawl /search API\u003cbr/\u003eAlt Query 1\"]:::firecrawl\n        FC5[\"Firecrawl /search API\u003cbr/\u003eAlt Query 2\"]:::firecrawl\n        FC6[\"Firecrawl /search API\u003cbr/\u003eAlt Query 3\"]:::firecrawl\n    end\n    \n    Alt1 --\u003e FC4\n    Alt2 --\u003e FC5\n    Alt3 --\u003e FC6\n    \n    Results2[\"SamMobile ✓ ($899 leak)\u003cbr/\u003e9to5Google ✓ ($100 more)\u003cbr/\u003ePhoneArena ✓ ($899)\"]:::source\n    \n    FC4 --\u003e Results2\n    FC5 --\u003e Results2\n    FC6 --\u003e Results2\n    \n    Final[\"All answers found ✓\u003cbr/\u003eS25 price: $899\"]:::good\n    \n    Results2 --\u003e Final\n    \n    Synthesis[\"LLM synthesizes response\"]:::synthesis\n    \n    Final --\u003e Synthesis\n    \n    FollowUp[\"Generate follow-up questions\"]:::primary\n    \n    Synthesis --\u003e FollowUp\n    \n    Citations[\"List citations [1-10]\"]:::primary\n    \n    FollowUp --\u003e Citations\n    \n    Answer[\"Complete response delivered\"]:::answer\n    \n    Citations --\u003e Answer\n    \n    %% No path - skip retry and go straight to synthesis\n    Retry --\u003e|No| Synthesis\n    \n    classDef query fill:#ff8c42,stroke:#ff6b1a,stroke-width:3px,color:#fff\n    classDef subq fill:#ffd4b3,stroke:#ff6b1a,stroke-width:1px,color:#333\n    classDef search fill:#ff8c42,stroke:#ff6b1a,stroke-width:2px,color:#fff\n    classDef source fill:#3a4a5c,stroke:#2c3a47,stroke-width:2px,color:#fff\n    classDef check fill:#ffeb3b,stroke:#fbc02d,stroke-width:2px,color:#333\n    classDef good fill:#4caf50,stroke:#388e3c,stroke-width:2px,color:#fff\n    classDef bad fill:#f44336,stroke:#d32f2f,stroke-width:2px,color:#fff\n    classDef strategy fill:#9c27b0,stroke:#7b1fa2,stroke-width:2px,color:#fff\n    classDef synthesis fill:#ff8c42,stroke:#ff6b1a,stroke-width:3px,color:#fff\n    classDef answer fill:#3a4a5c,stroke:#2c3a47,stroke-width:3px,color:#fff\n    classDef firecrawl fill:#ff6b1a,stroke:#ff4500,stroke-width:3px,color:#fff\n    classDef label fill:none,stroke:none,color:#666,font-weight:bold\n```\n\n### Process Flow\n\n1. **Break Down** - Complex queries split into focused sub-questions\n2. **Search** - Multiple searches via Firecrawl API for comprehensive coverage\n3. **Extract** - Markdown content extracted from web sources\n4. **Validate** - Check if sources actually answer the questions (0.7+ confidence)\n5. **Retry** - Alternative search terms for unanswered questions (max 2 attempts)\n6. **Synthesize** - GPT-4o combines findings into cited answer\n\n### Key Features\n\n- **Smart Search** - Breaks complex queries into multiple focused searches\n- **Answer Validation** - Verifies sources contain actual answers (0.7+ confidence)\n- **Auto-Retry** - Alternative search terms for unanswered questions\n- **Real-time Progress** - Live updates as searches complete\n- **Full Citations** - Every fact linked to its source\n- **Context Memory** - Follow-up questions maintain conversation context\n\n### Configuration\n\nCustomize search behavior by modifying [`lib/config.ts`](lib/config.ts):\n\n```typescript\nexport const SEARCH_CONFIG = {\n  // Search Settings\n  MAX_SEARCH_QUERIES: 12,        // Maximum number of search queries to generate\n  MAX_SOURCES_PER_SEARCH: 4,     // Maximum sources to return per search query\n  MAX_SOURCES_TO_SCRAPE: 3,      // Maximum sources to scrape for additional content\n  \n  // Content Processing\n  MIN_CONTENT_LENGTH: 100,       // Minimum content length to consider valid\n  SUMMARY_CHAR_LIMIT: 100,       // Character limit for source summaries\n  \n  // Retry Logic\n  MAX_RETRIES: 2,                // Maximum retry attempts for failed operations\n  MAX_SEARCH_ATTEMPTS: 2,        // Maximum attempts to find answers via search\n  MIN_ANSWER_CONFIDENCE: 0.7,    // Minimum confidence (0-1) that a question was answered\n  \n  // Timeouts\n  SCRAPE_TIMEOUT: 15000,         // Timeout for scraping operations (ms)\n} as const;\n```\n\n### Firecrawl API Integration\n\nFiresearch leverages Firecrawl's powerful `/search` endpoint:\n\n#### `/search` - Web Search with Content\n- **Purpose**: Finds relevant URLs AND extracts markdown content in one call\n- **Usage**: Each decomposed query is sent to find 6-8 relevant sources with content\n- **Response**: Returns URLs with titles, snippets, AND full markdown content\n- **Key Feature**: The `scrapeOptions` parameter enables content extraction during search\n- **Example**:\n  ```\n  POST /search\n  {\n    \"query\": \"iPhone 16 specs pricing\",\n    \"limit\": 8,\n    \"scrapeOptions\": {\n      \"formats\": [\"markdown\"]\n    }\n  }\n  ```\n\n### Search Strategies\n\nWhen initial results are insufficient, the system automatically tries:\n- **Broaden Keywords**: Removes specific terms for wider results\n- **Narrow Focus**: Adds specific terms to target missing aspects\n- **Synonyms**: Uses alternative terms and phrases\n- **Rephrase**: Completely reformulates the query\n- **Decompose**: Breaks complex queries into sub-questions\n- **Academic**: Adds scholarly terms for research-oriented results\n- **Practical**: Focuses on tutorials and how-to guides\n\n## Example Queries\n\n- \"Who are the founders of Firecrawl?\"\n- \"When did NVIDIA release the RTX 4080 Super?\"\n- \"Compare the latest iPhone, Samsung Galaxy, and Google Pixel flagship features\"\n\n## License\n\nMIT License","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmendableai%2Ffiresearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmendableai%2Ffiresearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmendableai%2Ffiresearch/lists"}