{"id":25295648,"url":"https://github.com/langchain-ai/open_deep_research","last_synced_at":"2025-04-10T13:56:55.733Z","repository":{"id":267364896,"uuid":"891642747","full_name":"langchain-ai/open_deep_research","owner":"langchain-ai","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-21T20:05:44.000Z","size":643,"stargazers_count":2685,"open_issues_count":22,"forks_count":359,"subscribers_count":31,"default_branch":"main","last_synced_at":"2025-04-03T09:05:38.321Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/langchain-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-20T17:37:22.000Z","updated_at":"2025-04-03T08:36:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"28125f20-0d4f-4dc8-98ff-ded17b8af3d0","html_url":"https://github.com/langchain-ai/open_deep_research","commit_stats":{"total_commits":18,"total_committers":2,"mean_commits":9.0,"dds":0.05555555555555558,"last_synced_commit":"7bf657b9ec17e80ffd7137abab3e8a31324c9844"},"previous_names":["langchain-ai/report-maistro","langchain-ai/open_deep_research"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fopen_deep_research","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fopen_deep_research/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fopen_deep_research/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/langchain-ai%2Fopen_deep_research/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/langchain-ai","download_url":"https://codeload.github.com/langchain-ai/open_deep_research/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248230082,"owners_count":21069051,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-13T02:22:28.777Z","updated_at":"2025-04-10T13:56:55.716Z","avatar_url":"https://github.com/langchain-ai.png","language":"Jupyter Notebook","funding_links":[],"categories":["Python","Literature Review and Research Discovery","LangGraph Extensions","🟢 Official LangGraph Projects 🦜","References","Functional Roles of AI Scientists","Foundation Models, LLMs, and Agents","A01_文本生成_文本对话"],"sub_categories":["DSGE and Structural Models","🟩 Specialized Agent Libraries 🤖","Deep Research","AI Agents","大语言对话模型及数据"],"readme":"# Open Deep Research\n \nOpen Deep Research is an open source assistant that automates research and produces customizable reports on any topic. It allows you to customize the research and writing process with specific models, prompts, report structure, and search tools. \n\n![report-generation](https://github.com/user-attachments/assets/6595d5cd-c981-43ec-8e8b-209e4fefc596)\n\n## 🚀 Quickstart\n\nEnsure you have API keys set for your desired search tools and models.\n\nAvailable search tools:\n\n* [Tavily API](https://tavily.com/) - General web search\n* [Perplexity API](https://www.perplexity.ai/hub/blog/introducing-the-sonar-pro-api) - General web search\n* [Exa API](https://exa.ai/) - Powerful neural search for web content\n* [ArXiv](https://arxiv.org/) - Academic papers in physics, mathematics, computer science, and more\n* [PubMed](https://pubmed.ncbi.nlm.nih.gov/) - Biomedical literature from MEDLINE, life science journals, and online books\n* [Linkup API](https://www.linkup.so/) - General web search\n* [DuckDuckGo API](https://duckduckgo.com/) - General web search\n* [Google Search API/Scrapper](https://google.com/) - Create custom search engine [here](https://programmablesearchengine.google.com/controlpanel/all) and get API key [here](https://developers.google.com/custom-search/v1/introduction)\n\nOpen Deep Research uses a planner LLM for report planning and a writer LLM for report writing: \n\n* You can select any model that is integrated [with the `init_chat_model()` API](https://python.langchain.com/docs/how_to/chat_models_universal_init/)\n* See full list of supported integrations [here](https://python.langchain.com/api_reference/langchain/chat_models/langchain.chat_models.base.init_chat_model.html)\n\n### Using the package\n\n```bash\npip install open-deep-research\n```\n\nAs mentioned above, ensure API keys for LLMs and search tools are set: \n```bash\nexport TAVILY_API_KEY=\u003cyour_tavily_api_key\u003e\nexport ANTHROPIC_API_KEY=\u003cyour_anthropic_api_key\u003e\n```\n\nSee [src/open_deep_research/graph.ipynb](src/open_deep_research/graph.ipynb) for example usage in a Jupyter notebook:\n\nCompile the graph:\n```python\nfrom langgraph.checkpoint.memory import MemorySaver\nfrom open_deep_research.graph import builder\nmemory = MemorySaver()\ngraph = builder.compile(checkpointer=memory)\n```\n\nRun the graph with a desired topic and configuration:\n```python\nimport uuid \nthread = {\"configurable\": {\"thread_id\": str(uuid.uuid4()),\n                           \"search_api\": \"tavily\",\n                           \"planner_provider\": \"anthropic\",\n                           \"planner_model\": \"claude-3-7-sonnet-latest\",\n                           \"writer_provider\": \"anthropic\",\n                           \"writer_model\": \"claude-3-5-sonnet-latest\",\n                           \"max_search_depth\": 1,\n                           }}\n\ntopic = \"Overview of the AI inference market with focus on Fireworks, Together.ai, Groq\"\nasync for event in graph.astream({\"topic\":topic,}, thread, stream_mode=\"updates\"):\n    print(event)\n```\n\nThe graph will stop when the report plan is generated, and you can pass feedback to update the report plan:\n```python\nfrom langgraph.types import Command\nasync for event in graph.astream(Command(resume=\"Include a revenue estimate (ARR) in the sections\"), thread, stream_mode=\"updates\"):\n    print(event)\n```\n\nWhen you are satisfied with the report plan, you can pass `True` to proceed to report generation:\n```python\nasync for event in graph.astream(Command(resume=True), thread, stream_mode=\"updates\"):\n    print(event)\n```\n\n### Running LangGraph Studio UI locally\n\nClone the repository:\n```bash\ngit clone https://github.com/langchain-ai/open_deep_research.git\ncd open_deep_research\n```\n\nThen edit the `.env` file to customize the environment variables according to your needs. These environment variables control the model selection, search tools, and other configuration settings. When you run the application, these values will be automatically loaded via `python-dotenv` (because `langgraph.json` point to the \"env\" file).\n```bash\ncp .env.example .env\n```\n\nSet whatever APIs needed for your model and search tools.\n\nHere are examples for several of the model and tool integrations available:\n```bash\nexport TAVILY_API_KEY=\u003cyour_tavily_api_key\u003e\nexport ANTHROPIC_API_KEY=\u003cyour_anthropic_api_key\u003e\nexport OPENAI_API_KEY=\u003cyour_openai_api_key\u003e\nexport PERPLEXITY_API_KEY=\u003cyour_perplexity_api_key\u003e\nexport EXA_API_KEY=\u003cyour_exa_api_key\u003e\nexport PUBMED_API_KEY=\u003cyour_pubmed_api_key\u003e\nexport PUBMED_EMAIL=\u003cyour_email@example.com\u003e\nexport LINKUP_API_KEY=\u003cyour_linkup_api_key\u003e\nexport GOOGLE_API_KEY=\u003cyour_google_api_key\u003e\nexport GOOGLE_CX=\u003cyour_google_custom_search_engine_id\u003e\n```\n\nLaunch the assistant with the LangGraph server locally, which will open in your browser:\n\n#### Mac\n\n```bash\n# Install uv package manager\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Install dependencies and start the LangGraph server\nuvx --refresh --from \"langgraph-cli[inmem]\" --with-editable . --python 3.11 langgraph dev\n```\n\n#### Windows / Linux\n\n```powershell\n# Install dependencies \npip install -e .\npip install -U \"langgraph-cli[inmem]\" \n\n# Start the LangGraph server\nlanggraph dev\n```\n\nUse this to open the Studio UI:\n```\n- 🚀 API: http://127.0.0.1:2024\n- 🎨 Studio UI: https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024\n- 📚 API Docs: http://127.0.0.1:2024/docs\n```\n\n(1) Provide a `Topic` and hit `Submit`:\n\n\u003cimg width=\"1326\" alt=\"input\" src=\"https://github.com/user-attachments/assets/de264b1b-8ea5-4090-8e72-e1ef1230262f\" /\u003e\n\n(2) This will generate a report plan and present it to the user for review.\n\n(3) We can pass a string (`\"...\"`) with feedback to regenerate the plan based on the feedback.\n\n\u003cimg width=\"1326\" alt=\"feedback\" src=\"https://github.com/user-attachments/assets/c308e888-4642-4c74-bc78-76576a2da919\" /\u003e\n\n(4) Or, we can just pass `true` to accept the plan.\n\n\u003cimg width=\"1480\" alt=\"accept\" src=\"https://github.com/user-attachments/assets/ddeeb33b-fdce-494f-af8b-bd2acc1cef06\" /\u003e\n\n(5) Once accepted, the report sections will be generated.\n\n\u003cimg width=\"1326\" alt=\"report_gen\" src=\"https://github.com/user-attachments/assets/74ff01cc-e7ed-47b8-bd0c-4ef615253c46\" /\u003e\n\nThe report is produced as markdown.\n\n\u003cimg width=\"1326\" alt=\"report\" src=\"https://github.com/user-attachments/assets/92d9f7b7-3aea-4025-be99-7fb0d4b47289\" /\u003e\n\n## 📖 Customizing the report\n\nYou can customize the research assistant's behavior through several parameters:\n\n- `report_structure`: Define a custom structure for your report (defaults to a standard research report format)\n- `number_of_queries`: Number of search queries to generate per section (default: 2)\n- `max_search_depth`: Maximum number of reflection and search iterations (default: 2)\n- `planner_provider`: Model provider for planning phase (default: \"anthropic\", but can be any provider from supported integrations with `init_chat_model` as listed [here](https://python.langchain.com/api_reference/langchain/chat_models/langchain.chat_models.base.init_chat_model.html))\n- `planner_model`: Specific model for planning (default: \"claude-3-7-sonnet-latest\")\n- `writer_provider`: Model provider for writing phase (default: \"anthropic\", but can be any provider from supported integrations with `init_chat_model` as listed [here](https://python.langchain.com/api_reference/langchain/chat_models/langchain.chat_models.base.init_chat_model.html))\n- `writer_model`: Model for writing the report (default: \"claude-3-5-sonnet-latest\")\n- `search_api`: API to use for web searches (default: \"tavily\", options include \"perplexity\", \"exa\", \"arxiv\", \"pubmed\", \"linkup\")\n\nThese configurations allow you to fine-tune the research process based on your needs, from adjusting the depth of research to selecting specific AI models for different phases of report generation.\n\n### Search API Configuration\n\nNot all search APIs support additional configuration parameters. Here are the ones that do:\n\n- **Exa**: `max_characters`, `num_results`, `include_domains`, `exclude_domains`, `subpages`\n  - Note: `include_domains` and `exclude_domains` cannot be used together\n  - Particularly useful when you need to narrow your research to specific trusted sources, ensure information accuracy, or when your research requires using specified domains (e.g., academic journals, government sites)\n  - Provides AI-generated summaries tailored to your specific query, making it easier to extract relevant information from search results\n- **ArXiv**: `load_max_docs`, `get_full_documents`, `load_all_available_meta`\n- **PubMed**: `top_k_results`, `email`, `api_key`, `doc_content_chars_max`\n- **Linkup**: `depth`\n\nExample with Exa configuration:\n```python\nthread = {\"configurable\": {\"thread_id\": str(uuid.uuid4()),\n                           \"search_api\": \"exa\",\n                           \"search_api_config\": {\n                               \"num_results\": 5,\n                               \"include_domains\": [\"nature.com\", \"sciencedirect.com\"]\n                           },\n                           # Other configuration...\n                           }}\n```\n\n### Model Considerations\n\n(1) You can pass any planner and writer models that are integrated [with the `init_chat_model()` API](https://python.langchain.com/docs/how_to/chat_models_universal_init/). See full list of supported integrations [here](https://python.langchain.com/api_reference/langchain/chat_models/langchain.chat_models.base.init_chat_model.html).\n\n(2) **The planner and writer models need to support structured outputs**: Check whether structured outputs are supported by the model you are using [here](https://python.langchain.com/docs/integrations/chat/).\n\n(3) With Groq, there are token per minute (TPM) limits if you are on the `on_demand` service tier:\n- The `on_demand` service tier has a limit of `6000 TPM`\n- You will want a [paid plan](https://github.com/cline/cline/issues/47#issuecomment-2640992272) for section writing with Groq models\n\n(4) `deepseek-R1` [is not strong at function calling](https://api-docs.deepseek.com/guides/reasoning_model), which the assistant uses to generate structured outputs for report sections and report section grading. See example traces [here](https://smith.langchain.com/public/07d53997-4a6d-4ea8-9a1f-064a85cd6072/r).  \n- Consider providers that are strong at function calling such as OpenAI, Anthropic, and certain OSS models like Groq's `llama-3.3-70b-versatile`.\n- If you see the following error, it is likely due to the model not being able to produce structured outputs (see [trace](https://smith.langchain.com/public/8a6da065-3b8b-4a92-8df7-5468da336cbe/r)):\n```\ngroq.APIError: Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.\n```\n\n## How it works\n   \n1. `Plan and Execute` - Open Deep Research follows a [plan-and-execute workflow](https://github.com/assafelovic/gpt-researcher) that separates planning from research, allowing for human-in-the-loop approval of a report plan before the more time-consuming research phase. It uses, by default, a [reasoning model](https://www.youtube.com/watch?v=f0RbwrBcFmc) to plan the report sections. During this phase, it uses web search to gather general information about the report topic to help in planning the report sections. But, it also accepts a report structure from the user to help guide the report sections as well as human feedback on the report plan.\n   \n2. `Research and Write` - Each section of the report is written in parallel. The research assistant uses web search via [Tavily API](https://tavily.com/), [Perplexity](https://www.perplexity.ai/hub/blog/introducing-the-sonar-pro-api), [Exa](https://exa.ai/), [ArXiv](https://arxiv.org/), [PubMed](https://pubmed.ncbi.nlm.nih.gov/) or [Linkup](https://www.linkup.so/) to gather information about each section topic. It will reflect on each report section and suggest follow-up questions for web search. This \"depth\" of research will proceed for any many iterations as the user wants. Any final sections, such as introductions and conclusions, are written after the main body of the report is written, which helps ensure that the report is cohesive and coherent. The planner determines main body versus final sections during the planning phase.\n\n3. `Managing different types` - Open Deep Research is built on LangGraph, which has native support for configuration management [using assistants](https://langchain-ai.github.io/langgraph/concepts/assistants/). The report `structure` is a field in the graph configuration, which allows users to create different assistants for different types of reports. \n\n## UX\n\n### Local deployment\n\nFollow the [quickstart](#-quickstart) to start LangGraph server locally.\n\n### Hosted deployment\n \nYou can easily deploy to [LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/#deployment-options). \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangchain-ai%2Fopen_deep_research","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flangchain-ai%2Fopen_deep_research","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flangchain-ai%2Fopen_deep_research/lists"}