{"id":15039330,"url":"https://github.com/internlm/mindsearch","last_synced_at":"2025-04-25T14:43:05.560Z","repository":{"id":250802709,"uuid":"834688418","full_name":"InternLM/MindSearch","owner":"InternLM","description":"🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)","archived":false,"fork":false,"pushed_at":"2025-01-08T09:34:38.000Z","size":4338,"stargazers_count":6323,"open_issues_count":47,"forks_count":634,"subscribers_count":45,"default_branch":"main","last_synced_at":"2025-04-24T04:15:02.812Z","etag":null,"topics":["ai-search-engine","gpt","llm","llms","multi-agent-systems","perplexity-ai","search","searchgpt","transformer","web-search"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InternLM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-28T03:54:50.000Z","updated_at":"2025-04-24T02:49:50.000Z","dependencies_parsed_at":"2024-08-16T07:24:45.715Z","dependency_job_id":"87cadccf-bf53-475e-9e96-076650dc8aea","html_url":"https://github.com/InternLM/MindSearch","commit_stats":null,"previous_names":["internlm/mindsearch"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2FMindSearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2FMindSearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2FMindSearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternLM%2FMindSearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InternLM","download_url":"https://codeload.github.com/InternLM/MindSearch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250560056,"owners_count":21450173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-search-engine","gpt","llm","llms","multi-agent-systems","perplexity-ai","search","searchgpt","transformer","web-search"],"created_at":"2024-09-24T20:42:28.890Z","updated_at":"2025-04-24T04:15:10.903Z","avatar_url":"https://github.com/InternLM.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv id=\"top\"\u003e\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"assets/logo.svg\" style=\"width: 50%; height: auto;\"\u003e\n\n[📃 Paper](https://arxiv.org/abs/2407.20183) | [💻 Demo](https://internlm-chat.intern-ai.org.cn/)\n\nEnglish | [简体中文](README_zh-CN.md)\n\n\u003chttps://github.com/user-attachments/assets/44ffe4b9-be26-4b93-a77b-02fed16e33fe\u003e\n\n\u003c/div\u003e\n\u003c/p\u003e\n\n## ✨ MindSearch: Mimicking Human Minds Elicits Deep AI Searcher\n\n## 📅 Changelog\n\n- 2024/11/05: 🥳 MindSearch is now deployed on Puyu! 👉 [Try it](https://internlm-chat.intern-ai.org.cn/) 👈\n  -  Refactored the agent module based on [Lagent v0.5](https://github.com/InternLM/lagent) for better performance in concurrency.\n  -  Improved the UI to embody the simultaneous multi-query search.\n\n\n## ⚽️ Build Your Own MindSearch\n\n### Step1: Dependencies Installation\n\n```bash\ngit clone https://github.com/InternLM/MindSearch\ncd MindSearch\npip install -r requirements.txt\n```\n\n### Step2: Setup Environment Variables\n\nBefore setting up the API, you need to configure environment variables. Rename the `.env.example` file to `.env` and fill in the required values.\n\n```bash\nmv .env.example .env\n# Open .env and add your keys and model configurations\n```\n\n### Step3: Setup MindSearch API\n\nSetup FastAPI Server.\n\n```bash\npython -m mindsearch.app --lang en --model_format internlm_server --search_engine DuckDuckGoSearch --asy \n```\n\n- `--lang`: language of the model, `en` for English and `cn` for Chinese.\n- `--model_format`: format of the model.\n  - `internlm_server` for InternLM2.5-7b-chat with local server. (InternLM2.5-7b-chat has been better optimized for Chinese.)\n  - `gpt4` for GPT4.\n    if you want to use other models, please modify [models](./mindsearch/agent/models.py)\n- `--search_engine`: Search engine.\n  - `DuckDuckGoSearch` for search engine for DuckDuckGo.\n  - `BingSearch` for Bing search engine.\n  - `BraveSearch` for Brave search web api engine.\n  - `GoogleSearch` for Google Serper web search api engine.\n  - `TencentSearch` for Tencent search api engine.\n  \n  Please set your Web Search engine API key as the `WEB_SEARCH_API_KEY` environment variable unless you are using `DuckDuckGo`, or `TencentSearch` that requires secret id as `TENCENT_SEARCH_SECRET_ID` and secret key as `TENCENT_SEARCH_SECRET_KEY`.\n- `--asy`: deploy asynchronous agents.\n\n### Step4: Setup MindSearch Frontend\n\nProviding following frontend interfaces,\n\n- React\n\nFirst configurate the backend URL for Vite proxy.\n\n```bash\nHOST=\"127.0.0.1\"  # modify as you need\nPORT=8002\nsed -i -r \"s/target:\\s*\\\"\\\"/target: \\\"${HOST}:${PORT}\\\"/\" frontend/React/vite.config.ts\n```\n\n```bash\n# Install Node.js and npm\n# for Ubuntu\nsudo apt install nodejs npm\n\n# for windows\n# download from https://nodejs.org/zh-cn/download/prebuilt-installer\n\n# Install dependencies\n\ncd frontend/React\nnpm install\nnpm start\n```\n\nDetails can be found in [React](./frontend/React/README.md)\n\n- Gradio\n\n```bash\npython frontend/mindsearch_gradio.py\n```\n\n- Streamlit\n\n```bash\nstreamlit run frontend/mindsearch_streamlit.py\n```\n\n## 🌐 Change Web Search API\n\nTo use a different type of web search API, modify the `searcher_type` attribute in the `searcher_cfg` located in `mindsearch/agent/__init__.py`. Currently supported web search APIs include:\n\n- `GoogleSearch`\n- `DuckDuckGoSearch`\n- `BraveSearch`\n- `BingSearch`\n- `TencentSearch`\n\nFor example, to change to the Brave Search API, you would configure it as follows:\n\n```python\nBingBrowser(\n    searcher_type='BraveSearch',\n    topk=2,\n    api_key=os.environ.get('BRAVE_API_KEY', 'YOUR BRAVE API')\n)\n```\n\n## 🐞 Using the Backend Without Frontend\n\nFor users who prefer to interact with the backend directly, use the `backend_example.py` script. This script demonstrates how to send a query to the backend and process the response.\n\n```bash\npython backend_example.py\n```\n\nMake sure you have set up the environment variables and the backend is running before executing the script.\n\n## 🐞 Debug Locally\n\n```bash\npython -m mindsearch.terminal\n```\n\n## 📝 License\n\nThis project is released under the [Apache 2.0 license](LICENSE).\n\n## Citation\n\nIf you find this project useful in your research, please consider cite:\n\n```\n@article{chen2024mindsearch,\n  title={MindSearch: Mimicking Human Minds Elicits Deep AI Searcher},\n  author={Chen, Zehui and Liu, Kuikun and Wang, Qiuchen and Liu, Jiangning and Zhang, Wenwei and Chen, Kai and Zhao, Feng},\n  journal={arXiv preprint arXiv:2407.20183},\n  year={2024}\n}\n```\n\n## Our Projects\n\nExplore our additional research on large language models, focusing on LLM agents.\n\n- [Lagent](https://github.com/InternLM/lagent): A lightweight framework for building LLM-based agents\n- [AgentFLAN](https://github.com/InternLM/Agent-FLAN): An innovative approach for constructing and training with high-quality agent datasets (ACL 2024 Findings)\n- [T-Eval](https://github.com/open-compass/T-Eval): A Fine-grained tool utilization evaluation benchmark (ACL 2024)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finternlm%2Fmindsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finternlm%2Fmindsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finternlm%2Fmindsearch/lists"}