{"id":13593292,"url":"https://github.com/namuan/dr-doc-search","last_synced_at":"2025-05-16T07:04:41.401Z","repository":{"id":65169002,"uuid":"584935389","full_name":"namuan/dr-doc-search","owner":"namuan","description":"Converse with book - Built with GPT-3","archived":false,"fork":false,"pushed_at":"2024-10-01T00:11:26.000Z","size":30202,"stargazers_count":601,"open_issues_count":7,"forks_count":60,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-06T17:11:22.571Z","etag":null,"topics":["gpt3","langchain","nlp-machine-learning","python","summarization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/namuan.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-03T22:31:17.000Z","updated_at":"2025-03-21T21:20:02.000Z","dependencies_parsed_at":"2024-02-08T23:03:10.252Z","dependency_job_id":"d1f67841-f889-4730-b6cc-001d88909c5f","html_url":"https://github.com/namuan/dr-doc-search","commit_stats":{"total_commits":64,"total_committers":2,"mean_commits":32.0,"dds":0.015625,"last_synced_commit":"5b64a27964691b1339b693bf300cd7b6153ef9b8"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fdr-doc-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fdr-doc-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fdr-doc-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namuan%2Fdr-doc-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/namuan","download_url":"https://codeload.github.com/namuan/dr-doc-search/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254485055,"owners_count":22078767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpt3","langchain","nlp-machine-learning","python","summarization"],"created_at":"2024-08-01T16:01:18.846Z","updated_at":"2025-05-16T07:04:36.390Z","avatar_url":"https://github.com/namuan.png","language":"Python","funding_links":[],"categories":["Langchain","Python","Research AI Tools","开源项目","Open Source Projects"],"sub_categories":["AI Tools for Research","其他聊天机器人","Other / Chatbots"],"readme":"# Doc Search\n\n[![PyPI](https://img.shields.io/pypi/v/dr-doc-search?style=flat-square)](https://pypi.python.org/pypi/dr-doc-search/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dr-doc-search?style=flat-square)](https://pypi.python.org/pypi/dr-doc-search/)\n[![PyPI - License](https://img.shields.io/pypi/l/dr-doc-search?style=flat-square)](https://pypi.python.org/pypi/dr-doc-search/)\n\nConverse with a book (PDF)\n\n![](assets/dr-doc-search-github-demo.gif)\n\nSee [tweet](https://twitter.com/deskriders_twt/status/1612088387984588802) for full demo.\n\n---\n\n**Documentation**: [https://namuan.github.io/dr-doc-search](https://namuan.github.io/dr-doc-search)\n\n**Source Code**: [https://github.com/namuan/dr-doc-search](https://github.com/namuan/dr-doc-search)\n\n**PyPI**: [https://pypi.org/project/dr-doc-search/](https://pypi.org/project/dr-doc-search/)\n\n---\n\n## Pre-requisites\n\n- [Tessaract OCR](https://github.com/tesseract-ocr/tesseract)\n- [ImageMagick](https://imagemagick.org/index.php)\n\n\u003e **Note:**\n\u003e If you are using Windows, then make sure that you set the location\n\u003e of ImageMagick executable in the `IMCONV` environment variable.\n\n```shell\n# For example, if you have installed ImageMagick in PROGRAMFILES\\ImageMagick-7.1.0-Q16-HDRI\nset IMCONV=\"%PROGRAMFILES%\\ImageMagick-7.1.0-Q16-HDRI\\magick\"\n```\n\n## Installation\n\n```sh\npip install dr-doc-search\n```\n\n## Example Usage\n\nThere are two steps to use this application:\n\n**1.** First, you need to create the index and generate embeddings for the PDF file.\nHere I'm using a PDF file generated from this page [Parable of a Monetary Economy\n   ](http://heteconomist.com/parable-of-a-monetary-economy/)\n\nBefore running this, you need to set up your OpenAI API key. You can get it from [OpenAI](https://beta.openai.com/account/api-keys).\n\n\u003e From version 1.5.0, you can skip OpenAI and use HuggingFace models to generate embeddings and answers.\n\n```shell\nexport OPENAI_API_KEY=\u003cyour-openai-api-key\u003e\n```\n\nThe run the following command to start the training process:\n\n```shell\ndr-doc-search --train -i ~/Downloads/parable-of-a-monetary-economy-heteconomist.pdf\n```\n\nUse `huggingface` for generating embeddings:\n\n```shell\ndr-doc-search --train -i ~/Downloads/parable-of-a-monetary-economy-heteconomist.pdf --embedding huggingface\n```\n\nThe training process generates some temporary files in the `OutputDir/dr-doc-search/\u003cpdf-name\u003e` folder under your home directory.\nHere is what it looks like:\n\n```text\n ~/OutputDir/dr-doc-search/parable-of-a-monetary-economy-heteconomist\n$ tree\n.\n├── images\n│ ├── output-1.png\n│ ├── output-10.png\n│ ├── output-11.png\n...\n│ └── output-9.png\n├── index\n│ ├── docsearch.index\n│ └── index.pkl\n├── parable-of-a-monetary-economy-heteconomist.pdf\n└── scanned\n    ├── output-1.txt\n    ...\n    └── output-9.txt\n```\n\n\u003e **Note:**\n\u003e It is possible to change the base of the output directory by providing the `--app-dir` argument.\n\n**2.** Now that we have the index, we can use it to start asking questions.\n\n```shell\ndr-doc-search -i ~/Downloads/parable-of-a-monetary-economy-heteconomist.pdf --input-question \"How did the attempt to reduce the debut resulted in decrease in employment?\"\n```\n\nOr You can open up a web interface (on port :5006) to ask questions:\n\n```shell\ndr-doc-search --web-app -i ~/Downloads/parable-of-a-monetary-economy-heteconomist.pdf\n```\n\nTo use `huggingface` model, provide the `--llm` argument:\n\n```shell\ndr-doc-search --web-app -i ~/Downloads/parable-of-a-monetary-economy-heteconomist.pdf --llm huggingface\n```\n\nThere are more options for choose the start and end pages for the PDF file.\nSee the help for more details:\n\n```shell\ndr-doc-search --help\n```\n\n## Acknowledgements\n\n- [anton/@abacaj](https://twitter.com/abacaj/status/1608163940726358024) for the idea\n- [LangChain](https://github.com/hwchase17/langchain)\n- [HoloViz Panel](https://panel.holoviz.org/)\n- [OpenAI](https://beta.openai.com/)\n\n## Development\n\n* Clone this repository\n* Requirements:\n  * Python 3.7+\n  * [Poetry](https://python-poetry.org/)\n\n* Create a virtual environment and install the dependencies\n```sh\npoetry install\n```\n\n* Activate the virtual environment\n```sh\npoetry shell\n```\n\n### Validating build\n```sh\nmake build\n```\n\n### Release process\nA release is automatically published when a new version is bumped using `make bump`.\nSee `.github/workflows/build.yml` for more details.\nOnce the release is published, `.github/workflows/publish.yml` will automatically publish it to PyPI.\n\n### Disclaimer\n\nThis project is not affiliated with OpenAI.\nThe OpenAI API and GPT-3 language model are not free after the trial period.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamuan%2Fdr-doc-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnamuan%2Fdr-doc-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamuan%2Fdr-doc-search/lists"}