https://github.com/daveshap/quickly_extract_science_papers
Scientific papers are coming out TOO DAMN FAST so we need a way to very quickly extract useful information.
https://github.com/daveshap/quickly_extract_science_papers
Last synced: 6 months ago
JSON representation
Scientific papers are coming out TOO DAMN FAST so we need a way to very quickly extract useful information.
- Host: GitHub
- URL: https://github.com/daveshap/quickly_extract_science_papers
- Owner: daveshap
- License: mit
- Created: 2023-07-06T15:35:46.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-07-09T23:59:54.000Z (almost 3 years ago)
- Last Synced: 2025-01-23T04:12:40.495Z (over 1 year ago)
- Language: HTML
- Size: 29 MB
- Stars: 145
- Watchers: 5
- Forks: 47
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Quickly_Extract_Science_Papers
Scientific papers are coming out TOO DAMN FAST so we need a way to very quickly extract useful information.
## Repo Contents
- `chat.py` - this file is a simple chatbot that will chat with you about the contents of `input.txt` (you can copy/paste anything into this text file). Very useful to quickly discuss papers.
- `generate_multiple_reports.py` - this will consume all PDFs in the `input/` folder and generate summaries in the `output/` folder. This is helpful for bulk processing such as for literature reviews.
- `render_report.py` - this will render all the reports in `output/` to a an *easier* to read file in `report.html`.
## EXECUTIVE SUMMARY
This repository contains Python scripts that automate the process of generating reports from PDF files using OpenAI's
GPT-4 model. The scripts extract text from PDF files, send the text to the GPT-4 model for processing, and save the
generated reports as text files. The scripts also include functionality to render the generated reports as an HTML
document for easy viewing.
## SETUP
1. Clone the repository to your local machine.
2. Install the required Python packages by running `pip install -r requirements.txt` in your terminal.
3. Obtain an API key from OpenAI and save it in a file named `key_openai.txt` in the root directory of the repository.
4. Place the PDF files you want to generate reports from in the `input/` directory.
## USAGE
1. Run the `generate_multiple_reports.py` script to generate reports from the PDF files in the `input/` directory. The
generated reports will be saved as text files in the `output/` directory.
2. Run the `render_report.py` script to render the generated reports as an HTML document. The HTML document will be
saved as `report.html` in the root directory of the repository.
3. You can modify the `prompts` in `generate_multiple_reports.py` to focus on any questions you would like to ask. In other words you can automatically ask any set of questions in bulk against any set of papers. This can help you greatly accelerate your literature reviews and surveys.
## NOTE
The scripts are designed to handle errors and retries when communicating with the OpenAI API. If the API returns an
error due to the maximum context length being exceeded, the scripts will automatically trim the oldest message and retry
the API call. If the API returns any other type of error, the scripts will retry the API call after a delay, with the
delay increasing exponentially for each consecutive error. If the API returns errors for seven consecutive attempts, the
scripts will stop and exit.