Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/zmactep/gpt-digest
A neural network generated daily digest based on RSS feeds
https://github.com/zmactep/gpt-digest
digest docker docker-compose gpt news rss telegram
Last synced: about 2 months ago
JSON representation
A neural network generated daily digest based on RSS feeds
- Host: GitHub
- URL: https://github.com/zmactep/gpt-digest
- Owner: zmactep
- License: bsd-3-clause
- Created: 2023-04-23T19:18:02.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2023-04-25T20:38:45.000Z (over 1 year ago)
- Last Synced: 2024-08-02T15:56:15.219Z (5 months ago)
- Topics: digest, docker, docker-compose, gpt, news, rss, telegram
- Language: Python
- Homepage:
- Size: 19.5 KB
- Stars: 5
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# GPT Digest
[Try it! (en)](https://t.me/gptbionews) [Try it! (ru)](https://t.me/gptbionewsrus)
> This project was created to read daily digests based on **biotech news**. You may need to edit prompts for your purpose.
This repository includes a project designed to download RSS feeds, filter them with regular expressions, and generate daily digests using OpenAI GPT models (GPT-3.5 and GPT-4 are used for different stages of the pipeline). Here's a quick overview:
1. Download RSS feeds: The project reads data from various RSS feeds.
2. Filter using regex: Filters the content of the feeds based on pre-defined regular expressions.
3. Detect topics: Utilizes OpenAI's GPT to analyze and categorize text into topics.
4. Generate daily digest: Rounds up the categorized content and generates a daily digest using OpenAI GPT models.
5. Translates digest to Russian language.
6. Publishes digests for web access.
7. Publishes highlights and link to the digest to Telegram channel.
Enjoy exploring the code and feel free to contribute!
*The text above is generated by GPT-4*
## Prerequires
- conda
- docker (with compose plugin)
- Telegram bot
- 2 or 3 Telegram channels
- public domain
## Installation
Create one or more CSV source files with RSS feeds using this format:
```csv
My Feed; https://example.com/rss
```
Create one or more TXT keywords files with regexps you want to use to filter:
```
PD-?1
```
Create a conda environment:
```bash
cconda env create -f environment.yml
```
Set these environment variables:
```bash
export DIGEST_PATH="..." # Path to the current repository local installation
export CONDA_PATH="..." # Path of conda installation
export OPENAI_API_KEY="sk-..." # OpenAI API Key, should support GPT-4
export TELEGRAM_DIGEST_KEY="..." # Telegram Bot API, obtained from @BotFather
export TELEGRAM_DIGEST_EN_CHANNEL="..." # English channel chat_id
export TELEGRAM_DIGEST_RU_CHANNEL="..." # Russian channel chat_id
export TELEGRAM_DIGEST_TEST_CHANNEL="..." # Test channel chat_id
export DIGEST_DOMAIN="example.com" # FQDN of the domain for web access
```
Create digest docker image:
```bash
docker build . -t digest
```
Copy `cron_task.sh` to cron directory with the period that you wish, e.g.:
```bash
cp cron_task.sh /etc/cron.daily/
```
Run a compose with
## Test usage
You can use a test mode to send all output to a test channel:
```bash
TEST_RUN=1 ./cron_task.sh
```
You are also able to use different components of the package:
```bash
% python -m digest.generate --help
usage: digest.generate [-h] [--output OUTPUT] [--summaries] [--highlights]
[--fix-links]
options:
-h, --help show this help message and exit
--output OUTPUT custom path to write digest
--summaries add topics summaries
--highlights add daily highlights
--fix-links fix meshed up links
% python -m digest.translate --help
usage: digest.translate [-h] [--input INPUT] [--output OUTPUT]
options:
-h, --help show this help message and exit
--input INPUT custom path of digest to translate
--output OUTPUT custom path to write translated digest
% python -m digest.telegram --help
usage: digest.telegram [-h] --input INPUT_PATH [--english] [--russian]
[--only-highlights]
options:
-h, --help show this help message and exit
--input INPUT_PATH input path of digest
--english work with english channel
--russian work with russian channel
--only-highlights send ony highlights and a URL
```