https://github.com/jiggy-ai/hn_summary

Summarizes top stories from Hacker News using a large language model and post them to a Telegram channel.
https://github.com/jiggy-ai/hn_summary

gpt gpt-3 gpt-3-prompts gpt3 hacker-news hackernews llm news telegram-bot

Last synced: 11 days ago
JSON representation

Summarizes top stories from Hacker News using a large language model and post them to a Telegram channel.

• Host: GitHub
• URL: https://github.com/jiggy-ai/hn_summary
• Owner: jiggy-ai
• License: apache-2.0
• Created: 2022-11-25T20:47:51.000Z (almost 2 years ago)
• Default Branch: master
• Last Pushed: 2024-05-15T01:58:44.000Z (6 months ago)
• Last Synced: 2024-07-31T08:14:09.148Z (3 months ago)
• Topics: gpt, gpt-3, gpt-3-prompts, gpt3, hacker-news, hackernews, llm, news, telegram-bot
• Language: Python
• Homepage:
• Size: 207 KB
• Stars: 242
• Watchers: 7
• Forks: 20
• Open Issues: 6
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome - jiggy-ai/hn\_summary - Summarizes top stories from Hacker News using a large language model and post them to a Telegram channel. (Python)
• awesome-ChatGPT-repositories - hn_summary - Summarizes top stories from Hacker News using a large language model and post them to a Telegram channel. (NLP)

README

        


 



**HN Summary**  is an open source bot which sumarizes top stories on Hacker News and publishes the summaries to a Telegram channel.

Join the **HN Summary** channel on Telegram to see the bot in action and enjoy the story summaries: \

https://t.me/hn_summary

Flag bad summaries on the telegram channel with 👎 to help mitigate and improve.

You can find summaries of the current top Hacker News articles here as well: \

[https://news.jiggy.ai](https://news.jiggy.ai)

Feel free to open PR/issue or dm me at @wskish on telegram or twitter with feedback.

## Overview

Whenever a new story appears on the Hacker News API /topstories.json endpoint, this bot summarizes it (currently using OpenAI gpt-3.5-turbo) and sends the Story title, summary, and url to the hn_summary channel on Telegram.

The purpose of this project is to help build intuition on the capabilities of the current generation of large language models while surfacing a broader swath of top Hacker News content.  It could also serve as a platform for experimentation with other language model capabilities such as semantic search.

## Limitations

Large language models such as GPT-3 are prone to crazy hallucinations and sometimes make things up while writing in a very authoritative tone.

The code for extracting text from html is very basic and error prone. (PR's welcome.)  In addition many sites (such as news sites) are either paywalled or make it difficult to extract text. We now attempt to catch this case via prompt engineering but when one does slip through we tend to get fanciful hallucinations based on just the title and FQDN.

Links to content types other than PDF and HTML are currently ignored.

Text extraction from reddit and twitter and other commercial links are broken and probably produce wildly hallucinated summaries. 

Telegram messages are limited to 4K. Currently the response is truncated to 4K.

## Major Dependencies

The following environment variables are used to inject credentials and other required configuration for the major dependencies:

**OpenAI**

* OPENAI_API_KEY # your OpenAI API key

**PostgresQL** 

Database for keeping track of items we have already seen and associated item info.

- HNSUM_POSTGRES_HOST  # The database FQDN

- HNSUM_POSTGRES_USER  # The database username

- HNSUM_POSTGRES_PASS  # The database password

**Telegram**

  

* HNSUM_TELEGRAM_API_TOKEN # The bot's telegram API token

* HNSUM_TELEGRAM_CHANNEL_ID # the telegram chat where the bot will post the summaries