https://github.com/polyrabbit/hacker-news-digest
:newspaper: Let ChatGPT Summarize Hacker News for You
https://github.com/polyrabbit/hacker-news-digest
chatgpt chatgpt-api crawler data-extraction extract-summaries hacker-news hacker-news-digest hacker-news-reader machine-learning news-aggregator openai openai-api python rss spider
Last synced: 1 day ago
JSON representation
:newspaper: Let ChatGPT Summarize Hacker News for You
- Host: GitHub
- URL: https://github.com/polyrabbit/hacker-news-digest
- Owner: polyrabbit
- License: lgpl-3.0
- Created: 2014-09-17T13:34:44.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2025-04-01T02:30:19.000Z (15 days ago)
- Last Synced: 2025-04-07T02:14:43.373Z (9 days ago)
- Topics: chatgpt, chatgpt-api, crawler, data-extraction, extract-summaries, hacker-news, hacker-news-digest, hacker-news-reader, machine-learning, news-aggregator, openai, openai-api, python, rss, spider
- Language: Python
- Homepage: http://hackernews.betacat.io/
- Size: 4.68 MB
- Stars: 713
- Watchers: 18
- Forks: 92
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-ChatGPT-repositories - hacker-news-digest - :newspaper: Let ChatGPT Summarize Hacker News for You (NLP)
- awesome-chatgpt - polyrabbit/hacker-news-digest - Let ChatGPT Summarize Hacker News for You (ChatGPT-based applications / Other sdk/libraries)
README
[Let ChatGPT Summarize Hacker News for You](https://hackernews.betacat.io/)
==================
[](https://github.com/polyrabbit/hacker-news-digest/actions/workflows/static.yml)
[](https://github.com/polyrabbit/hacker-news-digest/blob/master/LICENSE)
[](https://github.com/polyrabbit/hacker-news-digest/pulls)
[](https://hackernews.betacat.io/)
> [中文版](https://blog.betacat.io/post/2023/06/summarize-hacker-news-by-chatgpt/)
[Hacker News Summary](https://hackernews.betacat.io/) leverages AI technology to extract summaries
and illustrations from [Hacker News](https://news.ycombinator.com/)
articles, providing a seamless news scanning experience.
Summaries are primarily generated by
ChatGPT [gpt-3.5-turbo](https://platform.openai.com/docs/models/gpt-3-5) model, and fallback to
local [GoogleT5](https://huggingface.co/t5-large) model when ChatGPT is not available.
## Features
* Clear and easily understandable summaries generated by our advanced AI assistant
* Relevant illustrations make articles easily scannable and visually engaging
* Common video sites, PDFs, and GitHub gists are seamlessly embedded
* Flexibility to sort articles based on their points, comment count, or publication time
* Filter the topN articles based on their points.
* RSS feeds fully supported ([#14](https://github.com/polyrabbit/hacker-news-digest/issues/14), [#19](https://github.com/polyrabbit/hacker-news-digest/issues/19))
* Local translation (Chinese)
## Talk is cheap, show me the screenshot!
Emoji explained:
* ❤️: point - upvotes received from the Hacker News community
* 👤: user - Hacker News user who submitted this post
* 🕘: submission time - a human-readable time indicating when the post was submitted
* 💬: comment count - comments posted by the community, click to visit this comment page
* 🔗: source of the news - where the news originated
* 📰: summary model - which model is used to generate the summary, options
are `OpenAI`, `GoogleT5` and `Prefix`
## How it works
[Hacker News Summary](https://hackernews.betacat.io/) is a static site hosted on GitHub Pages. It
performs the following periodic actions:
1. Parsing the Hacker News page to obtain a list of news articles
2. Extracting the main content from each news article using
a [score algorithm](%5Btutorial%5D%20How-to-extract-main-content-from-web-pages-using-Machine-Learning.ipynb)
3. Finding the most suitable illustration for each article and making a local copy
4. Generating summaries of the article's content using OpenAI API or invoking a local model as a
fallback when the API is unavailable
5. Rendering a template that incorporates the illustrations and summaries, and deploying it to
GitHub Pages
## Localization
Translation is also performed by ChatGPT, with a single extra step in the prompt. Currently supported languages:
* [中文翻译](https://hackernews.betacat.io/zh.html)
## TODO
- [ ] A better way to scrap websites (maybe PhantomJS & Selenium)
- [ ] Also summarize comments ([see discussions on Hacker News](https://news.ycombinator.com/item?id=36260140))
- [ ] Switch to [Hacker News API](https://github.com/HackerNews/API)
- [ ] A more beautiful home page (maybe in HTML9)
- [ ] Discover an alternative local models for generating summaries
- [X] Sort articles by points/comments/time
- [X] Filter topN articles by points
- [X] RSS
- [X] Deploy on github pages
- [X] Have a good sleep !important