https://github.com/minhhungit/github-action-rss-crawler
Auto crawl RSS feeds using Github Action
https://github.com/minhhungit/github-action-rss-crawler
crawler csharp github-actions litedb netcore rss rss-crawler rss-items
Last synced: 11 months ago
JSON representation
Auto crawl RSS feeds using Github Action
- Host: GitHub
- URL: https://github.com/minhhungit/github-action-rss-crawler
- Owner: minhhungit
- Created: 2020-08-09T17:24:45.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2025-06-28T04:06:01.000Z (12 months ago)
- Last Synced: 2025-06-28T05:21:46.408Z (12 months ago)
- Topics: crawler, csharp, github-actions, litedb, netcore, rss, rss-crawler, rss-items
- Language: HTML
- Homepage: https://minhhungit.github.io/github-action-rss-crawler/
- Size: 6.32 GB
- Stars: 33
- Watchers: 2
- Forks: 12
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-rainmana - minhhungit/github-action-rss-crawler - Auto crawl RSS feeds using Github Action (HTML)
README
## Rss auto crawling using Github Action
**Github Action does all these steps automatically, it run rss crawler every 4 hours**
Steps:
- Github will pull repository, build and run crawler code (crawling code is C# (.net core), github will run it directly)
- Read channel urls from LiteDB
- Fetch rss feed items
- Insert feed items into LiteDB after checking blacklist and existing
- Generate all rss items to static page (index.html - https://minhhungit.github.io/github-action-rss-crawler/ )
- Commit change (litedb database & index.html page) and push to this repo
### Workflow
```yml
on:
schedule:
# Runs every 4h
- cron: '0 */4 * * *'
workflow_dispatch:
jobs:
update-readme-with-blog:
name: Crawl rss and generate static page
runs-on: windows-2019
steps:
- uses: actions/checkout@main
with:
repository: minhhungit/github-action-rss-crawler
token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/setup-dotnet@v1
with:
dotnet-version: 3.1.x
#- run: dotnet build DemoApp\DemoApp.sln
- run: dotnet run --project RssCrawler\RssCrawler.csproj
- run: git config --local user.email "it.minhhung@gmail.com"
- run: git config --local user.name "Jin"
- run: git add .
- run: git commit -m "Add changes"
- run: git push
```
---
### Demo
> https://minhhungit.github.io/github-action-rss-crawler/
### Donate ^^
**If you like my works and would like to support then you can buy me a coffee ☕️ anytime**
**I would appreciate it!!!**