https://github.com/krishpatel023/go-rss-scraper
A modern RSS feed aggregator that synchronizes feeds automatically, provides a clean UI, and ensures fast, reliable content fetching.
https://github.com/krishpatel023/go-rss-scraper
golang nextjs postgresql rss-aggregator rss-reader sqlc typescript
Last synced: 3 months ago
JSON representation
A modern RSS feed aggregator that synchronizes feeds automatically, provides a clean UI, and ensures fast, reliable content fetching.
- Host: GitHub
- URL: https://github.com/krishpatel023/go-rss-scraper
- Owner: krishpatel023
- Created: 2025-02-05T13:14:09.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-02-14T13:05:50.000Z (over 1 year ago)
- Last Synced: 2025-08-16T16:30:47.634Z (11 months ago)
- Topics: golang, nextjs, postgresql, rss-aggregator, rss-reader, sqlc, typescript
- Language: TypeScript
- Homepage:
- Size: 147 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# RSS Feed Scraper & Aggregator
[](https://nextjs.org/)
[](https://golang.org/)
[](https://www.typescriptlang.org/)
[](https://www.postgresql.org/)
> A modern RSS feed aggregator that helps you stay up-to-date with content from multiple sources in one place. Built with Next.js frontend and Go backend, featuring automatic feed synchronization and a clean, accessible interface.
## 🌟 Overview
This RSS Feed Scraper is a personal project designed to provide a clean, efficient way to track various blogs, news sites, and content creators. Unlike commercial RSS readers, it focuses on simplicity and speed, letting you aggregate all your favorite feeds in one place without unnecessary social features or recommendations.
## ✨ Features
### Feed Management
- **Multiple RSS Sources**: Add any number of RSS feeds from blogs, news sites, and podcasts with automatic feed URL validation and metadata extraction.
- **Automatic Synchronization**: Stay up-to-date with fresh content through our efficient background synchronization system that fetches new articles every minute. Currently limited to 10 feeds at a time but can be easily increased.
- **Feed Validation**: Robust error handling system that validates feed formats and maintains feed health monitoring.
## 🚀 Tech Stack
### Frontend
- Next.js 15.1 with App Router
- TypeScript 5.0
- Tailwind CSS for styling
- ShadcnUI components
- Zod for validation
- Axios for API requests
### Backend
- Go 1.23.2
- Chi router for HTTP routing
- PostgreSQL with SQLC
- JWT authentication
- Bcrypt for password hashing
## 📦 Installation
### Prerequisites
- Go 1.23.2 or higher
- Node.js 18 or higher
- PostgreSQL
- Make (for running Makefile commands)
### Backend Setup
1. Navigate to backend directory:
```bash
cd backend
```
2. Install Go dependencies:
```bash
go mod download
```
3. Create and configure environment variables:
```bash
cp .env.example .env
```
4. Configure your `.env` with:
```bash
PORT=8000
DATABASE_URL=postgresql://[user]:[password]@localhost:5432/[dbname]
```
5. Run database migrations:
```bash
make backend_migrate_up
```
6. Build and run the backend:
```bash
# Build the backend
make backend_build
# Run the backend
make backend_run
```
Additional backend commands:
```bash
# Clean up backend build artifacts
make backend_clean
# Generate SQLC code from SQL
make backend_sqlc
# Run database migrations down
make backend_migrate_down
```
### Frontend Setup
1. Navigate to frontend:
```bash
cd frontend
```
2. Install dependencies:
```bash
npm install
```
3. Create and configure environment variables:
```bash
cp .env.example .env
```
4. Configure your `.env` with:
```bash
NEXT_PUBLIC_BACKEND_URL=http://localhost:8000
```
## 🚀 Usage
### Starting the Frontend
```bash
cd frontend
npm run dev
```
Frontend will be available at `http://localhost:3000`
### Starting the Backend
```bash
make backend_run
```
Backend will be available at `http://localhost:8000`
## 🔒 Authentication Flow
The application uses API key-based authentication:
1. User signs up/logs in and receives an API key
2. API key is stored in HTTP-only cookies
3. Subsequent requests include the API key in Authorization header
4. Protected routes verify the API key before processing requests