{"id":31286893,"url":"https://github.com/mohammedmogeab/rssagger","last_synced_at":"2025-09-24T10:51:25.534Z","repository":{"id":316088250,"uuid":"1061911014","full_name":"MohammedMogeab/rssagger","owner":"MohammedMogeab","description":"this for rss for post with go routines","archived":false,"fork":false,"pushed_at":"2025-09-22T15:37:31.000Z","size":4845,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-22T17:23:43.368Z","etag":null,"topics":["go","golang","golang-api","golang-application","golang-package","rss"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MohammedMogeab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-22T14:49:26.000Z","updated_at":"2025-09-22T15:37:34.000Z","dependencies_parsed_at":"2025-09-22T17:24:02.013Z","dependency_job_id":"d0d9bc90-cef5-4731-8686-76f84f279961","html_url":"https://github.com/MohammedMogeab/rssagger","commit_stats":null,"previous_names":["mohammedmogeab/rssagger"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/MohammedMogeab/rssagger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedMogeab%2Frssagger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedMogeab%2Frssagger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedMogeab%2Frssagger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedMogeab%2Frssagger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MohammedMogeab","download_url":"https://codeload.github.com/MohammedMogeab/rssagger/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedMogeab%2Frssagger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276737507,"owners_count":25695699,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-24T02:00:09.776Z","response_time":97,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","golang-api","golang-application","golang-package","rss"],"created_at":"2025-09-24T10:51:23.502Z","updated_at":"2025-09-24T10:51:25.529Z","avatar_url":"https://github.com/MohammedMogeab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rssagger\n\nA minimal RSS aggregator written in Go. It exposes a small REST API to:\n\n- Create users and authenticate via API key (Bearer token)\n- Create and list RSS feeds\n- Follow/unfollow feeds per user\n- Fetch posts from followed feeds\n- Continuously scrape feeds in the background\n\nIt uses Chi for routing, Postgres for storage, and sqlc for type‑safe DB access.\n\n## Requirements\n\n- Go 1.24+\n- PostgreSQL 13+\n\n## Setup\n\n1) Create a Postgres database and run the migrations in `sql/schema` in order:\n\n- `001_users.sql`\n- `002_users_apikey.sql`\n- `003_feeds.sql`\n- `004_feedsfollow.sql`\n- `005_feeds_fetch.sql`\n- `006_posts.sql`\n\nYou can run them with your preferred tool (psql, a GUI, or goose). Example with `psql`:\n\n```bash\npsql \"$DB_URL\" -f sql/schema/001_users.sql\npsql \"$DB_URL\" -f sql/schema/002_users_apikey.sql\npsql \"$DB_URL\" -f sql/schema/003_feeds.sql\npsql \"$DB_URL\" -f sql/schema/004_feedsfollow.sql\npsql \"$DB_URL\" -f sql/schema/005_feeds_fetch.sql\npsql \"$DB_URL\" -f sql/schema/006_posts.sql\n```\n\n2) Create a `.env` file in the project root:\n\n```env\nPORT=8080\nDB_URL=postgres://user:pass@localhost:5432/rssagger?sslmode=disable\n```\n\n3) Install deps and run:\n\n```bash\ngo run .\n# or\ngo build -o rssagger\n./rssagger\n```\n\nServer starts on `:$PORT` and mounts API under `/v1`.\n\n## Authentication\n\n- Create a user to receive an API key.\n- Send the API key on protected routes using the `Authorization: Bearer \u003capi_key\u003e` header.\n\n## API\n\nBase URL: `http://localhost:\u003cPORT\u003e/v1`\n\n- Health: `GET /healthz`\n  - Returns `{}` (currently implemented as an empty JSON response).\n\n- Test error: `GET /error`\n  - Returns `{ \"error\": \"something is wrong\" }` with status 400.\n\n- Create user: `POST /users`\n  - Body: `{ \"name\": \"Alice\" }`\n  - Response: user object including `api_key`.\n\n- Get current user: `GET /users` (auth required)\n  - Header: `Authorization: Bearer \u003capi_key\u003e`\n  - Response: user object.\n\n- Create feed: `POST /feeds` (auth required)\n  - Body: `{ \"name\": \"Go Blog\", \"url\": \"https://blog.golang.org/index.atom\" }`\n  - Response: feed object.\n\n- List feeds: `GET /feeds`\n  - Response: array of feeds.\n\n- Follow a feed: `POST /feeds/follow` (auth required)\n  - Body: `{ \"feed_id\": \"\u003cfeed UUID\u003e\" }`\n  - Response: follow relationship object.\n\n- List my follows: `GET /feeds/follow` (auth required)\n  - Response: array of follow relationships.\n\n- Unfollow: `DELETE /feeds/follow/{feedfollowId}` (auth required)\n  - Path param: `feedfollowId` is the follow relationship UUID.\n\n- Get my posts: `GET /posts` (auth required)\n  - Returns posts for feeds you follow, ordered by `published_at` desc.\n  - Optional query: `?limit=10` (defaults to 4, max 100).\n\n### Example cURL\n\n```bash\n# Create user\ncurl -sX POST http://localhost:8080/v1/users \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"name\":\"Alice\"}'\n\n# Set API key\nAPI_KEY=... # from the response\n\n# Create a feed\ncurl -sX POST http://localhost:8080/v1/feeds \\\n  -H \"Authorization: Bearer $API_KEY\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"name\":\"Go Blog\",\"url\":\"https://blog.golang.org/index.atom\"}'\n\n# List feeds\ncurl -s http://localhost:8080/v1/feeds\n\n# Follow a feed\ncurl -sX POST http://localhost:8080/v1/feeds/follow \\\n  -H \"Authorization: Bearer $API_KEY\" \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"feed_id\":\"\u003cFEED_UUID\u003e\"}'\n\n# Get my posts\ncurl -s http://localhost:8080/v1/posts \\\n  -H \"Authorization: Bearer $API_KEY\"\n```\n\n## Background Scraper\n\nThe server starts a background worker that:\n\n- Periodically selects the next feeds to fetch (by `last_fetch_at` nulls‑first)\n- Fetches each feed concurrently\n- Parses items from the RSS/Atom feed\n- Inserts posts (skipping duplicates by unique URL)\n- Marks the feed as fetched (`last_fetch_at = now()`)\n\nConfigurable via env:\n\n- `SCRAPER_CONCURRENCY` (default 3)\n- `SCRAPER_INTERVAL_SECONDS` (default 60)\n\n## Project Structure\n\n- `main.go` – server setup, routes, middleware\n- `middleware.go` – API key auth wrapper\n- `UserHandler.go`, `handler_feed.go` – HTTP handlers\n- `json.go` – JSON helpers\n- `models.go` – response models and mappers\n- `rss.go` – RSS fetch/parse helpers\n- `scrapper.go` – background scraping logic\n- `internal/database` – sqlc‑generated DB layer\n- `sql/schema` – SQL migrations\n- `sql/queries` – SQL queries (sqlc input)\n\n## Development Notes\n\n- CORS is enabled for local development; see `main.go` for origins.\n- DB access code is generated with `sqlc` using `sqlc.yaml`.\n- Errors are returned as JSON `{ \"error\": \"...\" }` with appropriate HTTP status codes.\n\n## License\n\nNo license specified. Add one if needed before distributing.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohammedmogeab%2Frssagger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmohammedmogeab%2Frssagger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohammedmogeab%2Frssagger/lists"}